We have recently made an update to the data tracking logic used in the Training module on the platform.
The update will remove the ‘time played’ requirement for ‘Security Maturity’ and will align tracked metrics like “Accuracy”, “Time spent”, “Challenges played” and “Confidence Level” to those reported in the Courses module, ensuring a more consistent view of your developer’s overall progress and the success of your application security program.
If you have any questions on how this update may affect your reporting, please contact customer support.
This article details how the points system works in the Secure Code Warrior® platform to help increase engagement and measure your team's secure coding prowess.
Definitions
Accuracy
Accuracy is a ratio between the number of attempts made and those answered correctly or incorrectly.
Accuracy is calculated by dividing the number of correct attempts by the total of both incorrect and correct attempts and multiplying that by 100.
Accuracy = (# Correct attempts /# of total attempts) * 100 For example, a Player answering everything correctly the first time will have a high Accuracy score while a Player that answers incorrectly or 'guesses' more often would have a lower Accuracy score.
Confidence
Confidence is a ratio between the total number of Hints available and those used.
Confidence = (#total hints - #hints used)/#total hints
#total hints are hints that are available for challenges that the user has completed. For example: if a player doesn't use any Hints they'll have a high confidence score. Players who frequently use Hints will have a lower confidence score.
How is Time measured?
Challenge Time Played is the amount of time a player has spent actively in training. The time is tracked once a challenge has started and stopped once they submit an answer. If the player navigates away from the challenge without submitting an answer or is inactive (ie. no mouse movement, keyboard interaction, or touch gesture) for more than 5 minutes, the time spent on that specific challenge is discarded.
Team Managers and Company Administrators can also see the total time a player interacts on the platform in Learning Resources (Videos), Training, Assessments, Tournaments. This is referred to as Time Spent.
Time Spent is determined by monitoring the active Playing mode every 30 seconds. If the player navigates away from the platform (closes the browser window or switches browser tabs) or is inactive for more than 10 minutes, the time spent is discarded.
How are experience Points calculated?
The possible number of Points earned for each challenge is determined by the Secure Code Warrior Security Competency Algorithm Metric or SCAM, much like the famous Google Algorithm for search. SCAM calculates a number of factors such as; Playing Stage, Challenge Difficulty, Application Type, Hint Used, and Failed Attempts (or guesses!) to derive the Players' Accuracy, Confidence, and Points.
Training Mode
Training Points are the unit of score in Secure Code Warrior and they're earned by completing missions in the Training Ground.
The total of all earned points is called the Points Score and is the major contributor to leveling up a developer's Security Maturity level to earn certifications and bragging rights.
Note: Security Maturity might be disabled by a Company Administrator
Tournament Mode
In Tournament Mode players compete against each other to see who can score the most points in a given period of time by correctly completing a stage. The harder the stage, the higher the potential points available.
There are two types of activities in tournaments:
Challenges:
There are eight (8) challenge levels. A Challenge Stage is defined as; Locate, Identify, Fix, Locate & Fix, or Identify & Fix.
Tournament Players can view the detailed scoring rules by clicking on the Information icon to expand the detailed scoring rules as shown below.
There are three preset tournament scoring modes that allow more or fewer attempts to be played. These can be configured by the Team Manager or Company Administrator when setting up a Tournament.
The following table outlines the scoring for each.
Challenges (Level 1 - 8)
| Default | Forgiving | Aggressive | |
|---|---|---|---|
| Allowed Attempts | 3 | 5 | 2 |
| Maximum Points (per Task) | |||
| Easy | 100 | ||
| Medium | 200 | ||
| Hard | 300 | ||
| The Proportion of Points Awarded/Penalised | |||
| Attempt 1 | 100% | 100% | 100% |
| Attempt 2 | 60% | 60% | 60% |
| Attempt 3 | 30% | 30% | - |
| Attempt 4 | - | 10% | - |
| Attempt 5 | - | 5% | - |
| Hint Penalties | Locate | Identify | Fix |
| Hint 1 |
0% |
0% |
-35% |
| Hint 2 |
-5% |
-50% |
-33% |
| Hint 3 |
-35% |
- |
-34% |
| Hint 4 |
-60% |
- |
- |
Missions:
If enabled, this option activates five (5) playable Bonus levels within a tournament. Missions are hands-on, interactive coding simulations designed to immerse developers in real-world applications to see, first hand, the impact of when certain vulnerabilities are introduced in the code.
Tournament Players can view the detailed scoring rules by clicking on the Information icon to expand the detailed scoring rules as shown below.
Bonus Levels
| Default | Forgiving | Aggressive | |
|---|---|---|---|
| Maximum Points (per Challenge Stage) | |||
| Easy | 200 | ||
| Medium | 400 | ||
| Hard | 600 | ||
| Hint Penalties | Penalty as a % of maximum earned points | ||
| Small Hint | 10% | 10% |
10% |
| Medium Hint | 30% |
30% |
30% |
| Large Hint | 50% |
50% |
50% |
| Maximum cumulative penalty |
80% |
60% |
100% |
Assessment Mode
Assessments are calculated in a similar manner as Training.
Using SCAM, points are normalized to enable a fair comparison across different Language/Framework challenges. The Assessment Score is displayed as a percentage.
The Team Manager or Company Administrator can configure a 'pass/fail' score.
Courses Mode
Courses are skills-based learning pathways that progressively build secure coding competency for developers of all skill levels. Educational hands-on coding challenges and missions, build transferable skills that help your organization achieve compliance and reduce recurring vulnerabilities.
How are base rewards calculated?
| Difficulty | Challenges | Walkthrough Missions | Standard Mission |
| Easy | 100 points | 100 points | 200 points |
| Medium | 200 points | (Not Applicable) | 400 points |
| Hard | 300 points | (Not Applicable) | 600 points |
Points earned for additional attempts:
This happens when a user repeats a course or plays the same module again. This only applies to challenges, it doesn't apply to videos, custom activities, standard missions, or walkthrough missions.
| Challenge Difficulty | Points |
| Easy difficulty | 10 points |
| Medium difficulty | 20 points |
| Hard difficulty | 30 points |
- Hint usage in Courses doesn't reduce the points earned, it only affects confidence
- Incorrect attempts don't reduce points earned, they only affect accuracy. (Accuracy = (# Correct Attempts /# of attempts) * 100 )
- Watching videos doesn't affect points, confidence, and accuracy
- Missions and walkthroughs don't affect confidence and accuracy
How is the streak bonus calculated?
For every second challenge a user completes with no mistakes, a bonus of 10% will be added to the points earned. This only applies to Challenges. Streak Bonus doesn't apply to videos, custom activities, standard missions, or walkthrough missions.
Example:
Easy Challenge 1 (no mistakes made) - 100 points
Easy Challenge 2 (no mistakes made) - Streak bonus >> 100 points + (10% of 100) = 110 points
Sum = 210 points
Comments
0 comments
Article is closed for comments.