This article details how the points system works in the Secure Code Warrior® platform to help increase engagement and measure your team's secure coding prowess.
Metrics Definitions
Accuracy
Accuracy is the ratio between the number of attempts made and those answered correctly or incorrectly.
Accuracy is calculated by dividing the number of correct attempts by the total of both incorrect and correct attempts and multiplying that by 100.
Accuracy = (# Correct attempts /# of total attempts) * 100For example, a Player answering everything correctly the first time will have a high Accuracy score while a Player that answers incorrectly or 'guesses' more often would have a lower Accuracy score.
Confidence
Confidence is the ratio between the total number of Hints available and those used.
Confidence = (#total hints - #hints used)/#total hints
#total hints are hints that are available for challenges that the user has completed. For example: if a player doesn't use any Hints they'll have a high confidence score. Players who frequently use Hints will have a lower confidence score.
How is Time measured?
Challenge Time Played is the amount of time a player has spent actively in training. The time is tracked once a challenge has started and stopped once they submit an answer. If the player navigates away from the challenge without submitting an answer or is inactive (ie. no mouse movement, keyboard interaction, or touch gesture) for more than 5 minutes, the time spent on that specific challenge is paused.
Team Managers and Company Administrators can also see the total time a player interacts on the platform in Learning Resources (Videos), Training, Assessments, and Tournaments. This is referred to as Time Spent.
Time Spent is determined by monitoring the active Playing mode every 30 seconds. If the player navigates away from the platform (closes the browser window or switches browser tabs) or is inactive for more than 10 minutes, the time spent is paused.
How are experience Points calculated?
The possible number of Points earned for each challenge is determined by the Secure Code Warrior Security Competency Algorithm Metric, much like the famous Google Algorithm for search. It calculates a number of factors such as; Playing Stage, Challenge Difficulty, Application Type, Hint Used, and Failed Attempts (or guesses!) to derive the Players' Accuracy, Confidence, and Points.
Training Mode
Training Points are the unit of score in Secure Code Warrior and they're earned by completing missions in the Training Ground.
The total of all earned points is called the Points Score and is the major contributor to leveling up a developer's Security Maturity level to earn certifications and bragging rights.
Note: Security Maturity might be disabled by a Company Administrator
Tournament Mode
In Tournament Mode players compete against each other to see who can score the most points in a given period of time by correctly completing a stage. The harder the stage, the higher the potential points available.
There are two types of activities in tournaments:
Challenges:
There are eight (8) challenge levels. A Challenge Stage is defined as; Locate, Identify, Fix, Locate & Fix, or Identify & Fix.
Tournament Players can view the detailed scoring rules by clicking on the Information icon to expand the detailed scoring rules as shown below.
There are three preset tournament scoring modes that allow more or fewer attempts to be played. These can be configured by the Team Manager or Company Administrator when setting up a Tournament.
The following table outlines the scoring for each.
Challenges (Levels 1 - 8)
| Default | Forgiving | Aggressive | |
| Allowed Attempts | 3 | 5 | 2 |
| Base Stage Score (score of stages for each challenge in the tournament) | |||
| Easy | 100 | ||
| Medium | 200 | ||
| Hard | 300 | ||
| Attempts per Stage. The proportion of points awarded | |||
| Attempt 1 | 100% | 100% | 100% |
| Attempt 2 | 60% | 60% | 60% |
| Attempt 3 | 30% | 30% | - |
| Attempt 4 | - | 10% | - |
| Attempt 5 | - | 5% | - |
| Hint Penalties (consistent across all scoring algorithms (Default, Forgiving, Aggressive) | |||
| Locate Vulnerability | Identify vulnerability | Pick Solution | |
| Hint 1 | -5% | -50% | -33% |
| Hint 2 | -35% | -50% | -33% |
| Hint 3 | -60% | - | -34% |
Missions:
If enabled, this option activates five (5) playable Bonus levels within a tournament. Missions are hands-on, interactive coding simulations designed to immerse developers in real-world applications to see, first hand, the impact of when certain vulnerabilities are introduced in the code.
Tournament Players can view the detailed scoring rules by clicking on the Information icon to expand the detailed scoring rules as shown below.
Bonus Levels
| Default | Forgiving | Aggressive | |
| Maximum points per challenge stage | |||
| Easy | 200 | ||
| Medium | 400 | ||
| Hard | 600 | ||
| Hint Penalties (Penalty as a % of maximum earned points) | |||
| Small Hint | -10% | -10% | -10% |
| Medium Hint | -30% | -30% | -30% |
| Large Hint | -50% | -50% | -50% |
| Maximum cumulative penalty | -80% | -60% | -100% |
Assessment Mode
Assessments are calculated in a similar manner as Training.
Using the Secure Code Warrior Security Competency Algorithm Metric, points are normalized to enable a fair comparison across different Language/Framework challenges. The Assessment Score is displayed as a percentage.
The Team Manager or Company Administrator can configure a 'pass/fail' score.
Courses Mode
Courses are skills-based learning pathways that progressively build secure coding competency for developers of all skill levels. Educational hands-on coding challenges and missions, build transferable skills that help your organization achieve compliance and reduce recurring vulnerabilities.
How are base rewards calculated?
| Difficulty | Challenges | Walkthrough Missions | Standard Mission | Guideline |
| - | (Not Applicable) | (Not Applicable) | (Not Applicable) | 100 points |
| Easy | 100 points | 100 points | 200 points | (Not Applicable) |
| Medium | 200 points | (Not Applicable) | 400 points | (Not Applicable) |
| Hard | 300 points | (Not Applicable) | 600 points | (Not Applicable) |
Points earned for additional attempts:
This happens when a user repeats a course or plays the same module again. This only applies to challenges, it doesn't apply to videos, custom activities, standard missions, or walkthrough missions.
| Challenge Difficulty | Points |
| Easy difficulty | 10 points |
| Medium difficulty | 20 points |
| Hard difficulty | 30 points |
- Hint usage in Courses doesn't reduce the points earned, it only affects confidence
- Incorrect attempts don't reduce points earned, they only affect accuracy. (Accuracy = (# Correct Attempts /# of attempts) * 100 )
- Skipping challenges affects accuracy.
- Watching videos doesn't affect points, confidence, and accuracy
- Missions and walkthroughs don't affect confidence and accuracy
How is the streak bonus calculated?
For every second challenge a user completes with no mistakes, a bonus of 10% will be added to the points earned. This only applies to Challenges. Streak Bonus doesn't apply to videos, custom activities, standard missions, or walkthrough missions.
Example:
Easy Challenge 1 (no mistakes made) - 100 points
Easy Challenge 2 (no mistakes made) - Streak bonus >> 100 points + (10% of 100) = 110 points
Sum = 210 points
Comments
0 comments
Article is closed for comments.