The SCW Trust Score launched in May 2024 and we are thrilled by the outcomes it is driving for our customers. Secure Code Warrior customers are using Trust Score to measure the effectiveness of their security program against their peers, highlight areas for targeted improvement and demonstrate the value that secure coding learning is bringing to their AppSec program.
To make the SCW Trust Score the most accurate reflection of a company’s security posture, we are further enhancing the SCW Trust Score algorithm to include more signals of developer comprehension. This update to the algorithm, along with a number of improvements to the Trust Score report, makes up the “2025 Update”.
With the 2025 Update, SCW Trust Score will take into account:
- Accuracy in Attempts - Learners that take fewer attempts on a given topic will generate a greater contribution to their Skill Level score. This update rewards demonstrated comprehension and seeks to prevent “brute forcing” in playmodes that allow it.
- Selected Language - Learners working in real programming languages will receive higher scores than those studying in pseudocode
- All Learning Activities Included - SCW Trust Score will include all learning activity performed in Quests, Explore, and during the recent Cybermon event.
Note: Trust Scores will likely go down for most companies. This is because we are measuring more things, so it’s harder for each developer to get “full marks” as far as the Trust Score algorithm is concerned. It is important to note that this update will roll-out to all SCW customers, so for most customers, rankings within the industry benchmarks visible in the Trust Score report should not change significantly.
Frequently Asked Questions:
- Why is this important?
If you’re using the SCW Trust Score to benchmark your program, identify areas of improvement, or demonstrate ROI - this update will make the score do this job even better, honing in more acutely on demonstrated knowledge acquisition and encompassing additional inputs.
It will also have the net effect of lowering scores generally for developers. This will allow you to better separate the truly high performers based on the improved activity signals SCW Trust Score is gathering.
If you’ve been on the fence about using reporting on Trust Score, or had a look at found it measured your developer cohort too generously, take another look!
- When is this going to happen?
The Trust Score 2025 Update will roll out on 21 November, 2024, following the rollout plan described below.
- How will the roll-out take place?
The 2025 Update will launch ON by default.
Once live, you will have the ability to switch back to the old version of SCW Trust Score if you prefer. This, and ability to switch back and forth between the old version and 2025 Update will remain in effect until the old version is sunset at the end of Q2 2025.
For those using the Reporting API to create custom reports, or integrate your SCW Trust Score into other dashboards, the score the API delivers will reflect the currently selected version. This means that the score will be reflective of the 2025 Update and will update accordingly if you revert to the old version in the UI.
- How will I know which version I am using?
There will be prominent banners on the SCW Trust Score report page displaying whether the 2025 Update has been applied, or whether you still need to opt in.
- Why is this happening all in a single update?
We have had a number of improvements come together in a short period of time. By grouping them into a single update and providing flexibility to choose between the two versions our goal is to make it easy to upgrade to the new version while minimizing disruption. Incremental roll-out of individual improvements updates would create a scenario where most users would see multiple changes in their SCW Trust Score making it more difficult to assess and communicate if the change is attributable to a change in learning behavior or a small update.
- Will I always be able to switch back and forth? Can I stay on the old version?
The 2025 Update is the future of the SCW Trust Score. We want to make sure everyone has the time to roll out the change in a manner convenient to their organization. As such, we are giving organizations the ability to “go back” to the old version of SCW Trust Score until the end of Q2 2025 so they can minimize disruptions and transition on their preferred pace.
It’s important to remember though that by staying on the old version you will be missing out on the new features, including new visualizations - Making it easier to explain and demonstrate impact of your Trust Score.
- Trust Score Over Time - Showcasing how your Trust Score tracks over time to see the impact of your learning program!
- Vulnerability Concept Coverage - See how your learners have been covering the range of vulnerability topics available, identifying areas to focus on for continued improvement
- Learner Skill Levels - Displaying a Skill Level to learners in the new Quest module coming in Q1 2025.
And all future improvements we release to the Trust Score.
- Will I be able to retrieve the updated score via the API?
Yes, the Reporting API will match the selected version you see in the Trust Score Report, returning the currently enabled version of the Trust Score.
If you have built dashboards that use the Trust Score, you may want to have their dashboards stay stable, while you inspect and get comfortable with the new score from the SCW portal. To do so, there is a new parameter that has been added to the Reporting API called getLatestVersion.
After the 21st of November, any queries to the TrustScores endpoint of the Reporting API will return the new version of the score, unless this parameter is set to false. If you want to keep your dashboards stable, the easiest way is to:
- Go to the Trust Score report and use the switcher to go back to the old version.
- Update your dashboard queries to send getLatestVersion=false.
- Now, you have time to get comfortable with the new score, and can switch back and forth in the SCW portal without impacting your dashboards.
- When you are ready, you can set the flag in your dashboard queries to getLatestVersion=true.
- What if I want to opt-out?
It is important to keep in mind that you will be able to revert back to the current version of SCW Trust Score anytime between the launch date and the end of Q2 2025 when the current version will be sunset.
However if you would still prefer for the upcoming update to be OFF by default for your organisation, please contact our support team.
Comments
0 comments
Please sign in to leave a comment.