ChaptersEventsBlog
We're exploring how organizations adapt IAM to AI. Take the AI Identity and Risk Readiness Survey by September 5 →

Announcing RiskRubric.ai: A Clear Scorecard for Every AI Model

Published 08/04/2025

Announcing RiskRubric.ai: A Clear Scorecard for Every AI Model
Written by Caleb Sima, Chair of CSA AI Safety Initiative.
Originally published on RiskRubric.ai.
 

As data science and AI engineering teams mix general purpose LLMs from foundation model developers with dozens of specialist models like Mistral and Qwen, they leave their security leaders asking:

“Can I trust this model for my data and my customers?”

This question of trust isn't just academic; it creates daily operational hurdles and tangible business risks. Approval bottlenecks, engineers waiting, and risk decisions becoming guesswork. The idea for RiskRubric.ai was born from hearing these problems repeatedly described by CISOs and security teams.

 

We built RiskRubric.ai to provide a clear, objective standard.

Our platform grades each model against six security-centric pillars and publishes the results as a one-page “report card.” At launch, our process includes:

  • Models Covered: 40+, refreshed monthly
  • Tests Per Model: 1,000+ reliability prompts, 200+ adversarial prompts, analysis of risk indicators, and model card reviews.
  • **Rating:**A letter-grade (A–F) risk rating composed of multiple independent “Risk Pillars,” each assigned its own sub-rating.

 

The Six Pillars

  • Transparency: We check published data sources, training disclosures, and license terms.
  • Reliability: We run a repeatability suite and measure output variance.
  • Security: We red-team for prompt injections, jailbreaks, and hostile code execution.
  • Privacy: We test models' willingness to request personal data and probe for training-data leakage.
  • Safety: We evaluate harmful-content filters with structured adversarial prompts.
  • Reputation: We track a model's history and improvement over time.

 

How This Helps Security Leaders

This data-driven approach provides immediate, actionable value:

  • Transparent Risk Management: Get immediate clarity into each model’s risk profile -enabling smarter risk management, informed model selection, and alignment with your organization’s risk tolerance without running your own time-consuming and costly tests.
  • Assured Model Approvals: Provide leadership and customers with confidence that each model’s use is backed by a structured, evidence-based risk rating - removing the guesswork and inconsistency from approval decisions.
  • Audit-Ready Evidence: Download the report cards today and detailed test logs in the coming months for your risk register and compliance needs.

 

Try It Today

View the scorecards at RiskRubric.ai. The core ratings are free to browse, and full test data will be available soon for enterprise users. This is just the beginning of our roadmap, and I invite you to request a model we haven’t covered yet.

Straight answers, no marketing gloss. That’s the rubric.

Share this content on your favorite social network today!

Unlock Cloud Security Insights

Unlock Cloud Security Insights

Choose the CSA newsletters that match your interests:

Subscribe to our newsletter for the latest expert trends and updates