Standard : Experiments test assumptions before scaling

Purpose and Strategic Importance

This standard ensures teams test risky assumptions through small experiments before committing significant investment. Experiments de-risk innovation and provide evidence to scale with confidence.

It supports "Test Before Scale", while reinforcing "Bias for Experimentation" and "Discovery Before Delivery". Without this standard, teams risk large-scale failures and wasted effort.

Strategic Impact

  • Safer, evidence-based scaling of products
  • Reduced waste from unvalidated ideas
  • Increased confidence in investment decisions

Risks of Not Having This Standard

  • Expensive failures due to untested assumptions
  • Low adoption of scaled products
  • Stakeholder mistrust in product direction

CMMI Maturity Model

Level 1 – Initial

  • People & Culture: Teams rush to build and scale without testing.
  • Process & Governance: Experiments rare or informal.
  • Technology & Tools: No tools for rapid testing.
  • Measurement & Metrics: Success measured by delivery speed.

Level 2 – Managed

  • People & Culture: Teams occasionally run small tests.
  • Process & Governance: Some guidance for experiments exists.
  • Technology & Tools: Prototypes and pilots created manually.
  • Measurement & Metrics: Basic metrics (e.g. adoption rates) captured post-launch.

Level 3 – Defined

  • People & Culture: Teams consistently test assumptions with small experiments.
  • Process & Governance: Clear frameworks for hypothesis-driven development.
  • Technology & Tools: Tools support prototyping, A/B testing, and pilots.
  • Measurement & Metrics: Experiments linked to decision-making.

Level 4 – Quantitatively Managed

  • People & Culture: Experimentation embedded as cultural norm.
  • Process & Governance: Systematic experimentation informs investment decisions.
  • Technology & Tools: Integrated experimentation platforms provide rapid feedback.
  • Measurement & Metrics: Quantitative analysis of experiment success/failure rates.

Level 5 – Optimising

  • People & Culture: Experimentation fuels innovation across organisation.
  • Process & Governance: Practices continuously refined for speed and safety.
  • Technology & Tools: AI predicts outcomes and optimises experiment design.
  • Measurement & Metrics: Continuous improvement cycles refine experimentation ROI.

Key Measures

  • % of initiatives tested before scaling
  • Number of experiments run per quarter
  • Ratio of validated vs invalidated hypotheses
  • Investment avoided or redirected due to experiments