Standard : Tests provide meaningful confidence in code changes
Purpose and Strategic Importance
This standard ensures that tests provide reliable, actionable confidence in code changes, allowing teams to move quickly without compromising stability. Effective testing practices catch issues early, reduce rework, and underpin safe, continuous delivery.
Aligned to our "Engineering Excellence First" and "Architect for Change" policies, this standard promotes technical assurance, repeatable quality, and accelerated learning. Without it, teams ship with uncertainty, increasing the risk of defects, regressions, and user impact.
Strategic Impact
- Increased delivery confidence and reduced failure rates
- Faster feedback loops and shorter time to resolve defects
- Higher release frequency with less risk of incident
- Stronger alignment between testing and business-critical flows
Risks of Not Having This Standard
- Low confidence in the stability of code changes
- Slow feedback cycles, delayed defect discovery
- Increased rework, technical debt, and defect backlog
- Erosion of developer morale and trust in quality pipelines
- Reduced stakeholder confidence in product reliability
CMMI Maturity Model
Level 1 – Initial
| Category |
Description |
| People & Culture |
Testing is inconsistent, informal, or skipped under pressure. |
| Process & Governance |
No formal testing strategy or test review process. |
| Technology & Tools |
Testing is largely manual or based on legacy scripts. |
| Measurement & Metrics |
Test coverage or defect leakage is not tracked. |
Level 2 – Managed
| Category |
Description |
| People & Culture |
Awareness of testing value emerges but execution varies. |
| Process & Governance |
Basic automated tests exist (e.g. unit tests) but lack consistency. |
| Technology & Tools |
CI tools trigger automated tests, but coverage and reliability are low. |
| Measurement & Metrics |
Informal measurement of test pass/fail and post-release bugs. |
Level 3 – Defined
| Category |
Description |
| People & Culture |
Teams adopt test-first or shift-left mindsets; test ownership is clear. |
| Process & Governance |
Strategies for unit, integration, and end-to-end testing are documented. |
| Technology & Tools |
Automation frameworks are standardised across teams. |
| Measurement & Metrics |
Coverage thresholds and defect trends are reviewed regularly. |
Level 4 – Quantitatively Managed
| Category |
Description |
| People & Culture |
Testing is prioritised to prevent regressions and support refactoring. |
| Process & Governance |
Test health, flakiness, and defect escape rates are monitored. |
| Technology & Tools |
Tools enforce test quality gates and provide feedback in near real-time. |
| Measurement & Metrics |
Metrics drive improvements in test effectiveness, stability, and speed. |
Level 5 – Optimising
| Category |
Description |
| People & Culture |
Testing is seen as an enabler of innovation and rapid delivery. |
| Process & Governance |
Continuous improvement of testing strategies through retrospectives. |
| Technology & Tools |
Tests are self-maintaining, intelligent, and integrated into CD pipelines. |
| Measurement & Metrics |
Test impact on lead time, MTTR, and release confidence is quantified. |
Key Measures
- Test coverage by layer (unit, integration, E2E) and by business-critical flow
- Change failure rate linked to insufficient or ineffective testing
- Mean time to detect and resolve test-related issues
- Ratio of escaped defects vs. defects caught pre-release
- Developer confidence in tests as measured through surveys or retrospectives
- Stability of tests (e.g. flake rate) and test execution time in CI/CD pipelines