Tracking metrics to measure performance is a fundamental best practice of DevOps. After all, how can you know if DevOps is working, or measure progress toward goals, without benchmarking and tracking metrics? Monitoring key performance indicators (KPIs) is critical, to understand how successful your existing DevOps processes are, and to identify areas for improvement.
Use the tips below to show leadership in your organization on how DevOps and your teams have impacted processes and productivity to increase job security and confidence in your department.
Some KPIs that you should consider tracking include:
1. Cycle time
Delivering software quickly and efficiently is one of the prime motivators of DevOps adoption. Cycle time measures how long it takes an application to go from the initial commit to functioning in a production environment with minimal to no friction in the process.
Every time a process is updated, or a new tool is added, cycle time should be analyzed to ensure that a change results in a positive outcome. If a new function – for example, test automation – is added to the process, it should result in a reduction in cycle time. If it doesn’t, there is a problem. Carefully review the process (and personnel) to identify the source of delay, and correct it to improve efficiency and reduce overall cycle time.
2. Deployment frequency
One of the pillars of DevOps is frequent deployments – incremental improvements released several times per day, as opposed to traditional, bundled or waterfall updates released once per quarter, or once per year. When DevOps processes are implemented, deployment frequency is one of the first indicators that process changes are working. However, as DevOps matures, deployment frequency should continue to improve.
Benchmark this KPI and monitor it continuously to ensure that deployment frequency is continually improving. If the rate of improvement drops off, it is an early indication of a potential issue that should be identified and addressed – an issue that may go undetected if deployment frequency is not tracked.
3. Rate of failure
A DevOps process measured only by a reduction in cycle time and deployment frequency would seem successful – but say end-user satisfaction has declined… what could the problem be? Look to rate of failure. More frequent deployments offer more opportunities for issues to arise, which could require a rollback for additional work. The rate of failure should be kept as low as possible. Tracking this metric allows visibility into successes and failures regarding changes made. An increasing rate of failure affects user experience, and rollbacks eat into resources that could be channeled to higher-level, possible revenue-generating objectives.
4. Mean time to detection (MTTD)
Mean time to detection refers to how long it takes to find a problem – the rate of failure may be low, but the longer it takes to diagnose and fix an issue, the more it will affect users and the system as a whole. MTTD should decrease as DevOps processes are refined and become more mature – if it stalls or increases, monitoring, testing and deployment processes should be reviewed.
5. Mean time to recovery (MTTR)
Once a problem or failure is detected, MTTR describes how long it takes to resolve the issue and deploy a successful fix. This measures the team’s ability to effectively roll back a process to a previous iteration, and to create, implement, and deploy a working version.
MTTR should always be a focus for KPI monitoring in DevOps, as improving MTTR contributes to better customer satisfaction, accelerated application delivery, and improved cost control.
Conclusion:
DevOps isn’t a sprint; it is a marathon. Effective DevOps is a long-term process, requiring change management, a cohesive vision, support from key stakeholders, and an agile approach. This flexibility will be called into action – as benchmarks are established and KPIs are measured over time, an adaptable approach will help to continuously improve and support your DevOps progression to maturity.
