Introduction

Over the last few decades, the idea of augmenting your software development team’s capacity and capabilities using remote team members from a trusted near-shore provider has become normalised. More and more organisations use this strategy to provide them with a cost-effective method of supplementing their internal software development “horsepower” to create and maintain key applications, whether it be for their internal use (gaining a competitive advantage) or taking their solutions to market.

Mitrais has been providing just such team augmentation services for over 30 years, working with many well-known organisations. Some of these have been working in this mode, with blended teams comprising local and remote team members completely integrated, for decades.

What is clear is that highly experienced development managers within these partner organisations have some common traits. One of the most critical is the realisation that successfully executing this strategy requires careful planning and monitoring. There is no doubt that managing an outsourced or blended development team to a high level of performance requires more than just “gut feel”. Fortunately, there are a wide range of metrics available that allow team managers to get a continuously updated picture of their team’s performance.

In this series of articles, we will look in detail at a range of metrics that can be used to measure the success of your team over time. It is important to understand that software development is far from a “one size fits all” exercise. Naturally, development processes and policies vary from organisation to organisation, so there is no “magic bullet” combination that is appropriate for every case. The right mix will be one that serves the needs of your organisation and your management, and are tools for understanding and improving outcomes specific to you. But by looking at the most common measures used throughout the software industry, we hope that your decision regarding which to prioritise will be an easier one.

Importance of Choosing the Right Metrics

Selecting the right metrics to gauge the success of a development team is like selecting the right instruments for a scientific experiment. Just as the right instruments give you access to accurate and reliable information in an experiment, well-chosen metrics can offer informative insights into your augmented team’s performance, efficiency and productivity. The right metrics can allow you to see where your team is succeeding, and where improvements can be made, all before issues arise.

The suite of metrics you choose can act as a compass for you in navigating the complexities of software project management, ensure your team remains aligned with your project goals, and foster an environment of continuous improvement.

The appropriateness of each metric can, however, vary significantly depending on the nature and phase of your project. For instance, the Feature Delivery Rate may be a pertinent measure for a greenfield or enhancement project, where the focus is on rolling out new features. However, it may not hold the same relevance for a project in the maintenance phase, where the emphasis shifts towards stability and issue resolution.

You can see that this underlines the importance of selecting metrics that reflect the current priorities of your individual project, and even consider altering the mix at different stages of development to suit. Knowing that the right metrics mix can be somewhat dynamic, it’s a delicate balancing act. It is vital to choose metrics that reflect your current needs and that serve your unique project scenario TODAY, and avoid the trap of relying on generic “one size fits all” indicators.

Also, it is imperative to be clear that metrics are tools to aid your understanding to improve TEAM performance and processes, rather than definitive indicators of an individual’s performance within it. Team metrics should inform and guide, rather be used to judge or label. Using a balance of quantitative and qualitative factors is essential for an accurate assessment of team productivity. By doing so, you not only measure what matters, but also cultivate a team environment that values growth, learning, and continuous improvement.

Individual metrics can sometimes only reveal their significance when read in combination with others, so over-reliance on just one or two can lead to misinterpretation. A perfect example might be Technical Debt. While important to performance and stability, Technical Debt must be considered in conjunction with other metrics like Bug Rate and Code Compliance to fully understand it’s significance. Again, the selection of the right metrics suite is critical.

Indicative Metrics

In subsequent papers in this series we will do a deeper dive into the most used metrics and their pros and cons, but in general software development metrics can grouped into several discrete classes based on what they intend to measure.

Briefly, development metrics can be classified as:

1. Code Quality
   • Bug Rate: Crucial for product reliability and measured by tracking the number of bugs per unit of code over time.
   • Technical Debt: Significant to development speed and stability but must be considered in conjunction with other metrics (as previously stated).
   • Code Coverage: Important in assessing test effectiveness and measures the percentage of code actually executed during testing.
   • Static Code Analysis: Essential for early detection of potential vulnerabilities and issues.
   • Coding Standards Compliance: Ensues uniformity and readability which is vital for maintenance.
   • Code Complexity: Sometimes referred to as Cyclomatic Complexity, directly impacts code maintainability.
   • Code Review Coverage: Vital for Quality Assurance and Knowledge Sharing, measures the percentage of changes reviewed before integration.
   • Code Churn: Important, since it reflects the stability and maturity of code. Measures the changes to the codebase over time.
     
2. Productivity
   • Velocity: A metric specific to Agile (SCRUM) methodologies. Measures the amount of work that a team can handle in a single sprint, and directly impacts delivery timelines.
   • Throughput: Used in KANBAN and other flow-based Agile methodologies, measures the number of team tasks completed during a period.
   • Bug Fix Rate: Important for maintaining product quality. Measures the number of bugs resolved in a period.
   • Feature Delivery Rate: Vital to team management and product owners and can impact customer satisfaction and competitive advantage. Measures the number of features deployed in a period.
   • Automated Test Pass Rate: Reflects the health of the codebase. High pass rates can correlate with higher productivity, but should be analysed alongside other Code Quality metrics.
   • Commit Frequency: Indicates developer activity by measuring code commits in a period. Impacted by the complexity of tasks assigned, and does not necessarily correlate with quality or value. Analyse alongside other metrics.
   • Team Turnover Rate: A reflection of the health and stability of the team. Measures the rate at which team members leave the team and are replaced. High turnover can be associated with lower productivity.
   • Net Promoter Score: A simple measure used to gauge satisfaction and enthusiasm for the teams performance from key stakeholders.
   • Burn Down/Up Charts: A useful visualisation tool for monitoring project conduct.
   • Team Member Satisfaction Rate: A less-used metric, but evidence does suggest that team satisfaction strongly correlates with team productivity.
     
3. Innovation and Problem Solving
   • Problem Resolution Time: Critical for maintaining project momentum. Measured by the time taken to resolve issues from identification to closure.
   • Number of New Ideas Submitted: Indicates innovative thinking within the team, but relies on ideas being appropriate and adopted. (See Number of Successful Innovations/Ideas)
   • Number of Successful Innovations/Ideas: A good indicator of level of creativity within a development team. Measures the number of successfully implemented new ideas that contribute to the project’s (or client’s) goals.

How to Use Metrics Effectively

In general terms, there are two major takeaways regarding the implementation of a metrics suite on your software development project.

Firstly, while metrics are certainly a powerful toolset, it is vital to remember that Context Is King! If some metrics seem to indicate variations from what you expect of your team, they should represent the starting point of your investigation into the root causes of the variations, not the end point. It is not uncommon for individual measures metrics to diverge during the course of a project’s execution, and the divergence can have many possible causes. If one metric indicates an issue while others do not, consider what that means. Try not to get fixated on one particular number; consider the “big picture” of your overall team performance, and think about what the outlier result is really telling you.

Secondly, understand the correlation that the metrics you are measuring in your suite have with the goals of your project and organisation. Ideally, each metric within your suite has a very tight relationship with one or more of your overall goals, and provides you with timely and targeted insights into how your team is delivering on them. Metrics never represent all the answers; they should suggest places where you can find improvements. If they aren’t doing that, consider revisiting items that you are measuring to align them better with your desired outcomes.

Conclusion

Software Development Project success comes as a result of a myriad of elements, and effective performance tracking is one important element. Focussing on the right metrics as part of that performance tracking not only gives you an objective way to assess the performance of your augmented team, but should also help you provide an environment that fosters sustained success and continuous improvement.

The goal of a metrics suite goes well beyond simply measuring and reporting numbers. It is to provide you with a way to understand how your team performs under different circumstances, to engage with the team on how to optimise results, and to evolve the team composition, processes and management to achieve high performance.

Those who have been successfully managing augmented teams for years are proof that guiding your project with the aid of the right metrics suite represents a key element in long-term achievement and positive outcomes.

Watch for the next article in this series which will go deeper into the Code Quality Metrics, examining their significance and utility in more detail. Following that, there will be further articles covering Productivity and Innovation Metrics, and some Case Studies showing how clients have implemented complete metrics suites.

If you would like to learn more about the effective implementation and use of a metrics suite, please contact Mitrais by filling out the form below.