How to measure productivity effectively? Adolfo Cruz, Scio’s very own Project Management Office director, offers insights into a question many in the business have in their minds all the time.
By Adolfo Cruz.
Building an exact image of productivity is a question with no definitive answer yet. Defining what “productivity” means for someone starting a business for the first time is a complex process. From everything I’ve learned these 14 years at Scio, each person needs to discover what works for them, considering the specific context in which they are trying to succeed.
The thing is, when trying to measure productivity in something as fluid as software development, every seemingly useful metric comes from a different perspective (be it from the client, the developer, or the managers), so if you want to establish a baseline, I would recommend defining priorities first, taking into account your business needs.
How do we measure productivity in Scio?
Early in Scio’s history, around 2008 or so, we tried to implement Agile Methodologies along with some number crunching, like the hours put on a project or the number of bugs, to know exactly how productive our teams were. However, the number of variables involved, like realizing that a specific team had better outcomes because they had certain chemistry between them, made it difficult to translate the results into measures that could be implemented in any project.
So we moved away from that, instead of looking at the results from the client’s perspective, thus defining our productivity as the value our developers are adding to a product during development. Our measures became more focused on people than metrics because helping a client to build a product, adding value at every step, and therefore growing their business, was our main priority.
To this end, building a perfectly calibrated development team that proposes new things, learning and evolving together was the best approach for Scio. A lot of the time it’s not even necessary to keep a register of everything happening during development because the team itself realizes when something’s not right and works out any issue; a team of developers capable of self-management is always a good sign.
If you achieve such a team, you don’t need to measure specific metrics for each collaborator, instead, you evaluate the team as a whole, looking for relevant information that can help them grow, so you should think of questions that help you get the information you need.
For example number, many User Stories were completed in a given time?
- Was there any serious bug in the development of the product?
- How many bugs surfaced?
- What priority did these bugs have?
- How did the team respond to them?
- How long did it take?
However, these questions need to be constructed with the project’s context in mind; although there are a lot of metrics you can use to generate specific “productivity” numbers (like the Personal Software Process popular in the 90s that tried to define productivity through the number of lines of code written), if they don’t reflect reality, then the team will not know how to use this information, as many of the challenges they face can be subjective.
So avoid trying to arrive at exact numbers, and instead, I recommend you to look at the trends emerging from one project to another. A trend interpreted correctly should give you a “big picture” of both performance and productivity, showing places where your attention or some improvement is needed.
First, look for a certain range of information in a development cycle; If the team completed between 5 and 10 stories in a sprint, but only 1 or 2 the next one, there’s probably a red flag somewhere in need of attention. Then dig a little into it, and maybe you’ll find some unforeseen challenges the team needed to prioritize to keep making progress. And finally, make a note out of it, so you can improve the circumstances around an issue so it doesn’t reappear later; that’s how an understanding of productivity begins to emerge.
Most of the trends we watch at Scio are about how our developers contribute to our client’s goals, and we developed some tools to measure that. For example, we use the Team Self-Assessment, a guideline for collaborators to reflect on their work, that we adopted and modified according to what our experiences and specific contexts dictate.
In our TSA, there’s a section where we inquire about bugs surfacing during development, asking how they are preventing issues during development, if they are doing Pair Testing practices, if they are showing their additions to someone else before moving on, and generally if they are focusing on things that matter.
When a team reviews these questions, they realize their areas of improvement, and we work together to avoid falling into them again in future projects, because what you want to do is push your teams into adopting excellence as a natural part of their processes, encouraging them to add their experiences and expertise to the final product.
The subjectivity of a productive team
Productivity is not always a purely technical approach; sometimes you need to look at the business side of things, involving the stakeholders and product owners in the creation of the product to reconcile a multitude of perspectives, converting them into priorities to define the best course of action.
Just remember that you need to experiment in software development, build an iteration of the product to validate its potential, and look for features that can be improved to maximize its success. Your development team will probably need to go back and rework a lot of stuff, updating their methods to create a more efficient version of the product, so your definition of productivity here has to account for it; if stuff is being redone, at least quality should always be improving.
So, to recap, a good measure of productivity is established by defining your business priorities and then choosing the metrics that best reflect the context in which the product will be built. A productive development process gives enough flexibility to iterate and experiment without neglecting business needs, and having your client’s goal as the focus of your team is a great starting point.
Just remember that if you aren’t adapting what you learn into your specific context, capturing the information your team needs, then productivity might suffer. Software is mostly a creative enterprise, so some subjectivity will always be needed to properly define what makes a successful project, fine-tuning every cycle to achieve the value you want for a specific product. With an approach like that, you can be sure your organization will always be improving.