Data-driven software engineering: how to avoid common problems


In today’s digital world, data is constantly being generated, evaluated and updated. It also plays an important role in the work of software engineers by providing accurate and actionable feedback that helps engineers understand where and how to make improvements to a product or process.

Data also helps IT managers visualize how work is done, the quality and quantity of results, and how it can improve the lives of staff. And it’s an essential part of any digital transformation.

Many organizations implement Key Performance Indicators (KPIs) or Key Objectives and Results (OKRs) based on metrics that encourage teams to think about business value and strategic outcomes in their day-to-day work.

When used correctly, KPIs and OKRs are valuable tools for data-driven software engineering. The right metrics create visibility into the success (or not) of the business and each person can see how their individual work has contributed.

But there is a lot to be discovered when it comes to data-driven software engineering. Here are the key elements, challenges and best practices for getting it right.

People are the key

The key to successfully implementing a metrics-based program often depends more on how well your teams collaborate, and not on focusing on numbers and charts. People need to engage with data, and ideally, they should be the ones asking for more data to keep improving their work.

This tends to happen only when the human aspect of engineering is taken into account at the start of a KPI or OKR initiative. With that in mind, a successful data-driven software engineering organization will determine what motivates its employees to be successful and what they need to do their jobs better.

There is a downside to a metrics-centric approach. Problems tend to arise when you haven’t executed KPIs properly, especially when:

  1. Creation and deployment are clunky and focus on management without listening to engineers, understand what they need or want, and respond to their fears. Teams will be afraid of a data-driven organization, so leadership must engage the fear or risk the disengagement that will follow.
  2. The company, department or team has a year / quarter / month off. In this situation, it’s hard to keep team members engaged and prevent KPIs or data-driven management from feeling bossy.
  3. The balance between the contributions of teams and individuals is out of balance. Organizations need a healthy balance between team metrics that measure (and reward) team success and those that recognize individual contributions.

You can avoid many of these problems by carefully building metrics or OKR / KPI systems. To begin the process, your leadership needs to understand the four pillars that enable data-driven software and how to successfully implement them.

Anatomy of a Data-Driven Engineering Organization

Here’s how the key elements of data-driven software engineering work together:

Figure 1. The four pillars of data-driven software engineering should be guided by your business vision. Source: Exadata

In all, there are five main components to enable data-driven software engineering:

Company vision

Having a strong business vision is not technically one of the pillars, but it is nonetheless essential, and it has to come from leadership. The vision strongly influences each pillar and should determine (especially in the case of KPIs and OKRs) what these pillars are. The business vision is the “why” of the organization and should be strongly reflected in your KPIs and OKRs.

KPI

These measure the ongoing business performance of your organization, including profitability and how well it achieves its vision. If your KPIs are missing any of these marks, your employees may become disconnected from the company’s vision.

OKR

These are measurable goals that are more transitory than KPIs. Your OKRs need to measure what’s going on right now (this month, this quarter, this year) to achieve and improve results. Strong OKRs translate into improved KPIs. At their best, they show you’ve chosen the right job and done it right.

Engineering metrics

These are often elusive. What is the measure of good software development? What is the measure of a great developer? Good engineering measures should result in agreed standards for software engineers, a high bar for the quality of work, and the production of more and better features to support more valuable work.

Positive behavioral measures

These metrics require additional explanation beyond this bulleted list in order to better understand why they are important and how they work. When you have a failing shift or a struggling team, what energizes your employees and pushes them to the top to deliver anyway? What keeps them going? What makes them feel it’s worth it? Each of them is guided by positive behavioral measures.

How to do it

While this framework offers a way to overcome some of the key issues in data-driven software engineering, you may find it impossible to follow these pillars.

It’s understandable: a leader’s time and energy is a precious commodity, and a bunch of extra paperwork doesn’t help anyone. So here is a cost effective way to approach this challenge, start and grow it by determining which items are most valuable to you:

Most KPIs should be an accounting function

Those that are not either should be, or they should be integrated into the software product as a key management function. Beware of special cases and subdivisions that make reporting on KPIs more cumbersome. Everyone should have the same measurement. Large organizations with real business divisions can be subdivided.

Experiment with OKRs and scale as needed

Tools like Weekdone offer free options that you can use to get started. Another solution is to create basic OKRs for teams each quarter and use a spreadsheet to track them. Then periodically review and discuss the spreadsheet results.

Get an engineering measurement tool

You will need one to track successful engineering measurements, and several are available. Engaging your development team to better understand their work is key here. An effective tool will help them understand their work and get more done. What are the roadblocks? What is going well? What is difficult ?

Try to get your teams to engage with the numbers and suggest ways to use them. In-house metrics that your team sees as serving them or their fellow developers are the way to go. You can make suggestions, but the more it’s up to the people who create your software, the better.

For positive behavioral measures, using a tool is helpful, but not 100% necessary

The right tool can help spread positive energy to more people by sending automated emails, texts, or messages. These give managers a boost, show employees and bosses that managers radiate this energy and also give them a boost.

Just emailing a person and tracking the information in a spreadsheet doesn’t have the same effect. I have built tools that have this kind of functionality, and the fundamentals of this solution provide multi-person recognition, upstream recognition, general positive energy flow, and principle-based recognition. . (I am currently developing an open-source tool that I plan to share soon). Other employee recognition tools are also available, and you can find them by doing a quick search online.

If there isn’t a tool in place, just find ways to diffuse as much positive energy as possible and practice reporting (sending kudos to that person’s person and boss or to the boss of the boss, etc.). As the CEO I’ve worked for, said: “One of the best parts of my job is recognizing people for all the great things they do.” So emailing him about my team’s successes wasn’t difficult for me. It made him feel good, it did my employees good and it made me feel good. It is positive energy.

Go beyond the numbers

Data-driven software engineering delivers benefits far beyond the numbers if you have an effective process or framework that reduces problems and drives success. As your organization moves towards digital transformation, you will need to focus more on creating value. With these measures in place, the value you deliver will be measurable and your employees will be engaged and happy to contribute to the success of the organization.

Keep learning


Gordon K. Morehouse