In my previous blog post, I wrote about data-driven experiments. Unfortunately, it lacked any practical application, and today I want to do something about that. But let’s get one thing straight first. This isn’t my favorite topic. Why? It focuses so heavily on data and so little on humans. When we get too wrapped up in data, we lose sight of the human dimension, and that’s bad. Never lose sight of what matters. That’s not you. That’s not data. It’s your teams. Keep this in mind when you run data-driven experiments.
Before I show you some of my own experiments, let me explain the questions I ask myself when building one:
- What problem (or behavior) are we attempting to solve (or highlight)? This is the heart of your experiment. Never lose sight of this. We will, and when we do, immediately bring it back to center as often as necessary.
- What data can we collect that addresses the question above? Collect this data and no more. The more data we collect, the more time it’ll take, and we risk muddying the focus. We also risk inundating our teams with too much data so it’s imperative pare it down to the absolutely minimum.
- Who will we share this data with? Honesty in your data is important. If the team feels that they will be judged by those in power, they may unconsciously game the system and manufacture the data we want or expect to see.
- How can we create simple view into my data? The simpler, the better. Consider the burn down chart. With just a few simple data points, it can inspire a great conversation. It’s these conversations that make for a powerful experiment.
Work In Progress (WIP) Experiment
Multi-tasking has a cost, and it’s often a cost that many teams overlook. In fact, this was one of my top 10 tips in a previous blog post. While I was a scrum master for three teams, I wanted to highlight how much work each team had open on every day of the sprint. Once a day and at the same time, we recorded the percentage of stories in progress by each team and created the graph you see below. Notice that I also included an “optimal” range. This range isn’t based in any science. It’s simply where my gut told me our teams should be. Here’s what they looked like after the first sprint of the experiment.