Most of the “Agile” and “Lean” product design and development practices that we follow in the modern age can be directly linked to the lean manufacturing movement from the 1940s and 1950s, largely attributed to the work of W. Edward Deming and his influence on the post-war Japanese manufacturing culture. Deming relied on a “Plan-Do-Study-Act” methodology that likely sounds extremely familiar to those who have implemented true Agile and Lean methodologies in their own work, and his influence is clearly seen in such famous processes as the Toyota Production System, Six Sigma, and Kaizen practices.
One of the things that all of these practices have in common, that is often missing from software development teams, is that they are focused on continuous or continual improvement of not only the product, but the process as well. While many software companies do somewhat focus on continuous improvement of the product, many neglect the equally important factor of working to continually improve the process as well.
For the purposes of this post, I’m going to use “continuous” improvement as a synonym for “continual” improvement — there’s a minor but important distinction, in that “continual” implies a step-based practice, whereas continuous implies a non-stop practice. Most approaches to this ideal, such as Scrum, take a step-based, continual approach — I personally believe that even in Scrum we should be continuously assessing our performance and making personal changes well before we bring those issues to the Retrospective. Thus, I’ll speak to “continuous” improvement, understanding that there’s a subtle difference between the two.
What is Continuous Improvement?
“A system must be managed. It will not manage itself.”
W. Edward Deming
The fundamental basis of any continuous improvement practice lies primarily in the concept that every process, procedure, or check that has been developed by a person has been done at a given moment in time, based on their knowledge of the system or the output at that point. And, additionally, that as we move through that system, we learn more about it every single time, and this information should be regularly reviewed to confirm that the system is working as efficiently and effectively as possible. If not, then we should alter the process, procedure, or check so that it becomes more efficient and effective. And then we should test out those changes, and review the impacts — if the system is better, we keep the changes; if it’s worse, we try another set of changes. Then we repeat the process.
Continuous Improvement is both a philosophy and a set of practices. One can bring in all the Six Sigma blackbelts you want to study your work, draw up hundreds of fishbone diagrams, drive to root-cause analysis, and then leave you with some well-conceived proposals…but if you don’t deeply ingrain the concept into everything that your company does, those efforts will fall flat shortly after the consultants leave. I know, I’ve witnessed it first-hand. Just as being agile is a value that you either have or don’t, continuous improvement is something that simply has to be a part of your culture in order for you to successfully implement it as a practice — every employee has to see quality in the product and the process as part of their job, and every employee needs to be empowered to identify and try improvements to their processes on a regular basis.
In many lean manufacturing processes, there’s the idea of a Big Red Button that anyone can press at any time to stop the line entirely in order to raise some problem with the system. This is the amount of trust and faith that these massive companies have in their workers — anyone can stop the line at any time. Do you trust your employees that much? Why not?
Why is Continuous Improvement Important?
“Sure we have to solve problems. Certainly stamp out the fire. Stamp out the fire and get nowhere. Stamp out the fires puts us back to where we were in the first place.”
W. Edward Deming
Continuous improvement is important because it’s based on reality, not some fiction that we create for ourselves.
Every system created by man is fallible. Every solution that we create is based on incomplete information. New information is created every single time we follow a process or procedure that’s been established. Ignoring this information results in time, money, and effort lost to waste based on ignorance and ego. Reacting to the failures after they occur places us in the role of a fire-fighter, arriving on the scene after the blaze has happened, stamping out the fire, and returning to our firehouse to await the next flare-up.
Continuous improvement eschews the practice of being reactive to issues, and instead asks us to be proactive to prevent these issues from occurring. Every single run through a process is an opportunity to learn, to assess, and to improve. Every single run through a process tests the hypothesis that you’ve established as the “rules” for that process. And every single run through that process provides you with data about the process itself, as well as the end result of that process.
Too many people simply toss that data aside, because the result is “good enough” and because nothing actually “broke down” getting from point A to point B. But that’s simply not good enough — and we know it. We perform regular maintenance on our cars to ensure they’re able to reliably move us around; we get regular physical exams from our doctors to ensure that we’re on the path to good health; why don’t we apply those same regular checkpoints to our software development processes? Why don’t we ask the very people who are doing the work what can be done better, faster, more efficiently, and with higher quality? Why don’t we trust people and empower them to work better?
Simply put, continuous improvement is important because it’s the best way to ensure that we’re doing things the most efficient, effective, and productive way, every single day of the week, and on every single component that we’re working on. As the saying goes, those who ignore history are doomed to repeat it; and we make history every single time we complete a sprint.
How Can We Achieve Continuous Improvement?
“Massive training is required to instill the courage to break with tradition. Every activity and every job is a part of the process.”
W. Edward Deming
The saddest part of this whole discussion is how simple it is to begin and follow some basic practices of continuous improvement — yet many people eschew it as too “touchy-feely” or as “waste” itself. After all, goes the logic, looking in the rearview mirror means you’re not watching where you’re going.
But that view ignores the simple fact that we don’t know everything when we decide on how we want to start doing something. There’s always more to learn, more to understand, and those things inevitably lead to opportunities for improvement — not just in how we do things, but in what we do and what we create. Taking a half-hour a month to reflect on how things have been going, what opportunities for improvement we have, and what things we’re going to try for the next month is an essential part of the machine. We cannot expect the system that we’ve built to simply improve itself over time — if we know anything from the history of lean manufacturing, it’s that when left to itself, the system will degrade over time, rather than improve.
The Scrum retrospective meeting is a good example of how a small team can engage in high-quality and actionable continuous improvement practices with low overhead. A thirty minute meeting at the end of each sprint that is solely focused on:
- What went well?
- What didn’t go well?
- What can we improve?
- What are we going to try next sprint?
Of course, there are a few ground rules that have to apply:
- No blaming.
- Everyone participates.
- Everyone commits.
It’s not a big deal, really — and when the teams feel empowered to experiment, to try things, to better themselves as well as the product and the company, the engagement happens naturally, as does the commitment. People feel as though they’re actually a valuable part of a bigger whole, not just a cog in the machine, pushing forward without their input or support.
Great things don’t just happen. Great teams make things happen. And part of making things happen is adjusting your approach based on new information.
