How to advance lean software development (beyond the 'Toyota Way')

The Japanese word Muda loosely translates as waste. The core element of lean manufacturing is to eliminate waste--or, in more North American terms, to "cut the fat." While applying lean concepts to manufacturing may seem straightforward, there is little agreement on what that term even means for software, or if it applies.

I'll start at the beginning, explaining where lean manufacturing came from, and, apply the lean idea to software development and cover the implications of lean software.

Where 'Lean' Came From

You may know that lean comes from Japanese methods, most notably the Toyota Production System (TPS), which was largely created by Taichi Ohno, Toyota's chief engineer. The word lean, however, did not come from anywhere in the Far East but was first used by two American researchers, James Womack and Daniel Jones, in their 1990 book The Machine that Changed The World: The Story of Lean Production.

The great business success story of the day was McDonald's, who took a good burger and standardized it. Womack and Jones did the same thing to TPS; they took the practices they observed and made them rules, calling the collection of methods lean."

We'll talk about the practices in a moment. For now, let's talk about standards, heavily associated with "lean" and one principle of the 5S method.

The first great paradox of lean is the combination of standardization with continual improvement. After all, if you tell a team to do the same thing every time, how can it experiment with new methods and improve? British author and psychologist John Seddon takes the argument one step further. Not only did Womack and Jones study what the Japanese were doing at the moment and completely miss any improvement opportunities, Seddon says; they also viewed the Toyota Revolution through the lens of 1980's American business, which was caught in a love affair with standards.

Driving Out (Software) Waste

If practices are supposed to continually evolve, how exactly do you "do" lean? Mary Poppendieck, co-author of two books on lean software, gave me one clue in an interview. "Lean isn't something you do," she says. "It is a way of thinking."

The simple part of the lean philosophy is the waste. If you have technical staff sitting around, doing nothing, waiting for a build, that is waste. If they are waiting to set up a server, or need training to complete a task, that is also waste. Excess work in progress inventory is waste.

In some ways, waste is in the eye of the beholder. Metrics, measures, estimates, audits, plans and process may all add visibility to upper management--or keep the SOX auditor happy--but the technical team itself may see no benefit at all from the practices.

Under lean software development, your team defines waste in its own terms. If the government, management or a customer requires something that the team decides does not add value, then you do the absolute minimum to satisfy the requirement. After all, anything beyond that would be waste, right?

Using Lean Tools in a Software World

It is far too easy to approach lean as a sort of toolbox or checklist of techniques--apply them all, you might think, and woo hoo, you are lean! Seddon once famously pointed out that the lean "toolbox" was designed to produce cars at a rate to match demand--a very different sort of business model than most of us who produce software. Seddon suggests that lean should be about a way of thinking. I agree.

However, many of the tools of lean production techniques described by Womack and Jones do still apply to software testing. Consider them solutions that came from a different kind of thinking about manufacturing. The challenge of lean software development is not to copy the tools below; rather, it is to apply the same sort of thinking to our work and drive improvements in the flow of value.

• Create continuous, one-piece flow.
• Achieve pull.
• Limit work in progress.
• Improve the work area to eliminate needless motion.
• Limit scrap (work thrown away because of defective products).
• Focus on continuous improvement.

Let's look at each of these carefully.

Create continuous, one-piece flow. Traditional manufacturing methods reduce cost per part by pressing large batches of parts at the same time. Ironically, optimizing one part of the process leads to inefficiencies and a decrease in overall flow. Lean manufacturing focuses on delivering one part, end-to-end. This is sometimes called one-piece flow.

Limit work in progress. To get to one-piece flow, we'll eliminate multi-tasking. Technical contributors (or pairs) will work on one thing at a time. But what happens when developers want to hand off work, but the testers are busy?

Create a pull system. Instead of "pushing" work to the next step in the chain, we pull it. This means testers don't take on work until they are ready for it. The programmers in the example above can't pull work, though, as that would break the work in progress limits. They have to figure out how to help the testers-- by testing themselves, perhaps, or by writing test tools or "drivers." Rather than identify a bottleneck and complain about it, you pull systems expose the bottleneck and fix it, thus increasing throughput. See how it's starting to work together?

Improve the work area to eliminate needless motion. The classic example is a worker walking several feet to grab a letter, sort it into one of five bins and repeat the process for a whole pile of mail. Instead, bring all the work items together so he can sort without walking.

The objects in software are different--compilers, databases, continuous improvement (CI) frameworks and version control--but the reality is the same. Too many programmers live with code on different environments, with build systems that take too long and with design changes sent via email and out of date before they are even read.

If your technical staff is asking questions about "correct" system behavior, and waiting hours or days to get an answer, then you have the same problem. The lean software solution is to get the person who can answer the question in the room--either by embedding the customer directly in the team room, or, for a technical question, pairing new programmers with more experienced staff so they dont get stuck.

Limit scrap. The most obvious example of this is reducing the amount of defects that escape to production--and, if possible, preventing them even from getting to a tester. After all, if testers find a defect, they have to stop, reproduce the problem, document it, pass it back to a developer to fix, wait and then retest. Preventing the defect eliminates handoffs, dead time and rework.

While these tasks can't always be eliminated, they can often be streamlined. The tester might mention in passing the defect and work directly with the developer to get it fixed, instead of a more formal (read: slow) document/triage/handoff/retest process.

Focus on continuous improvement. "Continuous improvement is the basic hygiene of knowledge work" Tom DeMarco and Tim Lister wrote in the landmark book PeopleWare. The heart of continuous improvement is change and differentiation, which is exactly what standardization is designed to prevent.

If continuous improvement is to be a core value in lean--and it is--then teams need the freedom to adapt the process over time in order to improve outcomes. This repaints standards not as a rule, but, rather, as a sort of guideline that's defined by the team itself, once technical staff believe the benefits of standards exceed their limitations; these standards can also be removed at any time. (The same concept applies for inter-team communication and services, which are continually improved through collaboration with management.)

Here's one example: When I started at one recent client, the team was doing "book" Scrum, with two-week iterations and bi-weekly ritual meetings that included retrospective, planning and estimating meetings. These meetings often felt forced and artificial. In the retrospective, for example, the team had to talk, so it brought up either minor issues or repeating, hard-to-address issues.

The iteration boundaries, at one time helpful in creating a sense of urgency, weren't adding much value. Some meetings were straight-up waste. We dropped iterations and moved the meetings to a just-in-time format. This meant sizing each story right before it was worked on, tracking improvement ideas on sticky notes and scheduling a meeting when the list reached six sticky notes.

Since we made that switch, two other teams switched to this continuous flow approach; three other teams continue to use iterations. There was no mandate from management or standard to follow. Certain teams simply saw waste and wanted to eliminate it.

Lean Software Development for Tomorrow: Moving Beyond Toyota

In short, lean thinking means a constant flow of improvement ideas can come from anywhere, at any time, as long as they improve outcomes or reduce risk.

The bullet list above is not magic. It's just a summary of thinking at the time. Ten years after lean first came to the software community, we see new tools emerging that allow teams to reduce waste further. Automatic build tools, cloud computing and software as a service can all reduce time spent waiting and allow the team to make forward progress. Over time we should expect to see many lean software implementations, ones tied not to the rules of Toyota in 1986 but instead to the critical success factors of the organization from which they sprang.

As Poppendeick says, it's not a thing you do. It's a way of thinking; for some, perhaps, a way of life.

The author would like to thank Joseph Ours and Bob Corrick for their assistance in producing this article.

Matthew Heusser is a consulting software tester and self-described software process naturalist who develops, tests and manages software projects. Matt is a contributing editor for Software Test & Quality Assurance Magazine and his blog "Creative Chaos" focuses on software writing. An elected member of the Board of Directors of the Association for Software Testing, Matt recently served as lead editor for "How to Reduce the Cost of Software Testing" (Taylor and Francis, 2011). You can follow Matt on Twitter @mheusser or email him.

Follow everything from CIO.com on Twitter @CIOonline, on Facebook, and on Google +.