Hamilton at NASA, Photo courtesy of Creative Commons

A Brief History of Agile

The story of Agile has only just begun. An approach to software found across the world from Airbnb to Apple, startups to multinational companies focused on iterative development.

Agile is not really about computers or software, it’s about the organization of people.

In 1965, Margaret Hamilton, a developer at NASA, coined the term “software engineering.” On November 22, 2016, Hamilton was awarded the Presidential Medal of Freedom by US President Barack Obama for her work leading the development of on-board flight software for NASA’s Apollo Moon missions

But similar to the days of the “human computers” at Harvard, software was seen as a kind of second-class lower science, less interesting work. Hamilton was a big proponent of trying to bring discipline from hardware engineering into the real of software engineering.

She famously said, “The space mission software had to be man-rated. Not only did it have to work, it had to work the first time. And not only did the software itself have to be ultra-reliable, it needed to be able to perform error detection and recovery in real time.” A significant, intense ask that bring us today to have some abstractions helping us with our software delivery today.

We can see that many of the early software projects were about the critical importance of maintaining safety.

Sending people into space has to be right the first time. Space missions are incredibly expensive and there’s a huge amount of government funding and accountability.

 

A Risky Implementation That Invites Failure

In 1970 Winston Royce published a paper called Managing the Development of Large Software Systems where he draws the following graph, which looks very much like a waterfall going from system requirements, analysis, coding, testing, and operations.

 

Immediately after this graphic he says, “I believe in this concept, but the implementation described above is risky and invites failure. Required design changes are likely to be so disruptive that the software requirements upon which the design is based and which provides the rationale for everything are violated.”

In other words, Royce is saying that cost increases as time goes on, and this graph really represents that. If the cost of change is exponential in these systems, then it demands that you have to know exactly what’s happening in the previous step. It has to be accurate.

It has to be validated before going to the next step because as you move through each step, it gets exponentially more expensive if there is changed required, whether that is an error or whether that is a change of the requirements.

 

 

Tragic Consequences

With Waterfall, there’s a strict structure which means every decision has to be right, and every advancement requires a formal sign-off. What follows are some case studies (some more extreme than others) where this perfectionism a waterfall process broke down:

An example can be seen in London’s computer-aided dispatch in 1992. This was a computer system, which aimed to replace a paper-based system and increase the efficiency for the dispatch of London ambulances.

Great idea but the result was a disaster. It saw multiple units being sent to the same address and no units being sent to others. Calls got lost, which resulted in repeat calls, which were logged in the system individually. This congested system couldn’t handle the volume of calls, which was unexpectedly high because they were able to accurately dispatch the ambulances.

There was no rollback. There was no plan B. The process only included one plan, with no adaptability. In the media that followed, there were reports of between 30 and 45 deaths, all due to a software release process gone horribly wrong.

 

Takeaways From The Failures of Waterfall

Assumptions About The Accuracy Of the Previous Step

If we go back to the waterfall process you can see the major failings: the inaccurate assumptions, and the clunky sign-off process. But these are related. Often in Waterfall, the people who are signing off for a team to get to the next step are not the most familiar with the project.

They’re just looking through requirements and saying if everything looks good. This top-down approach defaults to a process where those who sign-off are making assumptions, and they don’t have the day-to-day experience of working on the project to help their judgment.

Too Much Pressure Around Fixed Scope, Time, and Cost

When managers push developers and their teams to work longer and harder to meet their contractual demands, it’s ultimately going to result in producing poorer quality software. They’re going to make sure they’re getting that sign-off process without necessarily making sure whether what they are building is the right implementation.

Integrated Testing Happens Too Late, If At All

We must perform end-to-end testing. The first time that everything worked together should be in the lab environment and not in real, which is quite a scary thought. When there is no A/B test, no small trial, just a really big launch that proved many people didn’t understand what was really needed to solve the problem accurately the story failed.

Process, Not People-Centric

Ultimately, Waterfall’s biggest failing is that it puts its trust in a system, not the people working on a product. If your system does not empower the team to test ideas and change requirements based on these tests, then it is simply structure for the sake of structure.

This reduces the accountability of individuals and puts more emphasis on successfully passing a gate, over building the right product.

 

Source: https://content.pivotal.io/blog/a-brief-history-of-agile-part-1-the-rise-of-waterfall