Kai Gilb... Agile is dead?

So there's this fellow out there named Kai Gilb who is writing a series of articles about how bad Agile is. To increase the visibility of his arguments, and because his site requires log in, I'm commenting back here. I also don't want to lose the old thought-a-roos...

So, many of his points so far in the series are spot on. I will say that. In fact, given the so called "State of Agile", I'd give Kai probably an 85-90 out of 100. There are, however, a few things that I think merit attention in his arguments. This probably isn't my best writing, call it draft one.

Rebuttals

Hey, Quit Picking On the Manifesto

I'm a little concerned whether Kai understands the historical context of the Agile Manifesto. The Agile Manifesto was written in a time period when many organizations were drowning in process molasses. Kai really rips into the Manifesto, particularly around the idea that the Manifesto does not focus on delivering Business Value. This is a great example of how someone can be oh so right and still wrong.

I would first make the very simple observation that the Manifesto was written in 2001, that's nine years ago. It came out of a discussion culminating in a small conference between like minded individuals who all were fed up with the stifling environment being created by bloated, bureaucratic processes. The mindset at the time was that 1/3 or more of the effort on the project should be on requirements gathering and design. Many, many organizations interpreted this to mean that you should do all analysis work upfront, before development began. This was largely driven by a group of studies which indicated that the reason most projects fail is due to improper understanding of the requirements. The consensus answer was that we had to do more thorough analysis before we tried to build it. Guess what? Projects still failed (and failure to deliver is not the only dimension of failure). Why did they fail? I've been party to several project failures (either in the trenches, or coming after the fact to pick up the pieces). The reasons have not been for lack of understanding the technology. The technology is usually the one thing you can account for uncertainty in. The reasons have come down to various flavors and degrees of the following (nothing new here)...

1. Took too long/cost too much (time = money or opportunity cost)
2. Requirements were poorly understood.
3. It was built according to spec, but spec didn't meet the need.
4. The business just didn't want it anymore.

When you read the Manifesto, it really comes down to "If you follow the process, and don't deliver you lose. If you don't follow the process, and deliver, you win." What Kai rightly objects to is the lack of focus on "business value" and instead the focus on "product". However, the Manifesto was written at a time where "business value" was something simply calculated as Return on Investment (ROI). Since Agile has opened up the dialog, "business value" has become "Business Value" and I argue that what the initial signatories of the Manifesto had in mind was "Business Value". Jim Highsmith's commentary on the history of the Manifesto all but confirms as much:

At the core, I believe Agile Methodologists are really about "mushy" stuff about delivering good products to customers by operating in an environment that does more than talk about "people as our most important asset" but actually "acts" as if people were the most important, and lose the word "asset"...

Now, certainly, reading that, you pick up on Kai's argument that the Manifesto is more about the developer than the stakeholder, but you also see that it is about enabling the developer to satisfy the stakeholder (customers). The goal is "delivering good products to customers". The means is "operating in an environment...". Everything I read in that time period (say 2001-2003) about Agile from the signatories was ALL about the customer. No one had anything else in mind. We, as people who were practicing or fighting with their bosses (me) about practicing Agile, were concerned about nonsensical processes that seemed to be more about justifying jobs for particularly incompetent business analysts and 'architects' (washed up senior developers who couldn't hack management) and heading off the dreaded FUD (fear, uncertainty, doubt) than satisfying the people who needed the product.

In short, Kai's being rather unfair to the Manifesto and its authors/original signatories. His Claims in part 2 actually sound suspiciously similar to similar statements being made 9 years ago.

Great Points, But Don't Feed the Trolls

My second concern in his arguments has nothing to do with content, but is about the approach. He is taking a rather hyperbolic approach which will inevitably be taken up by certain brands of trolls who want to be in total control of everything to say "Agile isn't working and so we should go back to the 'good old days'." They won't read any further. They will just take his headlines and run off with them.

That's about as far as I'll go with taking significant exception to his arguments so far (up to post 2).

Let's Agree to Agree

This always happens

In the last three years, the idiot brigade has taken over Agile. Scrum (which is pre-Agile in all actuality) has become "The Agile Methodology" or "The Agile Process". I honestly believe if you jumped back 8-9 years to the early Agilists, you'd scare them out of their pants if you told them Scrum was pretty much it (go read that history again). Their whole point was more to do what was sensible for your organization, your project, and your customer than to follow someone else's cookbook. Sure, idea theft is the sincerest form of flattery, but slavishly following it is not what was in mind. These days, Scrum Master certification is almost mandatory to be considered an Agile practitioner. I've never seen certification in anything actually shed any light on anyone's actual competence. I've seen people without a relevant certification who were absolutely brilliant. I've seen people with technical certification who shouldn't be allowed anywhere near a computer, let alone operating one.

Useless User Stories

User Stories used to be what Use Cases were before Use Cases became too bloated. Now User Stories are what Use Cases became. Generally speaking, over-specification kills. My 'text book' case of over-specification was about 5-6 years ago.

Coming on to a project for a month or so to help out in their final phases of development, I got this requirement: "We need to be able to output all of the Web Pages associated with the client to PDF." I was a bit scared by this. At the time, there really weren't a whole lot of good PDF generators for server environments (particularly Windows) and knew that our usual purchasing procedures ran 2-3 months anyway. Not exactly what the project management wanted to hear as they were, as usual, way behind and way over budget. A few more conversations with the folks on the requirements side got the wheels turning. Back at my 'desk' (a folding table in temporary space), the light bulb finally warmed up and lit up. Tromping back across the maze of cables taped across the floor of our temporary accommodations, I cornered the BA.

"What do they do with the PDF," I asked.

"To print it of course," he said.

"So you don't need a PDF, you need to be able to print the pages in a decent format."

"Ummmmmm yes. But we can't do that."

"That's what style sheets are for. So can we agree that we need to make it printable, not generate PDFs?"

"Yes."

Yes, coming in at the tail end of a project and trying to make sense of all the style sheets is a real pain. But there was no way we'd be able identify, purchase, and implement a PDF component in time. It wasn't fun, but it was mission accomplished. Lesson learned... never put off printing til the end and always question the requirement until your certain you've separated out needs from someone else's solution to the needs.

This is where small user stories, use cases, what ever the heck they are called this go 'round really come into play. If you force them to keep it on an index card (in large print!), then you're more likely to get the need than their idea of a solution to the need. Don't let them start the problem solving without you! Sure, let them play, but really, the developer is the expert in what can and can't be done here.

If You Need Software Created, You Need A Developer

Really, we are back on the same roads that we were before. The same clueless wonders continue to adhere to the patently false impression that software development is a factory process. According to this mindset, developers are a commodity and you can drop someone who didn't even touch a computer until they started learning how to program 4 years ago into the same spot as someone who has been at it for 10 years and was using a computer 10 years before that. Fundamentally, there's a misconception (in my mind) that software development is a group of problems solved by thorough analysis. Once you understand the need, you will be able to get any reasonable competent coder to crank out an app to address the need. NONSENSE! If all you are is analytical, you will figure out what the problem is, but you will often not be able to solve your way out of it. The problems we face require creativity, an understanding of how it has been solved in the past, a certain level of battle scars, and just plain experience. Software development, like it or not, is about the people doing the work. I've seen plenty of cases where the team that was 'too small to get it done' got it done and the 'project that was too important to fail' did because management panicked and threw people at it until the project was paralyzed with meetings, documentation, and above all, Business Analysts. BA's are project killers in my opinion. I did an inventory of projects a short while ago that I've worked on or had intimate knowledge about and the results were startling:

• Without fail, the success level of the project was inversely proportional to the level of BA involvement. (i.e. BA's did not help)
• The successful projects without fail had the developers talking directly to people who would use the product (or were past that point in their career and were now managing those people).

Conclusion: Despite the all the FUD, developers actually do a pretty good job of listening to the user and know how to turn that into an app.

Drive-by Bullet Points

1. Agile is now firmly in the death grip of cargo cult mentality. Most 'adopters' are using Scrum and XP in a cookbook fashion without analyzing the philosophy behind it. When confronted with the problem, they blame process and not their own thinking. Agile is all about adapting to your customer's/organization's needs, not slavishly following the process to the detriment of what you are trying to do.
2. This cargo cult mentality is exactly what dooms everything in organizational methodology. Six Sigma worked for the early adopters who really bought into it and adopted not only the process, but the philosophy. Ditto that for RUP, CMM, XP and anything else you care to think about. It's when the success stories make it to the general population that the methodology starts to break down. The successes weren't aberrations, they just had the right conditions to be successful.
3. Agile got started with people who were really just saying "hook me up with the customer so I can deliver what they need. Get the cruft out of the way. Great things can be done with great people".
4. Kai seems to be on the road to making the very same argument. In fact, the Craftsman movement is starting to shape up (although this is the nascent form) and based on Kai's arguments so far, I think he's really leading in this direction. At the least, he's starting out on the same foot. I'm going out on a limb... within 10 years, someone will be writing something like "I have nothing but respect for Kai Gilb, but..."
5. Kai is right, the state of Agile is bad. It is now as bad as what it replaced. But, real Agile folks won't disagree. Not only are we not afraid to hear it, we've been saying it all along!
6. This is a case of Evolve or Die. If we tried to stick to 9 year old ideas without modification, either those were true geniuses (who should be working on cures for cancer, or at least quantum algos) or we're true idiots. The landscape has greatly changed in 9 years. Our approaches need to change as well. But the fundamental concept of getting what the customer needs into the customer's hands hasn't. Call it Business Value, call it Working Software, or whatever. It really is the same thing.