The KIPOS principle.

Working in software development for too long now, I've had the "joy" of working under a few dominant project management philosophies. These include (if you're interested in this, I probably don't need the definitions, but here it goes anyway):

• "ad-hoc" - The "get 'er" done principle. Do what seems to make sense at the moment and don't really think about it. Maybe after a version release you do a review and establish best practices, maybe not. Can be surprisingly successful with small teams even though it may seem like no one knows what's happening next week, or it's based on a lot of hope (as in "I hope I don't have to do a kachillion hours of overtime next month to meet the deadline").
  
• Waterfall (aka "the Software Development Life Cycle")- This is a linear model where you Gather Requirements, Design, Develop, Test, Release... over long cycles, often a year or more. This is a gated model, so you don't begin development before you do design, and don't do design before you start gathering requirements. Lots of documentation, most of which is out of date before the code even starts getting written. Generally favored by those who believe in top down management and who think of people as "resources".
  
• Agile(including Scrum, XP)- Agile favors much shorter release cycles and thus lots of early on testing. Agile is based on the ideas that you minimize those things that add limited value (lots of documentation, rigid plans) and focus on those things that get you to working software(focusing on the software vs. huge specifications, communication over contracts, etc.) Those practicing Agile generally view themselves as members of a "post-waterfall" society. They may be charitable and say that waterfall was a needed step along the way. They may be very much uncharitable and say that waterfall was... well we'll leave that alone.
  

The short answer is that no matter what methodology you use, your chances of success are slim if you think the methodology is the main source of your hope. Hope can never be found in a THING. Hope can only be found in someone you put your hope in (best placed in Jesus if you ask me, but I'm here to talk about getting software done at the moment, more on that in another post). Projects don't succeed because of process. Dan North wrote an article a while back on Infoq that will drive most of my discussion here. It looks at the Dreyfus model of learning and applies it to software development.

The main takeaway for this in relation to what I'll call the KIPOS principle is the following... Waterfall methodologies rely on preventing the damage that the inexperienced, incompetent, etc can do to your project. This comes at the expense of limiting the good that the experts and event intermediate/journeyman can do for you.

Agile, on the other hand, relies on the team members to manage the team members... helping the inexperience and incompetent get better. In particular, XP emphasizes pair programming for just this reason. However, if the team loses sight of someone or some team members collaborate to cheat or game the system, then it can fall apart.

Having worked under all of these paradigms, I can tell you I positively hate waterfall if you can't dial back the process to a rational level. I've been on some projects that were in the hundreds of man hours that required the same level of documentation, artifacts, etc as projects of 10s of thousands, even hundreds of thousands of hours. When you're producing long documents to communicate with yourself, there's something wrong here.

On the flip side, I've seen Agile where the team members just don't seem to get it. They are accustomed to working with a rigid spec and when asked to work outside those boundaries, they may have a problem. They may be smart, competent on the coding side, follow the rules, but they just seem to struggle anyway. In following AGILE the methodology family, some teams seem to lose sight of what agile means, at least according to the dictionary. And so, without further ado...

The KIPOS Principle

The KIPOS principle is simply this "Keep It People Oriented, Stupid". Look at your team: What do you have? If you have a bunch of people who have only ever been handed a spec to work off of, then you need to build a spec. If you have people who have only ever worked in a small shop with little idea of formal methodology, or who like a cleaner canvas to work on, then sure, go for the less documentation approach. If you have a mix, then pair them up! But above all, you can still hang on to some of those Agile values and principles. Keep in mind that the Agile Manifesto never said "we never do documentation" etc, it said "we value working software over comprehensive documentation".

I'll probably expand as time goes on into specific cases, but please, above all: Look at your team. Do what works to get the software out the door. Two things I can tell you that will always be a part of do what works:

1. Get the software into a testable state early and frequently send out releases to testing, however you decide to test. If this is more than a month between test cycles, something is wrong.
   
2. Continuously Improve (I'll probably have a post on this later, but meanwhile) This means always look for ways to improve the behaviors, processes, tools, training, etc in your team. Never stop improving and don't sweat not being perfect... you're always improving, so you can adjust later!