Jim Leonardo's Blog

Stop SOPA!!!

2012-01-18T10:12:00.000-08:00

Needless to say, sign me up with those who want to stop the SOPA and PIPA bills. Both bills are heavy fisted attempts by Big Content to protect their intellectual property without any concern for the ramifications.

Even if you strongly believe piracy is wrong (and I do), Congress's attempts to legislate technology to protect piracy have been ineffective at best and unconstitutional at worst. How we justify 100,000 USD per violation of copyright for a 0.99USD song under "no cruel and unsual punishment" is beyond me. How SOPA and PIPA won't produce both a chilling effect on free speech is also beyond.

More importantly, both bills CLEARLY violate the concept of Due Process. If you think this is minor, you should investigate some of the cases that have been brought under the old DMCA (Digital Millenium Copyright Act). Just the Righthaven fiasco should be a cautionary tale(and Righthaven at least had some justification for pursuing at least some of the cases, they just did it wrong). Even before DMCA, we've seen small websites (and bulletin boards before that) bullied by content owners into shutting off fan created content over copyright and other intellectual property concerns even though they clearly would fall under Fair Use. Software applications have been gutted to comply with a content owners perception that it could potentially be used for pirating content. Now, some people think that prior to trial and with little review of the merits of a case, a site should be effectively shut down.

The sad part is that it's completely ineffective from a technical perspective. All that will happen is that the pirates will move on to other means of contacting one another whilst the legitimate Fair Users or accidental infringers get clobbered by content owners who have their nose out of joint. As the content owners won't be able to stop it with their current laws, they will pursue an even more draconian set of laws in another 5-10 years time.

But: think how the copyright trolls will use this! Instead of their quasi-legal extortion letter saying "pay us or we sue", it will say "pay us or we'll have your site taken off the internet".

We've seen that there have been reductions in the Due Process violating provisions due to all of the pushback, but still, the idea that I would have to monitor every site that I link to in order to determine if it hosts violating content is nuts. What it means is that I will not be able to cite any references in any blog post for fear of the site I'm referring having some smattering of violating content. I not only need to check it before hand, but after as well. What does this mean for blogger, wordpress, tumblr, etc? All blog hosting websites would basically be put on notice that they need to monitor every single site that one of their users posts a link to. While we're at it, why not pass a law making restaurant guests liable for dining at a restaurant that buys its tomatoes from a distributor that buys from a farm that uses illegal immigrant labor. It's about the same thing.

Anyway, there's others who have articulated this much more clearly than I have. I'd link to them or mention them, but that would be a bad idea in a SOPA world.

Migrating a bootable VHD to Fusion in Mac

2011-11-17T18:25:00.001-08:00

I just got a new Macbook Pro for work use and wanted to move a windows image which I used for client work. The image was a bootable VHD file (like this) and that presented a couple of issues for a simple move.
First, the bootable VHD is not the same as the vhd file as used for Microsoft's virtual machine technologies (whether Virtual PC or HyperV). VHD just means "virtual hard disk", but that does not mean the machine image contained within that disk is built for being run as a guest inside another machine. The bootable VHD machine is configured to be run as it's own machine, it's just running via a packaged up file sitting inside of another hard drive partition. The result is you cannot use VMWare Fusion's Import feature to convert this disk from a VHD to a VMWare image.
What you could do is use Fusion's "Migrate Your PC" option to migrate the machine, with a bootable VHD there may be a gotcha. The gotcha is that the Migration option will automatically try to import the host (naked) hard drive in addition to the VHD image if it's connected to your VHD machine. Your VHD image will not be big enough to hold it's parent hard drive! I suppose you could detach the drive, but doing so would require to alter the original machine. That doesn't seem such a good idea if you want to go back to that machine, even to keep as a back up.
What you can do is run the VMWare Standalone Converter from within the Windows machine you're trying to move over. This will allow you to tell it to not try to package up any drives you don't want. It also will let you put the packages elsewhere, not force you to fit it inside of the VHD's space. This creates the virtual machine for you, ready to run in just about any of VMWare's products. You can then simply copy the files over to your mac and then launch it in fusion.

If you did not take the drive with your boot launcher info, you will need your windows CDs (or isos if it's from MSDN) to run a quick Windows Repair to fix the boot info inside the new VM. It's pretty painless, just follow the instructions.
This is a new machine, you may need to run windows or office activation again depending on original activation requirements. OEM installs of windows will probably give you grief, but it's unlikely you were running that from a VHD to begin with.

Overall, once I figured out the need to run the stand alone converter, this was a pretty easy process. But, it did take a bit of noodling and googling to figure out.

2011-09-07T15:38:00.000-07:00

A new little toy out there... Wordle. Feed it some text (or a blog) and it creates a word cloud for you.

Apparently I want to Just Get Work Going People.

Clicking through may cause issues in some browsers, seems to work fine in Opera though.

A dice tool for 7th Sea.

2011-08-15T19:00:00.000-07:00

I've put the first version of dice roller for 7th Sea up on my gaming related site, smellyoldgamer.com. As of right now, smellyoldgamer is still just an alias for jimleo.com, so you can just use jimleo.com as the root too. The full address to the dice roller is http://www.smellyoldgamer.com/sog/7thSea.html.

I've built it to play nice with keyboard, mouse, and touch displays, so please check out. Notes on keyboard use are on the bottom.

I kept the page completely self contained, so you can just save it and use it offline. It doesn't save anything, post it back to the server, or anything like that. It's completely intended to be locally run because everywhere your device goes may not have an internet connection. Simply save it as an html file and load it up in your browser for later.

A possible enhancement over today's version would be to keep track of the last x number of rolls so you wouldn't need to reenter them.

I've tested it using the major browsers, and as usual, IE is the only one annoying the snot out of me. There's a bit of page "dancing" going on that causes one of the sections to move a bit unexpectedly that I'm still working on. Firefox seemed to be the pickiest and managed to find bugs in markup that the other browsers compensated for.

A Few Very Useful Tools–Mar 2011

2011-03-17T22:43:00.001-07:00

Wow, I missed a YEAR’s worth of A Few Very Useful Tools. You are now entitled to your subscription fee back. Given I pay most of you to read this, you owe me money.
[Warning, philosophy ahead]
Seriously though, a year ago I was running out of things to post, so it was time to let it rest for a little while. There’s really only so much you can use well enough to get a good idea of whether it’s worth using. Lots of reviews represent 5-10 hours of use of a product (if that) and often come down to what I euphemistically call “counting cup holders”. Feature count doesn’t matter if those features aren’t well implemented. I would much rather use a bunch of small utilities that did one thing well than one mega-app that did it all but not very well. Granted, there are times where the whole is greater than the sum of its parts, but that requires true integration.
Without further ado, some gems from the last year.

Git – Source control the way it should be (I believe I heard that from Linus Torvalds). Most of all, branching and merging are so easy you stop thinking about it. You also can ship changes directly to someone else without creating ugly patches/shelves and without sending it through the central server. Why? Git is distributed from day one. A central repository is a decision you make as a project team, not requirement of the software. You commit to a local repository, then merge that repository with a central or with your peers. Git is much more a command line tool, graphic tools are lacking. They’re available, just lacking. Mercurial is a close second and has better graphical tool support if you’re on windows, but Git’s the king right now in terms of functionality.
Resharper – Ok, it was on my Dishonorable Mentions list a year or so ago. That said, I think they’ve improved it, but I’ve also found that on a “real” machine it rocks. That said, a “real” machine is a Windows 7 box with min 4GB ram and a beefy CPU. An SSD and 6GB of ram makes for a nice experience with “R#”. This configuration will neuter my main complaint from a year ago, which was performance. The refactoring tools, the test running tools, and the various navigation tools make for a completely different world of development.
Remote Desktop Connection– Silly as it sounds, this little tool is just what the doctor ordered if you have to work with multiple machines. I’ve stopped trying to manage multiple machines with a Keyboard-Video-Mouse switch. Using Remote Desktop, you can just open a window on one machine into another machine. It is not screen sharing. It let’s you open another desktop as if you were another user logged into the system (note number of users is limited). As far as I know, there’s nothing like this for mac, although there is for Linux. XRDP let’s you pull this off in a Linux world and share between the two as well! Mac doesn’t get to play, sorry.
Moq – Unit testing is the sign of a professional developer. It’s also the sign of a sane developer. Moq is a mocking framework that makes unit testing in C# absolute cake. I would favor this over any pre-VS2008 frameworks as it makes extensive use of lambdas to make mocking out dependent classes absolutely easy. Of all the things I can say about it, the best one is probably this: just about anything I’ve thought would be hard to mock has already had a way to do it in Moq. You really don’t need much more than the Quick Start page for this, trust me.

Well, that’s it. Four stellar tools in one year, only two of which I’d say are new-to-me in the year. Back to the grind…

DateTime is Dead, Long Live DateTimeOffset!

2011-02-02T19:13:00.000-08:00

Ok, maybe I’m late to the presses with this one, but given that no one I’ve talked to recently knows about this, I think it’s worthy of more press…

The .Net DateTime structure is a nice handy structure that we basically can’t escape. Funny thing about users, they have a tendency to want to know when they did something, when they should do something, and how far overdue it is when they didn’t do it.

The problem with .Net’s DateTime is the lack of time zone information in the structure. Believe it or not. Yeah, really.

In .Net 1, we had no indicator of what time zone it was for. In .Net 2, they introduced the notion of “kind”, but that was limited to indicating “Local”, “UTC” (i.e. GMT or Zulu), and “Unspecified”. Even with .Net 2, this led to most of us just taking the convention of immediately converting everything to UTC. Unfortunately, that led to more than one bug when someone forgot to convert or double converted (or triple or quadruple or…).

MS’s answer was always that DateTime was a 64 bit structure that they used something like 62 bits for it. They were able to add in the “kind” by using the extra 2 bits, but that only allows for 4 possible values if you recall your bitwise arithmetic (00, 01, 10, 11). Hence, Local(10), UTC(01), Unspecified (00). Maybe you’re thinking that “we only need 24”, but that doesn’t account for India or Newfoundland (something + .5 hours offset) and leaves you scratching your head at what to do with daylight savings time.

Worse, SQL Server’s DateTime only stored the date and time with no idea on time zone. However, there at least we could just say “server time” and be done with it.

This is manageable and usually not too much of a problem in self contained systems. Where it becomes painful is in situations where you need to serialize data and send it somewhere else. Then you really need to worry about that time zone and it gets really goofy when you’re working with datasets. Once I get over the psychological trauma of dealing with datasets and time zones, I might be able to blog about that.

Since VS2008/.Net 3.5 and SQL 2008, we’ve had a new and vastly improved way of handling this. This is called DateTimeOffset. Unfortunately, the name seems to throw everyone off. DateTimeOffset does not just store the offset (I.e. –5 from UTC), it stores the entire date time (2/2/2011 9:56:12 PM -05:00). Now,the interesting part is that this takes a TimeSpan, so you really can make this as flexible as you want it. TimeZoneInfo helps you with finding timezones and applying the various rules such as daylight savings time. Both just plain work with SQL server.

Short of it is that I plan on forgetting all about DateTime and only using DateTimeOffset from now on. Now I’ll always know what time zone I meant and not have to figure out what rules I need to apply to get it into local time.

Once I figure out the gotchas, I’ll relay those too.

See:
http://msdn.microsoft.com/en-us/library/system.datetimeoffset.aspx
http://msdn.microsoft.com/en-us/library/system.timezoneinfo.aspx
http://msdn.microsoft.com/en-us/library/system.timezone.aspx

Rescuing yourself...

2010-03-08T23:42:00.000-08:00

Lately I've been blogging about the dimensions software project failure and what kinds of personalities you see in projects that have failed or are about to fail. Now, I'm the kind of person that says you need to find the negative to fix the negative, so I'm not a pessimist as much as a tinker. Everything we do can be done better than we're doing it now. That doesn't mean we should try to be totally perfect, just that we should be better than last time.

I feel that I need to offer up a little disclaimer before we go any further: Discussing project health and management is very difficult to do with some co workers and bosses. Please go careful. Charging ahead convinced that you are right is likely to only end badly. Open ears lead to persuasiveness. Closed ears lead to a pink slip. Reading body language is just as important.

The Roads We're On

I've seen projects that were successful all the way through. Any hiccups were just bumps on the road and even freshly laid pavement's going to have a bump or two. Then there's driving in NYC... you manage to get through, but the going is rough and suicidal taxi drivers seem like they are trying to ram you into the East River. These are those projects that seem to succeed in spite of themselves. Other times, you feel like Roscoe P Coltrain in a Dukes of Hazzard episode... you just know you're not going to make the jump over the creek yet you try because if you don't Boss Hogg is going to tear into you. These are the projects that are going and going, but everyone knows what's coming and yet people keep pushing for whatever reason until finally the plug is pulled and you're all scrambling to not be the one who gets to be one of the ones canned.

If life were simple, you'd know which type of project you were on from the get go. If it was all going well, you'd happily keep with it til the end and pat yourself on the back. If it was rough, but survivable, you might stick it through figuring you might get some kudos for pulling it off. In reality, projects rarely stay on the same footing, but you rarely see it trend upwards. Either a project is good all the way through, or it just gets steadily worse. Call it project entropy... unless something else influences it, it tends to degrade over time. However, if we understand what's going on and keep a higher level view of what's going on, we might be able to act to correct it.

What goes wrong?

Phaser Burns

This downhill trend of projects is something I 've seen tend to happen in phased projects. During phase 1, you're primarily focused on getting something done. Proving the basic premise, getting a rough architecture mapped out and executing on the first set of use cases/user stories/whatever you call it. This tends to be busy, but the atmosphere isn't uptight and things tend to go ok. Phase 2 is where trouble begins. Brooks calls it "the Second System Effect" but I think there's maybe more to it. Please go read Mythical Man Month. It may not be spot on all the time, but the observations are still often valid 30+ years later. His main point in Second System Effect, and I pretty much agree with it, is that everything that was highly desirable but not mandatory got cut out for version/phase 1. Unsurprisingly enough, phase 1 goes reasonably well and delivers on time. In version/phase 2, there's a tendency to pull in far more than can be accomplished and failure mode starts to set in. What was once a great example of how to develop software now becomes a boondoggle.

A few things contribute to this. Obviously, biting off more than you can chew is a huge contributor. But how exactly does that happen? We're all intelligent adults and most of us aren't too terribly masochistic. Thus, we can cut out the overt psychological oddities and focus on more subtle issues.

The Inevitable Gold Plating - Gold plating is of course a major curse in software development in general. It tends to be at its worst in phase 2. All the 'nifty stuff we didn't have time for' now is in scope and the laser like focus you had in phase 1 diffuses as every 'brilliant' idea someone had starts to get to be a must have. In reality, business value's not there, but resume stacking is.

Good Old 80/20 - Quite simply, there's a subtle mental push that everyone feels even coming from within themselves. In phase 1, where your focus was on what was an absolute must have, you probably were cutting edge cases left and right. This leaves phase 2 with an incredible backlog of 80/20 ruled features. You know the 80/20 rule: 80% of the features are done with 20% of the work and 20% of the features take 80% of the work. Yeah, its just nice round numbers, but it communicates the point... edge cases often consume far more time than the main features yielding a low return on time invested. Contributing to this is that subtle mental push I referred to. Let's call it the Feature Counting Effect. This is brutally simple and goes like this: "In phase 1, we were able to deliver 20 features. Therefore, in phase 2, accounting for variability, we can plan on at least 15 features." Alternately, you'll think "you've learned some lessons" and instead plan for 25 or 30.

Go read that bit about the 80/20 rule again... if that's true you should only be planning FOUR features! After all, you've 80/20 ruled out some hard, but still pretty important, features. Go ahead, tell that to the boss and let me know how it works out... Of course you'll have to use your personal email address for me to reply to you because after they can your rear end, I won't be able to use your corporate email.

Cheaters Get Caught - Little known... err... widely known fact: No software project gets done without a compromise or three hundred along the way. Ok, calling it cheating is harsh, but the reality is that very often, to get past today, you don't have time to polish everything up to the point that you're really happy with it and you know it's going to come round to bite you. Sometimes, this turns out to be Gold Plating and YAGNI. But, almost always, there turns out to be a few features that have turned into dead end sin terms of being able to be extended to accomodate new functionality and you'll have to re-work them. Ok, this often occurs irrespective of what phase of development you are in. However, it's more likely to become an issue when new requirements and scope come into play. At any rate, we should always be willing to refactor to improve the internal quality of our code. Sometimes, it's just a lot more expensive and a lot less optional than others. All of this adds to the investment you need to make in phase 2... the larger the amount of code you have and the more users of the system, the more it costs to make any changes. (One of the prime drivers for refactoring in the first place... clean your junk up).

Your Boss's Boss (Political Commitments) - Let's face it, external commitments get made. Sometimes, to get agreement to go ahead with phase 1, the Boss needs to tell the Big Boss that he's going to get a feature in phase 2. Sometimes it's not Big Boss, but the Other Sponsor, etc. Anyway, there's a reason sometimes that 'not quite mandatory, but we better get it' feature is pushed out to phase 2: it's hard! Or, it's really not a good return on investment. The only value is because someone with a lot of weight to throw around wants it and this is going to drain time away from other looked for features in phase 2. At any rate, the premise here is that sometimes phase 2 ends up 'oversold' before phase 1 is hardly under way. Said another way: phase 2 is already doomed while phase 1 is still a long way from even being code complete let alone tested and shipped. This is all because the idea that "we'll deliver it in phase 2" was the convenient answer to anything that couldn't get put into phase 1 leading to significant overselling. Been there, done that, now you're reading my novel.

Piling on the "Help" - When phase 1 is seen as primarily exploratory, the temptation of the Bosses to throw more people into the mix can become irresistible. Surely, we can accelerate development by throwing more bodies at the keyboards, right? This what I call "World War I" methodology. It is this sort of thinking: if you have to cross an empty field with machine guns at the other end, then obviously the best way is to get as many people to cross the field at once hoping there will be too many targets. Needless to say, it didn't work in WWI and it doesn't work in software development. Sure, there are times to use more people, but I've not seen too many cases where growing the team rapidly works once the project is in flight.

All analogies aside, what happens is that in phase 1, you've usually got a small team where communication, knowledge sharing, group design, etc all happen naturally. During this time, people settle into roles and everyone knows who knows what. The team gets to know one another and start to achieve some level of that nirvana of not needing to talk to one another to know what they will do as a group next (I did say some level). Now, double the team. This is putting new wine in an old wine skin... it will not work. All those connections that have been forged over time will be broken.

Furthermore, I don't care how much time you waste on documentation. There will always be bits of the system that will primarily be stuck in the head or heads of the team. I might be able to document the system I built to the point that when there is a bug, some random person can read the document, figure out where the bug is and fix it. But, take the same random person and ask them to add a new feature. Sure, they can do it, but they will do it by integrating their brand new code into the system. They will not know the 'tweak this, extend that, refactor over there' that will get it done in the context of the system in a fraction of the time. Sure, over time, they will get it. I'm just saying it's foolish to think they'll get there quickly.

It Was All Bad

Sometimes, it's not a downward spiral. It's not even a slippery slope. It always is bad and that's all there is to it. You've probably already decided that the reasons to tough it out exceed the reasons to bail out and are going to ride this horse for what it's worth. But, what's going on here? We're all intelligent adults and most of us aren't too terribly masochistic... wait, I'm repeating myself.

Anyway, usually, you'll see the same factors creep in while you're still in phase 1 or whatever you call it that you see in phase 2+ as factors that doom you. Certainly, over commitment is a problem anytime you run into as is Gold Plating and other activities that don't add value. So, we can put all those reasons on the list.

Above that, there's a few other issues that can come into play.

Analysis Paralysis - When you get so wrapped up in thinking about the problem, you over look the truth in the old cliche "the best is the enemy of good enough". This is so classic, I'm not even talking about it any further here. Go here instead.

This Project That is too Important to Fail - If ever you hear those words about a project you're on, run. Don't walk. RUN! "This project is too important to fail" is a code for "Paranoia, Sloppiness and/or Egotism will cause us to make one bad decision after another." Seriously, I think I've seen at least three projects fall into this category. The project is "mission critical", so no resource will be spared so when there's the inevitable hiccup: the meetings that follow will distract the team and suck away crucial resources. Furthermore, inevitably, "momentum" and "positive attitude" start to take over and it's almost eerie how the decision making process begins to cave in. Add in all the other factors, and you're playing jenga on top of only one block. It's not the importance of the project that causes it to fail as much as the fear of failure. Basically, no one wants to sound like the one that is being negative and contradicting leadership or take on the political heavyweights. So, the small problems grow and grow until they consume the project, and the result is inevitable: Projects that are too important to fail usually do.

Everyone Wants In - These projects often also are the ones "too important to fail." Guess the story here: It's important so lots of people want in on the action. It's going to make the careers of people who are involved, so the proverbial wood work is working overtime. Everyone who can fabricate a reason to be on the project or better yet, 'advising' the project is there. The kindest way of looking at it is that you run into the Piling on the "Help" issue. The unkind way of looking at it (and more true) is that many (not all, or even most) are assessing their political/advancement prospects far more than their actual skill and contribution. At best, they are dead weight. At worst, they aren't content to follow the leader and want to be the top dog. I can't complain too much about someone who is making a full contribution even though their motivations may be bad. What causes problems is when they spend more time on their ambitions than the team and the project.

Now, teamwork really just comes down to mutual interest. Altruism rarely factors in for most people. We all are on a given project because it benefits us to do so. However, when someone is only on the project to reap and cares little for sowing, it's now a problem. What really becomes a problem is when someone's idea of reaping is not even letting the crops grow! Now, it's common to see large, important projects have some level of observer/adviser appointed by senior management to be their eyes and ears. That's ok, if a pain. But when someone places themselves in the role and refuses to budge, you now have a problem. This is the kind of person that makes Everyone Wants In projects bad. They don't contribute, but they sure make noise. When they start degrading other people's ideas simply because they aren't their own, the project is all but doomed. Avoid these folks, and Everyone Want In may not be all that bad. Presumably, it means you get to pick the most talented people available. However, if the talent selection process fails to weed out those who are here to take but not give, it's trouble.

Solving All the World's Problems - Ok, realistically, no one starts up a project thinking it will solve all the world's problems. However, quite often, the project's goals are overextended before it even begins. Honestly? Every project probable has more ambition than it's going to actually deliver on. It really only becomes a problem when the level of ambition turns into expectations that will not be satisfied. One way I've seen this manifest on several occasions is when what I like to call "false reuse" occurs. This happens when you have two or more stakeholders who describe their problem statements in very similar terms. In short order, the impression that everyone's needs can be satisfied with one project begins to get stuck in everyone's heads. Unfortunately, their problem statements may have been "I need to be able to produce a document I can print". They miss out that whole bit about what the document is and you end up trying to build a spread sheet and word processor all at once. We also run into this idea of "false reuse" at a code level, but it's a lot less damaging (still bad though).

We Can Solve Our Own Problems - This one is may be the worst. On one hand, the team may be facing an issue that it can't resolve by itself, but keeps trying to resolve itself. On the other hand, the team may be asking for help, but for whatever reason, the message never makes it out. I identified this in Faces of Failure as the "All Mouth, No Ears" type. There's a project manager who has sold the higher ups on a vision, a time line, a budget, whatever. But, reality and vision aren't lining up and they won't take the message out. Frustratingly enough, when I've run into this in the past I've also seen instances where the higher ups would have been happy to hear about the problem and readjust expectations! The project ends up dying because someone without the brass to look the Big Boss in the eye and say "it's not going to plan" early is in between the team and the ultimate decision maker (this isn't always the Project Manager, they are just in the role where it's easiest to do this).

Of course, if you are the project manager, you pretty much are going to get blamed anyway. Avoid the failure if you can and so read on. Keep in mind I'm not really writing towards PMs, I'm writing towards the everyone involved in software projects which by shear dint of percentages is mostly the developers, testers, implementers, etc.

Throw Yourself a Life Line... or... Go Down Guns Blazing

So, another long article and I've managed to talk mostly about failing again. Wasn't this one supposed to be "Rescuing Yourself?"

Oh yeah.

You see, it doesn't have to be a downward spiral. On occasion, I've seen a project rescued. So what do you do when you find yourself in the middle of a project that is dying? Or, what was good starts to become bad?

Don't Go There - I've never seen a project where the team worked well together (at that nirvana of knowing what to do together often without discussing it) fail. Note, that is my experience, I'm sure someone will contradict me. However, I've seen projects where the team that worked well together got split apart by all those other factors creeping in. Working well today is no sign of working well tomorrow. In general though, always keep checking the pulse. It's a truism, but it's easier to solve a problem when it's still small. But, that's not really the answer you were looking for, was it? You're already in failure mode, which leads me to:

Don't just do something, stand there! - That's the old HazMat advice and poking around the interwebs it looks like someone wrote a management book to that effect. I haven't read it and so can't recommend/endorse it. Anyway, if things are looking grim, first thing is to ask yourself "Why?" Look around for some of the factors I've discussed in this series. Some of them have obvious answers. Other's aren't so obvious. But, you need to understand the cause before you solve the problems.

Talk! - I wrote about the Complainer in the post on the Faces of Failure. You don't want to just complain, but you do want to open a dialog with whomever you can. Ideally, you get everyone involved. Often, this won't work. You may have an "All Mouth, No Ears" somewhere in the chain of command. You may have a Pacifist who doesn't want to see conflict, or a Warrior who is spoiling for anything to fight about. But, for the most part, you may be surprised to find out that even if others aren't seeing what you're seeing per se, they really want to talk about it. Lunch is the best team building exercise on the planet.

Blow up the Dam - A bit the opposite of "Don't just do something, stand there". Sometimes, you just need to stop talking and do something! You've ended up in a situation where for whatever reason, the project is just gaining any ground. Once you've identified an impediment or two, get rid of it. The larger the project, the more likely you are to have needless bureaucracy (see Everybody Wants In). Do what you can to get rid of it. Also applies to Analysis Paralysis. Sometimes, the best way to solve the problem is to start writing some code.

Use People Effectively - Outside of obvious things like not engaging people in useless tasks, think of more creative solutions when you need more people. Rather than adding yet another coder, you may need someone just to manage the build process and deployments to staging. You may be surprised how much time this eats up. Rather than having it take 10% of 10 people's days, have it take 100% of one person's day (ok, the math rarely adds up so evenly, you get the point though).

Be Formal Enough - A myth of Agile Software Development is that there is no documentation in an Agile project. The reality is that the Agile Manifesto says only that we should value Working software over comprehensive documentation. You are seriously misreading the Manifesto if you take that to mean no documentation. One of my Rules Regarding Everything is that "the more people you have, the more formal you need to be". In short, the more people you have, the more you need of things like documentation, task tracking, etc. Add distance into the equation, and it only goes up. Trying to substitue email and Excel for proper tracking doesn't scale, and trying to keep it all in people's heads is just begging for trouble if nothing else. In short, not adding a little formality can actually cause you to do more writing.

All Requirements Are Subject To Change - Seriously. Work on understanding needs not requirements. Lots of times, someone put a requirement down because they were trying to envision a way of solving the problem. If you have an easier, cheaper, or better way, talk to them, and...

Above All, Chutzpah - I have to throw some Yiddish in somewhere, don't I? After all, I grew up in the Borscht Belt. Anyway, Chutzpah is simply "audacity".

Without courage, you are just saying "walk over me". Many failed projects are savable, but someone doesn't stand up to say "enough". If you're the developer and all you do is say "yes I can do that, yes I can do that, yes I can do that." Don't be surprised when you can't do it anymore as work piles up. Meanwhile, you're only transmitting the message "I have more capacity." If you don't have capacity, if you have to work lot of overtime to get it done, SAY SO! This is probably a separate article by itself, but people can work O/T only so much before they melt down mentally.

If you see there are other misbehaviors/malfunctions in the team, you need to step up to the team and constructively take on the issues. If external influences are driving the issues, you need to start talking up the food chain. I've seen it happen many times that people just didn't know how complex etc something was that they were asking for. When I've traced the requirement back to the source (this can take some fighting believe it or not), its very often very negotiable. Talk and be courageous!

Being courageous is not being pigheaded and it certainly isn't being a cowboy. You need to listen to others, be ready to admit you're wrong, etc. However, while I say it's not being a cowboy, it may take some creativity. For example: sometimes your problems are processes that don't make any sense for what you're doing on this project. I've generally been successful by complying, but isolating the impact of those processes on day to day activities. Showing compliance in the battles you can't win also helps you in the battles that you can win. If folks think you're reasonable, they're more likely to listen when you say something is a problem.

Keep Your Skills Sharp, Soft And Broad (Ready to Relocate?) - All statements about courage etc aside... if you have a sense of project failure looming, you better load up the resume machine. You probably don't know what will happen in the end. Most software folks I know who have a hard time getting reemployed have let themselves get over specialized. Keep your skills broad, not just all technical. I've also found being ready to relocate helps a lot, but isn't practical for everyone. Also, keep in mind that changing jobs/projects isn't always the same as leaving the company! One more reason to be courageous, but level headed.

Anyway, this ran a LOT longer than I thought it would. As usually, probably 30 typos that I'll fix tomorrow or something. Adios!

Faces of Failure

2010-03-05T00:02:00.000-08:00

My last post discussed what I called "Dimensions of Failure". At the end of that post, I promised I'd look into some causes of failure this time. Unlike the last post which drew heavily on knowledge I'd accumulated from other authors in addition to seeing how things played out in real life, this one will draw mostly on those real life experiences.

It really needs to be said: Most software projects do not fail for reasons related to technology (please notice the use of the qualifier "Most"). Most fail for other reasons, those reasons are either a failure to understand the real nature of the need that is driving the project in the first place, or people issues. I'm going to look more at the people aspect this time around.

These may be machines we work on, but software development is a craft. There's nothing scientific about it at all. It is not created by artists, but rather artisans. I've ceased thinking of myself as an engineer (regardless of job titles), an architect (blah), an analyst (that's a job function, not a job position), and certainly not a manager (that sound was me running screaming out of the room). I am a creator of software.

I say that because the impact of the people that you have on a project is critical to the outcome of the project. Understanding what kinds of people you have is also critical. You may have had 3 really super developers last time, this time you have 5 wet behind the ears college grads. Last time you had some real passive aggressive types, this time you have people who are willing to fight to the last over any point. Without understanding the capabilities and personalities you have, you will operate far less efficiently. When a project begins failing, the personality types of the people on that project will dictate how they contribute to that failure. Usually, the very strengths that make them an asset when things are going well also dictate how they will be a detriment when things are failing. So, what kinds of people do you have on a failing project (again, from what I've seen)?

The Hero

If you just had three more people like the Hero, you'd be successful, right? If it's hyper-critical, if it looks hard, you give it to the Hero. The Hero has probably been around the project the longest, knows exactly what to do, gets it done quickly, etc.

The problem with the Hero in failure mode is precisely the fact that everyone starts leaning on him or her. This turns into a situation where Hero can't keep up to all of the demands. The situation is further exacerbated by the Hero becoming more and more intimate with the inner workings of the application while everyone else becomes less and less intimate because you're leaning so heavily on the Hero to do anything. You need the Hero knowledge sharing, mentoring, and driving, not being isolated to "just get things done".

Needless to say, the Hero becomes a high risk for getting another job at exactly the worst possible time, or turning into the Burn Out, Lone Wolf etc. Anyway you slice it, the Hero is an asset, so really, addressing the negative aspects of the Hero consists of getting the project back on track. Of course, that may not be possible. If that's the case, guard the Hero against slipping into one of the more dangerous types.

The Martyr

Related to the Hero, and perilously close to being a Wannabe Hero is the Martyr. The Martyr wants to be a Hero, but unlike the Wannabe Hero, isn't actually out and out destructive. The problem with the Martyr is they are willing to grind themselves (any anyone else willing to join them) into the ground to get things done. They put in long hours even when not needed. They volunteer for the worst tasks. They are the ones on the plane to Chicago in January and so on.

They are close to being the Hero, but they don't have the skills and/or knowledge of the application to quite make it to Hero status. Meanwhile, the long hours, the constant throwing themselves in front of the bus "for the team's sake" exacts a toll. Inevitably the quality of work suffers because they are tired and if you're lucky, they turn into a Burn Out. If you're not lucky, they turn into the Wannabe Hero.

Often, building trust with the Martyr can help them to get out of the habit of killing themselves. If you can show them that they can be appreciated for doing a more normal level of work, then they'll cut it out. Sometimes, that's not enough though because this really is part of their make up. In that case, I don't have an answer for you other than to try to understand their motivations.

The Wannabe Hero

The Wannabe Hero is pure evil. The things that make someone a Wannabe Hero are not at all strengths. They are selfishness, ambition and unmerited pride personified. The Wannabe Hero is the person who appears to be a Hero, or at least a Martyr to everyone but anyone who has to get intimate with the work they do. The work they do is pure garbage, causes more problems than it solves, and yet they get credit for a "wonderful job" and "working hard".

All to often, the reason they are "working hard" is because they are incompetent or mentally lazy. They are making mistakes and having to put in long hours to make up for it. Hang around the software business and you'll hear a story similar to this over and over again...

"I joined a new company where everyone talked about so and so and how wonderful so and so was. Whenever something broke, so and so knew just how to fix it. So and so put in long hours and was the darling of management."
"After looking at the work so and so did, I realized that all of the problems were caused by so and so. Within two weeks, I was able to straighten out almost all of the issues. Of course, I never got so much as a pat on the back for it, but I also slept at night and got weekends off."

Sometimes, you get a Wannabe Hero out of the person who was once the Hero because they got overwhelmed and pride got in the way. They couldn't keep up, but the promotions, bonuses, etc kept coming until the Peter Principle reigned supreme again. They don't know the way out without being looked on as failures, so they end up in a death spiral.

If you can catch the Wannabe Hero early enough, then you can probably provide a soft landing for them and get them out of the death spiral. Otherwise, you're going to end up having to "repurpose" them or flat out get rid of them. In short, avoid the Peter Principle, and the Wannabe Hero usually isn't an issue.

The Lone Wolf

The Lone Wolf was often a Hero at some point. Now, they are afraid of letting their work depend on anyone else, so they do it all. The main problems with the Lone Wolf is they never mentor, they want nothing to do with any kind of meeting or status reports, and after a while, no one really knows what they are doing. The Lone Wolf is a particular breed of Hero, Wannabe Hero, or Martyr in all reality. The main thing that sets them apart is the fact that they are really good at working by themselves and very bad at working with others. They kill projects because no one knows what's going on, but are relying on them anyway. Or they kill projects because they don't work with anyone else, so they only integrate their stuff at the last minute.

Force them into the light. They will either bail on you or get it together and work with the team. But, don't expect it to be overnight in many cases, so don't try to go too fast with the lightbulb. If they are a Lone Wolf because in real life they are a loner, it's going to be a slow process and you really are just looking for them to be communicative. If they are a Lone Wolf because of trust issues, work on the trust issues. If you find yourself a Lone Wolf, ask yourself why? Really, most people don't suck, and if you think you are that good, let me tell you, most of the really good people I've met are really humble as well.

The Complainer

It sucks. Whatever it is, it sucks. You know that meeting we had yesterday? It sucked. You know the team dinner we're supposed to have tomorrow night? It sucks.

The new tools? They suck. The PCs we have? They suck.

You get the drift. The Complainer can't see the silver lining in anything. They destroy morale so quickly that they turn everyone else into Complainers or Burn Outs. When you're particularly unlucky, the Complainer is also a Warrior, but usually they're more passive-agressive than that.

Now, I grew up in New York, so I'm legally entitled to complain. Complaining itself isn't horrible. When things are going badly, you need the cathartic effect of complaining to just air it out. Or, the person's got some other issues going on and complaining about work is 'safe'. Shoot, sometimes they're just the voice of the rest of the team and act as a safety valve. If that's the case, let them blow the steam! If it gets too much, then my advice is stop and listen to them! Sometimes, the Complainer complains constantly because they don't know how to affect change and they will calm it down if they know others are willing to listen and act. Other times, they can't deal with reality. If that's the case, put them on the hook for solving problems instead of just raising the issue and they might just learn the reason why everything isn't perfect is because there's just not enough hours in the day. Either way, this is actually a person who can be handled much of the time.

A subset of Complainers are actually Blamers. Whereas the Complainer is frustrated in someway, the Blamer is usually either afraid of losing their job or has low self esteem. The Complainer can haul a project into the ground because they destroy everyone else's drive. The Blamer shoots the project down by causing friction and division. The Complainer can sometimes be handled easily, the Blamer usually is a lot harder to deal with. The Blamer is usually a Blamer because of a lack of trust that they won't be the first tossed under the bus when it comes along, so you need to work on that trust with them.

The Warrior

Got someone willing to fight it out for a worthy cause? You've got a potential Warrior. The Warrior is a great asset to have when things are going well because they won't let people derail you. When things are going bad, they are also fighting to "make things better". Hey, they can remember when they stuck to their guns and it worked, so it's got to work again, right?

The Warrior often gets to a point where they get frustrated and they fight with everyone. That's where the problem starts. They can drive a team apart quite quickly. Sometimes you get lucky though, they get everyone else pulling together because they all hate the Warrior. They also burn a ton of energy fighting and frequently sit and stew when they can't fight for whatever reason. This leads to lost productivity in lots of ways.

Really, the only way to deal with the Warrior that I know of is to come at them head on. Confront them with what they're doing and let them know you don't like it. If they listen, great. If not, you're in trouble. Otherwise, try to deal with them like a Complainer, because they probably are just a more agressive Complainer anyway.

The Pacifist

Worse than the Warrior is the Pacifist. When things are going well, the Pacifist calms people down when they are getting a bit excitable. The problem is that it's not because the Pacifist is a sworn peacemaker. The problem is the Pacifist is afraid of conflict and can't stand when there isn't peace. When it comes to Fight or Flight, they flee. Always.

The problem is they can create a team situation where no one can air the dirty laundry. When a problem needs to be dealt with that will cause disagreement, as soon as the volume goes up, the Pacifist tries to table the discussion. The sense of repressed feeling builds until no one is talking to one another. No communication makes failure a very short road. If a project is in trouble, you need to deal with problems and sometimes, or oftentimes, there will be disagreement. If you can't get people talking, you won't solve the problems.

A manager who is a Pacifist and not a peace maker can turn a successful project into a failure just by ending every disagreement by declaring it over before everyone who has a point of view can say it. Problems are swept under the carpet and never dealt with until the backlog of problems overwhelms the team.

Short term, the team needs to figure out how to not punch the Pacifists Fight or Flight button when in a disagreement. Some Pacifists will be able to get over their pacifism if they see the team is able to argue without imploding. Others never will. Work on calm disagreement.

The Defender

You know the Defender. They have their territory and won't let anyone else into it if they can help it. In this way, they can be a bit Lone Wolfish, but the Defender is probably willing to share their territory with someone else who they perceive as having the same project focus. Usually, the obvious place for defenders is as DBAs. Business Analysts and UI designers often can turn into Defenders as well. The short of it is that they probably have some special skill, or special understanding of an area of the application and are afraid to trust anyone with less knowledge than them. Sometimes this also just comes out of pride... They think they did a really good job in building "x" and don't want anyone else messing with their "baby".

Defenders can sometimes be dealt with by requiring anyone who needs to work in the Defender's "territory" to have their work reviewed by the Defender. If they see that it's not the end of the world, they will loosen up. If they find lots of problems, maybe they weren't so wrong!

The Burn Out

Anyone can turn into the Burn Out. You know the Burn Out. They've fought and worked hard, but got nowhere. I once heard a pastor describe burn out this way: "Working hard doesn't cause burn out. Burn out is caused by working hard and not getting anywhere." Now, the Burn Out just coasts if you're lucky. If you're not lucky, the Burn Out can't really do quality work anymore and primarily operate as a bug factory.

Usually, giving them a break from the same old grind is sufficient to snap them out of it. Vacation can help acute Burn Outs, but worn in chronic Burn Outs probably need re-tasking. Even a week of re-tasking can sometimes help dramatically. Or, shake up the project in other ways.

Overall, Burn Out is usually a condition, not a personality type. A change of pace, scenery or a breather can remedy the condition.

All Mouth, No Ears

I've saved this one for last because unlike the rest, this is a real project killer. All Mouth and No Ears is that decision maker who has the plan and doesn't mind telling you. The problem is that they won't listen to anyone else. When contradictory evidence comes up, they raise the reality-denial-bubble and pretend it will go away.

This is a big problem when this manifests in management and stakeholders. The team can scream til they're blue in the face that the deadlines are unreasonable, but they will still find themselves committed to the same death march. In fairness, often this manifests due to the downhill nature of pressure and may not be avoidable in that instance.
A phrase to look out for is "This project is too important to fail." You are dealing with All Mouth, No Ears. In this case, they will just try throwing more people, working longer hours, etc, etc even when alternatives are presented. "This project is too important to fail" often leads to singleminded-ness. I'm here to tell you that every time I've heard that, the project failed.

What to do? Initially, fight the battle. I would not focus on just shouting about them trying to shove 10 pounds into a 5 pound sack. I would seriously recommend trying to find an alternate route or a minimum route. There may be some gold plating someone is trying to inject. Get down to essentials, and it may be achievable.

If you can't win it, then either resign yourself to taking the paychecks or get another job. Ultimately, it's their money, they can do what they want. Your responsibility is to advise. To do otherwise is unprofessional. It's well within your rights to refuse to death march, but the right way to do that is to not sabotage the project. Sometimes, you're just being nervous and things will work out. However, your resume is up to date, right?

Firing in the Dark

Now before you think I'm being a snob, I'm sure I fit into a couple of the categories above when things go bad. So do you. What is important is to realize what's happening and you can redirect the strengths of yourself and others back into a positive direction.

I once posted a comment on InfoQ containing this little rule of thumb...

CMM and its ilk are designed to minimize harm by bad behaviors and less skilled people, whereas an Agile methodology seeks to empower its people. CMM=minimize risk, Agile=maximize productivity.

The short of it is that when you have a team you can't really trust, you need to be heavier handed. I hope that lack of trust is just that there's less experience and folks need more guidance. Otherwise, why are they still around? I don't think this means waterfall type methodologies, but I do think you need to be, umm, agile in how you approach how light or heavyweight a process you use depending on relative skill level of the team.

Above all, don't treat people like machines. Treat them as individuals and you may get lucky. Most people in development actually want to be contributors. They want to reap what they sow. Make sure they can, and you will even be able to be successful when things do fail. At the very least, being able to say you gave it your best shot is worth something.

A last line disclaimer... I'm up way too late typing this. Some of this may not make much sense, so comment or private message me. I'm not afraid to revise. If I've offended you, I have no intention of doing so. I'm sure something I've written will hit close to the mark for some. As I said earlier, your strengths often dictate how you react when things go wrong. As such, if I am hitting close to the mark, I'm probably speaking as much to your strengths as weaknesses. We're all human. Failure is our normal state!

Dimensions of Failure

2010-03-03T21:57:00.001-08:00

I made a slight reference to dimensions of failure in my last post. What I said there was failure to deliver was not the only dimension of failure. I would like to expand on this a bit more because I think it is extremely important from my perspective in the trenches. If this goes right, you'll probably be thinking "Jim, good job, but you missed..."

All of these are failures because they impede your ability to deliver value to the customer/stakeholder both today and in the future. Now that I've said that, I'm not going to dwell too much on that bit of terminology. I'll also say that you'll probably find much of this isn't really new thoughts of my own. There's plenty of people like Fred Brooks who've hit on at least some of these ideas themselves and Roger Pressman has covered most of these in one form or another in his software engineering textbook (it's good as a survey of ideas out there, I wouldn't recommend it as a manual on how to attack a software project).

Some Dimensions of Failure

Well, We Tried (Delivery Failure)

At the end of the day, Failure to Deliver is the most significant failure. Right? After all, you are saying that you undertook the project, but it didn't deliver. How can it be worse than that?

Well, there's a simple fallacy inherent in this thinking. Sometimes, it is perfectly ok to not deliver working software (or whatever) at the end of the project. You may embark on a project knowing that the problem you are trying to solve is extremely difficult, that others have tried it, and it hasn't succeed. Your customer may be perfectly happy to throw money at the problem on the off chance that this time a good solution presents itself. In this case, reducing the uncertainty that they could be taking advantage of something that they are missing out on has its own particular value.

Finally, Here You Go (Schedule/Resource Failure)

This is probably the most easily understood failure dimension and is certainly the most common. The project got done, was acceptable, everyone loves using it etc. However it was late even past the ideal date, the "we gotta have by" date, and three further "drop dead deadlines". It cost twice as much, and it involved, by the end, three times as many people.

The obvious cost overruns are bad enough. Far worse is the long term psychological impact on the organization as it doesn't ever want to go through that again or loses it's taste for taking on risky projects. The long term cost of not taking on good, but risky ideas is hard to measure and has led more than one organization into fatal stagnation.

We Gotta Keep The Contractors For A While Longer(Completeness/Correctness Failure)

Completeness/Correctness Failure (do not confuse this with Iterative Development!) takes a couple of forms. The easy one is that the project was 'ready' at the end of the projected time, but there were key features missing. Inevitably, there's at least some features that won't make it in that you were hoping to get in to a release. That's ok. I'm talking about key features here. Think building the next Word replacement and not being able to print. That's a key failure, because very few users are going to want to use a word processor and not be able to print. They might be ok with manual duplex features missing, but if they can't kill a tree, they're not going to touch it. This may be related to the other form of this failure mode...

More commonly, this takes the form of a project that was delivered with the majority of features, but bugs abound. There's no such thing as a significant project that releases without some bugs. However, you want to keep them down to the sort that are cosmetic, are not stop work, don't cause data loss, and don't corrupt the system they are on. It all comes down to the impact on the user. If every 40th page, your printing suddenly inverts black and white, you're gonna have annoyed users to say the least. There's a bit of wisdom about waiting for the first service pack out there, but you need to also build some credibility. If you were hoping to direct the team in to Version 2 right after releasing Version 1, but have to have 90% working on bug fixes and not the V2 features, something is drastically wrong. What's worse is if you were planning on working on that Snow Leopard port and now have to miss a whole market segment because you have to put people on bug fixing.

We Built It, They Didn't Come (Market Failure)

People will say anything at a trade show if it's going to get them a free mug, even if all they ever drink are diet cokes. Market research done in a haphazard way has caused more than one organization to blow large sums of cash on a product no one wanted as badly as they said. Other times, people are dead convinced they need an app to help trim their toenails because they hate doing it, but the reality is no matter what you build, it's either still going to be inconvenient or the reality is they are so used to doing it that they can't fathom paying money to improve the situation. This last one happens a LOT. The other way this often goes wrong is that the understanding of the problem was totally wrong. Either you don't solve enough of the problem to have a marketable product or you have something that is just missing that magic 5% that makes people want to use it.

Market Failure can be incredibly frustrating. The development team does exactly what it was supposed to. Perhaps everyone involved, not just those inside the code, does a super job. But because of some flawed understand of the need, that perfectly working, bug free code turns out to be worth zilch.

What, You Don't Need It Anymore?(Opportunity Failure)

In the 1800s, a new type of sailing vessel called a Clipper was developed. These were the fastest wooden sailing ships devised and for a period were the kings of the global trade routes. However, very shortly after they came to prominence, this new fangled thing called a steam engine started becoming popular. The opening of the Suez Canal only sped up the process of the clipper becoming obsolete as steamships could navigate it with ease, whereas a sailing vessel had a hard time in narrow waters with uncertain winds.

Many projects suffer a similar fate. They are built. They are built on time, on budget, meet the initial understanding of what was to be delivered, and by the early definition, they should have been successful. However, in the meantime, the window of opportunity has closed. Opportunity Failure occurs because of sea changes in what people want, competing products come out, new regulations come into law, etc. But overall, the theme is that there was a right time, but that right time has come and gone. This is often exacerbated by or entirely because of Schedule/Resource Failure.

In short, those Vancouver 2010 t shirts may be really great, but if you didn't have them ready until the day after the games ended, they really aren't going to make you a bunch of money.

Now Can I Get Some Sleep?(Team Sustainability Failure)

This the one I hate most. This usually comes along with one of the other dimensions of failure. In order to get done on time, weekends cease being time spent recharging with family and people end up working 60, 70, 80 hours a week (yeah, do the math on 80 hours a week). They manage to deliver something, but usually it's still a Schedule/Resource Failure, often a Completeness / Correctness Failure, and frequently features other dimensions to boot.

I call this Team Sustainability Failure because this leads to burnt out people.

"That guy who was a real hero and had all the brilliant early ideas? Well he hasn't had a good idea in months. The woman who was solving every problem encountered almost instantly? Well, she's been stuck on the same bug for three days now. And what about that one developer actually born in this country ? What happened to him. He was really the best we had, wasn't he?

"Oh yeah, citizenship plus skills meant he got an offer from someone else 3 months ago and flew out the door. He hated the hours, and was just too blasted marketable. At least he didn't steal our source code, although I'm not sure why anyone would want to."

You probably see the problem here. When the project gets on a death march like this, the team's ability to mentally regenerate begins to drop precipitously. With drops in mental regeneration comes more and more mistakes. More and more mistakes lead to falling further and further behind, and you get the rest.

Parting Shots

It's rare to see only one of these dimensions of failure manifest by itself. If it does, it's probably Opportunity Failure or Market Failure. Usually, one dimension of failure is accompanied by others. Not all failures are easily visible enough to see what's coming early. However, most failures have plenty of warning signs along the way and plugs can be pulled before too much is wasted or rocking ship can be steadied before it capsizes. Catastrophe rarely occurs because only one thing went wrong.

Technology is rarely the reason projects fail. Next time... some reasons for failure from my own personal experience (and I won't just be ripping on how having too many Business Analysts and Architects spoils the broth). What I can share now is this simple fact... most dimensions of failure can be avoided by tight feedback loops in order to make sure you are constantly driving towards value in your product. Iterative development and continuous integration are just two ways to tighten up the feedback loop. Understanding the team you have and making sure you are doing what's right to keep that team productive is the other important factor.

Kai Gilb... Agile is dead?

2010-03-02T20:01:00.001-08:00

So there's this fellow out there named Kai Gilb who is writing a series of articles about how bad Agile is. To increase the visibility of his arguments, and because his site requires log in, I'm commenting back here. I also don't want to lose the old thought-a-roos...

So, many of his points so far in the series are spot on. I will say that. In fact, given the so called "State of Agile", I'd give Kai probably an 85-90 out of 100. There are, however, a few things that I think merit attention in his arguments. This probably isn't my best writing, call it draft one.

Rebuttals

Hey, Quit Picking On the Manifesto

I'm a little concerned whether Kai understands the historical context of the Agile Manifesto. The Agile Manifesto was written in a time period when many organizations were drowning in process molasses. Kai really rips into the Manifesto, particularly around the idea that the Manifesto does not focus on delivering Business Value. This is a great example of how someone can be oh so right and still wrong.

I would first make the very simple observation that the Manifesto was written in 2001, that's nine years ago. It came out of a discussion culminating in a small conference between like minded individuals who all were fed up with the stifling environment being created by bloated, bureaucratic processes. The mindset at the time was that 1/3 or more of the effort on the project should be on requirements gathering and design. Many, many organizations interpreted this to mean that you should do all analysis work upfront, before development began. This was largely driven by a group of studies which indicated that the reason most projects fail is due to improper understanding of the requirements. The consensus answer was that we had to do more thorough analysis before we tried to build it. Guess what? Projects still failed (and failure to deliver is not the only dimension of failure). Why did they fail? I've been party to several project failures (either in the trenches, or coming after the fact to pick up the pieces). The reasons have not been for lack of understanding the technology. The technology is usually the one thing you can account for uncertainty in. The reasons have come down to various flavors and degrees of the following (nothing new here)...

Took too long/cost too much (time = money or opportunity cost)
Requirements were poorly understood.
It was built according to spec, but spec didn't meet the need.
The business just didn't want it anymore.

When you read the Manifesto, it really comes down to "If you follow the process, and don't deliver you lose. If you don't follow the process, and deliver, you win." What Kai rightly objects to is the lack of focus on "business value" and instead the focus on "product". However, the Manifesto was written at a time where "business value" was something simply calculated as Return on Investment (ROI). Since Agile has opened up the dialog, "business value" has become "Business Value" and I argue that what the initial signatories of the Manifesto had in mind was "Business Value". Jim Highsmith's commentary on the history of the Manifesto all but confirms as much:

At the core, I believe Agile Methodologists are really about "mushy" stuff about delivering good products to customers by operating in an environment that does more than talk about "people as our most important asset" but actually "acts" as if people were the most important, and lose the word "asset"...

Now, certainly, reading that, you pick up on Kai's argument that the Manifesto is more about the developer than the stakeholder, but you also see that it is about enabling the developer to satisfy the stakeholder (customers). The goal is "delivering good products to customers". The means is "operating in an environment...". Everything I read in that time period (say 2001-2003) about Agile from the signatories was ALL about the customer. No one had anything else in mind. We, as people who were practicing or fighting with their bosses (me) about practicing Agile, were concerned about nonsensical processes that seemed to be more about justifying jobs for particularly incompetent business analysts and 'architects' (washed up senior developers who couldn't hack management) and heading off the dreaded FUD (fear, uncertainty, doubt) than satisfying the people who needed the product.

In short, Kai's being rather unfair to the Manifesto and its authors/original signatories. His Claims in part 2 actually sound suspiciously similar to similar statements being made 9 years ago.

Great Points, But Don't Feed the Trolls

My second concern in his arguments has nothing to do with content, but is about the approach. He is taking a rather hyperbolic approach which will inevitably be taken up by certain brands of trolls who want to be in total control of everything to say "Agile isn't working and so we should go back to the 'good old days'." They won't read any further. They will just take his headlines and run off with them.

That's about as far as I'll go with taking significant exception to his arguments so far (up to post 2).

Let's Agree to Agree

This always happens

In the last three years, the idiot brigade has taken over Agile. Scrum (which is pre-Agile in all actuality) has become "The Agile Methodology" or "The Agile Process". I honestly believe if you jumped back 8-9 years to the early Agilists, you'd scare them out of their pants if you told them Scrum was pretty much it (go read that history again). Their whole point was more to do what was sensible for your organization, your project, and your customer than to follow someone else's cookbook. Sure, idea theft is the sincerest form of flattery, but slavishly following it is not what was in mind. These days, Scrum Master certification is almost mandatory to be considered an Agile practitioner. I've never seen certification in anything actually shed any light on anyone's actual competence. I've seen people without a relevant certification who were absolutely brilliant. I've seen people with technical certification who shouldn't be allowed anywhere near a computer, let alone operating one.

Useless User Stories

User Stories used to be what Use Cases were before Use Cases became too bloated. Now User Stories are what Use Cases became. Generally speaking, over-specification kills. My 'text book' case of over-specification was about 5-6 years ago.

Coming on to a project for a month or so to help out in their final phases of development, I got this requirement: "We need to be able to output all of the Web Pages associated with the client to PDF." I was a bit scared by this. At the time, there really weren't a whole lot of good PDF generators for server environments (particularly Windows) and knew that our usual purchasing procedures ran 2-3 months anyway. Not exactly what the project management wanted to hear as they were, as usual, way behind and way over budget. A few more conversations with the folks on the requirements side got the wheels turning. Back at my 'desk' (a folding table in temporary space), the light bulb finally warmed up and lit up. Tromping back across the maze of cables taped across the floor of our temporary accommodations, I cornered the BA.

"What do they do with the PDF," I asked.

"To print it of course," he said.

"So you don't need a PDF, you need to be able to print the pages in a decent format."

"Ummmmmm yes. But we can't do that."

"That's what style sheets are for. So can we agree that we need to make it printable, not generate PDFs?"

"Yes."

Yes, coming in at the tail end of a project and trying to make sense of all the style sheets is a real pain. But there was no way we'd be able identify, purchase, and implement a PDF component in time. It wasn't fun, but it was mission accomplished. Lesson learned... never put off printing til the end and always question the requirement until your certain you've separated out needs from someone else's solution to the needs.

This is where small user stories, use cases, what ever the heck they are called this go 'round really come into play. If you force them to keep it on an index card (in large print!), then you're more likely to get the need than their idea of a solution to the need. Don't let them start the problem solving without you! Sure, let them play, but really, the developer is the expert in what can and can't be done here.

If You Need Software Created, You Need A Developer

Really, we are back on the same roads that we were before. The same clueless wonders continue to adhere to the patently false impression that software development is a factory process. According to this mindset, developers are a commodity and you can drop someone who didn't even touch a computer until they started learning how to program 4 years ago into the same spot as someone who has been at it for 10 years and was using a computer 10 years before that. Fundamentally, there's a misconception (in my mind) that software development is a group of problems solved by thorough analysis. Once you understand the need, you will be able to get any reasonable competent coder to crank out an app to address the need. NONSENSE! If all you are is analytical, you will figure out what the problem is, but you will often not be able to solve your way out of it. The problems we face require creativity, an understanding of how it has been solved in the past, a certain level of battle scars, and just plain experience. Software development, like it or not, is about the people doing the work. I've seen plenty of cases where the team that was 'too small to get it done' got it done and the 'project that was too important to fail' did because management panicked and threw people at it until the project was paralyzed with meetings, documentation, and above all, Business Analysts. BA's are project killers in my opinion. I did an inventory of projects a short while ago that I've worked on or had intimate knowledge about and the results were startling:

Without fail, the success level of the project was inversely proportional to the level of BA involvement. (i.e. BA's did not help)
The successful projects without fail had the developers talking directly to people who would use the product (or were past that point in their career and were now managing those people).

Conclusion: Despite the all the FUD, developers actually do a pretty good job of listening to the user and know how to turn that into an app.

Drive-by Bullet Points

Agile is now firmly in the death grip of cargo cult mentality. Most 'adopters' are using Scrum and XP in a cookbook fashion without analyzing the philosophy behind it. When confronted with the problem, they blame process and not their own thinking. Agile is all about adapting to your customer's/organization's needs, not slavishly following the process to the detriment of what you are trying to do.
This cargo cult mentality is exactly what dooms everything in organizational methodology. Six Sigma worked for the early adopters who really bought into it and adopted not only the process, but the philosophy. Ditto that for RUP, CMM, XP and anything else you care to think about. It's when the success stories make it to the general population that the methodology starts to break down. The successes weren't aberrations, they just had the right conditions to be successful.
Agile got started with people who were really just saying "hook me up with the customer so I can deliver what they need. Get the cruft out of the way. Great things can be done with great people".
Kai seems to be on the road to making the very same argument. In fact, the Craftsman movement is starting to shape up (although this is the nascent form) and based on Kai's arguments so far, I think he's really leading in this direction. At the least, he's starting out on the same foot. I'm going out on a limb... within 10 years, someone will be writing something like "I have nothing but respect for Kai Gilb, but..."
Kai is right, the state of Agile is bad. It is now as bad as what it replaced. But, real Agile folks won't disagree. Not only are we not afraid to hear it, we've been saying it all along!
This is a case of Evolve or Die. If we tried to stick to 9 year old ideas without modification, either those were true geniuses (who should be working on cures for cancer, or at least quantum algos) or we're true idiots. The landscape has greatly changed in 9 years. Our approaches need to change as well. But the fundamental concept of getting what the customer needs into the customer's hands hasn't. Call it Business Value, call it Working Software, or whatever. It really is the same thing.

Life updates

2010-02-20T20:11:00.001-08:00

Wow, I've been quiet for a while, now three posts in one day! I'm far, far more active on Facebook and have been starting to get into Twitter a very little lately as well.

Looking through my past posts, I see a little 'about me' post that begs for another 'about me' post as my life has changed a bit since then. I left PwC in October 2009 after 9 years (to the day!) with them. My tenure at PwC started working as a web/database developer 'in' the NY office (but really working from home 95%+ of the time) in 2000. After completing my M.S. at Pace, I was transfered to the Tampa, FL National Administrative Center for PwC in 2003. I was promoted to Manager in 2004 ('technical', not 'people' manager in theory). In 2006, I started on a fly-in commute project to Stamford, CT. Let me tell you, flying to NYC from Tampa, then an hour+ in the car to get to Stamford every week gets old fast. I was leading part of a team of 'architects' for the duration of that. Anyway, in 2007, there was a bit of project change and I ended up going on 'temporary assignment' to San Jose, CA. Somewhere along the line, I ended up in more of a management than development role. They moved me out here, and I left in 2009 to join MedeAnalytics, back as an engineer again.

Right now, I'm 'just' an engineer, but I don't know how long that 'just' part will last. I'm certainly the senior most engineer here in Emeryville at this point more because I'm not afraid to let my opinion be known than what I think of as any particular skill advantage. Sure I've been around, but it's better to know a little and use it correctly than know a lot and be on the sidelines. Anyway, I'm not managing people, so it's good. I see no point in being miserable while on the job. I might be tired from long hours, but as I once heard from a preacher "Burn out doesn't come from working hard. Burn out come from working hard AND feeling like you are getting nowhere."

Dishonorable Mentions

2010-02-20T15:44:00.000-08:00

I think I've settled on a nice theme with the "A Few Very Useful Tools" series. It helps me think about what I'm using, what I should be looking for, and so on. However, there's another side of the coin, and I think that I would be remiss to neglect to put down a few of the 'tools' that I've come across that have inspired revulsion in me.

OpenOffice - I know this is the darling of the Anti-Microsoft community, but seriously, it's been a miserable experience anytime I've tried this garbage-ware. I don't expect OSS to be as polished as commercial software, but when an application (or applications in this case) is positioned as not just an alternative, but the alternative to commercial apps, it really should be a good experience. Bad UI and missing features are one thing, but I've run into one bug after another anytime I've used it, no matter the platform or version. OpenOffice is more about the Anti-Microsoft agenda than about providing quality, functional software. How many hours do I have to lose before the cost of an MS product becomes cheap? Maybe I'm just an 'expert' user of Office and pushing the edges of what OpenOffice can do, but there's only so many exception dialogs I can put up with in an app that should be solid. It's not like I'm pro-MS... after all, I do like and use iWork on the Macs I own.
ReSharper - ReSharper positions itself as providing the "missing" functionality in Visual Studio. I will admit that I have mixed up feelings about ReSharper. Almost all of its features are really good, and I particularly like it's unit testing features as well as "collapse all" (ms... hint... you really need this in Solution Explorer). BUT anytime I've seen it used on a large solution (which is to say every project I've found myself in the last couple of years), its performance is LOUSY. I mean pathetic. It's bad enough that VS is struggling under large projects (probably worth another post on that topic), but once you turn on ReSharper, VS becomes unusable. If performance was acceptable, this would be a fantastic tool. When you are waiting 2-3 minutes for the screen to catch up to the keyboard several times a day and generally feeling like you're running in molasses, life is not good.
StyleCop - One of the worst things ever released by Microsoft, even if it is freeware. Seriously, there's little to nothing this 'tool' does that shouldn't just be done by the IDE. Instead, it adds to your build time, and then throws errors because you forgot to put an extra return after a closing '}' and similar silliness. It's particularly fun when you get out of sync with the rules on your local environment vs. the build server. You commit, then it pukes all because you exceeded 120 characters on a line (when we all have widescreen monitors!) What kills me most is that the default rules of StyleCop actually contradict Visual Studio's defaults in some cases. Why does MS produce an IDE with one set of rules and then create a tool to check style that does the exact opposite? It also requires comments all over the place. Comments are nice when they are useful, but they are actually harmful when someone is just putting in comments because the tool told them they had to do so. That leads to comments such as "Gets the name." for a method named "GetName". Err... duh. I've seen this used by people who think somehow they are increasing the quality of the application they are building, but the reality is that it doesn't affect the actual output of your compiler, just the code the developer sees. In the end, it is not a replacement for FXCop or other static analysis tools. The good news is that much of what it wants to whine about can be fixed with a quick CTRL-K, CTRL-D. The whole tool violates the philosophy of "Make it easy to do things right". Again, if it matters, why doesn't the IDE just do most of it automatically? Give me tools that help me, not constrain me!

So there's some things I recommend staying away from that may help you out. ReSharper is one I really would like to put on my "A Few Very Useful Tools" list, but bad execution is its Achilles Heel. The same can be said for OpenOffice (I would really like to see a viable multi-platform Office alternative). Style Cop is flawed from the get go. There's little it offers that FXCop doesn't or shouldn't just be made easy by the IDE. Maybe that means it's an admission of guild on MS's part? Ok, that's harsh, but it's inevitable to get to that thought!

A Few Very Useful Tools - Feb 2010

2010-02-20T15:07:00.001-08:00

Ok, ok, I missed the January Useful Tools. So sue me...

The last couple of months have seen me get a new iMac and create a new desk at home so I don't end up with stuff on top of stuff when working from home with the work laptop on the, umm, desk top. Seriously, why do we call them laptops? Just about any modern laptop will cook your lap if you try to hold it on your lap for any length of time. Notebook is also a misnomer when the thing weight 6+ lb..

In the interim, I've been contemplating what tools I've been finding most useful that I haven't already mentioned. There's a PC bias this go-round as I've settled into a bit of a routine with the macs.

Aptana Studio - General purpose, Eclipse based HTML, JavaScript, etc IDE. Actually, as it is based on Eclipse, it's really an anything editor as you can include any eclipse plug in. I've been using this as my IE "view source" option for the last month or so and really digging it. Built in support for things like jQuery really helps out. Available for just about any platform that also runs Eclipse and also offers a Ruby on Rails ide. Open source. For the most part, think of it as super NotePad ++. That is good, because it brings more features, but is bad because it's not quite as "instant". They complement one another very nicely though.
SQL Server Profiler - This tool is distributed with MS SQL Server. It is a MUST HAVE if you are developing against SQL Server and need to know what commands you're executing against your database server. Many performance issues can be resolved by consolidating your DB calls in to things that make more sense. Not at all relevant to anything that is not SQL Server.
CruiseControl.Net - A free, open source continuous integration (CI) server. Sets up easily, easy to maintain, and did I say FREE? If you have a team that is developing code together, once you start with CI, it will be difficult for you to go back in my opinion. Once you know you can 'get latest' without fear of suddenly being broken due to someone else's bad commit, you can be much more productive. What I like about CI servers in particular is that I can commit, wait for it to complete building, and go home knowing that I didn't miss a dependency when I committed. Anyway, CCNet is solid, dependable, even if it lacks the frills of commercial products.
AnkhSVN - An open source, free plugin for Visual Studio that provides real, honest to goodness source control features for SVN. It doesn't do everything that SVN supports, but it does the things you need to do on a regular basis. Most importantly to me, it brings support for "show history" and "pending changes" for SVN to Visual Studio. After not having source control integration in Visual Studio when starting with SVN, I felt like I was missing a thumb. Now I feel whole again. At least when coding.
Numbers - I've been using iWork more and more on the Macs, and while KeyNote and Pages are just so-so for me, I've really taken to Numbers. Numbers is the iWork spreadsheet and what I particularly like about it is that it's really easy to set up 'forms' in Numbers. In Excel, it's really kludgy to try to have different regions of the page that represent different data tables, etc. In Numbers, it's cake. It's what they call 'Intelligent Tables' and given a lot of the data analysis spreadsheets I saw while at PwC, I gotta say this is a missing feature in Excel!

Alright, there's a serious development bent to the above tools, but hey, it's what I do!

A Few Very Useful Tools part 2

2009-12-19T20:46:00.000-08:00

There's a few tools that of late I've realized I am really beginning to consider part and parcel of my life. My last post had a few key tools, here's a few more...

Paint.Net(getpaint.net) - This is firmly in my "are you kidding me?!?!" in a good way category. This is a freeware/donateware (recommended $12) graphic editor (of the jpg/png/bmp/gif sort). For free, THIS ROCKS. For windows, I would not recommend you buy nerf-ware like Photoshop Essentials, etc. Just download this, throw a 20 spot at them to thank them for a job well done and to encourage them, and have fun. While it won't replace Photoshop for professional graphic artists, the reality is that for my needs, it is way more than enough. My needs are retouching photos and doing images for web sites, buttons and the like. Seriously kids, check this out. I've not really seen free graphics editors that weren't quirky or buggy til I saw this. Use it. Love it.
Notepad++ - Alright, after close to 10 years, I've gotten over Text Pad. I'm now using NotePad++. This is open source, using the scintilla text editor component. All the to be expected joy of syntax highlighting, etc. What I haven't found yet is a wat to use the right mouse button to directly open a file in the app from Explorer for Win7 64 bit.
Scrivener - For Mac. Yes, I use a Mac at home. This tool is an Author's notebook essentially. It lets you organize your writing in a fairly intuitive way. I went looking for a Mac alternative to OneNote, and turned this up. It's reasonably cheap (~40US), and just plain works. No, it's not word processing/semi-desktop publishing like Word or iWork/Pages. It's what you do your brain work in to import into a desktop publishing or word processing application for final formatting.
Pandora - Ok, you might not think this is a utility, but really, it is for me. If I had to not listen to music and work, I'd go nuts. Pandora's a 'build your own radio station' web site. Basically, you create a station by telling it what kinds of songs/artists you want to hear and it figures out more to include from there. Tell it Dan Fogelberg and Dan Hill, Kenny Loggins and it will find Barry Manilow without you telling it! You can then tweak it so you fire Air Supply from your station and never hear their junk again. When you're in the mood for non-sappy, non-proto-emo, real music, you can create another station with Guns-N-Roses, Dropkick Murphys, Metallica, etc. Anyway, I don't know if there's a limit to how many stations you can have, meaning it's more than the 8 or so I have now. Primarily plays through a Flash Player embedded in the web site or a stand alone Adobe Air player.

The next post will probably include more Mac tools as I've been using that at home more than the Win7 machine. I've been using it enough that I get to work and have to remap my brain to use the correct key on the key board for cut-n-paste.

A Few Very Useful Tools

2009-09-17T22:58:00.001-07:00

del.icio.us Tags: Windows Development,Utilities

In my toolbox of nifty utilities that I use for my job, there’s a few that I think I couldn’t live without.

Sysinternals Suite- Now owned by Microsoft, this toolset developed by Mark Russinovich is hugely useful for anyone building applications for Windows or supporting Windows PCs. Particularly, ProcessExplorer and DebugView are go-to tools for me. ProcessExplorer is what TaskManager should be while DebugView will let you see trace statements emitted by applications.
TextPad – Ok, there’s a million text editors out there, and this is just one. But seriously, you’re not just getting by with Notepad are you? TextPad allows syntax highlighting and also support regular expression based find/replace which has made a lot of tedious, rote, work really simple on occasion.
NDepend – A great tool (alas, not free/shareware) that will help you in many ways to hunt down how your code is being used. CQL (Code Query Language) rocks the house! Want to know all the methods used directly or indirectly by Class A that are directly or indirectly using Class B so you know how big a job is going to be to replace some ugly bits (we all have them, you know that!), this will do it for you! A must on large projects in my book.

Many other nifty tools can be found on Scott Hanselman’s Blog here…

2009-03-17T21:12:00.000-07:00

Just a silly colorful face I made messing around with Photoshop CS4. Nothing fancy here.

Being rewarded for buying locally

2009-03-06T21:38:00.000-08:00

Talk about a relief.

It's been one of those weeks. I get back from vacation and try to get back to work, but find I've got some issues. I can't get data in our app synced with our server (we're using a new sync approach which is complete with NDAs so I can't tell you anything about it, at least yet), and then I start having issues with our VSTO customizations for Excel documents. I keep getting an error in GDI+ with nary a hint of stack trace of our code.

After much wrangling and reinstalling, I manage to get my PC back in order and working again. Something apparently got me fouled up with some other installing/reinstalling I'd been doing lately. HOWEVER I then find out that one of our business design team members is also having issues with Excel that are similar to mine. After much wrangling yesterday, I go home muttering something about "Format C:" as her issue is clearly one of something being fundamentally fouled up on her machine. (Today, we found out it was some third party stuff causing an issue in Excel, nothing to do with our app other than our app being victimized by bad behavior of this other app.)

Then I get home. And then I fire up my home PC. And then I get "there has been an error on Volume 0". In short... hard drive problem that's looking pretty serious. Windows hangs up. Then I restart. Windows seems to start, but hangs up again. Oh boy. The drive seems fouled up and my confidence that running CHKDSK would solve all the issues was pretty low. Unfortunately, I had decided to RAID 0 the disks so that my two physical hard disks appeared as one big one to windows. I still don't know why I did that instead of RAID 1/mirroring. 640GB should be big enough for a while for me. Anyway, I'm thinking I'm up the creek unless I buy TWO more 640 GB disks and then create two RAID 0's and mirror the two, then take the bad disk out and use the good original disk as a spare.

I decide that's a mess. Let me see first if I can get away with using a single 1.5 TB drive as mirror of the the original pair of connected drives so I can get a back up and then go from there. Now, I have no expectation of this working, mind you. I'm figuring my drive controller will expect each end of the mirror to have the same physical characteristics. Anyway, I pop over to Fry's to get a new drive. I know that I'll spend more money at Fry's than at Newegg, but my philosophy these days is to buy local and I'm also nearly in panic b/c I'm thinking I'll have to redo a lot of junk on this machine or may lose stuff. I try to back up important stuff to my eternal NAS drive, but there's always a chance of missing something.

I decide that what I'll do is buy TWO 1.5 TB drives (at 160 a pop) and then attempt to use the first one to mirror the existing drives (which are about 1.2 TB combined), take the old drives off, and mirror the second new drive with the first new drive.

Keep in mind, I have no expectation that my disk controller will even let me get away with this.

I get home with two new Seagate Barracuda 1.5 TB drives. RETAIL packaged Seagate Barracuda 1.5 TB drives (the old ones were Western Digital, just based on one arbitrary instance of bad luck, I figure I'll try Seagate this go round). And then the good Lord for some reason actually stops me and gets me to RTM (read the manual). It's then that I realize the CD that came with the drive isn't just a coaster. No, not at all. The CD has a program built to order for what I need... DiscWizard, complete with the ability to clone an existing drive. Furthermore, this program does this based on the logical volume (what Windows sees as the drive), not the physical disks. Therefore, these retail boxes I got at a higher price than I would have gotten "OEM" drives from Newegg just saved my bacon. I don't need to worry about whether the disk controller can support two different physical set ups for each copy of a mirror between drives. I can copy the existing data to the new drive, take out the old ones, and boot up from the new one.

In just about half an hour, I have the first new disk in, the utility has run, and the old disks are out. Back in action. I'll worry about setting up the mirror tomorrow... meanwhile, one big sigh of relief.

And looking at Newegg, I spent less than $60 extra over the same drives in OEM packaging. The software was worth that much to me!

Doing management, again.

2009-03-04T20:41:00.000-08:00

Well, after a valiant/vain attempt to avoid getting hauled into it, I'm in the team leadership business again. This time, I'm officially a Scrum Master of one of the 9 or 10 development teams for my current project (number varies with whether you consider one of the teams part of the main project). The result is that I'm about 50% hands on developing still and 50% sitting in on meetings. Fortunately, this project is reasonably disciplined about meetings and we don't run on and on and on and on. This does mean that I'll be in California at least a little while longer. It's not that bad, and there's no hurricanes, but I also haven't sold the house in Florida yet.

The good news is that we're using Scrum instead of a Waterfall methodology. For those of you who are mercifully insulated from software development processes, this basically means that we're working in monthly iterations instead of trying to figure out what ALL of our requirements are, then figure out the system design for the WHOLE system, then coding said WHOLE system, then going to test it and figuring out what we have isn't what we need or some other variation of the death march disaster theme. For a hilarious metaphor for scrum, check out Peter Stevens explanation to his parents. I'd argue that Mr. Stevens is really just describing the Agile ideal of iterative development in general rather than Scrum in particular, but we can be a little loosey goosey on occasion and let it slide just this once ;) .

Anyway, here I am managing people again and this time it's not just contractors. Don't they realize that my roommate is a one eyed, one horned, flying purple people eater? It's a good thing there's no purple people on the team. Although if anyone gets out of hand, I suppose I can call up a Witch Doctor to turn them purple!

How I plan to save the economy.

2009-02-23T17:20:00.000-08:00

After listening to our president present his "plan" (borrow, spend, tax, spend blah blah), I got to thinking and now I can confidently say:

I have a plan. A REAL plan.

Follow me on this one, and share it with your friends because it will only work if everyone gets on board.

First, you need to know a couple off the "Jim's Rules Regarding Everything". Maybe one day I'll codify the full and complete list... but it will have to stop growing and changing first. The complete set of rules really represents some hard and fast rules mixed in with some theories waiting for more data. Anyway, the ones you need to get familiar with are:

A healthy economy is not one where there's a lot of wealth but rather a lot of money moving around. A high level of personal wealth hoarding is very nearly opposite to a healthy economy.
The economy is driven primarily by emotion and faith. As long as the good vibes are flowing and people think it's going well, it will be well.
The economy is like the ocean. The water doesn't just sit still, but is bunched up in waves with highs and lows. A high can't last for ever and a low can't last forever.
The government can't help the economy, but it sure can hurt it (see the first point in this list). The mere presence of the government mucking about with the economy can induce panic.
Government's primary responsibility is to keep the playing field level for the relatively honest/long term players and to try to keep the cheaters out and to punish the cheaters when they can't keep em out. While not relevant to the topic, this one and the prior one lead to the rule that a moderate level of regulation is required to keep the economy functioning correctly.
The only economy that matters to you is your personal economy. You can be having a bad year when everyone else is having a good one and vice versa.

Now my plan is this:

No more "me first." Think about others before thinking about yourself.
No more hunkering down and tightening belts. We can hide in our trench while all the ones around us get overwhelmed, but where does that leave us?
No more overextending ourselves so that we get in trouble.
Therefore, continue to save, but also to spend rationally.
Spend locally, even if it means spending a wee bit more. Try to spend locally when it is practical to do so... i.e. buy your Shadowrun books at Game Kastle instead of Amazon.
Reward people for doing a good job. This can take many forms, but I'm primarily thinking that we need to ditch the discount mentality as a nation and demand quality.
When things get good again (and regardless of what the government does, this will be substantially turned around w/in 3 or so years), make special effort to save again.

This is based on nothing more than "Pay me so I can pay him so he can pay her so she can pay them so they can pay you." If I only watch my own back then I'm likely to find myself standing alone. If I'm looking out for others, hopefully they're looking out for me. Don't rely on the cavalry to save you... be the solution! Etc, etc, etc.

If enough of us start getting our personal economies flowing again, the rest of the economy will follow. Will it all look the same? I hope not. We need more sustainable practices in every way imaginable and probably need more mid-sized companies instead of fewer megacorps as part of a general theme of more decentralization. But at the end of the day, all I care about economy-wise is that all of us know where our next meal is coming from, have confidence that we can access healthcare etc, and have a roof over our heads. The treasures we store here are lost when we die, so the only treasure that matters is the treasure we can lay up for the next life. That's spirtual treasure and is measured by the good we do and so get out, take a risk, and spend!

That ends my latest screwball idea.

jimleo.com is back up... some how-to.

2009-02-20T22:22:00.001-08:00

Well kiddies, it's true. After WAY TOO LONG(in dog years no less!), I've finally updated my "main" website, jimleo.com.
Now that I'm using Blogger and delicious for blogging and links respectively, jimleo.com has a new mission in life. That mission is to be the "Jim portal." ATOM and RSS feeds from the various SN sites I'm involved on will be aggregated there so you can see what I'm blogging and what I've been recently keeping as reading and reference in one shot.

Dealing with Feeds

The under the hood bits involve a bit of ASP.NET to create an RSS/ATOM reader control and the ServiceModel.Syndication namespace to deal with the entry bits. This is certainly a lot easier in Visual Studio 2008/.NET 3.5. No more XPath, DOM, or other parsing magic to work with. Now you can do everything with some simple objects. The main magic is in the following:


XmlReader reader = XmlReader.Create(feedUriString);
SyndicationFeed feed = SyndicationFeed.Load(reader);

Of course, there should be some try{}catch{} magic surrounding that!

From there, it's a matter of just looping through feed.Items to render out each item.


foreach (SyndicationItem item in feed.Items)
{
    //your code for  rendering each item
}

The rest of it's just the various bits on what types of elements or user controls you want the items laid out into. I'm sure there's much more that you can think that can be done, but I kept it pretty simple. A heading (h4 to be precise) is created for each title with a link to the item. I then display the summary and/or the content for each item. If either is more than 275 characters, I cut it off at 250 and show "...more..." with a link to the actual item in the "...more..."

Now you may be asking yourself "Why is it 275 and 250? Why not 250 and 250 or 275 and 275?" The reason is simple... I didn't want to cut off the text and show a more link if there was only a few more words to show. The other bit of magic was that I strip out any HTML or other markup in the text. There's two reasons: 1) I don't want it screwing up my own layout and 2) Cross Site Scripting (XSS) attacks. While currently all feeds come from me, it's conceivable that I might want to use someone else's feeds. They could do something unexpected or malicious and I wouldn't want you to suffer dear reader. If you want the code for the user control, or the whole site (all one page), please let me know!

On to the WebPage (default.aspx)

Visually, the KISS princple rules the web page. Basically there's a completely non-inventive header, a main section with three "lists"... one what I call the "intro" and then beneath that two columns. The left has the links to blogger and the right has the delicious links. A footer with ye olde copyright lies underneath.

Now, from there, the simplicity takes a minor nose dive. Beyond the layout, there's a couple of requirements I had for myself:

The header should always appear on the top of the screen.
The footer should always appear on the bottom of the screen. Meaning: It should always be visible at the bottom regardless of length of any other content on screen.
The main section should scroll if the content can't fit between the header and footer.
Tables can't be used.

Why no Tables? They are not XHTML compliant when the usage is only for laying out a page. There's no legit reason to use a table on this page as I'm not showing tabular data anywhere.

Now #1 and #3 are relatively easily accomplished with HTML div elements with the right settings... making both the header and main section divs and setting main's style to include overflow: auto and position:fixed takes care of both of those.

The trick is #2. This would have been easy with tables, but with divs, it's not as simple as a setting and if anyone can figure out a pure CSS / style way to do it, please do tell! I ended up creating a bit of javascript to do this... it comes down to this following code called on window.resize (defined in the javascript file) and the onload for the body.


var footerHeight = document.getElementById('footer').offsetHeight;
var headerHeight = document.getElementById('header').offsetHeight;
document.getElementById('content').style.height = (getWindowHeight()-(headerHeight + 10) - (footerHeight + 10)) +'px';

Make sure to set the following style on the footer along with any other visual bits you want:


bottom: 0;
display: block;
height: auto;
position: fixed;

The rest of the CSS styling is pretty straightforward.

To download the javascript... click here.

To download the CSS... click here.

All in all, most of this CSS/JS magic was already done for smellyoldgamer.com, so I wasn't reinventing anything tonight. I think jimleo.com looks a little nicer though. SOG wasn't supposed to look nice. No one would believe I was a smelly old gamer if it did!

Testing/Known Issues

I've tested in both IE7 and Firefox 3.0.6, so it should render out on most browsers although I know it gets goofy if you make your window really narrow. This really shouldn't be much of a problem as you pretty much need to get it down to about 4 characters wide before the goofiness asserts itself. A larger problem is that once you get shorter than header + footer, the main section disappears. I'm still debating on what to do about that...maybe resize the header and footer?

Conclusion

This was much simplified in VS 2008/.NET 3.5 thanks to the Syndication namespace. I'd dealt with both RSS and ATOM at an XML level in the past and I'd always wonder if there's some scenario I'd missed. With the navigation of the feeds made simple, I was able to focus on how the site would be laid out and fiddling with the CSS. Given most of that had been done already on SOG, I was able to put it all together quickly. Reuse beats doing it by hand any day. Reuse also promotes fixing known issues as you know you can address more than one spot with one fix. My next enhancement will probably be to make the javascript a little more flexible or make it use the script.aculo.us library. If you have any ideas, please do comment!

The KIPOS principle.

2009-02-18T23:52:00.001-08:00

Working in software development for too long now, I've had the "joy" of working under a few dominant project management philosophies. These include (if you're interested in this, I probably don't need the definitions, but here it goes anyway):

"ad-hoc" - The "get 'er" done principle. Do what seems to make sense at the moment and don't really think about it. Maybe after a version release you do a review and establish best practices, maybe not. Can be surprisingly successful with small teams even though it may seem like no one knows what's happening next week, or it's based on a lot of hope (as in "I hope I don't have to do a kachillion hours of overtime next month to meet the deadline").
Waterfall (aka "the Software Development Life Cycle")- This is a linear model where you Gather Requirements, Design, Develop, Test, Release... over long cycles, often a year or more. This is a gated model, so you don't begin development before you do design, and don't do design before you start gathering requirements. Lots of documentation, most of which is out of date before the code even starts getting written. Generally favored by those who believe in top down management and who think of people as "resources".
Agile(including Scrum, XP)- Agile favors much shorter release cycles and thus lots of early on testing. Agile is based on the ideas that you minimize those things that add limited value (lots of documentation, rigid plans) and focus on those things that get you to working software(focusing on the software vs. huge specifications, communication over contracts, etc.) Those practicing Agile generally view themselves as members of a "post-waterfall" society. They may be charitable and say that waterfall was a needed step along the way. They may be very much uncharitable and say that waterfall was... well we'll leave that alone.

The short answer is that no matter what methodology you use, your chances of success are slim if you think the methodology is the main source of your hope. Hope can never be found in a THING. Hope can only be found in someone you put your hope in (best placed in Jesus if you ask me, but I'm here to talk about getting software done at the moment, more on that in another post). Projects don't succeed because of process. Dan North wrote an article a while back on Infoq that will drive most of my discussion here. It looks at the Dreyfus model of learning and applies it to software development.

The main takeaway for this in relation to what I'll call the KIPOS principle is the following... Waterfall methodologies rely on preventing the damage that the inexperienced, incompetent, etc can do to your project. This comes at the expense of limiting the good that the experts and event intermediate/journeyman can do for you.

Agile, on the other hand, relies on the team members to manage the team members... helping the inexperience and incompetent get better. In particular, XP emphasizes pair programming for just this reason. However, if the team loses sight of someone or some team members collaborate to cheat or game the system, then it can fall apart.

Having worked under all of these paradigms, I can tell you I positively hate waterfall if you can't dial back the process to a rational level. I've been on some projects that were in the hundreds of man hours that required the same level of documentation, artifacts, etc as projects of 10s of thousands, even hundreds of thousands of hours. When you're producing long documents to communicate with yourself, there's something wrong here.

On the flip side, I've seen Agile where the team members just don't seem to get it. They are accustomed to working with a rigid spec and when asked to work outside those boundaries, they may have a problem. They may be smart, competent on the coding side, follow the rules, but they just seem to struggle anyway. In following AGILE the methodology family, some teams seem to lose sight of what agile means, at least according to the dictionary. And so, without further ado...

The KIPOS Principle

The KIPOS principle is simply this "Keep It People Oriented, Stupid". Look at your team: What do you have? If you have a bunch of people who have only ever been handed a spec to work off of, then you need to build a spec. If you have people who have only ever worked in a small shop with little idea of formal methodology, or who like a cleaner canvas to work on, then sure, go for the less documentation approach. If you have a mix, then pair them up! But above all, you can still hang on to some of those Agile values and principles. Keep in mind that the Agile Manifesto never said "we never do documentation" etc, it said "we value working software over comprehensive documentation".

I'll probably expand as time goes on into specific cases, but please, above all: Look at your team. Do what works to get the software out the door. Two things I can tell you that will always be a part of do what works:

Get the software into a testable state early and frequently send out releases to testing, however you decide to test. If this is more than a month between test cycles, something is wrong.
Continuously Improve (I'll probably have a post on this later, but meanwhile) This means always look for ways to improve the behaviors, processes, tools, training, etc in your team. Never stop improving and don't sweat not being perfect... you're always improving, so you can adjust later!

Third time's a charm. Right?

2009-02-18T21:49:00.000-08:00

Hi all.
I tried Subtext.
I tried Word Press.
All hosted on http://www.jimleo.com.
The reality is that I never had enough traffic to keep my interest, so now a Blogger blog. I'm also hoping that Blogger does a much better job of filtering the comment spam so I don't have to.

If you know me, you can find me on Facebook. If you don't know me, well, that's what here is for. I have a Myspace profile, but who's using that anymore? I really don't use it, so don't bother.

Anyway... the usual spiel.
I do software for a living.
By software, I mean .Net development. Why .Net? Well, I don't like to dork around with endless config file after config file, but like the idea of using a general purpose programming language. Java's too dang cumbersome from the amount of things you need to do to make anything happen. Open source is something I like to use, but we have some constraints that make it hard to use in my firm. So the best world to get my day to day stuff done in is the .Net world.

The real living is my faith in a loving saviour, none other than Jesus Christ.

For other fun, I still game after all these years.

I hope this to be an eclectic mix of junk. And good stuff too.