Productivity Over Process

It seems like every software company you talk to will boast about how they use the latest development process du jour—Agile, Lean, XP, Kanban—pick your poison. What’s interesting is that the people evangelizing their chosen methodology are typically managers, not developers, almost emphasizing the process more than the product. Startups and other young tech companies seem to be particularly guilty of this (after all, every time someone utters the words “lean startup”, an angel investor gets his wings).

I’ve worked on a number of different teams, mostly Agile, ranging in size from 4 to 20 developers. I’ve used everything from TFS and JIRA to Pivotal Tracker and Trello to manage stories and track bugs. A process, the way in which you produce something, is often seen as necessary by project managers and a necessary evil by developers.

Are software developers just cowboys who want to build something, guns blazing? To an extent, probably, but the conclusion that I’ve drawn is that there is no silver bullet when it comes to crafting software. It’s not one size fits all. Agile is not the be-all, end-all solution, nor is anything else.

I’ve seen teams that said they were Agile when, in fact, they weren’t in any sense of the word. To some, it’s just a buzzword used to attract talent. I’ve also worked for companies where Agile was used across all teams, no exceptions. This led to problems with the way some groups operated. It worked great for feature teams who were engaged in completing user stories, but for some of the component teams, it just didn’t make sense for the type of work they were doing. My team, very much a backend architecture group of about six developers, was merely going through the Agile motions—a bad sign indeed. We switched to a less structured Kanban process as a result because it worked for us.

Too much emphasis is placed on the process and not the productivity of a team. Am I saying that Agile is bad? Absolutely not. In most cases, an Agile team is a productive team. When an Agile team hits its stride and really grooves, the results are impeccable, but a process needs to be peripheral. It should get out of your way as fast as possible so you can get work done. Don’t just go through the motions.

How is Software Valued?

I was talking to a friend a few weeks ago who was putting together a business presentation for potential investors. He was developing a plan for a campground kiosk system that would rely on GIS data to allow guests to view and check in to camp sites. The plan was reasonable enough and mostly feasible. He carefully considered all the costs—licensing for a third-party GIS, kiosk hardware, line trenching—and then there was software.

He allocated a mere $8,000 for the kiosk software, a low-ball figure by any definition of the word, and he estimated it to take about four weeks to complete from scratch.

“Where did you get that figure?” I asked him. The answer basically boiled down to “thin air.”

I didn’t have any kind of sudden realization, but this exchange did reinforce something many others have already observed: software is remarkably undervalued.

All too often clients say something along the lines of “You want me to pay you $X per hour to sit and type on your computer?!” What’s not obvious to many is that software engineers create extraordinary value for businesses. It’s almost ironic considering just about everything these days is driven by software, to the extent that it’s almost taken for granted, and it doesn’t somehow materialize out of thin air.

So why is this the case? Is it because software isn’t a physical good? Maybe. However, I think the issue is largely attributed to the disparate levels of productivity between software engineers and other areas of industry. A developer might write an accounting system that leaves a large number of accountants redundant or automate a process that otherwise takes a dozen employees to complete. Is it fair that they are compensated accordingly? Again, it’s about creating value, but the fact is, most developers aren’t paid in proportion to the value they create or their productivity. Consultant John Cook explains why this is the case:

A salesman who sells 10x as much as his peers will be noticed, and compensated accordingly. Sales are easy to measure, and some salesmen make orders of magnitude more money than others. If a bricklayer were 10x more productive than his peers this would be obvious too, but it doesn’t happen: the best bricklayers cannot lay 10x as much brick as average bricklayers. Software output cannot be measured as easily as dollars or bricks. The best programmers do not write 10x as many lines of code and they certainly do not work 10x longer hours.

It may also be due, at least in part, to software being endlessly enigmatic to non-software people. Is this auto mechanic ripping us off on our car? Is this developer ripping us off on our point-of-sale system? It’s easy for people to see what it takes to build a bridge—designing it, performing simulated load tests, pouring the concrete, assembling the steel, laying the superstructure—these are all tangible overheads.

What does it take to build software? It’s just some bit-twiddling, right? There’s no inventory that needs to be accounted for; there’s no manufacturing labor. As developers, we know it’s a lot more involved than that. The problem with software is that it’s a living thing. After you build a home, you don’t decide to move the bathroom to the other side of the house. The same cannot be said of software.

Product owners are fickle creatures. They don’t know what they want, except that Feature X needs to be changed to Feature Y and still ship in time. I’ve been on projects where this had become so problematic that developers started leaving Feature X implemented. That way, when NPD ultimately decided X was correct in the first place, we would be on schedule, but that’s tangential to this conversation.

What I’m getting at is that there’s a lot more to building software than what may be perceived. There’s still planning, and designing, and prototyping, and implementing, and testing. But unlike the bridge or the house, the process doesn’t stop when the software ships.

No self-respecting (or sane) software engineer would agree to build a complete system in four weeks’ time for $8,000. It’s almost insulting. But to someone which software is completely foreign to—and it is to most—it might not sound so outlandish. The problem is finding the appropriate level of value. It’s easy if you’re an independent consultant, but if you’re one of several hundred developers at a company, how is your value measured? As Cook explains, output, in terms of lines of code, is not a reliable metric. In fact, one could argue it’s inversely proportional to a developer’s ability.

The romantic image of an über-programmer is someone who fires up Emacs, types like a machine gun, and delivers a flawless final product from scratch. A more accurate image would be someone who stares quietly into space for a few minutes and then says “Hmm. I think I’ve seen something like this before.”

It’s for this reason, combined with the fact that programmer salaries don’t really vary dramatically, that many developers do consulting as a profession. They know exactly what their time is worth and the value they add to a business. Coming back to the problem described earlier, the downside of consulting is that many customers don’t recognize that value. As a consultant, it’s also your job to establish what it is and why.

I took on a contract last month to build some mobile software for a small engineering firm. They needed an Android application but didn’t have the resources in-house to do it. They met with a few software shops in the area but none of them specialized in mobile development. I build Android apps. This raised my value, and I already had a pretty good idea what the app would do for their business. At that point, it’s just letting economics work itself out.