Posted on

How is Software Valued?

I was talking to a friend a few weeks ago who was putting together a business presentation for potential investors. He was developing a plan for a campground kiosk system that would rely on GIS data to allow guests to view and check in to camp sites. The plan was reasonable enough and mostly feasible. He carefully considered all the costs—licensing for a third-party GIS, kiosk hardware, line trenching—and then there was software.

He allocated a mere $8,000 for the kiosk software, a low-ball figure by any definition of the word, and he estimated it to take about four weeks to complete from scratch.

“Where did you get that figure?” I asked him. The answer basically boiled down to “thin air.”

I didn’t have any kind of sudden realization, but this exchange did reinforce something many others have already observed: software is remarkably undervalued.

All too often clients say something along the lines of “You want me to pay you $X per hour to sit and type on your computer?!” What’s not obvious to many is that software engineers create extraordinary value for businesses. It’s almost ironic considering just about everything these days is driven by software, to the extent that it’s almost taken for granted, and it doesn’t somehow materialize out of thin air.

So why is this the case? Is it because software isn’t a physical good? Maybe. However, I think the issue is largely attributed to the disparate levels of productivity between software engineers and other areas of industry. A developer might write an accounting system that leaves a large number of accountants redundant or automate a process that otherwise takes a dozen employees to complete. Is it fair that they are compensated accordingly? Again, it’s about creating value, but the fact is, most developers aren’t paid in proportion to the value they create or their productivity. Consultant John Cook explains why this is the case:

A salesman who sells 10x as much as his peers will be noticed, and compensated accordingly. Sales are easy to measure, and some salesmen make orders of magnitude more money than others. If a bricklayer were 10x more productive than his peers this would be obvious too, but it doesn’t happen: the best bricklayers cannot lay 10x as much brick as average bricklayers. Software output cannot be measured as easily as dollars or bricks. The best programmers do not write 10x as many lines of code and they certainly do not work 10x longer hours.

It may also be due, at least in part, to software being endlessly enigmatic to non-software people. Is this auto mechanic ripping us off on our car? Is this developer ripping us off on our point-of-sale system? It’s easy for people to see what it takes to build a bridge—designing it, performing simulated load tests, pouring the concrete, assembling the steel, laying the superstructure—these are all tangible overheads.

What does it take to build software? It’s just some bit-twiddling, right? There’s no inventory that needs to be accounted for; there’s no manufacturing labor. As developers, we know it’s a lot more involved than that. The problem with software is that it’s a living thing. After you build a home, you don’t decide to move the bathroom to the other side of the house. The same cannot be said of software.

Product owners are fickle creatures. They don’t know what they want, except that Feature X needs to be changed to Feature Y and still ship in time. I’ve been on projects where this had become so problematic that developers started leaving Feature X implemented. That way, when NPD ultimately decided X was correct in the first place, we would be on schedule, but that’s tangential to this conversation.

What I’m getting at is that there’s a lot more to building software than what may be perceived. There’s still planning, and designing, and prototyping, and implementing, and testing. But unlike the bridge or the house, the process doesn’t stop when the software ships.

No self-respecting (or sane) software engineer would agree to build a complete system in four weeks’ time for $8,000. It’s almost insulting. But to someone which software is completely foreign to—and it is to most—it might not sound so outlandish. The problem is finding the appropriate level of value. It’s easy if you’re an independent consultant, but if you’re one of several hundred developers at a company, how is your value measured? As Cook explains, output, in terms of lines of code, is not a reliable metric. In fact, one could argue it’s inversely proportional to a developer’s ability.

The romantic image of an über-programmer is someone who fires up Emacs, types like a machine gun, and delivers a flawless final product from scratch. A more accurate image would be someone who stares quietly into space for a few minutes and then says “Hmm. I think I’ve seen something like this before.”

It’s for this reason, combined with the fact that programmer salaries don’t really vary dramatically, that many developers do consulting as a profession. They know exactly what their time is worth and the value they add to a business. Coming back to the problem described earlier, the downside of consulting is that many customers don’t recognize that value. As a consultant, it’s also your job to establish what it is and why.

I took on a contract last month to build some mobile software for a small engineering firm. They needed an Android application but didn’t have the resources in-house to do it. They met with a few software shops in the area but none of them specialized in mobile development. I build Android apps. This raised my value, and I already had a pretty good idea what the app would do for their business. At that point, it’s just letting economics work itself out.

Posted on

Bluetooth Blues

I spent the better part of two days working on Bluetooth connectivity for an Android app I’m developing. Going into it, I had virtually no experience working with Bluetooth, especially on Android. I quickly discovered some of the peculiarities of the platform’s Bluetooth API.

In addition to connecting to Bluetooth devices, the client wanted to pair and unpair from the app. The easy way out, and probably The Android Way™, would be to pass that responsibility off to the OS, à la an Intent:

This will bring up the Bluetooth settings menu, from which you can pair/unpair devices, but the problem is that it’s a complete context switch for the user—they are no longer in your application. I was looking to provide a more seamless experience so that the user didn’t have to leave the app at all to pair a device.

Device Discovery

The entry point for Bluetooth interaction in Android is through the BluetoothAdapter, which is used to orchestrate the device discovery process and fetch paired devices. Calling startDiscovery() will tell the adapter to start scanning for devices, and when one is found, an Intent will be fired off which can then be intercepted by a BroadcastReceiver.

The above code shows how the device discovery process is kicked off and how a BroadcastReceiver is registered to listen for discovery Intents. Note that the BroadcastReceiver is unregistered and discovery is canceled in onDestroy.

In order to react to discovery events, we must implement a BroadcastReceiver.

Device Pairing

Once you have a handle on the BluetoothDevice received in the BroadcastHandler, how do you actually pair with it? Looking at the documentation for the class, you’ll see that there are no methods for doing this. This is where things start to get a little strange.

Diving into the source code for BluetoothDevice, you’ll actually find that there is functionality for doing pairing and unpairing, but the methods are hidden from the API using the @hide annotation. What’s more interesting is that the methods are, in fact, public.

Evidently, device pairing is intended to be performed only by platform applications, which is a little curious considering the permission needed to perform pairing, android.permission.BLUETOOTH_ADMIN, is accessible by third-party applications. Nonetheless, this means we actually can pair a BluetoothDevice, just not in the way the Android engineers intended.

To access the BluetoothDevice methods needed, createBond and removeBond, we can use reflection.

The pairDevice method will prompt the user to enter a PIN for the discovered device, circumventing the need to open the Bluetooth settings. As such, the pairing does not actually complete until the correct PIN is entered. The boolean value returned from the method indicates whether the pairing process was successfully kicked off or not.

It goes without saying that this code, while functional, is volatile because these methods are technically not part of the public API, so they could change or disappear in future platform releases.

We can add an Intent filter to our BroadcastReceiver to listen for pairing events using the action BluetoothDevice.ACTION_BOND_STATE_CHANGED.

There are a few other hidden methods in BluetoothDevice, like cancelPairingUserInput, setPairingConfirmation, convertPinToBytes, and setPin, that you could potentially use to customize the pairing process or perform it programmatically, but use them at your own risk.

Once the devices are paired, they can be connected using one of BluetoothDevice’s createRfcommSocketToServiceRecord or createInsecureRfcommSocketToServiceRecord methods after determining the UUID to use, either with getUuids or fetchUuidsWithSdp (or, in most cases, using the well-known UUID 00001101-0000-1000-8000-00805F9B34FB).

It’s very likely that Android’s Bluetooth API is subject to change soon. It already has changed in some of the more recent releases, although I’m not entirely sure why Google isn’t providing a stable API for pairing. Jelly Bean 4.2 introduces a new Bluetooth stack, moving from BlueZ to a Broadcom solution, so my guess is that it’s related to this.

Posted on

Implementing Spring-like Classpath Scanning in Android

One of the things that Spring 2.5 introduced back in 2007 was component scanning, a feature which removed the need for XML bean configuration and instead allowed developers to declare their beans using Java annotations. Rather than this:

We can do this:

It’s a pretty simple idea since Java makes it very easy to introspectively check a class’s annotations at runtime through its reflection API. Spring’s component scan feature also allows you to specify the base package(s) to scan for beans.

The big question is how do we get access to the classes in the classpath, specifically, those in the desired package? Java SE doesn’t provide an API for doing it, but there are ways to accomplish this. The most common (if not the only) approach is to load classes by relying on the file system. We know that we can use the ClassLoader to load a class by its package-qualified name, so it becomes a matter of retrieving the file names.

Getting the classpath itself in Java SE is easy:

This will yield something that looks like “/Users/Tyler/Workspace/Test/bin:/Users/Tyler/Workspace/Test/lib/gson-2.1.jar”. Loading the files from here is pretty straightforward, as is filtering on the package name since it maps to a directory one-to-one.

Another similar approach is to use the ClassLoader to load the resources directly:

Transition to Android

Unfortunately, these solutions don’t lend themselves to Android, which made implementing classpath scanning a little more difficult for Infinitum. The reason for this is, more or less, because of the way Android’s Dalvik VM is designed. When an Android application is compiled, the Dalvik bytecode is packaged into a file called “classes.dex” inside the APK. The good news is that the Android SDK provides an API for interacting with DEX files through the DexFile class.

In order to access classes.dex, we need a handle on the APK itself, which is actually quite easy to do:

The above code opens a DexFile for the running APK. Of course, this can have some performance implications. Opening the DexFile will potentially cause the VM to pass classes.dex through a process known as “dexopt”, which is a program that performs bytecode verification and optimization. This is an expensive process, but since we’re opening a DexFile for the APK itself, classes.dex should have already undergone this process, meaning dexopt won’t be run again.

The DexFile gives us access to the classes contained in classes.dex as an enumeration of strings representing the package-qualified class names. With this, we can iterate over the class names and load any which match the desired package.

This gets the job done, and it’s essentially how Infinitum accomplishes component scanning. However, it’s a very expensive operation. DexFile.entries() yields every class in the classpath — that is, every class in classes.dex — which includes not just application binaries, but also those of any libraries included.

It’s great that we can introspect every class in the classpath, but if we’re only interested in classes of a particular package, we’re out of luck. Every class is compiled into classes.dex and, short of decompiling it ((Tools for decompiling DEX files exist, such as Baksmali, but doing such a thing at runtime — if it’s even possible — would arguably not gain you any performance benefits. Still, this is something worth exploring.)),  there’s no way to pull out the classes we want without iterating over the entire classpath.

So, for now we settle with this somewhat inefficient solution. Nonetheless, it accomplishes what it needs to at the cost of maybe a few hundred milliseconds ((On the emulator running on my MacBook Pro, the classpath scanning takes about 600 milliseconds, while on my Galaxy Nexus, it takes about 200 milliseconds.)), so maybe it’s not such a bad approach in the grand scheme of things.

Posted on

Introducing InfinitumFramework.com

rendering

Here’s a dose of shameless self-promotion. It’s coming up on a year since I started development on Infinitum, and I’m targeting its first full release on its birthday, February 11. Shortly before I moved the project to GitHub, they deprecated the downloads service, so I needed to fine a home for distributing the binaries as well as the Javadoc.

GitHub offers its pages service, but I figured I’d just host it myself. I threw together a website in a couple days and the result is www.infinitumframework.com. This website will be used to host the latest (and previous) releases of the framework, its documentation, and, in the future, announcements and updates for it.

Stay tuned as Infinitum approaches its first official release!

Posted on

He Sed, She Sed

Shortly after switching to GitHub, I decided to relicense Infinitum from GNU LGPL to Apache License 2.0. There aren’t really any implications except one: replacing the license and copyright header in every source file.

I’m far from being a Unix expert (more like amateur at best), but I figured sed would be the quickest and easiest way to do this. Sed is a Unix utility for processing text streams, and it allows you to replace string patterns in files. A simple string replacement using sed is quite easy:

This will replace “foo” with “bar”. The “g” indicates that every matching occurrence in file.txt will be replaced, and “-i” means it will do the replacement in place.

In my case, I wanted to find every occurrence of the following string in every Java source file:

And I wanted to replace it with this:

I needed to do a multi-line replacement across a couple hundred files. Feeding lots of files to sed is actually pretty simple:

This command will pass all of the Java files in the current directory (and all sub-directories)  to sed. The reason xargs is needed is because it lets us avoid the “Argument list too long” problem.

In order to replace multiple lines, I needed to use an additional feature of sed. The “c” command lets you replace a range of lines:

There’s a caveat that I have so far ignored. Many Unix utilities have idiosyncrasies or differences between platforms, and sed is no exception. I failed to mention that I was doing this on Mac OSX, whose implementation of sed, as I encountered, had some peculiar quirks. The “-e” in the above command is one such quirk as it’s needed to perform an in-place pattern replacement on OSX.

So, I had a way to process a bunch of files at once and a way to replace multiple lines in a file. Now I just needed to combine these two techniques to replace the license header in all of my project files.

This replaces the range of lines covering the original license with the new license. It works, but the formatting becomes slightly off. That’s because OSX’s sed does not preserve leading whitespace, so the space before each asterisk is stripped. Fortunately, GNU sed does preserve leading whitespace, so building that and using it in place of OSX’s sed solved the problem. Also, GNU sed doesn’t require “-e” for in-place replacement.

Sed is a very handy little tool that every developer should have in his or her toolbelt. Admittedly, I don’t leverage Unix’s utilities nearly enough (although I’m working on it!), but tools like grep, sed, find, and xargs are immensely powerful and pretty simple to use. I think some developers have a tendency to over-engineer solutions for problems that could otherwise be solved using a trivial combination of these tools — I know I have! Of course, it’s helped that I’ve started to do all my programming, both work and play, on Mac. It’s my goal to become a better Unix developer!