Posted on

Discipline in Prototyping

Writing software doesn’t require discipline, but writing good software does. I would argue that the vast majority of tech debt in projects results from PoCs/prototypes/spikes. The code from these typically aren’t intended to make it into production, but they almost invariably do in some capacity.

“I won’t bother writing unit tests for this code, it’s purely exploratory.”

The code grows…

“It’s just a rough proof-of-concept.”

…and grows…

“It won’t make it to production!”

…and grows, until pressure from above or other factors bulldoze it into a release.

This problem is not difficult to avoid, it’s entirely disciplinary. Spikes should always be timeboxed, but truthfully, this problem rarely occurs because of spikes—it happens at the onset of projects. Nearly every project gets its start as a proof-of-concept, which becomes a prototype, which becomes a product.

It’s during this process, as a project reaches its infancy, where tech debt has a tendency to accumulate like a cancerous growth. Less seasoned developers might skip out on writing unit tests or forgo code reviews because, well, it’s a prototype. This is especially tempting when you’re working in a codebase by yourself. I myself have been caught in this rut before.

I’m not saying you need to use TDD for every line of code you write (or, for that matter, at all), but, for the love of god, write unit tests around your code. If you’re writing code that’s in source control, it’s set in stone for everyone to see and, more important, maintain (and if it’s not in source control, it doesn’t exist). Write code like it will end up in production (because frankly, it probably will).

“Minimum viable product” is a popular buzzword these days but holds merit when used in the right context. What’s important to note is that an MVP is not a proof-of-concept nor a prototype. A PoC is a great way to sketch out an early draft, but an MVP is not a draft. Building something under the guise of an MVP and renouncing standard development procedures is a wildly mistaken approach.

It takes discipline. It might even take getting yourself caught in a tech-debt-riddled project before realizing just how crucial it is, but it will make you a better software developer and certainly make your projects more sustainable.

Posted on

Productivity Over Process

It seems like every software company you talk to will boast about how they use the latest development process du jour—Agile, Lean, XP, Kanban—pick your poison. What’s interesting is that the people evangelizing their chosen methodology are typically managers, not developers, almost emphasizing the process more than the product. Startups and other young tech companies seem to be particularly guilty of this (after all, every time someone utters the words “lean startup”, an angel investor gets his wings).

I’ve worked on a number of different teams, mostly Agile, ranging in size from 4 to 20 developers. I’ve used everything from TFS and JIRA to Pivotal Tracker and Trello to manage stories and track bugs. A process, the way in which you produce something, is often seen as necessary by project managers and a necessary evil by developers.

Are software developers just cowboys who want to build something, guns blazing? To an extent, probably, but the conclusion that I’ve drawn is that there is no silver bullet when it comes to crafting software. It’s not one size fits all. Agile is not the be-all, end-all solution, nor is anything else.

I’ve seen teams that said they were Agile when, in fact, they weren’t in any sense of the word. To some, it’s just a buzzword used to attract talent. I’ve also worked for companies where Agile was used across all teams, no exceptions. This led to problems with the way some groups operated. It worked great for feature teams who were engaged in completing user stories, but for some of the component teams, it just didn’t make sense for the type of work they were doing. My team, very much a backend architecture group of about six developers, was merely going through the Agile motions—a bad sign indeed. We switched to a less structured Kanban process as a result because it worked for us.

Too much emphasis is placed on the process and not the productivity of a team. Am I saying that Agile is bad? Absolutely not. In most cases, an Agile team is a productive team. When an Agile team hits its stride and really grooves, the results are impeccable, but a process needs to be peripheral. It should get out of your way as fast as possible so you can get work done. Don’t just go through the motions.

Posted on

How is Software Valued?

I was talking to a friend a few weeks ago who was putting together a business presentation for potential investors. He was developing a plan for a campground kiosk system that would rely on GIS data to allow guests to view and check in to camp sites. The plan was reasonable enough and mostly feasible. He carefully considered all the costs—licensing for a third-party GIS, kiosk hardware, line trenching—and then there was software.

He allocated a mere $8,000 for the kiosk software, a low-ball figure by any definition of the word, and he estimated it to take about four weeks to complete from scratch.

“Where did you get that figure?” I asked him. The answer basically boiled down to “thin air.”

I didn’t have any kind of sudden realization, but this exchange did reinforce something many others have already observed: software is remarkably undervalued.

All too often clients say something along the lines of “You want me to pay you $X per hour to sit and type on your computer?!” What’s not obvious to many is that software engineers create extraordinary value for businesses. It’s almost ironic considering just about everything these days is driven by software, to the extent that it’s almost taken for granted, and it doesn’t somehow materialize out of thin air.

So why is this the case? Is it because software isn’t a physical good? Maybe. However, I think the issue is largely attributed to the disparate levels of productivity between software engineers and other areas of industry. A developer might write an accounting system that leaves a large number of accountants redundant or automate a process that otherwise takes a dozen employees to complete. Is it fair that they are compensated accordingly? Again, it’s about creating value, but the fact is, most developers aren’t paid in proportion to the value they create or their productivity. Consultant John Cook explains why this is the case:

A salesman who sells 10x as much as his peers will be noticed, and compensated accordingly. Sales are easy to measure, and some salesmen make orders of magnitude more money than others. If a bricklayer were 10x more productive than his peers this would be obvious too, but it doesn’t happen: the best bricklayers cannot lay 10x as much brick as average bricklayers. Software output cannot be measured as easily as dollars or bricks. The best programmers do not write 10x as many lines of code and they certainly do not work 10x longer hours.

It may also be due, at least in part, to software being endlessly enigmatic to non-software people. Is this auto mechanic ripping us off on our car? Is this developer ripping us off on our point-of-sale system? It’s easy for people to see what it takes to build a bridge—designing it, performing simulated load tests, pouring the concrete, assembling the steel, laying the superstructure—these are all tangible overheads.

What does it take to build software? It’s just some bit-twiddling, right? There’s no inventory that needs to be accounted for; there’s no manufacturing labor. As developers, we know it’s a lot more involved than that. The problem with software is that it’s a living thing. After you build a home, you don’t decide to move the bathroom to the other side of the house. The same cannot be said of software.

Product owners are fickle creatures. They don’t know what they want, except that Feature X needs to be changed to Feature Y and still ship in time. I’ve been on projects where this had become so problematic that developers started leaving Feature X implemented. That way, when NPD ultimately decided X was correct in the first place, we would be on schedule, but that’s tangential to this conversation.

What I’m getting at is that there’s a lot more to building software than what may be perceived. There’s still planning, and designing, and prototyping, and implementing, and testing. But unlike the bridge or the house, the process doesn’t stop when the software ships.

No self-respecting (or sane) software engineer would agree to build a complete system in four weeks’ time for $8,000. It’s almost insulting. But to someone which software is completely foreign to—and it is to most—it might not sound so outlandish. The problem is finding the appropriate level of value. It’s easy if you’re an independent consultant, but if you’re one of several hundred developers at a company, how is your value measured? As Cook explains, output, in terms of lines of code, is not a reliable metric. In fact, one could argue it’s inversely proportional to a developer’s ability.

The romantic image of an über-programmer is someone who fires up Emacs, types like a machine gun, and delivers a flawless final product from scratch. A more accurate image would be someone who stares quietly into space for a few minutes and then says “Hmm. I think I’ve seen something like this before.”

It’s for this reason, combined with the fact that programmer salaries don’t really vary dramatically, that many developers do consulting as a profession. They know exactly what their time is worth and the value they add to a business. Coming back to the problem described earlier, the downside of consulting is that many customers don’t recognize that value. As a consultant, it’s also your job to establish what it is and why.

I took on a contract last month to build some mobile software for a small engineering firm. They needed an Android application but didn’t have the resources in-house to do it. They met with a few software shops in the area but none of them specialized in mobile development. I build Android apps. This raised my value, and I already had a pretty good idea what the app would do for their business. At that point, it’s just letting economics work itself out.

Posted on

Bluetooth Blues

I spent the better part of two days working on Bluetooth connectivity for an Android app I’m developing. Going into it, I had virtually no experience working with Bluetooth, especially on Android. I quickly discovered some of the peculiarities of the platform’s Bluetooth API.

In addition to connecting to Bluetooth devices, the client wanted to pair and unpair from the app. The easy way out, and probably The Android Way™, would be to pass that responsibility off to the OS, à la an Intent:

This will bring up the Bluetooth settings menu, from which you can pair/unpair devices, but the problem is that it’s a complete context switch for the user—they are no longer in your application. I was looking to provide a more seamless experience so that the user didn’t have to leave the app at all to pair a device.

Device Discovery

The entry point for Bluetooth interaction in Android is through the BluetoothAdapter, which is used to orchestrate the device discovery process and fetch paired devices. Calling startDiscovery() will tell the adapter to start scanning for devices, and when one is found, an Intent will be fired off which can then be intercepted by a BroadcastReceiver.

The above code shows how the device discovery process is kicked off and how a BroadcastReceiver is registered to listen for discovery Intents. Note that the BroadcastReceiver is unregistered and discovery is canceled in onDestroy.

In order to react to discovery events, we must implement a BroadcastReceiver.

Device Pairing

Once you have a handle on the BluetoothDevice received in the BroadcastHandler, how do you actually pair with it? Looking at the documentation for the class, you’ll see that there are no methods for doing this. This is where things start to get a little strange.

Diving into the source code for BluetoothDevice, you’ll actually find that there is functionality for doing pairing and unpairing, but the methods are hidden from the API using the @hide annotation. What’s more interesting is that the methods are, in fact, public.

Evidently, device pairing is intended to be performed only by platform applications, which is a little curious considering the permission needed to perform pairing, android.permission.BLUETOOTH_ADMIN, is accessible by third-party applications. Nonetheless, this means we actually can pair a BluetoothDevice, just not in the way the Android engineers intended.

To access the BluetoothDevice methods needed, createBond and removeBond, we can use reflection.

The pairDevice method will prompt the user to enter a PIN for the discovered device, circumventing the need to open the Bluetooth settings. As such, the pairing does not actually complete until the correct PIN is entered. The boolean value returned from the method indicates whether the pairing process was successfully kicked off or not.

It goes without saying that this code, while functional, is volatile because these methods are technically not part of the public API, so they could change or disappear in future platform releases.

We can add an Intent filter to our BroadcastReceiver to listen for pairing events using the action BluetoothDevice.ACTION_BOND_STATE_CHANGED.

There are a few other hidden methods in BluetoothDevice, like cancelPairingUserInput, setPairingConfirmation, convertPinToBytes, and setPin, that you could potentially use to customize the pairing process or perform it programmatically, but use them at your own risk.

Once the devices are paired, they can be connected using one of BluetoothDevice’s createRfcommSocketToServiceRecord or createInsecureRfcommSocketToServiceRecord methods after determining the UUID to use, either with getUuids or fetchUuidsWithSdp (or, in most cases, using the well-known UUID 00001101-0000-1000-8000-00805F9B34FB).

It’s very likely that Android’s Bluetooth API is subject to change soon. It already has changed in some of the more recent releases, although I’m not entirely sure why Google isn’t providing a stable API for pairing. Jelly Bean 4.2 introduces a new Bluetooth stack, moving from BlueZ to a Broadcom solution, so my guess is that it’s related to this.

Posted on

Implementing Spring-like Classpath Scanning in Android

One of the things that Spring 2.5 introduced back in 2007 was component scanning, a feature which removed the need for XML bean configuration and instead allowed developers to declare their beans using Java annotations. Rather than this:

We can do this:

It’s a pretty simple idea since Java makes it very easy to introspectively check a class’s annotations at runtime through its reflection API. Spring’s component scan feature also allows you to specify the base package(s) to scan for beans.

The big question is how do we get access to the classes in the classpath, specifically, those in the desired package? Java SE doesn’t provide an API for doing it, but there are ways to accomplish this. The most common (if not the only) approach is to load classes by relying on the file system. We know that we can use the ClassLoader to load a class by its package-qualified name, so it becomes a matter of retrieving the file names.

Getting the classpath itself in Java SE is easy:

This will yield something that looks like “/Users/Tyler/Workspace/Test/bin:/Users/Tyler/Workspace/Test/lib/gson-2.1.jar”. Loading the files from here is pretty straightforward, as is filtering on the package name since it maps to a directory one-to-one.

Another similar approach is to use the ClassLoader to load the resources directly:

Transition to Android

Unfortunately, these solutions don’t lend themselves to Android, which made implementing classpath scanning a little more difficult for Infinitum. The reason for this is, more or less, because of the way Android’s Dalvik VM is designed. When an Android application is compiled, the Dalvik bytecode is packaged into a file called “classes.dex” inside the APK. The good news is that the Android SDK provides an API for interacting with DEX files through the DexFile class.

In order to access classes.dex, we need a handle on the APK itself, which is actually quite easy to do:

The above code opens a DexFile for the running APK. Of course, this can have some performance implications. Opening the DexFile will potentially cause the VM to pass classes.dex through a process known as “dexopt”, which is a program that performs bytecode verification and optimization. This is an expensive process, but since we’re opening a DexFile for the APK itself, classes.dex should have already undergone this process, meaning dexopt won’t be run again.

The DexFile gives us access to the classes contained in classes.dex as an enumeration of strings representing the package-qualified class names. With this, we can iterate over the class names and load any which match the desired package.

This gets the job done, and it’s essentially how Infinitum accomplishes component scanning. However, it’s a very expensive operation. DexFile.entries() yields every class in the classpath — that is, every class in classes.dex — which includes not just application binaries, but also those of any libraries included.

It’s great that we can introspect every class in the classpath, but if we’re only interested in classes of a particular package, we’re out of luck. Every class is compiled into classes.dex and, short of decompiling it ((Tools for decompiling DEX files exist, such as Baksmali, but doing such a thing at runtime — if it’s even possible — would arguably not gain you any performance benefits. Still, this is something worth exploring.)),  there’s no way to pull out the classes we want without iterating over the entire classpath.

So, for now we settle with this somewhat inefficient solution. Nonetheless, it accomplishes what it needs to at the cost of maybe a few hundred milliseconds ((On the emulator running on my MacBook Pro, the classpath scanning takes about 600 milliseconds, while on my Galaxy Nexus, it takes about 200 milliseconds.)), so maybe it’s not such a bad approach in the grand scheme of things.