The Importance of Being Idle

“Practice not-doing and everything will fall into place.”

It’s good to be lazy. Sometimes, in programming, it can also be hard to be lazy. It’s this paradox that I will explore today — The Art of Being Lazy. Specifically, I’m going to dive into a design pattern known as lazy loading by discussing why it’s used, the different flavors it comes in, and how it can be implemented.

Lazy loading is a pretty simple concept: don’t load something until you really need it. However, the philosophy can be generalized further: don’t do something until you need to do it. It’s this line of thinking that has helped lead to processes like Kanban and lean software development (and also probably got you through high school). Notwithstanding, this tenet goes beyond the organizational level. It’s about optimizing efficiency and minimizing waste. There’s a lot to be said about optimizing efficiency in a computer program, which is why The Art of Being Lazy is an exceedingly relevant principle.

They Don’t Teach You This in School

My first real job as a programmer was working as a contractor for Thomson Reuters.  I started as a .NET developer (having no practical experience with it whatsoever) working on a web application that primarily consisted of C# and ASP.NET. The project was an internal configuration management database, which is basically just a big database containing information pertaining to all of the components of an information system (in this case, Thomson’s West Tech network, the infrastructure behind their legal technology division).

This CMDB was geared towards providing application-impact awareness, which, more or less, meant that operations and maintenance teams could go in and see what applications or platforms would be affected by a server going down (hopefully for scheduled maintenance and not a datacenter outage), which business units were responsible for said applications, and who the contacts were for those groups. It also provided various other pieces of information pertaining to these systems, but what I’m getting at is that we were dealing with a lot of data, and this data was all interconnected. We had a very complex domain model with a lot of different relationships. What applications are running on what app servers? Which database servers do they depend on? What NAS servers have what NAS volumes mounted on them? The list goes on.

Our object graph was immense. You can imagine the scale of infrastructure a company like Thomson Reuters has. The crux of the problem was that we were persisting all of this data as well as the relationships between it, and we wanted to allow users of this software to navigate this vast hierarchy of information. Naturally, we used an ORM to help manage this complexity. Since we were working in .NET, and many of us were Java developers, we went with NHibernate.

We wanted to be able to load, say, an application server, and see all of the entities associated with it. To the uninitiated (which, at the time, would have included myself), this might seem like a daunting task. Loading any given entity would result in loading hundreds, if not thousands, of related entities because it would load those directly related, then those related to the immediate neighbors, continuing on in what seems like a never-ending cascade. Not only would it take forever, but we’d quickly run out of memory! There’s simply no way you can deal with an object graph of that magnitude and reasonably perform any kind of business logic on it. Moreover, it’s certainly not scalable, so obviously this would be a very naive thing to do. The good news is that, unsurprisingly,  it’s something that’s not necessary to do.

It’s Good to be Lazy

The solution, of course, as I’ve already hit you across the face with, is a design pattern known as lazy loading. The idea is to defer initialization of an object until it’s truly needed (i.e. accessed). Going back to my anecdote, when we load, for example, an application server entity, rather than eagerly loading all its associated entities, such as servers, applications, BIG-IPs, etc., we use placeholders. Those related entities are then loaded on-the-fly when they are accessed.

Lazy loading can be implemented in a few different ways, through lazy initialization, ghost objects, value holders, and dynamic proxies — each has its own trade-offs. I’ll talk about all of them, but I’m going to primarily focus on using proxies since it’s probably the most widely-used approach, especially within the ORM arena.

Lazy initialization probably best illustrates the concept of lazy loading. With lazy initialization, the object to be lazily loaded is represented by a special marker value (typically null) which indicates that the object has yet to be loaded. Every call to the object will first check to see if it has been loaded/initialized, and if it hasn’t, it gets loaded/initialized. Thus, the first call to the object will load it, while subsequent calls will not need to. The code below shows how this is done.

Ghost objects are simply entities that have been partially loaded, usually just having the ID populated so that the full object can be loaded later. This is very similar to lazy initialization. The difference is that the related entity is initialized but not populated.

A value holder is an object that takes the place of the lazily loaded object and is responsible for loading it. The value holder has a getValue method which does the lazy loading. The entity is loaded on the first call to getValue.

The above solutions get the job done, but their biggest problem is that they are pretty intrusive. The classes have knowledge that they are lazily loaded and require logic for loading. Luckily, there’s an option which helps to avoid this issue. Using dynamic proxies ((For more background on proxies themselves, check out one of my previous posts.)), we can write an entity class which has no knowledge of lazy loading and yet still lazily load it if we want to.

This is possible because the proxy extends the entity class or, if applicable, implements the same interface, allowing it to intercept calls to the entity itself. That way, the object need not be loaded, but when it’s accessed, the proxy intercepts the invocation, loads the object if needed, and then delegates the invocation to it. Since proxying classes requires bytecode instrumentation, we need to use a library like Cglib.

First, we implement an InvocationHandler we can use to handle lazy loading.

Now, we can use Cglib’s Enhancer class to create a proxy.

Now, the first call to any method on foo will invoke loadObject, which in turn will load the object into memory. Cglib actually provides an interface for doing lazy loading called LazyLoader, so we don’t even need to implement an InvocationHandler.

ORM frameworks like Hibernate use proxies to implement lazy loading, which is one of the features we took advantage of while developing the CMDB application. One of the nifty things that Hibernate supports is paged lazy loading, which allows entities in a collection to be loaded and unloaded while it’s being iterated over. This is extremely useful for one-to-many and, in particular, one-to-very-many relationships.

Lazy loading was also one of the features I included in Infinitum’s ORM, implemented using dynamic proxies as well. ((Java bytecode libraries like Cglib are not compatible on the Android platform. Android uses its own bytecode variant.)) At a later date, I may examine how lazy loading is implemented within the context of an ORM and how Infinitum uses it. It’s a very useful design pattern and provides some pretty significant performance optimizations. It just goes to show that sometimes being lazy pays off.

Dalvik Bytecode Generation

Earlier, I discussed the use of dynamic proxies and how they can be implemented in Java. As we saw, a necessary part of proxying classes is bytecode generation. From its onset, something I wanted to include in Infinitum was lazy loading. I also wanted to provide support for AOP down the road. Consequently, it was essential to include some way to generate bytecode at runtime.

The obvious choice would be to use a library like Cglib or Javassist, but sadly neither of those would work. That’s because Android does not use a Java VM, it uses its own virtual machine called Dalvik. As a result, Java source code isn’t compiled into Java bytecode (.class files), but rather Dalvik bytecode (.dex files). Since Cglib and Javassist are designed for Java bytecode manipulation, they do not work on the Android platform. ((ASMDEX, a Dalvik-compatible bytecode-manipulation library was released in March 2012, meaning Cglib could, in theory, be ported to Android since it relies on ASM.))

What’s a programmer to do? Fortunately, some Googlers developed a new library for runtime code generation targeting the Dalvik VM called Dexmaker.

It has a small, close-to-the-metal API. This API mirrors the Dalvik bytecode specification giving you tight control over the bytecode emitted. Code is generated instruction-by-instruction; you bring your own abstract syntax tree if you need one. And since it uses Dalvik’s dx tool as a backend, you get efficient register allocation and regular/wide instruction selection for free.

Even better, Dexmaker provides an API for directly creating proxies called ProxyBuilder. If you followed my previous post on generating proxies, then using ProxyBuilder is a piece of cake. Similar to Java’s Proxy class, ProxyBuilder relies on an InvocationHandler to specify a proxy’s behavior.

Dexmaker enabled me to implement lazy loading and AOP within the Infinitum framework. It also opens up the possibility of using Mockito for unit testing in an Android environment because Mockito relies on proxies for generating mocks. ((Infinitum is actually unit tested using Robolectric, which allows for testing Android code in a standard JVM.))

Proxies: Why They’re Useful and How They’re Implemented

I wanted to write about lazy loading, but doing so requires some background on proxies. Proxies are such an interesting and useful concept that I decided it would be worthwhile to write a separate post discussing them. I’ve talked about them in the past, for instance on StackOverflow, so this will be a bit of a rehash, but I will go into a little more depth here.

What is a proxy? Fundamentally, it’s a broker, or mediator, between an object and that object’s user, which I will refer to as its client. Specifically, a proxy intercepts calls to the object, performs some logic, and then (typically) passes the call on to the object itself. I say typically because the proxy could simply intercept without ever calling the object.

proxy

A proxy works by implementing an object’s non-final methods. This means that proxying an interface is pretty simple because an interface is merely a list of method signatures that need to be implemented. This facilitates the interception of method invocations quite nicely. Proxying a concrete class is a bit more involved, and I’ll explain why shortly.

Proxies are useful, very useful. That’s because they allow for the modification of an object’s behavior and do so in a way that’s completely invisible to the user. Few know about them, but many use them, usually without even being aware of it. Hibernate uses them for lazy loading, Spring uses them for aspect-oriented programming, and Mockito uses them for creating mocks. Those are just three (huge) use cases of many.

JDK Dynamic Proxies

Java provides a Proxy class which implements a list of interfaces at runtime. The behavior of a proxy is specified through an implementation of InvocationHandler, an interface which has a single method called invoke. The signature for the invoke method looks like the following:

The proxy argument is the proxy instance the method was invoked on. The method argument is the Method instance corresponding to the interface method invoked on the object.  The last argument, args, is an array of objects which consists of the arguments passed in to the method invocation, if any.

Each proxy has an InvocationHandler associated with it, and it’s this handler which is responsible for delegating method calls made on the proxy to the object being proxied. This level of indirection means that methods are not invoked on an object itself but rather on its proxy. The example below illustrates how an InvocationHandler would be implemented such that “Hello World” is printed to the console before every method invocation.

This is pretty easy to understand. The invoke method will intercept any method call by printing “Hello World” before delegating the invocation to the proxied object. It’s not very useful, but it does lend some insight into why proxies are useful for AOP.

An interesting observation is that invoke provides a reference to the proxy itself, meaning if you were to instead call the method on it, you would receive a StackOverflowError because it would lead to an infinite recursion.

Note that the InvocationHandler alone is of no use. In order to actually create a proxy, we need to use the Proxy class and provide the InvocationHandler. Proxy provides a static method for creating new instances called newProxyInstance. This method takes three arguments, a class loader, an array of interfaces to be implemented by the proxy, and the proxy behavior in the form of an InvocationHandler. An example of creating a proxy for a List is shown below.

The client invoking methods on the List can’t tell the difference between a proxy and its underlying object representation, nor should it care.

Proxying Classes

While proxying an interface dynamically is relatively straightforward, the same cannot be said for proxying a class. Java’s Proxy class is merely a runtime implementation of an interface or set of interfaces, but a class does not have to implement an interface at all. As a result, proxying classes requires bytecode manipulation. Fortunately, there are libraries available which help to facilitate this through a high-level API. For example, Cglib (short for code-generation library) provides a way to extend Java classes at runtime and Javassist (short for Java Programming Assistant) allows for both class modification and creation at runtime. It’s worth pointing out that Spring, Hibernate, Mockito, and various other frameworks make heavy use of these libraries.

Cglib and Javassist provide support for proxying classes because they can dynamically generate bytecode (i.e. class files), allowing us to extend classes at runtime in a way that Java’s Proxy can implement an interface at runtime.

At the core of Cglib is the Enhancer class, which is used to generate dynamic subclasses. It works in a similar fashion to the JDK’s Proxy class, but rather than using a JDK InvocationHandler, it uses a Callback for providing proxy behavior. There are various Callback extensions, such as InvocationHandler (which is a replacement for the JDK version), LazyLoader, NoOp, and Dispatcher.

This code is essentially the same as the earlier example in that every method invocation on the proxied object will first print “Hello World” before being delegated to the actual object. The difference is that MyClass does not implement an interface, so we didn’t need to specify an array of interfaces for the proxy.

Proxies are a very powerful programming construct which enables us to implement things like lazy loading and AOP. In general, they allow us to alter the behavior of objects transparently. In the future, I’ll dive into the specific use cases of lazy loading and AOP.