Posted on

Building User Interfaces with React

If you follow me on Twitter, you’ve probably heard me raving about React. React is described as a “JavaScript library for building user interfaces” and was open sourced by Facebook about a year ago. Everybody and their mom has a JavaScript framework, so what makes React so interesting? Why would you use it over mainstays like Backbone or Angular?

There are a few things that make React worth looking at. First, React is a library, not a framework. It makes no assumptions about your frontend stack, and it plays nicely with existing codebases, regardless of the tech you’re using. This is great because you can use React incrementally for new or legacy code. Write your whole UI with it or use it for a single feature. All you need is a DOM node to mount.

React is delightfully simple (contrasted with Angular, which is a nightmare for beginners, and Backbone, which is relatively simple but still has several core concepts). It’s built around one idea: the Component, which is merely a reusable unit of UI. React was designed from the ground-up to be composable—Components are composed of other Components. Everything in the DOM is a Component, so your UI consists of a hierarchy of them.

Components can be built using JSX, an XML-like syntax that compiles down to regular JavaScript. As such, they can also be specified using plain-old JavaScript. The result is the same, but JSX makes it easy to visualize your DOM.

React does not do two-way data binding ((It’s worth noting that React provides a small add-on for getting the conciseness of two-way binding with the correctness of its one-way binding model.)). This is by design. It uses the von Neumann model of dataflow, which means data flows in only one direction. Two-way data binding makes it difficult to reason about your code. The advantage of the one-way model that React adopts is that it essentially turns your UI into a deterministic state machine. On the surface, it behaves as if the entire UI is simply re-rendered based on the current state of your data model. If you know what state your data is in, you know exactly what state your UI is in. Your UI is predictable. The React mantra is “re-render, don’t mutate.”

Re-rendering the entire DOM sounds expensive, but this is where React really shines. In order to draw your UI, it maintains a virtual DOM which can then be diffed. React’s diffing algorithm determines the minimum set of Components that need to be updated. It also batches reads and writes to the real DOM. This makes React fast.

Data is modeled two ways in a React component, props and state, which highlights the one-way data flow described earlier. Props consist of data that is passed from parent to child. A Component’s props can only be set by its parent. State, on the other hand, is an internal data structure that is accessed and modified only from within a Component. A Component is re-rendered when either its props or state is updated.

Once again, this makes it really easy to reason about your code (and unit test). Also note the use of the onClick handler. React provides a synthetic event system that gives you cross-browser compatible event listeners that you can attach to Components.

React and Backbone’s Router is a surprisingly powerful, yet effortless, combination for building single-page applications.

React makes it trivial to build small web apps, but because of its affinity for reusability and data modeling, it also scales well for large, complex UIs. You don’t have to use it for a new project, just start replacing small pieces of your UI with React Components. This makes it a lot easier for developers to adopt.

Posted on

Real-Time Client Notifications Using Redis and Socket.IO

Backbone.js is great for building structured client-side applications. Its declarative event-handling makes it easy to listen for actions in the UI and keep your data model in sync, but what about changes that occur to your data model on the server? Coordinating user interfaces for data consistency isn’t a trivial problem. Take a simple example: users A and B are viewing the same data at the same time, while user A makes a change to that data. How do we propagate those changes to user B? Now, how do we do it at scale, say, several thousand concurrent users? What about external consumers of that data?

One of our products at WebFilings called for real-time notifications for a few reasons:

  1. We needed to keep users’ view of data consistent.
  2. We needed a mechanism that would alert users to changes in the web client (and allow them to subscribe/unsubscribe to certain events).
  3. We needed notifications to be easily consumable (beyond the scope of a web client, e.g. email alerts, monitoring services, etc.).

I worked on developing a pattern that would address each of these concerns while fitting within our platform’s ecosystem, giving us a path of least resistance with maximum payoff.

Polling sucks. Long-polling isn’t much better. Server-Sent Events are an improvement. They provide a less rich API than the WebSocket protocol, which supports bi-directional communication, but they do have some niceties like handling reconnects and operating over traditional HTTP. Socket.IO provides a nice wrapper around WebSockets while falling back to other transport methods when necessary. It has a rich API with features like namespaces, multiplexing, and reconnects, but it’s built on Node.js, which means it doesn’t plug into our Python stack very easily.

The solution I decided on was a library called gevent-socketio, which is a Python implementation of the Socket.IO protocol built on gevent, making it incredibly simple to hook in to our existing Flask app. 

The gevent-socketio solution really only solves a small part of the overarching problem by providing a way to broadcast messages to clients. We still need a way to hook these messages in to our Backbone application and, more important, a way to publish and subscribe to events across threads and processes. The Socket.IO dispatcher is just one of potentially many consumers after all.

The other piece of the solution is to use Redis for its excellent pubsub capabilities. Redis allows us to publish and subscribe to messages from anywhere, even from different machines. Events that occur as a result of user actions, task queues, or cron jobs can all be captured and published to any interested parties as they happen. We’re already using Redis as a caching layer, so we get this for free. The overall architecture looks something like this:

pubsub

Let’s dive into the code.

Hooking gevent-socketio into our Flask app is pretty straightforward. We essentially just wrap it with a SocketIOServer.

The other piece is registering client subscribers for notifications:

NotificationsNamespace is a Socket.IO namespace we will use to broadcast notification messages. We use gevent-socketio’s BroadcastMixin to multicast messages to clients.

When a connection is received, we spawn a greenlet that listens for messages and broadcasts them to clients in the notifications namespace. We can then build a minimal API that can be used across our application to publish notifications.

Wiring notifications up to the UI is equally simple. To facilitate communication between our Backbone components while keeping them decoupled, we use an event-dispatcher pattern relying on Backbone.Events. The pattern looks something like this:

This pattern makes it trivial for us to allow our views, collections, and models to subscribe to our Socket.IO notifications because we just have to pump the messages into the dispatcher pipeline.

Now our UI components can subscribe and react to client- and server-side events as they see fit and in a completely decoupled fashion. This makes it very easy for us to ensure our client-side views and models are updated automatically while also letting other services consume these events.

Posted on

No More Ninjas

How does a software company attract talent? Compensation? That’s how they attract people. Free lunches and foosball tables? Keep guessing.

The most effective way for a company to bag top-tier engineers is simple: let developers be developers.

Engineers Drive Innovation

Software is a symbiotic creature, and there are two ways a product is built: from the top down and the bottom up. Top-down development means that requirements are curated by and flow from the business, product owners, and managers, downward to the engineers. With the latter, developers explore new ideas and use technology that may be outside the organization’s standard gamut. Ideas for new products or features are born and pushed up to product owners where they can be cultivated.

Both of these methods must strike a balance for a company and its products to be successful. Without a focused vision, a product will fail. Without embracing new ideas and technology, a company will become irrelevant.

Open Source is King

How an organization perceives OSS is paramount to developers. Companies that contribute and maintain open source projects tend to attract talented software engineers. Just look at some of the top repositories on GitHub. Unfortunately, there can sometimes be an eternal struggle between engineers and lawyers.

It’s equally important that a company supports the use of OSS in its own products. I’ve worked on projects where the use of open source code was discouraged in favor of homegrown solutions. This is dangerously senseless considering most mature, open source projects are battle tested and have large community support.

Developers want to work at companies that embrace open source, which is a push-pull dynamic.

Let Me Work, Dammit

I do all of my development on a MacBook Pro running OSX. Some of my coworkers use similar hardware configurations with various flavors of Linux. Back when I worked with .NET, I used Windows. The CLI is now my home, however, and Vim is Old Reliable sitting in the driveway. Some prefer IntelliJ, some PyCharm, and yet others Sublime Text—to each his own.

What I’m getting at is everyone has their preferred setup for building software. If your product does not have technical constraints resulting from the stack you’re using, don’t force me out of my element. Let me use the environment I’m comfortable with because it’s the environment I’m productive with.

No More Ninjas

Tech recruiters: no more code ninjas, no more Jedis, no more rock stars, no more gurus. These words are an unfortunate by-product of the startup culture. You may have the best of intentions, but to me, these words are red flags and marginally patronizing.

Perhaps I’m overly cynical, but I tend to translate job postings of this nature as “Seeking to exploit naive, young programmer to build a startup by working unsustainable hours. You’ll get equity!” It’s the hard sell.

We get it, you’re looking for skilled programmers, but no self-respecting programmer will call him or herself a rock star. And if they do, not only does it blow expectations out of proportion, it makes them look like a dick. Let your company, its products, and its culture speak for themselves. If you aren’t finding the “rock stars” you were looking for, it wasn’t meant to be.

Let Developers Be Developers

Yes, I’m biased. Developers are opinionated—I certainly am. Nonetheless, I would hazard to guess many top tech companies tend to address several of the points I’ve touched on. Call it headstrong, obstinate, whatever, but these are things I like to get a feel for when interviewing with a company. Others may disagree, but I suspect you wouldn’t be too hard-pressed to find many who take the craft seriously share similar views.

Posted on

The Art of Software

I stared at my coffee as a friend asked a mildly profound question: “what is your greatest passion in life?” Strangely, I thought of an exchange that occurred just a few months earlier while I was at the bank, opening an account for my software consulting company.

“What is the nature of your company’s business?” the banker asked.

“Building software,” my partner and I managed to respond with. Now suddenly fascinated, she looked away from her computer and at us, as if we had abruptly sprouted wings.

“Like Facebook?”

I did my best to suppress a laugh and simply nodded. I could hardly blame her. The 30-something-year-old banker’s knowledge of the domain likely consisted of the horrendous software they used at the bank, Angry Birds, and whatever insight she gleaned from watching The Social Network. About equivalent to my knowledge of banking.

I looked up from my coffee. Naturally, my answer would immediately gravitate towards software, but how to frame it in a way that made sense?

Building something that impacts people—making their lives easier, more enjoyable, or something to that effect—that’s really the goal of any engineer, but it’s an amazing ability that software developers possess. The modern-day alchemist, quite literally turning electrons into social media and flight simulators and bank transactions—things that delight or captivate; things that help us exist. We’re the janitors of data, or is it plumbers? But we’re also artists.

Software is a craft, an art, and not one that everyone is capable of doing (but certainly capable of learning). But what is art? Something that exhibits beauty in aesthetics, perhaps visually, perhaps audibly. We think of Michelangelo’s painting of the Sistine Chapel or Beethoven’s Fifth Symphony. We probably don’t think of Twitter or Facebook and certainly not the ATM you withdrew money from.

Web designers may argue that websites can be beautiful, and they’re right. Software can be both a visual and aural stimulus, but these are really just superficial observations. The same way I say “Huh, that’s pretty cool” after seeing the Sistine Chapel. Others may appreciate it more, or less.

Many would argue that the purpose of art is to evoke emotion from its audience. There’s probably a lot of truth to that, but to me, art exists on several different levels. There’s the immediate aesthetic beauty—what the eyes see and what the ears hear. There’s the emotions that it evokes from its audience—this makes me happy, this makes me sad, this makes me intrigued. And then there’s the appreciation of the craft: painters admiring a painting, not because of its obvious beauty, but because of the technical prowess it demonstrates. Software is no different.

I read something once that said software developers are never impressed. I don’t think that’s entirely true, or at least the entirety of the truth. Beyond the superficial veneer of beauty, there are the inner workings of incredibly complex systems, riddled with algorithms, design patterns, and all the things that make programmers swoon. And they run like well-oiled machines. That is art, but art that only others who share the trade can enjoy. There is elegance in code like there is elegance in the prose of a novel. The difference is in those who are able to marvel at it.

The great thing about engineering, software or otherwise, is the problem solving involved. There is no end to the amount of exceptionally difficult problems that need solving, and the way in which those problems are solved is an art in itself.

I doubt many people associate the words “art” and “software,” but there is as much creativity and craftsmanship in building software as there is engineering. It’s this drive, to build great products that affect people, that I consider my passion and my art.

Posted on

Discipline in Prototyping

Writing software doesn’t require discipline, but writing good software does. I would argue that the vast majority of tech debt in projects results from PoCs/prototypes/spikes. The code from these typically aren’t intended to make it into production, but they almost invariably do in some capacity.

“I won’t bother writing unit tests for this code, it’s purely exploratory.”

The code grows…

“It’s just a rough proof-of-concept.”

…and grows…

“It won’t make it to production!”

…and grows, until pressure from above or other factors bulldoze it into a release.

This problem is not difficult to avoid, it’s entirely disciplinary. Spikes should always be timeboxed, but truthfully, this problem rarely occurs because of spikes—it happens at the onset of projects. Nearly every project gets its start as a proof-of-concept, which becomes a prototype, which becomes a product.

It’s during this process, as a project reaches its infancy, where tech debt has a tendency to accumulate like a cancerous growth. Less seasoned developers might skip out on writing unit tests or forgo code reviews because, well, it’s a prototype. This is especially tempting when you’re working in a codebase by yourself. I myself have been caught in this rut before.

I’m not saying you need to use TDD for every line of code you write (or, for that matter, at all), but, for the love of god, write unit tests around your code. If you’re writing code that’s in source control, it’s set in stone for everyone to see and, more important, maintain (and if it’s not in source control, it doesn’t exist). Write code like it will end up in production (because frankly, it probably will).

“Minimum viable product” is a popular buzzword these days but holds merit when used in the right context. What’s important to note is that an MVP is not a proof-of-concept nor a prototype. A PoC is a great way to sketch out an early draft, but an MVP is not a draft. Building something under the guise of an MVP and renouncing standard development procedures is a wildly mistaken approach.

It takes discipline. It might even take getting yourself caught in a tech-debt-riddled project before realizing just how crucial it is, but it will make you a better software developer and certainly make your projects more sustainable.