Fast Topic Matching

Wed, 28 Dec 2016 17:52:30 -0600

A common problem in messaging middleware is that of efficiently matching message topics with interested subscribers. For example, assume we have a set of subscribers, numbered 1 to 3:

Subscriber

Match Request

forex.usd

forex.*

stock.nasdaq.msft

And we have a stream of messages, numbered 1 to N:

Message

Topic

forex.gbp

stock.nyse.ibm

stock.nyse.ge

forex.eur

forex.usd

…

stock.nasdaq.msft

We are then tasked with routing messages whose topics match the respective subscriber requests, where a “*” wildcard matches any word. This is frequently a bottleneck for message-oriented middleware like ZeroMQ, RabbitMQ, ActiveMQ, TIBCO EMS, et al. Because of this, there are a number of well-known solutions to the problem. In this post, I’ll describe some of these solutions, as well as a novel one, and attempt to quantify them through benchmarking. As usual, the code is available on GitHub.

Solving the Referential Integrity Problem

Sat, 01 Dec 2012 13:37:46 -0600

“A man with a watch knows what time it is. A man with two watches is never sure.”

I’ve been developing my open source Android framework, Infinitum, for the better part of 10 months now. It has brought about some really interesting problems that I’ve had to tackle, which is one of the many reasons I enjoy working on it so much.

Chicken or the Egg

Although it’s much more now, Infinitum began as an object-relational mapper which was loosely modeled after Hibernate. One of the first major issues I faced while developing the ORM component was loading object graphs. To illustrate what I mean by this, suppose we’re developing some software for a department store. The domain model for this software might look something like this:

Hashmaps on Brave New Geek

Fast Topic Matching

Solving the Referential Integrity Problem

Chicken or the Egg