Developer Champion
Saturday was, hopefully, my last conference of the year. My lucky readers should start to see some posts which are not simply me gushing about another opportunity to hang out with awesome people and learn about interesting "stuff".
Last Tuesday Mike and I unveiled our brand shiny new presentation: Understanding the Disruptor, a Beginner's Guide to Hardcore Concurrency. This was a preview of the talk we'll be doing at JAX London on the 2nd November.
A video of the session is available, as are the slides. I promise not to say "so" anywhere near as many times when I repeat my performance at JAX (is there anything more painful than watching yourself on video?).
Continue reading "Mike and I debut our new Disruptor presentation"
Having been back in London for a few days I've had some time to digest the madness that was last week.
My lasting impression of JavaOne is almost entirely positive. Granted, it was my first major conference, so maybe I'm just not jaded yet. But let me tell you what I loved about it (yes, I did cover some of these in my last post):
Here's a photo of me representing LMAX as I pick up the Duke Award we won for the Disruptor (Thanks to Martijn for taking the photo)
So I've been at JavaOne for the better part of three days, it's time to record some of my observations so far:
Martin recently announced version 2.0 of the Disruptor - basically there have been so many changes since we first open-sourced it that it's time to mark that officially. His post goes over all the changes, the aim of this article is to attempt to translate my previous blog posts into new-world-speak, since it's going to take a long time to re-write each of them all over again. Now I see the disadvantage of hand-drawing everything.
My recent slow-down in posting is because I've been trying to write a post explaining memory barriers and their applicability in the Disruptor. The problem is, no matter how much I read and no matter how many times I ask the ever-patient Martin and Mike questions trying to clarify some point, I just don't intuitively grasp the subject. I guess I don't have the deep background knowledge required to fully understand.
So, rather than make an idiot of myself trying to explain something I don't really get, I'm going to try and cover, at an abstract / massive-simplification level, what I do understand in the area. Martin has written a post going into memory barriers in some detail, so hopefully I can get away with skimming the subject.
Continue reading "Dissecting the Disruptor: Demystifying Memory Barriers"
We mention the phrase Mechanical Sympathy quite a lot, in fact it's even Martin's blog title. It's about understanding how the underlying hardware operates and programming in a way that works with that, not against it.
We get a number of comments and questions about the mysterious cache line padding in the RingBuffer, and I referred to it in the last post. Since this lends itself to pretty pictures, it's the next thing I thought I would tackle.
Continue reading "Dissecting the Disruptor: Why it’s so fast (part two) – Magic cache line padding"
Martin Fowler has written a really good article describing not only the Disruptor, but also how it fits into the architecture at LMAX. This gives some of the context that has been missing so far, but the most frequently asked question is still "What is the Disruptor?".
I'm working up to answering that. I'm currently on question number two: "Why is it so fast?".
Continue reading "Dissecting the Disruptor: Why it’s so fast (part one) – Locks Are Bad"
So now I've covered the ring buffer itself, reading from it and writing to it.
Logically the next thing to do is to wire everything up together.
I talked about multiple producers - they have the producer barrier to keep them in order and under control. I've talked about consumers in a simple situation. Multiple consumers can get a little more involved. We've done some clever stuff to allow the consumers to be dependent on each other and the ring buffer. Like a lot of applications, we have a pipeline of things that need to happen before we can actually get on with the business logic - for example, we need to make sure the messages have been journalled to disk before we can do anything.
The Disruptor paper and the performance tests cover some basic configurations that you might want. I'm going to go over the most interesting one, mostly because I needed the practice with the graphics tablet.
Continue reading "Dissecting the Disruptor: Wiring up the dependencies"
This is the missing piece in the end-to-end view of the Disruptor. Brace yourselves, it's quite long. But I decided to keep it in a single blog so you could have the context in one place.
The important areas are: not wrapping the ring; informing the consumers; batching for producers; and how multiple producers work.
Continue reading "Dissecting the Disruptor: Writing to the ring buffer"