mlrd: DynamoDB-Compatible API on MySQL
Introducing mlrd (“mallard”) to the world: a DynamoDB-compatible API on MySQL.
Crazy, but it works really well and I’m confident it will help a lot of business save a lot of money.
Here’s why.
Introducing mlrd (“mallard”) to the world: a DynamoDB-compatible API on MySQL.
Crazy, but it works really well and I’m confident it will help a lot of business save a lot of money.
Here’s why.
This has results for the sysbench benchmark on a 2-socket, 24-core server. A post with results from 8-core and 32-core servers is here.
tl;dr
(QPS for some version) / (QPS for base version)
This has results for MySQL versions 5.6 through 9.5 with the Insert Benchmark on a small server. Results for Postgres on the same hardware are here.
tl;dr
| dbms | l.i0 | l.x | l.i1 | l.i2 | qr100 | qp100 | qr500 | qp500 | qr1000 | qp1000 |
|---|---|---|---|---|---|---|---|---|---|---|
| 5.6.51 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| 5.7.44 | 0.91 | 1.53 | 1.16 | 1.09 | 0.83 | 0.83 | 0.83 | 0.84 | 0.83 | 0.83 |
| 8.0.44 | 0.60 | 2.42 | 1.05 | 0.87 | 0.69 | 0.62 | 0.70 | 0.62 | 0.70 | 0.62 |
| 8.4.7 | 0.59 | 2.54 | 1.04 | 0.86 | 0.68 | 0.61 | 0.68 | 0.61 | 0.67 | 0.60 |
| 9.4.0 | 0.59 | 2.57 | 1.03 | 0.86 | 0.69 | 0.62 | 0.69 | 0.62 | 0.70 | 0.61 |
| 9.5.0 | 0.59 | 2.61 | 1.05 | 0.85 | 0.69 | 0.62 | 0.69 | 0.62 | 0.69 | 0.62 |
| dbms | l.i0 | l.x | l.i1 | l.i2 | qr100 | qp100 | qr500 | qp500 | qr1000 | qp1000 |
|---|---|---|---|---|---|---|---|---|---|---|
| 5.6.51 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| 5.7.44 | 0.91 | 1.42 | 1.52 | 1.78 | 0.84 | 0.92 | 0.87 | 0.97 | 0.93 | 1.17 |
| 8.0.44 | 0.62 | 2.58 | 1.56 | 1.81 | 0.76 | 0.88 | 0.79 | 0.99 | 0.85 | 1.18 |
| 8.4.7 | 0.60 | 2.65 | 1.54 | 1.82 | 0.74 | 0.87 | 0.77 | 0.98 | 0.82 | 1.17 |
| 9.4.0 | 0.61 | 2.68 | 1.52 | 1.76 | 0.75 | 0.86 | 0.80 | 0.97 | 0.85 | 1.16 |
| 9.5.0 | 0.60 | 2.75 | 1.53 | 1.73 | 0.75 | 0.87 | 0.79 | 0.97 | 0.84 | 1.17 |
This has results for Postgres versions 12.22 through 18.1 with the Insert Benchmark on a small server.
Postgres continues to be boring in a good way. It is hard to find performance regressions.
tl;dr for a cached workload
This post has results for RocksDB on an Arm server. I previously shared results for RocksDB performance using gcc and clang. Here I share results using clang with LTO.
RocksDB is boring, there are few performance regressions.
tl;dr
Software
I used RocksDB versions 6.29, 7.0, 7.10, 8.0, 8.4, 8.8, 8.11, 9.0, 9.4, 9.8, 9.11 and 10.0 through 10.8.
I compiled each version clang version 18.3.1 with link-time optimization enabled (LTO). The build command line was:
flags=( DISABLE_WARNING_AS_ERROR=1 DEBUG_LEVEL=0 V=1 VERBOSE=1 )# for clang+LTOAR=llvm-ar-18 RANLIB=llvm-ranlib-18 CC=clang CXX=clang++ \make USE_LTO=1 "${flags[@]}" static_lib db_bench
I used a small Arm server from the Google cloud running Ubuntu 22.04. The server type was c4a-standard-8-lssd with 8 cores and 32G of RAM. Storage was 2 local SSDs with RAID 0 and ext-4.
Benchmark
Overviews on how I use db_bench are here and here.
The benchmark was run with 1 thread and used the LRU block cache.
Tests were run for three workloads:
Relative QPS
Many of the tables below (inlined and via URL) show the relative QPS which is:
(QPS for my version / QPS for RocksDB 6.29)
The base version varies and is listed below. When the relative QPS is > 1.0 then my version is faster than RocksDB 6.29. When it is < 1.0 then there might be a performance regression or there might just be noise.
The spreadsheet with numbers and charts is here. Performance summaries are here.
Results: byrx
This has results for by byrx workload where the database is cached by RocksDB.
RocksDB 10.x is faster than 6.29 for all tests.
Results: iobuf
This has results for by iobuf workload where the database is larger than RAM and RocksDB used buffered IO.
Performance in RocksDB 10.x is about the same as 6.29 except for overwrite. I think the performance decreases in overwrite that arrived in versions 7.x and 8.x are from new correctness checks and throughput in 10.8 is 7% less than in 6.29. The big drop for fillseq in 10.6.2 was from bug 13996.
Results: iodir
This has results for by iodir workload where the database is larger than RAM and RocksDB used O_DIRECT.
Performance in RocksDB 10.x is about the same as 6.29 except for overwrite. I think the performance decreases in overwrite that arrived in versions 7.x and 8.x are from new correctness checks and throughput in 10.8 is 7% less than in 6.29. The big drop for fillseq in 10.6.2 was from bug 13996.
I drive my daughter to school as part of a car pool. Along the way, I am learning a new language, Brainrot.
So what is brainrot? It is what you get when you marinate your brain with silly TikTok, YouTube Shorts, and Reddit memes. It is slang for "my attention span is fried and I like it". Brainrot is a self-deprecating language. Teens are basically saying: I know this is dumb, but I am choosing to speak it anyway.
What makes brainrot different from old-school slang is its speed and scale. When we were teenagers, slang spread by word of mouth. It mostly stayed local in our school hallways or neighborhood. Now memes go global in hours. A meme is born in Seoul at breakfast and widespread in Ohio by six seven pm. The language mutates at escape velocity and gets weird fast.
Someone even built a brainrot programming language. The joke runs deep, and is getting some infrastructure.
Here are a few basic brainrot terms you will hear right away.
I can anticipate the reactions to this post. Teenagers will shrugg: "Obviously. How is this news?" Parents of teens will laugh in recognition. Everyone else will be lost and move away.
I have seen things. I am not 50 yet, but I am getting there. I usually write about distributed systems and databases. I did not plan to write this post. This post insisted on being written through me. We will return to our regularly scheduled programming.
But here is my real point. I think the kids are alright.
It is not uncommon for a generation to get declared doomed by the one before it, yet Gen Z and Gen Alpha may have taken the heaviest hit, and written off as lost causes. But what I see is a generation with sharp self-mocking humor. They have short attention spans for things they do not care about. I think they do this out of sincerity. They don't see the purpose in mundane things, or and for many things theyfeel they lack enough agency. But what is new is how hard they can lock in (focus) on what they care about, and how fast they can form real bonds around shared interests. They are more open with each other. They are more inclusive by default. They are a feeling bunch. They waste no patience on things they find pointless. But when it matters, they show up fully.
From the outside, their culture may looks absurd and chaotic. But, under the memes, I see a group that feels deeply, adapts quickly, and learns in public. They are improvising in real time. And despite all predictions to the contrary, they might actually know what they are doing.
From a letter to Valve Corporation’s CEO Gabe Newell, lightly edited for formatting and links.
Dear Mr. Newell,
Steam has been my main source for games for over twenty years. I am disheartened that you chose not to publish Santa Ragione’s recently released game, Horses.
I’ve read some substantive critique; Polygon and Rock Paper Shotgun’s among them. I have also read Santa Regione’s discussion of the ban. I bought Horses on Itch and played it through; I found it enjoyable and thought provoking, though not transformative. I was surprised to find a much tamer experience than I had been led to believe. I am particularly concerned that Steam found this title dangerous enough to ban it.
Is Horses unsettling? Yes, though you would see far worse in popular horror films. I find Hostel or Saw gut-wrenching to near-unwatchable; Horses felt almost cute by comparison. It is nowhere near Argento’s classic Suspiria or Aster’s harrowing Midsommar, which deals with similar themes. The game’s pixelated censorship, silly animations, and cutting away from its worst moments comes across as coy, even camp. I suspect this is intentional: the game is in large part concerned with authoritarianism and the reproductive dynamics (in all senses) of cinema and games themselves. It is also concerned with complicity: the character’s choices to voice disgust or approval have apparently no impact on the story. Its four explicit themes—laid out in the embedded narrative of a VHS tape you must watch and decode to progress—are the repression of patriarchy, religion, chastity, and silence.
Steam has long been willing to publish works engaging with brutal dehumanization and authoritarian violence. Games like Amnesia: A Machine for Pigs or SOMA depict humans involuntarily warped, both physically and mentally, beyond recognition. Like Horses, Amnesia uses horror as a lens for the industrial revolution and the power of the wealthy. Valve’s own work has not shied away from horror; Half Life 2 is entirely about the visceral subjugation of political and bodily autonomy. Valve gave us the headcrab and the stalker—both instances of forcible objectification—and the game’s camera shies away from neither.
What, then, makes Horses unpublishable? Surely not violence, or you’d have pulled half the Steam catalogue by now. It can’t be body horror: I flinched far more at Dead Space 2’s eyeball scene. Could it be nudity? Half Life 2’s stalkers are fully nude and completely uncensored; I find the image of their mutilated bodies far more visually disturbing than the titular horses. Is it sex? Steam publishes the wildly popular Cyberpunk 2077, which has no shortage of breasts, vaginas, penises, and first-person sex scenes. It also depicts rape, torture, murder, and young boys hooded, restrained, and tortured as livestock on a farm. Could Horses be at fault for flagellation? Surely not; Steam published Robert Yang’s charming Hurt Me Plenty, where players directly spank a simulated consensusal partner. Is it complicity in authoriarian abuse? Lucas Pope’s highly-regarded Papers Please, also on Steam, puts players in the increasingly disturbing role of enforcing a fascist state’s border. It too contains pixelated nudity and violence.
I love cute, safe, and friendly games; Astroneer is my jam! And as an adult, I also enjoy challenging, even harrowing narratives. For me, Horses falls somewhere in the middle—one might call it the Animal Farm of fascist horror parables. I think Steam ought to publish it, and more transgressive works as well.
Yours truly,
Kyle Kingsbury