In the recent open-source data landscape, Valkey has emerged as a prominent player. Born as a Linux Foundation-backed, fully open-source fork of Redis (following Redis’s recent licensing changes), Valkey serves as a high-performance, in-memory key-value data store. Whether Valkey is deployed as a primary database, an ephemeral cache, or a rapid message broker, a single … Continued

The post Scaling Your Cache: A Step-by-Step Guide to Setting Up Valkey Replication appeared first on Percona.

We’re thrilled to welcome the open source database community back in person for Percona Live 2026, taking place May 27–29 in the Bay Area. After the energy of past events, there’s nothing like being together again — swapping war stories over coffee, sketching architectures on napkins, and learning from the people building and running databases … Continued

The post Percona Live 2026 is Back in the Bay Area — Here’s Why You Don’t Want to Miss It appeared first on Percona.

Can Your Data Stack Run DOOM?

TL;DR

We ran a multiplayer DOOM server in pure SQL on different data stack architectures, recorded nice videos, and measured what breaks first.

• Click here to go directly to the benchmark page (with videos).
• Here is a video of CedarDB being “DOOMbench”-ed:

Why DOOM?

Pedantic note: The original DOOMQL uses raycasting, not BSP trees, making it technically more Wolfenstein than DOOM as some people pointed out.

Last year, we published DOOMQL: a multiplayer DOOM-like game running entirely inside SQL, using recursive CTEs for raycasting and a real client-server architecture where players connect directly to the database. We were very excited when it hit the front page of Hacker News. DOOMBench builds on DOOMQL and turns it into a stress-test for different data stacks. Latency numbers and throughput charts are easy to report but hard to feel. A video of a database struggling to render a video game frame can be felt instantly!

Let’s look at the three dimensions DOOMQL covers:

• Raw analytical performance: DOOMQL uses recursive CTEs to render a game world in ASCII-art using raycasting. That’s as number-crunchy as SQL gets! Some might interject that this is not a workload representative of the real world, to which I reply: Might be true, but there’s precedence.
• Transaction Processing: DOOMQL uses a client-server architecture. Clients connect directly to the database and insert their inputs into an inputs table: WASD to move, X to shoot. That’s not going to be a lot of transactions (think: 10 players sending an input every 200 ms each -> 50 transactions per second), but latency is a big issue here. If you’ve ever played a multiplayer shooter with a high ping, you understand. Furthermore, there’s also a game loop the server has to run multiple times a second. This can reach from 100 ticks per minute (Runescape) to 128 ticks per second (Valorant).
• Atomicity: Nothing feels worse than being shot by a player who was already dead on your screen. Good database systems can execute transactions in an atomic fashion. Either everything applies or nothing: There cannot be a player that has 0 or less hitpoints but hasn’t been killed and respawned yet. This is not really a metric to measure: It either applies, or it doesn’t. Fortunately, nearly all serious databases implement such ACID guarantees nowadays.

The interesting part is that it’s very hard for your data stack to be good at both analytical and transactional processing. Analytics wants to crunch a lot of data and is usually bandwidth-bound (memory, disk, caches). Transaction processing wants writes to feel snappy and is usually latency-bound. Both approaches traditionally use different data layouts, data structures, and system architecture. Let’s explore them in the context of DOOM!

What does a database running DOOM look like?

Let’s use Postgres as an example. Here’s a video of it running DOOMbench:

Let’s go over what we see:

• The main view on the left shows the player’s view: The Raycasted game view itself, a minimap with the player’s sight cone, and a score screen including player health, ammo and kill count.
• Right of that is a minimap of the world state. It’s the state Postgres is currently in, i.e. not the view rendered to the client, but the state of the database at the current tick.
• Below that, we see the player input, as well as some performance numbers: the server tick rate and FPS, both current and as historic chart.

So how does data flow through the whole system? Let’s look at what has to happen for a new view to be rendered:

1. Inputs

The player presses ‘W’ which appends a new row into the inputs table: insert into inputs(player_id, action, timestamp) values (47, 'W', now()).

2. Game tick

The next game tick will then read that row, update the player position (as long as the player isn’t dead, the move is blocked by walls, etc…). We limit the server tick rate to 35 ticks per second (same as the original DOOM). Ticks and input are processed in lockstep.

3. Rendering

Clients can request a frame by querying a view that does all the raycasting behind the scenes on demand: select full_row from frames_by_row where player_id = 47 order by f.row asc (see here)

The rendering loop is decoupled from the game tick loop (game design 101), leading to a true mixed workload: Every client wants to maximize its own FPS (no VSYNC here, analytical workload), while the server also must be able to still process all the inputs at 35 ticks per second (transactional workload), while players continue to happily send their inputs whenever they please (transactional workload #2).

Since rendering is very(!) expensive and the game tick loop is decoupled from the rendering loop, the player’s view can get heavily out of sync with the game state.

You can see this with Postgres above: While it can process about 10 ticks per second, rendering the view takes multiple seconds. The player makes a smooth 360 degree turn, as visible in the game state, but the output never catches up. While the server already knows what happened to you, you don’t! I think we can all agree that a Counter-Strike pro wouldn’t call this playable.

OLAP: Let’s do everything in a data lake!

Okay, while Postgres is a battle-tested database system, it doesn’t seem to be a fit for our workload since it’s just not fast enough to push enough frames per second. Let’s go with a pure OLAP system instead! They are purpose-built to answer complex analytical queries, so they should push a lot of FPS, right?

True, but unfortunately they are really bad at transactions and usually don’t have a way to do live transaction processing at all. Where’s the value in rendering a lot of frames if your input isn’t processed?

There are two ways to get around this limitation: Two systems with an ETL pipeline in between (Extract-Transform-Load) or what I come to lovingly call the nesting doll approach. Let’s look at both approaches!

ETL

The concept is pretty simple: Let’s use a system that’s really good at transaction processing, and a system that is really good at analytical processing and insert a pipeline in the middle replicating the data (a so-called ETL pipeline). The transactional system can focus on processing all the player inputs and running the game loop, the analytical system can push the FPS.

Here’s such a set up. It uses Postgres as transactional system and DuckDB as analytical system. A simple CDC (Change-Data-Capture) loop runs once a second and copies over all tables from Postgres to DuckDB. Here’s the result:

Doesn’t look much better, right?

But it is, in fact, much better! If you look at the chart, you see that DuckDB pushes a respectable 10 FPS (compared to the 0.3 of Postgres). But since the game state visible to DuckDB only updates once a second, 9 out of 10 frames just render the same view!

This system split is awesome for raw analytical performance but kind of useless if you need a tight feedback loop.

The Nesting Doll Approach

Having a second system and an ETL pipeline sucks: Apart from the replication lag we just encountered, you also have to maintain multiple systems and the pipeline in-between. There is another approach, though: If everyone really likes using Postgres, but Postgres is not fast enough on analytical workloads, why not just co-locate a fast analytical engine inside your Postgres?

One such approach is pg_clickhouse which we already discussed in a previous post on this blog. It provides access to the ClickHouse database engine from inside Postgres and can push table scans to the far more capable ClickHouse analytical engine. Here’s DOOMQL on pg_clickhouse:

As you can see, it shows more or less the same performance as Postgres. Improving table scan performance with modern engines is great, but in the end it’s still the Postgres query optimizer and execution engine which are the bottleneck. This is especially true for DOOMQL where tables are pretty small (so no need for fast table scans), but the queries themselves are very complex.

HTAP is the holy grail?

So far, we’ve seen: Postgres handling transactions, but being unable to push frames. DuckDB behind an ETL pipeline pushes frames but renders stale state. Bolting a fast analytical engine onto Postgres doesn’t help because the bottleneck is query executor (and its execution model), not the bare table scans.

So what if your database was just good at both? That’s the premise of HTAP (Hybrid Transactional/Analytical Processing). A database built in such a way that it can handle writes with low latency and run complex analytical queries on the same data in parallel, without an ETL pipeline. So no replication lag, stale reads, and especially no second system and data pipeline to maintain.

But if that’s so desirable, why aren’t all systems like that? For most of database history, the hardware made you pick “either/or”. OLTP systems were designed around the catastrophically bad random I/O of spinning disks. To make sure that a newly inserted record touched as few different places on disk as possible, OLTP systems are usually row-oriented and new rows are just appended.

OLAP systems obviously have to work with the same disk limitations, but want to scan a lot of data in as few reads as possible. Since they were usually bottlenecked by the measly throughput of an HDD, they try to read as little as possible, making extensive use of compression schemes and structuring their data layout into columns. A query usually doesn’t touch all fields of a record (I’m not interested in the player’s password if I just want to find out their position on the map), this massively reduces the amount of data to be scanned. Unfortunately, this is terrible for OLTP: Adding a new row now means I have to update all its columns which are spread all over the disk.

Nowadays, entire companies exist just to move data between OLTP and OLAP systems. But the foundational assumptions of those systems don’t hold anymore: A single server can have dozens of cores and hundreds of GiB of RAM, which is often enough to keep your entire hot dataset in main memory. Modern NVMe SSDs can do hundreds of thousands of random IOPS. So, the hardware constraint that forced specialization is mostly gone, but most database architectures haven’t caught up. They are still organized around the tradeoffs of the 90’s and 00’s. CedarDB is built from scratch for the new reality: The storage layer, query optimizer, and execution engine are all designed to handle both workloads natively. Instead of bolting an analytical engine onto a transactional one or vice versa, CedarDB follows one coherent architecture that assumes fast storage, abundant DRAM, and many cores.

But enough theory, let’s see CedarDB in action:

The difference is immediately visible. CedarDB can push ~30 FPS at 30 ticks per second without replication lag: Each frame shows the current system state. DOOMbench records a median lag of 44 milliseconds, meaning it takes just 44 milliseconds for a keypress to lead to an observable outcome. Still not enough for a counterstrike pro, but enough to actually play the game!

Is it a contrived workload? Absolutely! But the underlying pattern (make observations on fresh data) shows up everywhere. Take for example dashboards, interactive analytics, or AI agents acting on their own decisions.

The DOOMbench web page

But enough about the videos. Head over to cedardb.com/doombench for the full result table, or keep reading for a summary.

DOOMbench measures four things:

• Tickrate: This is a pure OLTP measurement without rendering any frames. Four players move around and shoot while the server processes game ticks as fast as possible. How fast can your database run the game loop?
• Static FPS: This is a pure OLAP measurement without any movement or ticks. Four players query their rendered view as fast as possible. This is raw analytical query throughput. We sum up and report the FPS of all four clients.
• Median Lag: The metric every eSports gamer cares about. Time from button press to the rendered view reflecting that input. This captures OLTP performance, OLAP performance and replication lag in a single number.
• DOOMscore: The HTAP benchmark. Four clients playing the actual game with the game loop ticking at 35 Hz (original DOOM tick rate). How many combined frames per second can the database render while keeping up with the game loop? Systems that can’t sustain 35 ticks per second get penalized proportionally: If you only manage half the tick rate, your DOOMscore halves too.

Each system runs the same DOOMQL codebase on the same hardware. Since some systems (CockroachDB, DuckDB) have slight syntax deviations, DOOMbench allows you to declare database-specific SQL overrides. Apart from the four numbers above, DOOMbench also records a video replay of the same scene for every database which you can watch.

DOOMbench is open source. If you want to add additional systems, feel free to open a PR! DOOMbench currently only works with Postgres-compatible systems, but we’d like to add other systems as well.

Should I care?

About the benchmark?

Probably not! Every vendor pushes their own benchmark where they are the best to the surprise of absolutely no one. This benchmark isn’t different: It uses very obscure and arcane SQL features like recursion and very involved string manipulation. But it generates videos that make the tradeoffs in your data stack instantly visible. That has to account for something, right?

About HTAP?

Depends on your workload. Plenty of workloads are fine with stale data. If you’re reading the report after having a coffee anyway, you probably don’t care if it’s five minutes out of date. If you want to make decisions without a human in the loop, or use your database for interactive workloads (you, your customer, or your AI agent changes some parameter and expect an instant result), HTAP is a game changer! Or if you want to play DOOM, I guess…

What’s next?

We’re open-sourcing DOOMbench. Missing a system? Unhappy with our methodology? Open a pull request! I’m also working on a BSP tree implementation in recursive SQL, so we’ll hopefully have a real DOOM inside SQL soon.

If you want to run DOOMbench yourself, you can check out the code here. All database systems are dockerized and should work out of the box. Want to try out CedarDB in your own stack? Get started here or get in touch

We recently looked at how various open-source database engines maintain their secondary indexes (in a previous analysis) and found significant differences.  The maintenance of indexes is not the only aspect where storage engines differ, another significant difference is how they handle simple row updates.  These updates highlight how these open-source databases organize data and manage … Continued

The post Impacts of updates in open-source databases appeared first on Percona.

The latest release of the Percona Operator for MySQL, 1.1.0, is here. It brings point-in-time recovery, incremental backups, zstd backup compression, configurable asynchronous replication retries, and a set of stability fixes. This post walks through the highlights and how they help your MySQL deployments on Kubernetes.   Percona Operator for MySQL 1.1.0 Running stateful databases … Continued

The post Percona Operator for MySQL 1.1.0: PITR, Incremental Backups, and Compression appeared first on Percona.

Introduction: Recently I was having a conversation with a DB Enthusiast, and he mentioned that when he was a fresher, he tuned an ETL/reporting query that was running for 8-10 hours via a nightly job by 1/3rd. He went to his manager, saying that he reduced the query execution time, thinking that the manager would … Continued

The post PostgreSQL Performance: Is Your Query Slow or Just Long-Running? appeared first on Percona.

Having a separate DR cluster for production databases is a modern day requirement or necessity for tech and other related businesses that rely heavily on their database systems. Setting up such a [DC -> DR] topology for Percona XtraDB Cluster (PXC), which is a virtually- synchronous cluster, can be a bit challenging in a complex … Continued

The post Deploying Cross-Site Replication in Percona Operator for MySQL (PXC) appeared first on Percona.

AskTom Live is a great source of information from Oracle developer advocates and product managers, but I recently came across a clickbait marketing title ("Not All Binary Protocols Are Created Equal: The Science Behind OSON's 529x Performance Advantage") which compares apples to oranges, and it's an opportunity to explain what BSON is, the binary JSON format used by MongoDB.

TL;DR: If you want to compare with OSON, the Oracle Database datatype for JSON, you should compare the Mutable BSON Document which is the structure that MongoDB uses to access documents, reading and updating individual fields. Raw BSON is closer to protobuf: a compact serialization format for disk or network transfer, with access metadata removed and no blocks or headers.

I've left the following comment to the YouTube video but it seems that it is not publicly visible, so here it is.

Let me explain how Oracle Database and MongoDB handle disk-based data access, and you will understand the different design purposes of OSON and BSON, and why you are not testing the right thing to compare them.

Oracle Database, like many traditional databases, uses the same format on disk (blocks) and in memory (buffers), and must store all transient metadata that helps access it in memory on persistent storage. This applies to table blocks (which contain a table directory, a row directory, and even lock flags, ITLs, that need to be cleaned up later), and the same idea was used for OSON (header, dictionary, sorted field IDs, offset arrays). Think of it as a mini database with its catalog, like the Oracle database has its dictionary and segment headers, which map physical extents and blocks. Then accessing the on-disk OSON structure directly makes sense — it's designed to be used through buffers that match the disk blocks.

But MongoDB with WiredTiger uses a smarter cache where the in-memory structures are optimized for RAM: adding pointers instead of disk offsets, building an Elements Vector for O(1) field access, and adding skiplists to navigate fields, all when data is loaded into the database cache. So there are two formats: the mutable BSON that the database actually works on in memory for query processing and updates, and the on-disk raw BSON that, on purpose, strips any unnecessary metadata and compresses it, to maximize the OS filesystem cache usage, and fits to the major advantage of MongoDB for documents: read/write a document in a single I/O.

The raw BSON is a serialization format for disk and network, not to be accessed partially, because MongoDB has a powerful mutable BSON format in memory with O(1) access through its Elements Vector indexing. The O(n) sequential scan, the "no partial updates" limitation, and the field position penalties you describe — those are properties of the serialization format, not how MongoDB actually processes queries. And by definition, the serialization format is read sequentially, even though BSON can jump between fields. Don't do that except when you need a full document. Use the MongoDB server and drivers to access BSON, and learn how to use it correctly.

With this understanding, you can see that the "529x performance" clickbait title comes from a mistake: you used raw BSON to access individual fields, bypassing everything MongoDB does when serving a query. It would be like using BBED to query Oracle Datafiles without going through the instance — no buffer cache, no row directory navigation, no dictionary lookups — and then concluding that Oracle's storage format is slow.

Notably, the original OSON VLDB paper (Liu et al., 2020) by Zhen Hua Liu doesn't make the claims this video does. That paper honestly compares OSON against Oracle's own JSON text storage, not against MongoDB's query processing. It compares encoding sizes with BSON, which is legitimate for a serialization format comparison (though it overlooks that BSON in MongoDB is compressed on disk and over the network). The paper authors understood they were comparing serialization formats and storage approaches within Oracle, not benchmarking MongoDB's actual runtime performance. I believe OSON is the optimal format for Oracle because it was integrated into the existing instance, cache, and securefiles, which were created a long time ago. Conversely, BSON is ideal for MongoDB, as it capitalizes on the document database's purpose and the WiredTiger architecture.

Jepsen’s analysis of MySQL 8.0.34 walked through a set of concurrency and isolation anomalies in InnoDB. MariaDB, which inherits the same codebase, took the report seriously and shipped a response: a new server variable called innodb_snapshot_isolation, turned on by default starting in 11.8. The announcement claims that with the flag enabled, Repeatable Read in MariaDB … Continued

The post MariaDB’s Snapshot Isolation: A Fix That Breaks More Than It Fixes appeared first on Percona.

Previously: New Jobs.

Some readers are undoubtedly upset that I have not devoted more space to the wonders of machine learning—how amazing LLMs are at code generation, how incredible it is that Suno can turn hummed melodies into polished songs. But this is not an article about how fast or convenient it is to drive a car. We all know cars are fast. I am trying to ask what will happen to the shape of cities.

The personal automobile reshaped streets, all but extinguished urban horses and their waste, supplanted local transit and interurban railways, germinated new building typologies, decentralized cities, created exurban sprawl, reduced incidental social contact, gave rise to the Interstate Highway System (bulldozing Black communities in the process), gave everyone lead poisoning, and became a leading cause of death among young people. Many parts of the US are highly car-dependent, even though a third of us don’t drive. As a driver, cyclist, transit rider, and pedestrian, I think about this legacy every day: how so much of our lives are shaped by the technology of personal automobiles, and the specific way the US uses them.

I want you to think about “AI” in this sense.

Some of our possible futures are grim, but manageable. Others are downright terrifying, in which large numbers of people lose their homes, health, or lives. I don’t have a strong sense of what will happen, but the space of possible futures feels much broader in 2026 than it did in 2022, and most of those futures feel bad.

Much of the bullshit future is already here, and I am profoundly tired of it. There is slop in my search results, at the gym, at the doctor’s office. Customer service, contractors, and engineers use LLMs to blindly lie to me. The electric company has hiked our rates and says data centers are to blame. LLM scrapers take down the web sites I run and make it harder to access the services I rely on. I watch synthetic videos of suffering animals and stare at generated web pages which lie about police brutality. There is LLM spam in my inbox and synthetic CSAM on my moderation dashboard. I watch people outsource their work, food, travel, art, even relationships to ChatGPT. I read chatbots lining the delusional warrens of mental health crises.

I am asked to analyze vaporware and to disprove nonsensical claims. I wade through voluminous LLM-generated pull requests. Prospective clients ask Claude to do the work they might have hired me for. Thankfully Claude’s code is bad, but that could change, and that scares me. I worry about losing my home. I could retrain, but my core skills—reading, thinking, and writing—are squarely in the blast radius of large language models. I imagine going to school to become an architect, just to watch ML eat that field too.

It is deeply alienating to see so many of my peers wildly enthusiastic about ML’s potential applications, and using it personally. Governments and industry seem all-in on “AI”, and I worry that by doing so, we’re hastening the arrival of unpredictable but potentially devastating consequences—personal, cultural, economic, and humanitarian.

I’ve thought about this a lot over the last few years, and I think the best response is to stop. ML assistance reduces our performance and persistence, and denies us both the muscle memory and deep theory-building that comes with working through a task by hand: the cultivation of what James C. Scott would call metis. I have never used an LLM for my writing, software, or personal life, because I care about my ability to write well, reason deeply, and stay grounded in the world. If I ever adopt ML tools in more than an exploratory capacity, I will need to take great care. I also try to minimize what I consume from LLMs. I read cookbooks written by human beings, I trawl through university websites to identify wildlife, and I talk through my problems with friends.

I think you should do the same.

Refuse to insult your readers: think your own thoughts and write your own words. Call out people who send you slop. Flag ML hazards at work and with friends. Stop paying for ChatGPT at home, and convince your company not to sign a deal for Gemini. Form or join a labor union, and push back against management demands that you adopt Copilot—after all, it’s for entertainment purposes only. Call your members of Congress and demand aggressive regulation which holds ML companies responsible for their carbon and digital emissions. Advocate against tax breaks for ML datacenters. If you work at Anthropic, xAI, etc., you should think seriously about your role in making the future. To be frank, I think you should quit your job.

I don’t think this will stop ML from advancing altogether: there are still lots of people who want to make it happen. It will, however, slow them down, and this is good. Today’s models are already very capable. It will take time for the effects of the existing technology to be fully felt, and for culture, industry, and government to adapt. Each day we delay the advancement of ML models buys time to learn how to manage technical debt and errors introduced in legal filings. Another day to prepare for ML-generated CSAM, sophisticated fraud, obscure software vulnerabilities, and AI Barbie. Another day for workers to find new jobs.

Staving off ML will also assuage your conscience over the coming decades. As someone who once quit an otherwise good job on ethical grounds, I feel good about that decision. I think you will too.

And if I’m wrong, we can always build it later.

And Yet…

Despite feeling a bitter distaste for this generation of ML systems and the people who brought them into existence, they do seem useful. I want to use them. I probably will at some point.

For example, I’ve got these color-changing lights. They speak a protocol I’ve never heard of, and I have no idea where to even begin. I could spend a month digging through manuals and working it out from scratch—or I could ask an LLM to write a client library for me. The security consequences are minimal, it’s a constrained use case that I can verify by hand, and I wouldn’t be pushing tech debt on anyone else. I still write plenty of code, and I could stop any time. What would be the harm?

Right?

… Right?

Many friends contributed discussion, reading material, and feedback on this article. My heartfelt thanks to Peter Alvaro, Kevin Amidon, André Arko, Taber Bain, Silvia Botros, Daniel Espeset, Julia Evans, Brad Greenlee, Coda Hale, Marc Hedlund, Sarah Huffman, Dan Mess, Nelson Minar, Alex Rasmussen, Harper Reed, Daliah Saper, Peter Seibel, Rhys Seiffe, and James Turnbull.

This piece, like most all my words and software, was written by hand—mainly in Vim. I composed a Markdown outline in a mix of headers, bullet points, and prose, then reorganized it in a few passes. With the structure laid out, I rewrote the outline as prose, typeset with Pandoc. I went back to make substantial edits as I wrote, then made two full edit passes on typeset PDFs. For the first I used an iPad and stylus, for the second, the traditional pen and paper, read aloud.

I circulated the resulting draft among friends for their feedback before publication. Incisive ideas and delightful turns of phrase may be attributed to them; any errors or objectionable viewpoints are, of course, mine alone.

New Brand. Same Independence. If you read today’s announcement, you know Percona has a lot to say about what’s broken in modern data infrastructure. Lock-in dressed up as openness. Costs that climb while control shrinks. Vendors who made “managed” mean giving up visibility instead of gaining it. When we decided to stop being quiet about … Continued

The post Why A Goat? appeared first on Percona.

This is an external post of mine. Click here if you are not redirected.