This has results for HammerDB tproc-c on a small server using MySQL and Postgres. I am new to HammerDB and still figuring out how to explain and present results so I will keep this simple and just share graphs without explaining the results.

tl;dr

• Modern Postgres is faster than old Postgres
• Modern MySQL has large perf regressions relative to old MySQL, and they are worst at low concurrency for CPU-bound worklads. This is similar to what I see on other benchmarks.
• Modern Postgres is about 2X faster than MySQL at low concurrency (vu=1) and when the workload isn't IO-bound (w=100). But with some concurrency (vu=6) or with more IO per transaction (w=1000, w=2000) they have similar throughput. Note that partitioning is used at w=1000 and 2000 but not at w=100.

Relational database joins are, conceptually, a cartesian product followed by a filter (the join condition). Without that condition, you get a cross join that returns every possible combination. In MongoDB, you can model the same behavior at read time using $lookup, or at write time by embedding documents.

Example

Define two collections: one for clothing sizes and one for gender-specific fits:

db.sizes.insertMany([  
  { code: "XS", neckCm: { min: 31, max: 33 } },  
  { code: "S",  neckCm: { min: 34, max: 36 } },  
  { code: "M",  neckCm: { min: 37, max: 39 } },  
  { code: "L",  neckCm: { min: 40, max: 42 } },  
  { code: "XL", neckCm: { min: 43, max: 46 } }  
]);

db.fits.insertMany([
  {
    code: "MEN",
    description: "Straight cut, broader shoulders, narrower hips"
  },
  {
    code: "WOMEN",
    description: "Tapered waist, narrower shoulders, wider hips"
  }
]);

Each collection stores independent characteristics, and every size applies to every fit. The goal is to generate all valid product variants.

Cross join on read: $lookup + $unwind

In order to add all sizes to each body shape, use a $lookup without filter condition and, as it adds them as an embedded array, use $unwind to get one document per combination:

db.sizes.aggregate([
  {
    $lookup: {
      from: "fits",
      pipeline: [],
      as: "fit"
    }
  },
  { $unwind: "$fit" },
  { $sort: { "fit.code": 1, code: 1 } },
  {
    $project: {
      _id: 0,
      code: { $concat: ["$fit.code", "-", "$code"] }
    }
  }
]);

Here is the result:

Application-side

For such small static reference collections, the application may simply read both and join with loops:

const sizes = db.sizes.find({}, { code: 1, _id: 0 }).sort({ code: 1 }).toArray();
const fits  = db.fits.find({},  { code: 1, _id: 0 }).sort({ code: 1 }).toArray();

for (const fit of fits) {
  for (const size of sizes) {
    print(`${fit.code}-${size.code}`);
  }
}

While it's good to keep the reference in a database, such static data can stay in cache in the application.

Cross join on write: embed the many-to-many

Because sizes are inherently tied to body shapes (no size exists without a body shape), embedding them in the fits documents is often a better model:

db.fits.aggregate([
  {
    $lookup: {
      from: "sizes",
      pipeline: [
        { $project: { _id: 0, code: 1, neckCm:1 } },
        { $sort: { code: 1 } }
      ],
      as: "sizes"
    }
  },
  {
    $merge: {
      into: "fits",
      on: "_id",
      whenMatched: "merge",
      whenNotMatched: "discard"
    }
  }
]);

Here is the new shape of the single collection:

Once embedded, the query becomes straightforward, simply unwind the embedded array:

db.fits.aggregate([
  { $unwind: "$sizes" },
  {
    $project: {
      _id: 0,
      code: {
        $concat: ["$code", "-", "$sizes.code"]
      }
    }
  }
]);

You may embed only the fields required, like the size code, or all fields like I did here with the neck size, and then remove the size collection:

db.sizes.drop()

Although this may duplicate the values for each body shape, it only requires using updateMany() instead of updateOne() when updating it. For example, the following updates one size:

db.fits.updateMany(  
  {},  
  { $set: { "sizes.$[s].neckCm": { min: 38, max: 40 } } },  
  {  
    arrayFilters: [  
      { "s.code": "M" }  
    ]  
  }  
); 

Duplication has the advantage of returning all required information in a single read, without joins or multiple queries, and it is not problematic for updates since it can be handled with a single bulk update operation. Unlike relational databases—where data can be modified through ad‑hoc SQL and business rules must therefore be enforced at the database level—MongoDB applications are typically domain‑driven, with clear ownership of data and a single responsibility for performing updates.

In that context, consistency is maintained by the application's service rather than by cross‑table constraints. This approach also lets business rules evolve, such as defining different sizes for men and women, without changing the data model.

Conclusion

In a fully normalized relational model, all relationships use the same pattern: a one-to-many relationship between two tables, enforced by a primary (or unique) key on one side and a foreign key on the other. This holds regardless of cardinality (many can be three or one million), lifecycle rules (cascade deletes or updates), ownership (shared or exclusive parent), navigation direction (and access patterns). Even many-to-many relationships are just two one-to-many relationships via a junction table.

MongoDB exposes these same concepts as modeling choices—handled at read time with $lookup, at write time through embedding, or in the application—instead of enforcing a single normalized representation. The choice depends on the domain data and access patterns.

Academic writing has long been criticized for its formulaic nature. As I wrote about earlier, research papers are unfortunately written to please 3 specific expert reviewers who are overwhelmingly from academia. Given this twisted incentive structure (looking impressive for peer-review), the papers end up becoming formulaic, defensive, and often inpenetrable. 

Ironically, this very uniformity makes it trivially easy for LLMs to replicate academic writing. It is easy to spot LLM use in personal essays, but I dare you to do it successfully in academic writing. 

Aside: Ok, I baited myself with my own dare. In general, it is very hard to detect LLM usage at the paragraph level in a research paper. But LLM usage in research papers becomes obvious when you see the same definition repeated 3-4 times across consecutive pages.  The memoryless nature of LLMs causes them to recycle the same terms and phrases, and I find myself thinking "you already explained this to me four times, do you think I am a goldfish?" I have been reviewing a lot of papers recently, and this is the number one tell-tale sign. A careful read by the authors would clean this up easily, making LLM usage nearly undetectable. To be clear, I am talking about LLM assistance in polishing writing, not wholesale generation. A paper with no original ideas is a different beast entirely. They are vacuous and easy to spot. 

Anyway, as LLM use become ubiquitous, conference/journal reviewing is facing a big crisis. There are simply too many articles being submitted, as it is easy to generate text and rush half-baked ideas into the presses. I am, of course, unhappy about this. Writing that feels effortless because an LLM smooths every step deprives you of the strain that produces "actual understanding". That strain in writing is not a defect; it creates the very impetus for discovering what you actually think, rather than faking/imitating thought.

But here we are. We are at an inflection point in academic publishing.  I recently came across this post, which documents an experiment where an LLM replicated and extended a published empirical political science paper with near-human fidelity, at a fraction of the time and cost.

I have been predicting the collapse of the publishing system for a decade. The flood of LLM-aided research might finally break its back. And here is where I want to take you in this post. I want to imagine how academic writing may change in this new publishing regime. Call it a 5-10 year outlook, because at this day and age, who can predict anything beyond that.

I claim that costly signals of genuine intelligence will become the currency of survival in this new environment.

Costly signals work because they are expensive to fake, like a peacock’s tail or an elk’s antlers. And I claim academic writing will increasingly demand features that are expensive to fake. Therefore, a distinctive voice becomes more valuable precisely because it cannot be generated without genuine intellectual engagement. Personal narratives, peculiar perspectives, unexpected conceptual leaps, and field-specific cultural fluency are things that require deep immersion and creative investment that LLMs lack. These are the costly signals that will make a paper worth publishing. 

Literature reviews are cheap to automate, so they will shrink --as we are already seeing. But reviews with distinctive voice and genuine insight, ones that reflect on the author's own learning and thought process, will survive. Work that builds creative frameworks and surprising connections, which are expensive to produce, will flourish. When anyone can generate competent prose, only writing that screams "a specific human spent serious time thinking about this" will cut through.

So, LLMs may accidentally force academia toward what it always claimed to value: original thinking and clear communication. The costliest signal of all is having something genuinely new to say, and saying well. I am an optimist, as you can easily tell, if you are a long time reader of this blog.

“Simplicity and elegance are unpopular because they require hard work and discipline to achieve and education to be appreciated.”

-- Edsger W. Dijkstra

Prisma is an ORM (Object-Relational Mapper). With MongoDB, it acts as an Object Document Mapper, mapping collections to TypeScript models and providing a consistent, type-safe query API.

MongoDB is a document database with a flexible schema. Prisma does not provide schema migrations for MongoDB, but it supports nested documents and embedded types to take advantage of MongoDB’s data locality.

This article walks through a minimal “Hello World” setup on a Docker environment:

• Run MongoDB as a replica set
• Connect to it using Prisma
• Insert a "Hello World" document
• Read and display all documents

Start MongoDB as a replica set

Prisma requires MongoDB to run as a replica set. While MongoDB supports many operations without transactions, Prisma relies on MongoDB sessions and transactional behavior internally, which are only available on replica sets.

Start MongoDB in a Docker container with replica set support enabled:

docker run --name mg -d mongo --replSet rs0

Initialize the replica set (a single‑node replica set is sufficient for local development and testing):

docker exec -it mg mongosh --eval "rs.initiate()"

Start a Node.js container

Start a Node.js container that can access MongoDB using the hostname mongo:

docker run --rm -it --link mg:mongo node bash

Prepare the Node.js environment

Update the package manager, install an editor, update npm, disable funding messages, and move to the working directory:

apt-get update
apt-get install -y vim
npm install -g npm@11.9.0
npm config set fund false
cd /home

Install Prisma Client and enable ES modules

Install Prisma Client and enable ES modules by adding "type": "module" to package.json:

npm install @prisma/client@6.19.0
sed -i '1s/{/{\n  "type": "module",/' package.json

Using ES modules enables standard import syntax and aligns the project with modern Node.js tooling.

Install Prisma CLI and TypeScript tooling

Install the Prisma CLI and supporting tooling, and generate the initial Prisma configuration:

npm install -D prisma@6.19.0 @types/node 
npm install -D tsx
npx prisma init

Configure the Prisma schema

Edit prisma/schema.prisma, change the provider from postgresql to mongodb, and define a minimal Message model:

generator client {
  provider = "prisma-client"
  output   = "../generated/prisma"
}

datasource db {
  provider = "mongodb"
  url      = env("DATABASE_URL")
}

model Message {
  id        String   @id @default(auto()) @map("_id") @db.ObjectId
  content   String
  createdAt DateTime @default(now())
}

Prisma maps MongoDB’s _id field to a String backed by an ObjectId.
The prisma-client generator produces TypeScript output in a custom directory to avoid using @prisma/client.

Configure the database connection

Define the MongoDB connection string in .env:

DATABASE_URL="mongodb://mongo:27017/test"

Prisma reads DATABASE_URL at generation time, while the application reads it at runtime. Importing dotenv/config ensures both environments are consistent.

Generate the Prisma client

Generate the Prisma client from the schema:

npx prisma generate

This produces TypeScript client files in generated/prisma.

Write the “Hello World” program

Create prisma/index.ts:

import 'dotenv/config'
import { PrismaClient } from '../generated/prisma/client.ts'

const prisma = new PrismaClient()

async function main() {
  await prisma.$connect()
  console.log('Connected to MongoDB')

  await prisma.message.create({
    data: {
      content: 'Hello World',
    },
  })

  const messages = await prisma.message.findMany()

  console.log('Messages in database:')
  for (const message of messages) {
    console.log(`- ${message.content} at ${message.createdAt}`)
  }
}

main()
  .catch(console.error)
  .finally(() => prisma.$disconnect())

This program connects to MongoDB, inserts a “Hello World” document, and prints all stored messages.

Run the program

For running TypeScript directly in modern Node.js projects, tsx is generally preferred over ts-node due to better ESM support and faster startup.

Execute the TypeScript file:

npx tsx prisma/index.ts

Output:

Connected to MongoDB
Messages in database:
- Hello World at Wed Feb 11 2026 17:36:08 GMT+0000 (Coordinated Universal Time)

Conclusion and final note on schemas in MongoDB

This example shows a minimal Prisma + MongoDB setup:

• MongoDB running as a replica set
• Prisma configured for MongoDB
• A single model with one insert and one read

From here, you can add schema evolution, indexes, and more complex queries while keeping the same core configuration.

MongoDB is often called schemaless, but that’s misleading in practice, as we started to declare the database schema in schema.prisma and generate the client for it. Real‑world MongoDB applications are schema‑driven, with structure defined in the application layer through models, validation rules, and access patterns.

Unlike relational databases—where the schema is enforced in the database and then mapped into the application—MongoDB uses the same document structure across all layers: in‑memory cache, on‑disk storage, and application models. This preserves data locality, avoids ORM overhead and migration scripts, and simplifies the development.

Prisma makes this explicit by defining the schema in code, providing type safety and consistency while keeping MongoDB’s document model flexible as your application evolves.

The crux of this chapter is how to schedule tasks without perfect knowledge. If you remember from the previous chapter, the core tension in CPU scheduling is these two conflicting goals:

• Minimizing Turnaround Time: Usually achieved by running shorter jobs first (SJF).
• Minimizing Response Time: Usually achieved by Round Robin scheduling (RR). Essential for interactive users.

Unfortunately, the OS does not have a crystal ball. It doesn't know if a process is a short interactive job or a massive number-crunching batch job. The Multi-Level Feedback Queue (MLFQ) solves this by encoding/capturing information from history of the job, and assumes that if a job has been CPU-intensive in the past, it likely will be in the future. As we'll see below, it also gives a chance for jobs to redeem themselves through the boosting process.

I really enjoyed this chapter. MLFQ, invented by Corbato in 1962, is a brilliant scheduling algorithm. This elegant solution served as the base scheduler for many systems, including BSD UNIX derivatives, Solaris, and Windows NT and subsequent Windows operating systems.

(This is part of our series going through OSTEP book chapters.)

How MLFQ Works: The Basic Rules

The chapter constructs the MLFQ algorithm iteratively, starting with a basic structure involving distinct queues, each with a different priority level.

• Rule 1: If Priority(A) > Priority(B), A runs.
• Rule 2: If Priority(A) = Priority(B), they run in Round-Robin.

But how does a job get its priority?

• Rule 3: New jobs start at the highest priority.
• Rule 4 (Initial Version): If a job uses up its time allotment, it moves down a queue. If it gives up the CPU (e.g., for I/O) before the time is up, it stays at the same priority.

This setup cleverly approximates Shortest Job First. Because the scheduler assumes every new job is short (giving it high priority), true short jobs finish quickly. Long jobs eventually exhaust their time slices and sink to the bottom queues, where they run only when the system isn't busy with interactive tasks.

Patching the initial MLFQ rules

However, this basic version has fatal flaws.

• If too many interactive jobs flood the system, low-priority background jobs might starve.
• A clever user could rewrite a program to yield the CPU (say through I/O: writing to a dummy file) just before its time slice ends. This resets the job's allotment, allowing it to monopolize the CPU at the highest priority.
• A job that starts CPU-intensive but becomes interactive later (like a compiler finishing and waiting for input) would be stuck at the bottom priority.

To fix these issues, the chapter introduces two crucial modifications.

The Priority Boost: To prevent low-priority jobs from starving, the scheduler employs Rule 5: After a set time period (S), all jobs are moved back to the topmost queue. This "boost" ensures that CPU-bound jobs get at least some processing time and allows jobs that have become interactive to return to a high-priority state.

Better Accounting: To stop users from gaming the system, the scheduler rewrites Rule 4 regarding how it tracks time. Rule 4: Instead of resetting the allotment every time a job yields the CPU, the scheduler tracks the total time a job uses at a given priority level. Once the allotment is used up (regardless of how many times the job yielded the CPU) it is demoted.

Tuning MLFQ

The remaining piece of the puzzle is parameterization. An MLFQ requires choosing the number of queues, the time slice length for each, and the frequency of the priority boost. There are no easy answers to these questions, and finding a satisfactory balance often requires deep experience with specific workloads. For example, most implementations employ varying time-slice lengths, assigning short slices (e.g., 10 ms) to high-priority queues for responsiveness and longer slices (e.g., 100s of ms) to low-priority queues for efficiency. Furthermore, the priority boost interval is often referred to as a "voodoo constant" because it requires magic to set correctly; if the value is too high, jobs starve, but if it is too low, interactive performance suffers.

MLFQ is a milestone in operating systems design. It delivers strong performance for interactive jobs without prior knowledge of job length, while remaining fair to long-running tasks. As noted earlier, it became the base scheduler for many operating systems, with several variants refining the core idea. One notable variant is the decay-usage approach used in FreeBSD 4.3. Instead of using fixed priority tables (as in Solaris), it computes priority using a mathematical function of recent CPU usage. Running increases a job’s usage counter and lowers its priority, while the passage of time decays this counter. Decay plays the same role as periodic priority boosts. As usage fades, priority rises, ensuring long-running jobs eventually run and allowing jobs that shift from CPU-bound to interactive to regain high priority.

TLA+ model  

I used Gemini to write a TLA+ model of the MLFQ algorithm here. To run this MLFQ TLA+ model at Spectacle for visualization, click this link and it will open the model on your browser, no installation or plugin required. What you will see is the initial state. Click on any enabled action to take it, you can go back and forward on the right pane to explore the execution. And you can share a URL back with anyone to point to an interesting state or trace, just like I did here.

MongoDB guarantees durability—the D in ACID—over the network with strong consistency—the C in the CAP theorem—by default. It still maintains high availability: in the event of a network partition, the majority of nodes continue to serve consistent reads and writes transparently, without raising errors to the application.

A consensus protocol based on Raft is used to achieve this at two levels:

• Writes are directed to the shard's primary, which coordinates consistency between the collection and the indexes. Raft is used to elect one replica as primary, with the others acting as secondaries.
• Writes to the shard's primary are replicated to the secondaries and acknowledged once a majority has guaranteed durability on persistent storage. The equivalent of the Raft log is the data itself—the transaction oplogs.

It's important to distinguish the two types of consensus involved: one for controlling replica roles and one for the replication of data itself. By comparison, failover automation around monolithic databases like PostgreSQL can use a consensus protocol to elect a primary (as Patroni does), but replication itself is built into PostgreSQL and does not rely on a consensus protocol—a failure in the middle may leave inconsistency between replicas.

Trade-offs between performance and consistency

Consensus on writes increases latency, especially in multi-region deployments, because it requires synchronous replication and waiting on the network, but it guarantees no data loss in disaster recovery scenarios (RPO = 0). Some workloads may prefer lower latency and accept limited data loss (for example, a couple of seconds of RPO after a datacenter burns). If you ingest data from IoT devices, you may favor fast ingestion at the risk of losing some data in such a disaster. Similarly, when migrating from another database, you might prefer fast synchronization and, in case of infrastructure failure, simply restart the migration from before the failure point. In such cases, you can use {w:1} write concern in MongoDB instead of the default {w:"majority"}.

Most failures are not full-scale disasters where an entire data center is lost, but transient issues with short network disconnections. With {w:1}, the primary risk is not data loss—because writes can be synchronized eventually—but split brain, where both sides of a network partition continue to accept writes. This is where the two levels of consensus matter:

• A new primary is elected, and the old primary steps down, limiting the split-brain window to a few seconds.
• With the default {w:"majority"}, writes that cannot reach a majority are not acknowledged on the side of the partition without a quorum. This prevents split brain. However, with {w:1}, those writes are acknowledged until the old primary steps down.

Because the failure is transient, when the old primary rejoins, no data is physically lost: writes from both sides still exist. However, these writes may conflict, resulting in a diverging database state with two branches. As with any asynchronous replication, this requires conflict resolution. MongoDB handles this as follows:

• Writes from the new primary are preserved, as this is where the application has continued to make progress.
• Writes that occurred on the old primary during the brief split-brain window are rolled back, and it pulls the more recent writes from the new primary.

Thus, when you use {w:1}, you accept the possibility of limited data loss in the event of a failure. Once the node is back, these writes are not entirely lost, but they cannot be merged automatically. MongoDB stores them as BSON files in a rollback directory so you can inspect them and perform manual conflict resolution if needed.

This conflict resolution is a Recover To a Timestamp (RTT).

Demo on a Docker lab

Let's try it. I start 3 containers as a replica set:

docker network create lab
docker run --network lab --name m1 --hostname m1 -d mongo --replSet rs0
docker run --network lab --name m2 --hostname m2 -d mongo --replSet rs0
docker run --network lab --name m3 --hostname m3 -d mongo --replSet rs0
docker exec -it m1 mongosh --eval '
rs.initiate({
  _id: "rs0",
  members: [
    { _id: 0, host: "m1:27017", priority: 3 },
    { _id: 1, host: "m2:27017", priority: 2 },
    { _id: 2, host: "m3:27017", priority: 1 }
  ]
})
'
until
docker exec -it m1 mongosh --eval "rs.status().members.forEach(m => print(m.name, m.stateStr))" |
 grep -C3 "m1:27017 PRIMARY"
do sleep 1 ; done

The last command waits until m1 is the primary, as set by its priority. I do that to make the demo reproducible with simple copy-paste.

I insert "XXX-10" when connected to m1:

docker exec -it m1 mongosh --eval '
  db.demo.insertOne(
   { _id:"XXX-10" , date:new Date() },
   { writeConcern: {w: "1"}    }
)
'

{ acknowledged: true, insertedId: 'XXX-10' }

I disconnect the secondary m2:

docker network disconnect lab m2

With a replication factor of 3, the cluster is resilient to one failure and I insert "XXX-11", when connected to the primary:

docker exec -it m1 mongosh --eval '
  db.demo.insertOne(
   { _id:"XXX-11" , date:new Date() },
   { writeConcern: {w: "1"}    }
)
'

{ acknowledged: true, insertedId: 'XXX-11' }

I disconnect m1, the current primary, and reconnect m2, and immediately insert "XXX-12", still connected to m1:

docker network disconnect lab m1
docker network    connect lab m2

docker exec -it m1 mongosh --eval '
  db.demo.insertOne(
   { _id:"XXX-12" , date:new Date() },
   { writeConcern: {w: "1"}    }
)
'

{ acknowledged: true, insertedId: 'XXX-12' }

Here, m1 is still a primary for a short period before it detects it cannot reach the majority of replicas and steps down. If the write concern was {w: "majority"} it would have waited and failed, not able to sync to the quorum, but with {w: "1"} the replication is asynchronous and the write is acknowledged when written to local disks.

Two seconds later, a similar write fails because the primary stepped down:

sleep 2

docker exec -it m1 mongosh --eval '
  db.demo.insertOne(
   { _id:"XXX-13" , date:new Date() },
   { writeConcern: {w: "1"}    }
)
'

MongoServerError: not primary

I wait that m2 is the new primary, as set by priority, and connect to it to insert "XXX-20":

until
docker exec -it m2 mongosh --eval "rs.status().members.forEach(m => print(m.name, m.stateStr))" |
 grep -C3 "m2:27017 PRIMARY"
do sleep 1 ; done

docker exec -it m2 mongosh --eval '
  db.demo.insertOne(
   { _id:"XXX-20" , date:new Date() },
   { writeConcern: {w: "1"}    }
)
'

{ acknowledged: true, insertedId: 'XXX-20' }

No nodes are down, it's only a network partition, and I can read from all nodes as long as I don't connect through the network. I query the collection on each side:

docker exec -it m1 mongosh --eval 'db.demo.find()'
docker exec -it m2 mongosh --eval 'db.demo.find()'
docker exec -it m3 mongosh --eval 'db.demo.find()'

The inconsistency is visible, "XXX-12" is only in m1 and "XXX-20" only in m2 and m3:

I reconnect m1 so that all nodes can communicate and synchronize their state:

docker network    connect lab m1

I query again and all nodes show the same values:

"XXX-12" has disappeared and all nodes are now synchronized to the current state. When it rejoined, m1 rolled back the operations that occurred during the split-brain window. This is expected and acceptable, since the write used a { w: 1 } write concern, which explicitly allows limited data loss in case of failure in order to avoid cross-network latency on each write.

The rolled back operations are not lost, MongoDB logged them in a rollback directory in the BSON format, with the rolled back document as well as the related oplog.

I read and decode all BSON in the rollback directory:

docker exec -i m1 bash -c '
for f in /data/db/rollback/*/removed.*.bson
do
 echo "$f"
 bsondump $f --pretty
done
' | egrep --color=auto '^|^/.*|.*("op":|"XXX-..").*'

The deleted document is in /data/db/rollback/0ae03154-0a51-4276-ac62-50d73ad31fe0/removed.2026-02-10T10-40-58.1.bson:

{
        "_id": "XXX-12",
        "date": {
                "$date": {
                        "$numberLong": "1770719868965"
                }
        }
}

The deleted oplog for the related insert is in /data/db/rollback/local.oplog.rs/removed.2026-02-10T10-40-58.0.bson:

{
        "lsid": {
                "id": {
                        "$binary": {
                                "base64": "erR2AoFXS3mbcX4BJSiWjw==",
                                "subType": "04"
                        }
                },
                "uid": {
                        "$binary": {
                                "base64": "47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=",
                                "subType": "00"
                        }
                }
        },
        "txnNumber": {
                "$numberLong": "1"
        },
        "op": "i",
        "ns": "test.demo",
        "ui": {
                "$binary": {
                        "base64": "CuAxVApRQnasYlDXOtMf4A==",
                        "subType": "04"
                }
        },
        "o": {
                "_id": "XXX-12",
                "date": {
                        "$date": {
                                "$numberLong": "1770719868965"
                        }
                }
        },
        "o2": {
                "_id": "XXX-12"
        },
        "stmtId": {
                "$numberInt": "0"
        },
        "ts": {
                "$timestamp": {
                        "t": 1770719868,
                        "i": 1
                }
        },
        "t": {
                "$numberLong": "1"
        },
        "v": {
                "$numberLong": "2"
        },
        "wall": {
                "$date": {
                        "$numberLong": "1770719868983"
                }
        },
        "prevOpTime": {
                "ts": {
                        "$timestamp": {
                                "t": 0,
                                "i": 0
                        }
                },
                "t": {
                        "$numberLong": "-1"
                }
        }
}

Conclusion: beyond Raft

By default, MongoDB favors strong consistency and durability: writes use { w: "majority" }, are majority-committed, never rolled back, and reads with readConcern: "majority" never observe rolled-back data. In this mode, MongoDB behaves like a classic Raft system: once an operation is committed, it is final.

MongoDB also lets you explicitly relax that guarantee by choosing a weaker write concern such as { w: 1 }. In doing so, you tell the system: "Prioritize availability and latency over immediate global consistency." The demo shows what that implies:

• During a transient network partition, two primaries can briefly accept writes.
• Both branches of history are durably written to disk.
• When the partition heals, MongoDB deterministically chooses the majority branch.
• Operations from the losing branch are rolled back—but not discarded. They are preserved as BSON files with their oplog entries.
• The node then recovers to a majority-committed timestamp (RTT) and rolls forward.

This rollback behavior is where MongoDB intentionally diverges from vanilla Raft.

In classic Raft, the replicated log is the source of truth, and committed log entries are never rolled back. Raft assumes a linearizable, strongly consistent state machine where the application does not expect divergence. MongoDB, by contrast, comes from a NoSQL and event-driven background, where asynchronous replication, eventual consistency, and application-level reconciliation are sometimes acceptable trade-offs.

As a result:

• MongoDB still uses Raft semantics for leader election and terms, so two primaries are never elected in the same term.
• For data replication, MongoDB extends the model with Recover To a Timestamp (RTT) rollback.
• This allows MongoDB to safely support lower write concerns, fast ingestion, multi-region latency optimization, and migration workloads—without silently corrupting state.

In short, MongoDB replication is based on Raft, but adds rollback semantics to support real-world distributed application patterns. Rollbacks happen only when you explicitly allow them, never with majority writes, and they are fully auditable and recoverable.

Despite the ubiquity of the MongoDB aggregation framework, it has been lacking a formal mathematical framework/specification. This paper aims to fix this gap by providing a theoretical foundation, and proposes MQuery. The formalization in MQuery is largely based on the paper published at ICDT 2018 (for which the first author is involved), extending it to include more pipeline operators, relax the assumption that the JSON documents stored in the database comply to a predefined schema, and allow objects that are either ordered or unordered sets of key-value pairs.

Motivation

For decades, SQL proponents have flaunted the rigorous mathematical foundation of relational algebra (courtesy of Edgar Codd). The world of JSON document databases, however, has remained a bit of a Wild West in comparison. The analogy is apt because, like the frontier, there is immense opportunity here. JSON is the undisputed king of data exchange, and the MongoDB aggregation framework has emerged as the widely adopted query language for JSON collections. Thanks to its expressive pipeline model, massive developer base, and popularity, MongoDB aggregation framework has effectively become the de facto standard for querying JSON. The fact that major vendors (including Amazon, Microsoft, Oracle, and Google) seek to provide compatibility with the MongoDB API further underscores its recognition as a common lingua franca. (The authors' words, not mine, so don't think I'm bragging on behalf of MongoDB.)

To further motivate the need for a rigorous mathematical framework, the authors highlight current challenges. They argue that MongoDB's semantics are procedural rather than declarative. While the aggregation pipeline is pragmatic and powerful, its documentation often overlooks edge cases, leading to ambiguity.

The paper illustrates this with an example about query predicates. In MongoDB, the query origin: "UK" matches a document where origin is the string "UK". However, it also matches a document where origin is the array ["UK", "Japan"]. While this loose equality is convenient for developers, it is bad for mathematical logic, as it violates the property of transitivity: ["UK"] matches "UK", and "UK" matches "UK", yet [["UK"]] does not match "UK".

Furthermore, the paper argues MongoDB suffers from path polysemy. A path like origin.country is inherently ambiguous. Does it refer to a nested field in a single object? Or, if origin is an array, does it refer to the country field of every object inside that array? This leads to data-dependent behavior, where a valid query might throw a runtime error simply because a new document with a different structure was inserted into the collection.

MQuery

MQuery (which, admittedly, looks a lot like McQuery, and now that I've said this, you won't be able to read it any other way) serves as a formalized abstraction of the MongoDB language. MQuery formalizes the data model using "d-values" (document values), which encompass literals, arrays, and objects. It also defines 7 core pipeline stages that mirror the MongoDB aggregation framework: $match, $unwind, $project, $group, $lookup, $graphLookup, and $union.

By formalizing these stages, the authors confirm in Section 4 that "the MongoDB aggregation framework is very expressive: at least as expressive as full relational algebra (RA)". They mention:

• the match, unwind, project and group fragment captures RA over a single relation
• the match, unwind, project, group, and lookup fragment captures arbitrary RA

They demonstrate that $match corresponds to selection, $project to projection, and $lookup (or a combination of $unwind and $group) to joins. This confirms that document databases can theoretically perform every operation relational databases can, including complex joins and set operations. They also note the MongoDB aggregation framework goes beyond RA by handling Nested Relational Algebra and linear recursion via $graphLookup.

The Payoff: Algebraic Optimization

Why does all this math matter? The formal definition can help us safely optimize queries. The final section of the paper demonstrates algebraic rewriting rules. Thanks to the formal definitions, the authors can prove when it is safe to reorder pipeline stages without altering the result. They provide rules for filter anticipation (moving $match earlier to reduce data volume), unnesting postponement (moving $unwind later to save memory), and join optimization.

This is an external post of mine. Click here if you are not redirected.