This paper (SIGMOD '08) proposes a lightweight runtime technique to make Snapshot Isolation (SI) serializable without falling back to locking. The key idea behind Serializable SI (SSI) is to detect potentially dangerous (write-skew) executions at runtime and abort one of the transactions to guarantee serializability (SER).

The goal is to offer the strong guarantees of SER without sacrificing SI's high performance and non-blocking reads. But would it make sense to implement SER by layering on MVCC SI instead of implementing it directly? Do you think an SI-based implementation would be more performant than native 2PL-based SER implementations? What about compared to OCC-based SER? The evaluation section gives some answers.

The problem and insight

Let's back up.

Write-skew is the canonical anomaly under SI. And the canonical example for write-skew is the "doctors on-call" scenario. Consider two doctors removing themselves from a duty roster. Each transaction checks that at least one doctor remains on duty and proceeds to update its own status. Under SI, each transaction sees the old snapshot where the other doctor is still on duty. Both commit. The resulting state violates the application’s invariant, even though each transaction was locally consistent. Compare this to SER which ensures that integrity constraints are maintained even if those constraints are not explicitly declared to the DBMS. (Another famous write-skew example is reading savings and checking accounts both, and withdrawing from one. Yet another example could be graph coloring, reading neighbor's color to switch your own in response.)

A naive fix for preventing write-skew would be to track the read set of a transaction T1 and, at commit time, check whether any item read by T1 was overwritten by a concurrent writer. If so, we abort T1. This eliminates outgoing rw-dependencies and thus prevents write-skew cycles. It works, but it's overkill: You end up aborting transactions even when no cycle would form. For example, in the execution "b1 r1(x) w2(x) w1(y) c1 c2", T1 would be aborted because its read r1(x) is overwritten by T2's w2(x), but it is possible to serialize this execution as T1 followed by T2, as the writesets are not conflicting as per SI.

The paper proposes a smarter fix: only abort if a transaction has both an inConflict and an outConflict edges with other transactions. That is, it sits in the middle of two concurrent rw-dependencies (the paper calls this a pivot in a dangerous structure). Prior work by the authors (Fekete et al. 2005) proves that all SI anomalies contain such a pivot. So this check would suffice.

The approach still over-approximates somewhat: having both in and out conflicts doesn’t always imply a cycle. See Fig 9, for an example. Although T0 has both inConflict and outConflict, it is possible to serialize these transactions as TN, T0, and T1 order. The paper doesn't quantify the false positive rate or explore pathological workloads.

The paper says full serialization graph cycle detection would be too expensive, and this is the compromise. This is still a tighter and more efficient condition than the naive approach. The Ferro-Yabandeh Eurosys '12 paper uses something close to the naive approach. Same with naive OCC-based SER implementations. They abort based on single outConflict edge.

Implementation and evaluation

The implementation tracks two flags per transaction: inConflict and outConflict. It also adds a new lock mode, SIREAD, to record reads. Reads acquire SIREAD locks; writes acquire WRITE locks. When a read at T1 observes a version written by T2, we set T1.outConflict and T2.inConflict. This is called a rw-dependency (a.k.a antidependency) from T1 to T2. When a write at T1 detects an existing SIREAD by T2, we set T1.inConflict and T2.outConflict. See the pseudocode in Figures 5-8 for how this is implemented.

When both flags are set for any transaction, one transaction in the conflict chain is aborted. The implementation prefers to abort the pivot (the transaction with both in and out conflicts), unless it has already committed. Else an adjacent transaction is aborted. If there are two pivots in the cycle, whichever is detected first gets aborted. This choice is sound and gives room for policy tuning (e.g., prefer aborting younger transactions).

Note that all of this is layered on top of SI. Readers never block writers. Writers never block readers. The only cost is the extra bookkeeping to track conflicts.

The authors implemented SSI over Berkeley DB (which provides SI through MVCC) with minimal changes, around 700 lines of modified code out of a total of over 200,000 lines of code in Berkeley DB. The experiments with the SmallBank benchmark show that the performance is close to plain SI and significantly better than 2PL under contention.

The paper does not contrast SSI with OCC implementation of SER. But it argues that naive OCC implementations abort on any rw-conflict. SSI waits until both in and out conflicts are present, and would result in fewer false positives and better throughput.

My take

SI is attractive because it avoids blocking: readers don't block writers and vice versa. Maybe the takeaway is this: Stick with SI. But if you really must go serializable, consider layering SSI on top of SI instead of using 2PL or OCC based SER implementations.

But, still, there are many caveats to SSI as presented in this paper: phantoms are not addressed, false positives remain, B-tree level conflicts inflate aborts, and adapting this to distributed systems is non-trivial.

This paper is a research prototype after all. A VLDB'12 paper by Dan Ports followed this algorithm to provide a PostgreSQL implementation adds integration with real-world features like: replication and crash recovery, two-phase commit (tupac), subtransactions and savepoints, and memory bounding via transaction summarization. It also handled phantoms by leveraging SIREAD locks on index ranges to detect predicate read/write conflicts. 

Caveats and Open Questions

The paper punts on phantoms. It says that Berkeley DB uses page-level locking, so predicate-based anomalies don’t occur. But for engines with row-level locking, it suggests that SSI would need to acquire SIREAD locks on index ranges or use multigranularity locking.

The design assumes a single-node system. In a distributed DB, you would need to propagate SIREAD metadata and conflict flags across partitions. Detecting cross-partition dependencies and coordinating pivot aborts would be hard. The paper does not talk about implementation in distributed databases.

A known artifact in the Berkeley DB implementation is page-level locking. Updates to shared pages, like the B-tree root during a split, look like write-write conflicts across unrelated transactions. These inflate false positives and aborts under SSI.

My troubles with Figure 4 

I mean I have to complain about this (otherwise it won't be a blog post from Murat). Figure 4 was terribly confusing for me. I had a hard time figuring out why these two scenarios are not serializable. It turns out I got confused by the "helpful text" saying: "In Figure 4(a), both reads occur after the writes. In Figure 4(b), TN reads x before it is written by T0." No, that is irrelevant: the reads in SI happen from the snapshot at the transaction start time, so when the reads happen is not relevant. Confused by this I was wondering why we can't serialize Figure 4.a execution as T1, T0, TN.

Ok, let's focus on Fig 4.a. T0's serialization point should be before T1, because of rw-dependency from T0 to T1: T0' read happens before T1's write. T1 needs be serialized before TN, because TN reads z from T1's write. But TN is problematic. TN introduces rw-dependence from TN to T0. It reads x from its snapshot which is earlier than T0's commit point. So TN should come before T0. This conflicts with the two previous requirements that T0 precedes T1 and T1 precedes TN. We have a cycle that prevents serialization. 

Figure 4.b is very similar to Figure 4.a and has the same cycle. The only difference is TN's commit point comes a little later, but it doesn't change anything, TN does its read from the snapshot taken at the start of TN.

It would have been nice to supplement the figure with a text explaining the cycles, and giving some insights into the thought process. 

What It Takes to Get a Research Project Ready for Production

Concurrency control in databases has two main modes for handling conflicts. Traditionally, these are labeled "pessimistic" and "optimistic," but those names can be misleading. Let’s define them by what actually happens from a user point of view:

• Wait-on-Conflict: the operation blocks until the conflict is resolved, typically waiting for a concurrent transaction to either commit or abort and release its locks. For most Online Transaction Processing (OLTP) workloads, this process is usually quick, unless a long-running transaction is blocking progress or there is a deadlock in which two transactions are waiting on each other.
• Fail-on-Conflict: Instead of waiting, the operation immediately raises an error to the application. Now it’s up to the application to decide: retry after a short pause (hoping the conflict is gone), or just propagate the error. The application may also have done some non-transactional operations that need to be canceled or compensated.

In SQL databases, where multi-statement transactions are the norm because of normalization to multiple tables, you’ll see this as an explicit WAIT or NOWAIT on locking reads (LOCK, SELECT FOR UPDATE), or a consequence of the isolation level. READ COMMITTED tries to wait and detect deadlocks, SERIALIZABLE can fail with a serialization error.

MongoDB’s document-oriented approach is straightforward, providing developers the freedom to choose what best fits their needs.

• Single-document updates without an explicit transaction will wait on conflict, until the operation timeout. The document model encourages encapsulation of a business transaction within one document, keeping concurrency simple and fast.
• Multi-document operations within an explicit transaction, however, fail on conflict. Here, MongoDB expects your application to do the retry logic and error handling: since you started the transaction, you're responsible for finishing it.

The need to implement retry logic in the application is not a limitation of MongoDB. A database cannot transparently cancel a user-defined transaction and retry it because it lacks knowledge of the application's prior actions, like sending emails or writing to files. Only the application can cancel or compensate for actions taken before retrying the transaction.
In contrast to multi-statement transactions, single-statement transactions or auto-commit commands can be canceled and restarted before control is returned to the application. That's how MongoDB provides a lock-free wait-on-conflict behavior for single-document updates: it is internally a fail-on-conflict optimistic concurrency control (OCC), to avoid locks, but with internal retries to appear as a wait-on-conflict to the user.

Demonstration

Here’s what this looks like in practice. I start with a one-document collection:

AtlasLocalDev atlas [direct: primary] test> 
db.demo.drop();

true

AtlasLocalDev atlas [direct: primary] test> 
db.demo.insertOne({ _id: 'x' , value: 0 });

{ acknowledged: true, insertedId: 'x' }

I start a first transaction, in session A, that updates the document, but do not commit it yet:

//first transaction A updates one doc

AtlasLocalDev atlas [direct: primary] test> 
sessionA = db.getMongo().startSession();

{ id: UUID('ac8f6741-4ebb-4145-acb2-6b861a390c25') }

AtlasLocalDev atlas [direct: primary] test> sessionA.startTransaction();

AtlasLocalDev atlas [direct: primary] test> 
dbA = sessionA.getDatabase(db.getName());

test

AtlasLocalDev atlas [direct: primary] test> 
dbA.demo.updateOne({ _id: 'x' }, { $inc: { value: 2 } });

{
  acknowledged: true,
  insertedId: null,
  matchedCount: 1,
  modifiedCount: 1,
  upsertedCount: 0
}

Explicit transaction

While this transaction is ongoing, I start another one in session B and try to update the same document:

// second transaction B tries to update the same doc

AtlasLocalDev atlas [direct: primary] test> 
sessionB = db.getMongo().startSession();

{ id: UUID('a5302cab-9688-43db-badd-e3691b30a15b') }

AtlasLocalDev atlas [direct: primary] test> sessionB.startTransaction();

AtlasLocalDev atlas [direct: primary] test> 
dbB = sessionB.getDatabase(db.getName());

test

AtlasLocalDev atlas [direct: primary] test> 
Date();

Wed Jul 23 2025 10:59:37 GMT+0000 (Greenwich Mean Time)

AtlasLocalDev atlas [direct: primary] test> 
dbB.demo.updateOne({ _id: 'x' }, { $inc: { value: 4 } });

MongoServerError[WriteConflict]: Caused by :: Write conflict during plan execution and yielding is disabled. :: Please retry your operation or multi-document transaction.

AtlasLocalDev atlas [direct: primary] test> 
Date();

Wed Jul 23 2025 10:59:40 GMT+0000 (Greenwich Mean Time)

AtlasLocalDev atlas [direct: primary] test> sessionB.abortTransaction();

The conflict was immediately detected and the application gets a WriteConflict error and can retry it.

Single-document

This time, I'll retry without starting an explicit transaction:

// do the same without starting a transaction

AtlasLocalDev atlas [direct: primary] test> 
Date();

Wed Jul 23 2025 10:59:49 GMT+0000 (Greenwich Mean Time)

AtlasLocalDev atlas [direct: primary] test> 
dbB.demo.updateOne({ _id: 'x' }, { $inc: { value: 8 } });

The session blocks, it is waiting until the document is free. I leave the transaction in session A to show what happens with long transactions.

After one minute, the update is successful:

{
  acknowledged: true,
  insertedId: null,
  matchedCount: 1,
  modifiedCount: 1,
  upsertedCount: 0
}

AtlasLocalDev atlas [direct: primary] test> 
Date();

Wed Jul 23 2025 11:00:48 GMT+0000 (Greenwich Mean Time)

My two transactions are not serializable, as they update the same document at the same time, so if one was successful, the other must abort. This is what happened, the transaction in session A was automatically canceled after a one minute timeout. I can see that if I try to commit:

AtlasLocalDev atlas [direct: primary] test> 
sessionA.commitTransaction();

MongoServerError[NoSuchTransaction]: Transaction with { txnNumber: 2 } has been aborted.

AtlasLocalDev atlas [direct: primary] test> 
sessionA.endSession();

Finally, the result is consistent, with the changes committed by session B:

AtlasLocalDev atlas [direct: primary] test> 
db.demo.find();

[ { _id: 'x', value: 8 } ]

Internals

I looked at the call stack while it was waiting and here is the Flame Graph:

It shows the update attempts. updateWithDamages is not as terrible as it sounds, and is just an incremental update that logs the changes ("damages") rather than re-writing the whole document. logWriteConflictAndBackoff is the internal retry. Each write conflict in PlanExecutorExpress, increment an attempt counter and calls logWriteConflictAndBackoff → logAndBackoff → logAndBackoffImpl with the number of attempts. The code shows that the wait depends on it:

• For the first 4 attempts: No sleep, immediate retry.
• For attempts 4–9: Sleep 1 ms each retry.
• For attempts 10–99: Sleep 5 ms each retry.
• For attempts 100–199: Sleep 10 ms each retry.
• For 200 and greater: Sleep 100 ms each retry.

This mitigates resource contention by increasingly slowing down repeated conflicts with an exponential-to-linear backoff while still getting a chance to get the conflict resolved. In one minute, we can estimate that it has performed approximately 4 + 6 + 90 + 100 + 585 = 785 write conflict retries under this backoff schedule, spending most of that time in the 100 ms sleep interval.

Mongostat

I have run the wait-on-conflict situation again:

AtlasLocalDev atlas [direct: primary] test> 
Date();

Wed Jul 23 2025 08:48:58 GMT+0000 (Greenwich Mean Time)

AtlasLocalDev atlas [direct: primary] test> 
dbB.demo.updateOne({ _id: 'x' }, { $inc: { value: 8 } });

{
  acknowledged: true,
  insertedId: null,
  matchedCount: 1,
  modifiedCount: 1,
  upsertedCount: 0
}

AtlasLocalDev atlas [direct: primary] test> 
Date();
Wed Jul 23 2025 08:50:18 GMT+0000 (Greenwich Mean Time)

During these 80 seconds, I displayed the writeConflicts metric with mongostat:

mongostat -o="insert,query,update,delete,metrics.operation.writeConflicts.rate(),time"

insert query update delete writeConflicts.rate()                time
    *0    *0     *0     *0                     0 Jul 23 08:49:01.172
    *0    *0     *0     *0                     0 Jul 23 08:49:02.173
    *0    *0     *0     *0                     0 Jul 23 08:49:03.172
    *0    *0     *0     *0                     0 Jul 23 08:49:04.172
    *0    *0     *0     *0                     0 Jul 23 08:49:05.172
...
    *0    *0     *0     *0                     0 Jul 23 08:50:15.172
    *0    *0     *0     *0                     0 Jul 23 08:50:16.171
    *0    *0     *0     *0                     0 Jul 23 08:50:17.172
    *0    *0     *0     *0                   981 Jul 23 08:50:18.172
    *0    *0     *0     *0                     0 Jul 23 08:50:19.172
    *0    *0     *0     *0                     0 Jul 23 08:50:20.172

It shows 981 write conflict retries during those 80 second. Early retries go fast, adding just a few milliseconds. As time goes on, the loop spends the vast majority of those 80 seconds in the late-stage sleep(100ms) phase, pushing the retry count close to elapsed_time / 100ms (plus early faster backoffs).

Here is the exact calculation:

• First 4: 0ms
• Attempts 4–9 (6): 6×1=6ms
• Attempts 10–99 (90): 90×5=450ms
• Attempts 100–199 (100): 100×10=1,000ms
• Attempts 200–980 (781): 781×100=78,100ms The total time is 78 seconds and we have 981 write conflicts. The measure matches the algorithm.

Note that with mongostat you won't see individual retries until the moment the operation finally succeeds and increments the write conflict counter. If you see large spikes, it's time to look at data modeling patterns or long-running operations.

Profiling

Another way to look at it is with the MongoDB profiler:

db.setProfilingLevel(2);
Date();
dbB.demo.updateOne({ _id: 'x' }, { $inc: { value: 8 } });
Date();
db.setProfilingLevel(0);
db.system.profile.find().sort({ ts: -1 }).limit(1).pretty();

Here is the output:

[
  {
    op: 'update',
    ns: 'test.demo',
    command: {
      q: { _id: 'x' },
      u: { '$inc': { value: 8 } },
      multi: false,
      upsert: false
    },
    keysExamined: 1014,
    docsExamined: 1014,
    nMatched: 1,
    nModified: 1,
    nUpserted: 0,
    keysInserted: 0,
    keysDeleted: 0,
    writeConflicts: 1013,
    numYield: 1013,
    locks: {
      ReplicationStateTransition: { acquireCount: { w: Long('1015') } },
      Global: { acquireCount: { r: Long('1'), w: Long('1015') } },
      Database: { acquireCount: { w: Long('1015') } },
      Collection: { acquireCount: { w: Long('1015') } }
    },
    flowControl: { acquireCount: Long('1014') },
    readConcern: { level: 'local', provenance: 'implicitDefault' },
    storage: { data: { txnBytesDirty: Long('932') } },
    cpuNanos: 145071200,
    millis: 82965,
    planSummary: 'EXPRESS_IXSCAN { _id: 1 },EXPRESS_UPDATE',
    planningTimeMicros: 71,
    totalOplogSlotDurationMicros: 135,
    execStats: {
      isCached: false,
      stage: 'EXPRESS_UPDATE',
      keyPattern: '{ _id: 1 }',
      indexName: '_id_',
      keysExamined: 1014,
      docsExamined: 1014,
      nReturned: 0,
      nWouldModify: 1,
      nWouldUpsert: 0,
      nWouldDelete: 0
    },
    ts: ISODate('2025-07-23T17:18:48.082Z'),
    client: '172.17.0.1',
    appName: 'mongosh 2.5.0',
    allUsers: [],
    user: ''
  }
]

The single update of one document (Modified: 1) has been retried 1014 times (keysExamined: 1014, docsExamined: 1014) because the first 1013 were conflicting with the concurrent transaction (writeConflicts: 1013). It took in total an elapsed time of 83 seconds (millis: 82965) without consuming resources as each retry waited (numYield: 1013) so the CPU usage is minimal, less than 0.2% (cpuNanos: 145071200). The conflict resolution, with optimistic concurrency control, is lock-free while waiting. It only briefly acquires some write intent locks for each retry (acquireCount: { w: Long('1015') }).

The single update of one document (nModified: 1) was retried 1,014 times (keysExamined: 1014, docsExamined: 1014), as the first 1,013 attempts conflicted with a concurrent transaction (writeConflicts: 1013). The entire process took about 83 seconds (millis: 82965), but resource usage remained minimal: each retry yielded control (numYield: 1013), so CPU time used was less than 0.2% (cpuNanos: 145071200). This conflict resolution uses optimistic concurrency control and is lock-free while waiting. MongoDB only briefly acquires write intent locks for each retry (acquireCount: { w: Long('1015') }), never blocking other work for the whole time.

Summary

In MongoDB, single-document operations do not require explicit transactions and utilize a "wait-on-conflict" method, allowing for seamless, automatic conflict handling. This is achieved through optimistic concurrency control, where MongoDB retries updates on conflict with exponential backoff and operates in a lock-free manner while waiting.
In contrast, multi-document operations that involve explicit transactions follow a “fail-on-conflict” approach. If a conflict occurs, MongoDB immediately returns an error, placing the responsibility of canceling and retrying the transaction on your application, since it may involve some operations that are not visible to the database.

Concurrency control in databases has two main modes for handling conflicts. Traditionally, these are labeled "pessimistic" and "optimistic," but those names can be misleading. Let’s define them by what actually happens from a user point of view:

• Wait-on-Conflict: the operation blocks until the conflict is resolved, typically waiting for a concurrent transaction to either commit or abort and release its locks. For most Online Transaction Processing (OLTP) workloads, this process is usually quick, unless a long-running transaction is blocking progress or there is a deadlock in which two transactions are waiting on each other.
• Fail-on-Conflict: Instead of waiting, the operation immediately raises an error to the application. Now it’s up to the application to decide: retry after a short pause (hoping the conflict is gone), or just propagate the error. The application may also have done some non-transactional operations that need to be canceled or compensated.

In SQL databases, where multi-statement transactions are the norm because of normalization to multiple tables, you’ll see this as an explicit WAIT or NOWAIT on locking reads (LOCK, SELECT FOR UPDATE), or a consequence of the isolation level. READ COMMITTED tries to wait and detect deadlocks, SERIALIZABLE can fail with a serialization error.

MongoDB’s document-oriented approach is straightforward, providing developers the freedom to choose what best fits their needs.

• Single-document updates without an explicit transaction will wait on conflict, until the operation timeout. The document model encourages encapsulation of a business transaction within one document, keeping concurrency simple and fast.
• Multi-document operations within an explicit transaction, however, fail on conflict. Here, MongoDB expects your application to do the retry logic and error handling: since you started the transaction, you're responsible for finishing it.

The need to implement retry logic in the application is not a limitation of MongoDB. A database cannot transparently cancel a user-defined transaction and retry it because it lacks knowledge of the application's prior actions, like sending emails or writing to files. Only the application can cancel or compensate for actions taken before retrying the transaction.
In contrast to multi-statement transactions, single-statement transactions or auto-commit commands can be canceled and restarted before control is returned to the application. That's how MongoDB provides a lock-free wait-on-conflict behavior for single-document updates: it is internally a fail-on-conflict optimistic concurrency control (OCC), to avoid locks, but with internal retries to appear as a wait-on-conflict to the user.

Demonstration

Here’s what this looks like in practice. I start with a one-document collection:

AtlasLocalDev atlas [direct: primary] test> 
db.demo.drop();

true

AtlasLocalDev atlas [direct: primary] test> 
db.demo.insertOne({ _id: 'x' , value: 0 });

{ acknowledged: true, insertedId: 'x' }

I start a first transaction, in session A, that updates the document, but do not commit it yet:

//first transaction A updates one doc

AtlasLocalDev atlas [direct: primary] test> 
sessionA = db.getMongo().startSession();

{ id: UUID('ac8f6741-4ebb-4145-acb2-6b861a390c25') }

AtlasLocalDev atlas [direct: primary] test> sessionA.startTransaction();

AtlasLocalDev atlas [direct: primary] test> 
dbA = sessionA.getDatabase(db.getName());

test

AtlasLocalDev atlas [direct: primary] test> 
dbA.demo.updateOne({ _id: 'x' }, { $inc: { value: 2 } });

{
  acknowledged: true,
  insertedId: null,
  matchedCount: 1,
  modifiedCount: 1,
  upsertedCount: 0
}

Explicit transaction

While this transaction is ongoing, I start another one in session B and try to update the same document:

// second transaction B tries to update the same doc

AtlasLocalDev atlas [direct: primary] test> 
sessionB = db.getMongo().startSession();

{ id: UUID('a5302cab-9688-43db-badd-e3691b30a15b') }

AtlasLocalDev atlas [direct: primary] test> sessionB.startTransaction();

AtlasLocalDev atlas [direct: primary] test> 
dbB = sessionB.getDatabase(db.getName());

test

AtlasLocalDev atlas [direct: primary] test> 
Date();

Wed Jul 23 2025 10:59:37 GMT+0000 (Greenwich Mean Time)

AtlasLocalDev atlas [direct: primary] test> 
dbB.demo.updateOne({ _id: 'x' }, { $inc: { value: 4 } });

MongoServerError[WriteConflict]: Caused by :: Write conflict during plan execution and yielding is disabled. :: Please retry your operation or multi-document transaction.

AtlasLocalDev atlas [direct: primary] test> 
Date();

Wed Jul 23 2025 10:59:40 GMT+0000 (Greenwich Mean Time)

AtlasLocalDev atlas [direct: primary] test> sessionB.abortTransaction();

The conflict was immediately detected and the application gets a WriteConflict error and can retry it.

Single-document

This time, I'll retry without starting an explicit transaction:

// do the same without starting a transaction

AtlasLocalDev atlas [direct: primary] test> 
Date();

Wed Jul 23 2025 10:59:49 GMT+0000 (Greenwich Mean Time)

AtlasLocalDev atlas [direct: primary] test> 
dbB.demo.updateOne({ _id: 'x' }, { $inc: { value: 8 } });

The session blocks, it is waiting until the document is free. I leave the transaction in session A to show what happens with long transactions.

After one minute, the update is successful:

{
  acknowledged: true,
  insertedId: null,
  matchedCount: 1,
  modifiedCount: 1,
  upsertedCount: 0
}

AtlasLocalDev atlas [direct: primary] test> 
Date();

Wed Jul 23 2025 11:00:48 GMT+0000 (Greenwich Mean Time)

My two transactions are not serializable, as they update the same document at the same time, so if one was successful, the other must abort. This is what happened, the transaction in session A was automatically canceled after a one minute timeout. I can see that if I try to commit:

AtlasLocalDev atlas [direct: primary] test> 
sessionA.commitTransaction();

MongoServerError[NoSuchTransaction]: Transaction with { txnNumber: 2 } has been aborted.

AtlasLocalDev atlas [direct: primary] test> 
sessionA.endSession();

Finally, the result is consistent, with the changes committed by session B:

AtlasLocalDev atlas [direct: primary] test> 
db.demo.find();

[ { _id: 'x', value: 8 } ]

Internals

I looked at the call stack while it was waiting and here is the Flame Graph:

It shows the update attempts. updateWithDamages is not as terrible as it sounds, and is just an incremental update that logs the changes ("damages") rather than re-writing the whole document. logWriteConflictAndBackoff is the internal retry. Each write conflict in PlanExecutorExpress, increment an attempt counter and calls logWriteConflictAndBackoff → logAndBackoff → logAndBackoffImpl with the number of attempts. The code shows that the wait depends on it:

• For the first 4 attempts: No sleep, immediate retry.
• For attempts 4–9: Sleep 1 ms each retry.
• For attempts 10–99: Sleep 5 ms each retry.
• For attempts 100–199: Sleep 10 ms each retry.
• For 200 and greater: Sleep 100 ms each retry.

This mitigates resource contention by increasingly slowing down repeated conflicts with an exponential-to-linear backoff while still getting a chance to get the conflict resolved. In one minute, we can estimate that it has performed approximately 4 + 6 + 90 + 100 + 585 = 785 write conflict retries under this backoff schedule, spending most of that time in the 100 ms sleep interval.

Internally, WiredTiger enforces snapshot isolation at the storage layer. When a MongoDB operation tries to modify a document, WiredTiger checks that the current snapshot timestamp still matches what’s on disk. If another operation has modified the document since that snapshot was taken, WiredTiger detects that the update’s view is stale. Instead of acquiring a long-term lock, WiredTiger uses optimistic concurrency control and returns WT_ROLLBACK to MongoDB’s server layer to signal the conflict.
At this point, the MongoDB server recognizes the error, aborts the in-progress write attempt, and releases any snapshot or locks it has held on behalf of the operation. It then yields to allow other work, starts a new WiredTiger transaction with a fresh snapshot timestamp, and retries the update operation.
After a successful update (with a clean, current snapshot and no conflict), or if a timeout is reached, the process ends, and MongoDB will either commit the change or return an error to the application.

Mongostat

I have run the wait-on-conflict situation again:

AtlasLocalDev atlas [direct: primary] test> 
Date();

Wed Jul 23 2025 08:48:58 GMT+0000 (Greenwich Mean Time)

AtlasLocalDev atlas [direct: primary] test> 
dbB.demo.updateOne({ _id: 'x' }, { $inc: { value: 8 } });

{
  acknowledged: true,
  insertedId: null,
  matchedCount: 1,
  modifiedCount: 1,
  upsertedCount: 0
}

AtlasLocalDev atlas [direct: primary] test> 
Date();
Wed Jul 23 2025 08:50:18 GMT+0000 (Greenwich Mean Time)

During these 80 seconds, I displayed the writeConflicts metric with mongostat:

mongostat -o="insert,query,update,delete,metrics.operation.writeConflicts.rate(),time"

insert query update delete writeConflicts.rate()                time
    *0    *0     *0     *0                     0 Jul 23 08:49:01.172
    *0    *0     *0     *0                     0 Jul 23 08:49:02.173
    *0    *0     *0     *0                     0 Jul 23 08:49:03.172
    *0    *0     *0     *0                     0 Jul 23 08:49:04.172
    *0    *0     *0     *0                     0 Jul 23 08:49:05.172
...
    *0    *0     *0     *0                     0 Jul 23 08:50:15.172
    *0    *0     *0     *0                     0 Jul 23 08:50:16.171
    *0    *0     *0     *0                     0 Jul 23 08:50:17.172
    *0    *0     *0     *0                   981 Jul 23 08:50:18.172
    *0    *0     *0     *0                     0 Jul 23 08:50:19.172
    *0    *0     *0     *0                     0 Jul 23 08:50:20.172

It shows 981 write conflict retries during those 80 second. Early retries go fast, adding just a few milliseconds. As time goes on, the loop spends the vast majority of those 80 seconds in the late-stage sleep(100ms) phase, pushing the retry count close to elapsed_time / 100ms (plus early faster backoffs).

Here is the exact calculation:

• First 4: 0ms
• Attempts 4–9 (6): 6×1=6ms
• Attempts 10–99 (90): 90×5=450ms
• Attempts 100–199 (100): 100×10=1,000ms
• Attempts 200–980 (781): 781×100=78,100ms The total time is 78 seconds and we have 981 write conflicts. The measure matches the algorithm.

Note that with mongostat you won't see individual retries until the moment the operation finally succeeds and increments the write conflict counter. If you see large spikes, it's time to look at data modeling patterns or long-running operations.

Profiling

Another way to look at it is with the MongoDB profiler:

db.setProfilingLevel(2);
Date();
dbB.demo.updateOne({ _id: 'x' }, { $inc: { value: 8 } });
Date();
db.setProfilingLevel(0);
db.system.profile.find().sort({ ts: -1 }).limit(1).pretty();

Here is the output:

[
  {
    op: 'update',
    ns: 'test.demo',
    command: {
      q: { _id: 'x' },
      u: { '$inc': { value: 8 } },
      multi: false,
      upsert: false
    },
    keysExamined: 1014,
    docsExamined: 1014,
    nMatched: 1,
    nModified: 1,
    nUpserted: 0,
    keysInserted: 0,
    keysDeleted: 0,
    writeConflicts: 1013,
    numYield: 1013,
    locks: {
      ReplicationStateTransition: { acquireCount: { w: Long('1015') } },
      Global: { acquireCount: { r: Long('1'), w: Long('1015') } },
      Database: { acquireCount: { w: Long('1015') } },
      Collection: { acquireCount: { w: Long('1015') } }
    },
    flowControl: { acquireCount: Long('1014') },
    readConcern: { level: 'local', provenance: 'implicitDefault' },
    storage: { data: { txnBytesDirty: Long('932') } },
    cpuNanos: 145071200,
    millis: 82965,
    planSummary: 'EXPRESS_IXSCAN { _id: 1 },EXPRESS_UPDATE',
    planningTimeMicros: 71,
    totalOplogSlotDurationMicros: 135,
    execStats: {
      isCached: false,
      stage: 'EXPRESS_UPDATE',
      keyPattern: '{ _id: 1 }',
      indexName: '_id_',
      keysExamined: 1014,
      docsExamined: 1014,
      nReturned: 0,
      nWouldModify: 1,
      nWouldUpsert: 0,
      nWouldDelete: 0
    },
    ts: ISODate('2025-07-23T17:18:48.082Z'),
    client: '172.17.0.1',
    appName: 'mongosh 2.5.0',
    allUsers: [],
    user: ''
  }
]

The single update of one document (Modified: 1) has been retried 1014 times (keysExamined: 1014, docsExamined: 1014) because the first 1013 were conflicting with the concurrent transaction (writeConflicts: 1013). It took in total an elapsed time of 83 seconds (millis: 82965) without consuming resources as each retry waited (numYield: 1013) so the CPU usage is minimal, less than 0.2% (cpuNanos: 145071200). The conflict resolution, with optimistic concurrency control, is lock-free while waiting. It only briefly acquires some write intent locks for each retry (acquireCount: { w: Long('1015') }).

The single update of one document (nModified: 1) was retried 1,014 times (keysExamined: 1014, docsExamined: 1014), as the first 1,013 attempts conflicted with a concurrent transaction (writeConflicts: 1013). The entire process took about 83 seconds (millis: 82965), but resource usage remained minimal: each retry yielded control (numYield: 1013), so CPU time used was less than 0.2% (cpuNanos: 145071200). This conflict resolution uses optimistic concurrency control and is lock-free while waiting. MongoDB only briefly acquires write intent locks for each retry (acquireCount: { w: Long('1015') }), never blocking other work for the whole time.

Summary

In MongoDB, single-document operations do not require explicit transactions and utilize a "wait-on-conflict" method, allowing for seamless, automatic conflict handling. This is achieved through optimistic concurrency control, where MongoDB retries updates on conflict with exponential backoff and operates in a lock-free manner while waiting. With a proper document model, single-document operations are the general case, as there's no need to normalize to multiple tables like in SQL databases.

In contrast, multi-document operations that involve explicit transactions adopt a “fail-on-conflict” strategy. When a conflict arises, MongoDB immediately returns an error, requiring your application to handle cancellation and retrying the transaction. This is necessary because some operations may not be visible to the database. Such transactions must be kept short, and are canceled if they take longer than one minute. The reason for this limitation is that,like other MVCC databases, lengthy transactions consume more resources to maintain a consistent snapshot of the database state at the start of the transaction.

MongoDB sits at the crossroads between SQL, where developers can write queries on a logical model without naming indexes, and the NoSQL world, where developers are accustomed to querying directly by mentioning indexes in their queries.

Except for Atlas Search and Vector indexes, where you must explicitly indicate whether it uses approximate search, you typically query a collection, and the MongoDB query planner determines which index to use.

One exception to this rule happens when using an optimizer hint to reference an index. This feature is supported in most popular databases, except PostgreSQL, where it requires pg_hint_plan. When the hint isn't fully compatible with the query planner's execution plans, different databases may behave differently.

Here is my test case to show how it behaves in MongoDB:

mdb> db.demo.insertMany([
  { _id: 1, name: "Alice", age: 30 },           // has age
  { _id: 2, name: "Bob" },                      // missing age
  { _id: 3, name: "Charlie", age: null },       // age is null
  { _id: 4, name: "Dave", age: 35, status: "active" },    // age + status
  { _id: 5, name: "Eve", age: 28, status: "inactive" }    // age + status
]);

{
  acknowledged: true,
  insertedIds: { '0': 1, '1': 2, '2': 3, '3': 4, '4': 5 }
}

mdb> db.demo.find().sort({ age: 1 });

[
  { _id: 2, name: 'Bob' },
  { _id: 3, name: 'Charlie', age: null },
  { _id: 5, name: 'Eve', age: 28, status: 'inactive' },
  { _id: 1, name: 'Alice', age: 30 },
  { _id: 4, name: 'Dave', age: 35, status: 'active' }
]

mdb> db.demo.find(
  { age: { $gte: 28 } },      // WHERE age >= 28
  { _id:0, age:1, name: 1 }   // SELECT age, name
);

[
  { name: 'Alice', age: 30 },
  { name: 'Dave', age: 35 },
  { name: 'Eve', age: 28 }
]

Without any index, the query scans the collection:

mdb> db.demo.find(
  { age: { $gte: 28 } },      // WHERE age >= 28
  { _id:0, age:1, name: 1 }   // SELECT age, name
).explain("executionStats").executionStats

{
  executionSuccess: true,
  nReturned: 3,
  executionTimeMillis: 0,
  totalKeysExamined: 0,
  totalDocsExamined: 5,
  executionStages: {
    isCached: false,
    stage: 'PROJECTION_SIMPLE',
    nReturned: 3,
    executionTimeMillisEstimate: 0,
    works: 6,
    advanced: 3,
    needTime: 2,
    needYield: 0,
    saveState: 0,
    restoreState: 0,
    isEOF: 1,
    transformBy: { _id: 0, age: 1, name: 1 },
    inputStage: {
      stage: 'COLLSCAN',
      filter: { age: { '$gte': 28 } },
      nReturned: 3,
      executionTimeMillisEstimate: 0,
      works: 6,
      advanced: 3,
      needTime: 2,
      needYield: 0,
      saveState: 0,
      restoreState: 0,
      isEOF: 1,
      direction: 'forward',
      docsExamined: 5
    }
  }
}

With an index, the same query does an index scan:

mdb> db.demo.createIndex({ age: 1 }, { name: "idx_age" })

idx_age

mdb> db.demo.find(
  { age: { $gte: 28 } },      // WHERE age >= 28
  { _id:0, age:1, name: 1 }   // SELECT age, name
).explain("executionStats").executionStats

{
  executionSuccess: true,
  nReturned: 3,
  executionTimeMillis: 1,
  totalKeysExamined: 3,
  totalDocsExamined: 3,
  executionStages: {
    isCached: false,
    stage: 'PROJECTION_SIMPLE',
    nReturned: 3,
    executionTimeMillisEstimate: 1,
    works: 4,
    advanced: 3,
    needTime: 0,
    needYield: 0,
    saveState: 0,
    restoreState: 0,
    isEOF: 1,
    transformBy: { _id: 0, age: 1, name: 1 },
    inputStage: {
      stage: 'FETCH',
      nReturned: 3,
      executionTimeMillisEstimate: 1,
      works: 4,
      advanced: 3,
      needTime: 0,
      needYield: 0,
      saveState: 0,
      restoreState: 0,
      isEOF: 1,
      docsExamined: 3,
      alreadyHasObj: 0,
      inputStage: {
        stage: 'IXSCAN',
        nReturned: 3,
        executionTimeMillisEstimate: 1,
        works: 4,
        advanced: 3,
        needTime: 0,
        needYield: 0,
        saveState: 0,
        restoreState: 0,
        isEOF: 1,
        keyPattern: { age: 1 },
        indexName: 'idx_age',
        isMultiKey: false,
        multiKeyPaths: { age: [] },
        isUnique: false,
        isSparse: false,
        isPartial: false,
        indexVersion: 2,
        direction: 'forward',
        indexBounds: { age: [ '[28, inf.0]' ] },
        keysExamined: 3,
        seeks: 1,
        dupsTested: 0,
        dupsDropped: 0
      }
    }
  }
}

I create a better index, that covers the projection, and it is used:

mdb> db.demo.createIndex({ age: 1, name: 1 }, { name: "idx_age_name" })

idx_age_name

mdb> db.demo.find(
  { age: { $gte: 28 } },          // WHERE age >= 28
  { _id:0, age:1, name: 1 }       // SELECT age, name
).explain("executionStats").executionStats

{
  executionSuccess: true,
  nReturned: 3,
  executionTimeMillis: 1,
  totalKeysExamined: 3,
  totalDocsExamined: 0,
  executionStages: {
    isCached: false,
    stage: 'PROJECTION_COVERED',
    nReturned: 3,
    executionTimeMillisEstimate: 1,
    works: 5,
    advanced: 3,
    needTime: 0,
    needYield: 0,
    saveState: 0,
    restoreState: 0,
    isEOF: 1,
    transformBy: { _id: 0, age: 1, name: 1 },
    inputStage: {
      stage: 'IXSCAN',
      nReturned: 3,
      executionTimeMillisEstimate: 1,
      works: 5,
      advanced: 3,
      needTime: 0,
      needYield: 0,
      saveState: 0,
      restoreState: 0,
      isEOF: 1,
      keyPattern: { age: 1, name: 1 },
      indexName: 'idx_age_name',
      isMultiKey: false,
      multiKeyPaths: { age: [], name: [] },
      isUnique: false,
      isSparse: false,
      isPartial: false,
      indexVersion: 2,
      direction: 'forward',
      indexBounds: { age: [ '[28, inf.0]' ], name: [ '[MinKey, MaxKey]' ] },
      keysExamined: 3,
      seeks: 1,
      dupsTested: 0,
      dupsDropped: 0
    }
  }
}

I can decide to force the other index with a hint:

mdb> db.demo.find(
  { age: { $gte: 28 } },          // WHERE age >= 28
  { _id:0, age:1, name: 1 }       // SELECT age, name
).hint("idx_age").explain("executionStats").executionStats

{
  executionSuccess: true,
  nReturned: 3,
  executionTimeMillis: 0,
  totalKeysExamined: 3,
  totalDocsExamined: 3,
  executionStages: {
    isCached: false,
    stage: 'PROJECTION_SIMPLE',
    nReturned: 3,
    executionTimeMillisEstimate: 0,
    works: 4,
    advanced: 3,
    needTime: 0,
    needYield: 0,
    saveState: 0,
    restoreState: 0,
    isEOF: 1,
    transformBy: { _id: 0, age: 1, name: 1 },
    inputStage: {
      stage: 'FETCH',
      nReturned: 3,
      executionTimeMillisEstimate: 0,
      works: 4,
      advanced: 3,
      needTime: 0,
      needYield: 0,
      saveState: 0,
      restoreState: 0,
      isEOF: 1,
      docsExamined: 3,
      alreadyHasObj: 0,
      inputStage: {
        stage: 'IXSCAN',
        nReturned: 3,
        executionTimeMillisEstimate: 0,
        works: 4,
        advanced: 3,
        needTime: 0,
        needYield: 0,
        saveState: 0,
        restoreState: 0,
        isEOF: 1,
        keyPattern: { age: 1 },
        indexName: 'idx_age',
        isMultiKey: false,
        multiKeyPaths: { age: [] },
        isUnique: false,
        isSparse: false,
        isPartial: false,
        indexVersion: 2,
        direction: 'forward',
        indexBounds: { age: [ '[28, inf.0]' ] },
        keysExamined: 3,
        seeks: 1,
        dupsTested: 0,
        dupsDropped: 0
      }
    }
  }
}

A hint() is more than a hint here. It forces the index scan. I can even force its bounds with min() and max():

mdb> db.demo.find(
  { age: { $gte: 28 } },          // WHERE age >= 28
  { _id:0, age:1, name: 1 }       // SELECT age, name
).hint("idx_age").max({age:31})   // scan from begining to age: 30

[ { name: 'Eve', age: 28 }, { name: 'Alice', age: 30 } ]

Rather than the bounds defined by the filter ([ '[28, inf.0]' ]), I forced an index scan of range [ '[-inf.0, 30]' ]. Then the filter { '$gte': 28 } applied and the result has only two documents.

This is visible in the execution plan.

mdb> db.demo.find(
  { age: { $gte: 28 } },          // WHERE age >= 28
  { _id:0, age:1, name: 1 }       // SELECT age, name
).hint("idx_age").max({age:31}).explain("executionStats").executionStats

{
  executionSuccess: true,
  nReturned: 2,
  executionTimeMillis: 0,
  totalKeysExamined: 4,
  totalDocsExamined: 4,
  executionStages: {
    isCached: false,
    stage: 'PROJECTION_SIMPLE',
    nReturned: 2,
    executionTimeMillisEstimate: 0,
    works: 5,
    advanced: 2,
    needTime: 2,
    needYield: 0,
    saveState: 0,
    restoreState: 0,
    isEOF: 1,
    transformBy: { _id: 0, age: 1, name: 1 },
    inputStage: {
      stage: 'FETCH',
      filter: { age: { '$gte': 28 } },
      nReturned: 2,
      executionTimeMillisEstimate: 0,
      works: 5,
      advanced: 2,
      needTime: 2,
      needYield: 0,
      saveState: 0,
      restoreState: 0,
      isEOF: 1,
      docsExamined: 4,
      alreadyHasObj: 0,
      inputStage: {
        stage: 'IXSCAN',
        nReturned: 4,
        executionTimeMillisEstimate: 0,
        works: 5,
        advanced: 4,
        needTime: 0,
        needYield: 0,
        saveState: 0,
        restoreState: 0,
        isEOF: 1,
        keyPattern: { age: 1 },
        indexName: 'idx_age',
        isMultiKey: false,
        multiKeyPaths: { age: [] },
        isUnique: false,
        isSparse: false,
        isPartial: false,
        indexVersion: 2,
        direction: 'forward',
        indexBounds: {},
        keysExamined: 4,
        seeks: 1,
        dupsTested: 0,
        dupsDropped: 0
      }
    }
  }
}

The bounds are not correctly displayed, but it shows that 4 keys were examined, the 4 keys before {age:31}.

The most important to remember is that hints in MongoDB replace the query planner decision, and then may may give a different results. This is different from SQL hints where they only prune the query planner possibilities.

This is also visible with sparse and partial indexes:

// Named sparse index
db.demo.createIndex(
{ age: 1 },
{ sparse: true, name: "age_sparse" }
)

// Named partial index (only for status: 'active')
db.demo.createIndex(
  { age: 1 },
  {
    partialFilterExpression: { status: "active" },
    name: "age_active_partial"
  }
);

The result depends not only on the query, but also the index definition:

mdb> db.demo.find(
).sort({ age: 1 })                 // ORDER BY age NULLS FIRST

[
  { _id: 2, name: 'Bob' },
  { _id: 3, name: 'Charlie', age: null },
  { _id: 5, name: 'Eve', age: 28, status: 'inactive' },
  { _id: 1, name: 'Alice', age: 30 },
  { _id: 4, name: 'Dave', age: 35, status: 'active' }
]

mdb> db.demo.find(
).hint("age_sparse").sort({ age: 1 })  // sparse index

[
  { _id: 3, name: 'Charlie', age: null },
  { _id: 5, name: 'Eve', age: 28, status: 'inactive' },
  { _id: 1, name: 'Alice', age: 30 },
  { _id: 4, name: 'Dave', age: 35, status: 'active' }
<... (truncated)