The relational model was designed not to mirror application structures, but to optimize reasoning about data dependencies independently of access patterns. By restricting data to relations in First Normal Form (1NF) and enforcing closure, relational algebra makes query behavior mathematically derivable, so correctness, equivalence, and optimization follow from the model itself rather than implementation-specific rules.

Document databases take the opposite approach: they optimize representation for applications, network, and storage. Domain concepts such as ownership, lifecycle, and cardinality are embedded directly in the data’s shape. Without assuming value atomicity, algebra no longer applies, so semantics must be defined explicitly. This is what MongoDB did when defining query behavior in its document database.

Applications rarely start from relations

Most applications do not model data as relations. They model aggregates. It's obvious in Domain-Driven Design and Object-Oriented analysis, but it was true long before object‑oriented languages. Data structures in applications have always been hierarchical, with sub-structures and repeating groups:

• Record-based models include nested tables, like in the COBOL data division where relationships are expressed through containment and repetition:

01 EMPLOYEE.
   05 NAME        PIC X(20).
   05 SKILLS.
      10 SKILL    PIC X(10) OCCURS 5 TIMES.

• Object‑oriented models are more flexible but follow the same pattern with embedded objects, arrays, or lists:

@Entity
class Employee {
    @Id Long id;
    String name;
    @ElementCollection
    List<String> skills;
}

These models express semantics that do not exist in relational algebra:

• ownership: the skills belong to the employee
• lifecycle: the root aggregate is deleted with all its elements
• cardinality: this model supposes a short and bounded list of skills per employee, but a large and growing number of employees
• locality: employee's skills are queried with the employee, and stored together on disk

Those models are optimized for application behavior, not for algebraic reasoning. This is not new. It is how applications have always been written. Why do relational systems require data to be represented as flat relations at the logical level?

Why the relational model exists

When E. F. Codd published “Derivability, Redundancy and Consistency of Relations Stored in Large Data Banks” in 1969, his objective was not to help developers write applications, but to establish a formal mathematical foundation for data management.

He based the relational model and relational algebra on mathematics:

• Set theory for relations, with operations such as ∪ (union), ∩ (intersection), − (difference), and × (Cartesian product)
• First‑order predicate logic for constraints and querying: selection (σ) corresponds to logical predicates, and joins correspond to conjunction with implicit existential quantification (∃) over the join attributes
• A closed algebraic query language at the logical level, where every operation produces another relation

Within this framework, a relation is defined as:

• a set of tuples (unordered, with no duplicates)
• where all tuples share the same attributes
• and every attribute value is drawn from a simple (atomic) domain

These properties are not modeling advice. They are the definition.

First normal form

First Normal Form (1NF) is often presented as a design guideline. In Codd’s original work, it is not. It is a mandatory constraint to apply the first‑order predicate logic.

Without atomic attribute values, relations cease to be relations as defined by Codd, and the standard relational algebra no longer applies. Comparisons become ambiguous, predicates are no longer boolean, and algebraic equivalence rules break down.

Relational algebra is a closed system where inputs and outputs are relations. Its operators—selection (σ), projection (π), join (⨝)—all assume that:

• attributes can be compared using equality
• predicates evaluate to true or false
• each comparison involves exactly one value

This is what enables equivalence rules, join reordering, and provable optimizations, and the maths is defined for atomic values only.

Let's see some examples with SQL, which is inspired by the relational model (but is not itself a pure relational language).

1NF to apply mathematical operations to relations

Here is an example with a table of employees' skills:

CREATE TABLE employee_skill (
    employee_name TEXT,
    skill TEXT
);

INSERT INTO employee_skill VALUES
   ('Ann', 'SQL'),
   ('Ann', 'Java'),
   ('Ann', 'Python'),
   ('Bob', 'Java')
;

A simple selection involves a predicate comparing a column to a value:

SELECT DISTINCT *
   FROM employee_skill
   WHERE skill = 'Java'
;

It returns another relation with only the facts that verify the predicate:

 employee_name | skill
---------------+-------
 Ann           | Java
 Bob           | Java

This works because skill is atomic, equality has one meaning, and the result is still a relation.

This collapses without 1NF

Here is another model, simple, but that violates 1NF:

CREATE TABLE employee_array (
    name TEXT,
    skills TEXT[]
);

INSERT INTO employee_array VALUES
   ('Ann', ARRAY['SQL', 'Java', 'Python']),
   ('Bob', ARRAY['Java'])
;

The same predicate no longer applies:

SELECT DISTINCT *
   FROM employee_array
   WHERE skills = 'Java'
;


ERROR:  malformed array literal: "Java"
LINE 3: WHERE skills = 'Java';
                       ^
DETAIL:  Array value must start with "{" or dimension information.

PostgreSQL requires new operators:

SELECT DISTINCT *
   FROM employee_array
   WHERE 'Java' = ANY(skills)
;

 name |      skills
------+-------------------
 Ann  | {SQL,Java,Python}
 Bob  | {Java}

(2 rows)

SELECT DISTINCT *
   FROM employee_array
   WHERE skills @> ARRAY['Java']
;

 name |      skills
------+-------------------
 Ann  | {SQL,Java,Python}
 Bob  | {Java}

(2 rows)

These operators encode membership and containment. They are not part of relational algebra, and their exact semantics and syntax are vendor‑specific in SQL systems.

SQL/JSON does not restore relational semantics

JSON and JSONB datatypes do not change this:

CREATE TABLE employee_json (
    doc JSONB
);

INSERT INTO employee_json VALUES
  ('{"name": "Ann", "skills": ["SQL", "Java", "Python"]}'),
  ('{"name": "Bob", "skills": ["Java"]}')
;

Selections rely on path navigation and containment, with special operators specific to the datatype:

SELECT DISTINCT *
   FROM employee_json
   WHERE doc->'skills' ? 'Java'
;

                         doc
------------------------------------------------------
 {"name": "Ann", "skills": ["SQL", "Java", "Python"]}
 {"name": "Bob", "skills": ["Java"]}

(2 rows)

SELECT DISTINCT *
   FROM employee_json
   WHERE doc->'skills' @> '["Java"]'
;

                         doc
------------------------------------------------------
 {"name": "Ann", "skills": ["SQL", "Java", "Python"]}
 {"name": "Bob", "skills": ["Java"]}

(2 rows)

This involves again different datatypes, operators, semantic, and also indexing. A different type of index is required to serve those predicates, a GIN index in PostgreSQL, with different syntax and different capabilities.

JSON has been added to the SQL standard as SQL/JSON but this doesn't unify the semantics. For example, an SQL array starts at 1 and a JSON array starts at 0:

SELECT 
 skills[0] "0",
 skills[1] "1",
 skills[2] "2",
 skills[3] "3"
FROM employee_array
;

 0 |  1   |  2   |   3
---+------+------+--------
   | SQL  | Java | Python
   | Java |      |

(2 rows)

SELECT 
 doc->'skills'->0 "0",
 doc->'skills'->1 "1",
 doc->'skills'->2 "2",
 doc->'skills'->3 "3"
FROM employee_json
;

   0    |   1    |    2     | 3
--------+--------+----------+---
 "SQL"  | "Java" | "Python" |
 "Java" |        |          |

(2 rows)

JSON support in RDBMS extends SQL beyond relational algebra and introduces datatype‑specific semantics that are not algebraically closed. This is expected and was foreseen when enforcing the first normal form. Codd’s insight was that once attributes stop being atomic, mathematics no longer dictates behavior. Meaning must be defined explicitly.

MongoDB’s added semantics

MongoDB embraces the document model directly to match the data representation in the domain model and application structures:

db.employees.insertMany([
  { name: "Bob", skills: "Java" },
  { name: "Ann", skills: ["SQL", "Java", "Python"] }
]);

This is intentionally not 1NF because multiple entities and values may belong to the same aggregate. The relational operations cannot simply use the mathematical definition.

Selection resembles the relational operation, but when applied to a non‑1NF collection, MongoDB defines an explicit, extended semantics:

db.employees.find({ skills: "Java" })
;

[
  {
    _id: ObjectId('69a0ccaece22bf6640d4b0c2'),
    name: 'Bob',
    skills: 'Java'
  },
  {
    _id: ObjectId('69a0ccaece22bf6640d4b0c3'),
    name: 'Ann',
    skills: [ 'SQL', 'Java', 'Python' ]
  }
]

The same predicate applies to scalars and arrays. The document matches if the value or any array element satisfies the filtering condition. The same in SQL would require a union between a query using the SQL selection and another using the JSON containment, and casting the final result to the same datatype.

With MongoDB, indexes, comparisons, and sorting follow the same rule, as confirmed by execution plans. I create an index on skills that can index scalar values as well as array items, with one index type, and a generic syntax:

db.employees.createIndex({ skills: 1 })
;

It is easy to verify that the index is used to find the documents on a multi-key path (which means including an array in the path):

db.employees.find(
 { skills: "Java" }
).explain().queryPlanner.winningPlan

{
  isCached: false,
  stage: 'FETCH',
  inputStage: {
    stage: 'IXSCAN',
    keyPattern: { skills: 1 },
    indexName: 'skills_1',
    isMultiKey: true,
    multiKeyPaths: { skills: [ 'skills' ] },
    isUnique: false,
    isSparse: false,
    isPartial: false,
    indexVersion: 2,
    direction: 'forward',
    indexBounds: { skills: [ '["Java", "Java"]' ] }
  }
}

The multi-key index has one index entry for each item when it is an array. When a comparison operator is applied to an array field, MongoDB compares the query value to each array element individually.

I used an equality predicate but the indexBounds in the execution plan show that the same can apply to a range. The same index is used for non-equality predicates:

db.employees.find({ skills: { $gt: "M"} })
;

[
  {
    _id: ObjectId('69a0ccaece22bf6640d4b0c3'),
    name: 'Ann',
    skills: [ 'SQL', 'Java', 'Python' ]
  }
]

db.employees.find({ skills: { $lt: "M"} })
;

[
  {
    _id: ObjectId('69a0ccaece22bf6640d4b0c2'),
    name: 'Bob',
    skills: 'Java'
  },
  {
    _id: ObjectId('69a0ccaece22bf6640d4b0c3'),
    name: 'Ann',
    skills: [ 'SQL', 'Java', 'Python' ]
  }
]

Only Ann has a skill with a name after 'M' in the alphabet. Both Ann and Bob have a skill with a name before 'M' in the alphabet.

When sorting on an array field, MongoDB uses the minimum array element for ascending sort and the maximum array element for descending sort, according to BSON comparison order:

db.employees.find().sort({ skills: 1 })
;

[
  {
    _id: ObjectId('69a0ccaece22bf6640d4b0c2'),
    name: 'Bob',
    skills: 'Java'
  },
  {
    _id: ObjectId('69a0ccaece22bf6640d4b0c3'),
    name: 'Ann',
    skills: [ 'SQL', 'Java', 'Python' ]
  }
]

db.employees.find().sort({ skills: -1 })
;

[
  {
    _id: ObjectId('69a0ccaece22bf6640d4b0c3'),
    name: 'Ann',
    skills: [ 'SQL', 'Java', 'Python' ]
  },
  {
    _id: ObjectId('69a0ccaece22bf6640d4b0c2'),
    name: 'Bob',
    skills: 'Java'
  }
]

Here, 'Java' is the first in alphabetical order, so both employees are at the same rank in an ascending sort, but 'SQL' is the last in alphabetical order so 'Ann' appears first in a descending sort.

Again, the index is used:

db.employees.find().sort({ skills: 1 }).explain().queryPlanner.winningPlan
;

{
  isCached: false,
  stage: 'FETCH',
  inputStage: {
    stage: 'IXSCAN',
    keyPattern: { skills: 1 },
    indexName: 'skills_1',
    isMultiKey: true,
    multiKeyPaths: { skills: [ 'skills' ] },
    isUnique: false,
    isSparse: false,
    isPartial: false,
    indexVersion: 2,
    direction: 'forward',
    indexBounds: { skills: [ '[MinKey, MaxKey]' ] }
  }
}

MongoDB is optimized for scalable OLTP, index access is a must for equality and range predicates as well as sorting for pagination. In SQL databases, inverted indexes such as GIN are typically specialized for containment and equality predicates and offer more limited support for range ordering and pagination than B‑tree indexes.

Not forcing First Normal Form allows storage and indexing to remain efficient:

• compound index may include fields from multiple entities within one aggregate
• storage involves a single disk I/O per aggregate

By deviating from 1NF, closure is not guaranteed—by design. An explicit $unwind operation in an aggregation pipeline can normalize the result to a relation, in its mathematical sense, if needed.

With MongoDB, we can list the skills of employees who have a 'Java' skills, with all their skill, as a relational result:

db.employees.aggregate([  
  { $match: { skills: "Java" } },  
  { $unwind: "$skills" },
  { $project: { _id: 0, name: "$name", skill: "$skills" } }
])
;

[
  { name: 'Bob', skill: 'Java' },
  { name: 'Ann', skill: 'SQL' },
  { name: 'Ann', skill: 'Java' },
  { name: 'Ann', skill: 'Python' }
]

This simple query in MongoDB would be much more complex with the PostgreSQL examples above:

-- Correlated semi-join (EXISTS) over a normalized relation
SELECT DISTINCT es.employee_name AS name, es.skill  
  FROM employee_skill es  
  WHERE EXISTS (  
    SELECT 1  
    FROM employee_skill j  
    WHERE j.employee_name = es.employee_name  
      AND j.skill = 'Java'
;  

-- Existential quantification (ANY) over a non-1NF attribute (ARRAY) with explicit normalization (UNNEST)
SELECT DISTINCT ea.name, s.skill  
  FROM employee_array ea  
  CROSS JOIN LATERAL unnest(ea.skills) AS s(skill)  
  WHERE 'Java' = ANY (ea.skills)
;  

-- JSON containment predicate (@>) with explicit normalization (jsonb_array_elements)
SELECT DISTINCT doc->>'name' AS name, skill  
  FROM employee_json  
  CROSS JOIN LATERAL jsonb_array_elements_text(doc->'skills') AS s(skill)  
  WHERE doc->'skills' @> '["Java"]'
;  

Those queries must return all employees's skill, not only the one used for the filter, because they are part of the same aggregate. With those SQL queries, the object-relational mapping (ORM) must regroup those rows to build the aggregate.

In practice, the MongoDB query will not even $unwind to mimic a relation as it gets directly the aggregate:

db.employees.aggregate([  
  { $match: { skills: "Java" } },  
  { $project: { _id: 0, name: 1, skills: 1 } }
])
;

[
  { name: 'Bob', skills: 'Java' },
  { name: 'Ann', skills: [ 'SQL', 'Java', 'Python' ] }
]

With this query, MongoDB returns the binary BSON object directly to the application driver, instead of converting it into records or JSON like most SQL databases.

Conclusion

We exposed the enhanced semantics for selection over a non-1NF collection, as an example. MongoDB does more than enhance selection. All relational operations are extended with a document semantics:

• Selection works over scalars and arrays
• Projection reshapes documents
• Sort semantics are defined over arrays
• Indexes apply uniformly to scalars and array elements
• Joins exist ($lookup), and the semantics is defined even if the key is an array.

Relational theory is independent of physical implementation, but most RDBMS map each relation to a single table, and an index can cover the columns from a single table. Relational databases stem from mathematics and use normalization to structure storage. In contrast, applications center on aggregates, and MongoDB preserves these aggregates down to the storage layer.

First Normal Form (1NF) is required by the relational model—and therefore by relational SQL databases—because relational algebra assumes atomic attributes. In contrast, document databases relax these algebraic constraints in favor of explicit, document-oriented semantics, enabling similar operations to be performed directly on documents. So, when you hear that “data is relational” or feel you must apply normal forms to your domain model, recognize that this perspective is tied to a specific relational database implementation, not the nature of your data. The same data can support the same application with a document model.

Both models are valid but optimize for different abstraction layers: relational algebra offers a logical model for reasoning about data independently of the domain, while document databases model data as applications already do.

This is an external post of mine. Click here if you are not redirected.

I had written earlier that the first step of my paper reading process is actually printing the paper. I like to physically touch the paper and handwrite and doodle in the margins. For years, a Pilot Metropolitan fountain pen loaded with Waterman blue-black ink was my weapon of choice for wrestling with the papers. For thinking hard and for getting things out of my chest (exploring how I feel about something), that I also relied on that fountain pen.

This past year, I served as a Program Committee member for SOSP, OSDI, NSDI, and ATC. This meant reviewing about 15 heavy-duty papers for each conference. My HP laser printer is 20 years old and has finally gotten glitchy (the paper feed is broken, requiring me to feed pages carefully lest it jams). Facing the review workload, I looked for a solution that could save my sanity, and some trees as well.

So I bought a reMarkable Paper Pro (RMPP). This is an e-ink reader/writer with color and 11.8 inch screen. I knew what I was getting into:  muted colors, screen flashing, and UI slowness. There are no software apps on the tablet; it is strictly a digital paper substitute.

The claim is that this forced minimalism is the appeal of RMPP compared to an iPad, which comes feature-maxxed and distraction-heavy. After a year of using the RMPP, I agree.

Beyond focus, the writing feel is a selling point. The RMPP does well here, whereas the iPad still feels like writing on glass (though I hear that PaperLike screen protector for iPad improves the experience by providing paper-like friction). Does the RMPP fully replace the soul of a Pilot Metropolitan? No, it doesn't. But it gets close enough for doing some longhand thinking, and it is editable and searchable (unlike my fountain pen).  

Battery life is another interesting trade-off. E-ink is incredibly power efficient, but since the RMPP screen is somewhat dark (far from a bright paper-white background), I have to use it with the backlight. This reduces the battery duration from weeks to a couple days, depending on use. Well, a standard iPad would also last through a full day of work as well, which makes it also acceptable. While e-ink may be better on the eyes compared to the iPad's LCD screen, and works well under sunlight, let's be honest, how many times are you actually going to use the RMPP outside?

Overall, I can't complain about the RMPP. It solved my paper reviewing use case, and serves as a scratchpad during Zoom meetings. It has also taken over as my notebook for longhand writing. But at the end of the day, it remains a niche product. Considering the cost of RMPP versus iPad, an iPadAir with 13 inch screen is likely a better deal (if you can de-claw its distractions). The RMPP gels really well with some people, but its limitations will rub many the wrong way.

So, do your own research before you get a tablet. But as I wrote in my previous post, having a plan to escape the computer screen is a smart move. It is getting harder to do deep work in front of a glowing monitor, and securing an "analog" escape (be it a digital tablet, or a physical notebook) is important for doing more focused and centered thinking.

I quit my job at EnterpriseDB hacking on PostgreSQL products last month to start a company researching and writing about software infrastructure. I believe there is space for analysis that is more focused on code than TechCrunch or The Register, more open to covering corporate software development than LWN.net, and (as much as I love some of these folks) less biased than VCs writing about their own investments.

I believe that more than ever there is a need for authentic and trustworthy analysis and coverage of the software we depend on.

This company, The Consensus, will talk about databases and programming languages and web servers and everything else that is important for experienced developers to understand and think about. It is independent of any software vendor and independent of any particular technology.

Some people were surprised (in a positive way) to see me cover MySQL already, for example. But that is exactly the point.

I don't want The Consensus to be just "Phil's thoughts". I have already started working with a number of experienced developers who will be writing, and paid to write, for The Consensus.

I also hope that this is another way, beyond the many communities I already run, to give back to the community such as in highlighting the work of open-source developers (the first interview with a DataFusion developer is coming soon), and highlighting compelling events and jobs in the software infrastructure world.

The Consensus is entirely bootstrapped and will depend on the support of subscribers and, potentially, sponsors. The first few subscribers signed up just this past week.

You can read more about the background and goals here, you can read about how contributors will work with The Consensus here, and you can get a sense for where this is going by browsing the homepage of The Consensus already.

Thank you for your support in advance! Thank you to the folks who have subscribed already despite very little fanfare. Feedback is very welcome. I'm very excited and having quite a bit of fun already. We're all going to learn a lot.

Last year I shared many posts documenting MariaDB performance for vector search using ann-benchmarks. Performance was great in MariaDB 11 and this blog post explains that it is even better in MariaDB 12. This work was done by Small Datum LLC and sponsored by the MariaDB Foundation. My previous posts were published in January and February 2025.

tl;dr

• Vector search recall vs precision in MariaDB 12.3 is better than in MariaDB 11.8
• Vector search recall vs precision in Maria 11.8 is better than in Postgres 18.2 with pgvector 0.8.1
• The improvements in MariaDB 12.3 are more significant for larger datasets
• MariaDB 12.3 has the best results because it use less CPU per query

A few years ago I started getting issues of Modern Luxury in the mail. I had no idea why they started coming, and I tried to get them to stop. This should have been easy, and was instead hard. Here’s my process, in case anyone else is in the same boat.

First, if you use it, try to unsubscribe via PaperKarma. This is convenient and works for a decent number of companies. PaperKarma kept reporting they’d successfully unsubscribed me, but Modern Luxury kept coming.

Second, write to subscriptions@modernluxury.com. I got no response.

Third, call any numbers you can find associated with the company. Leave voicemails on anything that claims to be Modern Luxury related. Along this path I wound up discovering a Borgesian labyrinth of sketchy offers for life-alert style emergency devices and other things that felt vaguely like elder abuse; long story short, this did not work.

Fourth, Modern Luxury’s email format is [first initial][last name]@modernluxury.com. Start writing emails to a few names from your local edition that seem relevant, like the local publisher and editor. When they don’t respond, expand your emails to include everyone listed in the magazine. Start digging through corporate filings of their parent company, Cumulus Media, and emailing people there. Start short and simple; when that doesn’t work, try humor. This didn’t work either, but it was fun to write:

I love me some esoteric rich people nonsense. Fabergé eggs! Ominous lawn obelisks! Having oneself taxidermied and wheeled out for council meetings of University College London! Unfortunately, Modern Luxury contains nothing like this; perhaps rich people have forgotten how to be interesting. In any event, I would like you to stop. If you can figure out how to stop sending me magazines, I promise to stop sending you emails about it, and we can all go on to live happy lives.

Contraluxuriantly,

Kyle Kingsbury

Finally, cut out a suitable article from an issue of the magazine. Look up up the home address of the regional group publisher in city records. Mail the article back to the publisher, along with a letter asking them to stop.

Dear Mr. Uslan,

As the regional group publisher of Modern Luxury magazine, I would like you to stop publishing Modern Luxury to my home each month. I never asked for it, and I have been trying to unsubscribe for years. E-mails, phone calls, Paper Karma: nothing works. I appreciate your most recent column, entitled “Spirit of Generosity”, but please: it is possible to be too generous. Kindly stop sending these magazines.

Exhaustedly,

Kyle Kingsbury

This actually seems to have worked.

I think a lot about this idea of the Annoyance Economy—that modern life places ordinary people in contact with a dizzying array of opaque, nonresponsive bureaucracies, and that those bureaucracies have financial incentives to ignore you. This is why it’s so hard to replace a CPAP or get paid back when movers break things. This is why Redplum (one of those advertising/coupon mailers) ignored my unsubscribe requests for years, and only stopped when I started e-mailing the entire C-suite about it. I try to pick and choose these battles, but sometimes it’s hard to let it go. And goshdarnit, if nobody pushes back then bureaucratic indifference works, and we all have to live with it.

I don’t want to bother people like this; I think it’s unreasonably rude. I still start with the official support channels and escalate gradually. I like Patrick McKenzie’s strategy of presenting oneself as a boring, dangerous professional. However, I have also found that in the Annoyance Economy, one of the ways to get things done is to find specific people with power, and annoy them right back.

I hope this whole misadventure convinced Modern Luxury to build and document an easy unsubscribe process. If not, you know what to do.