This SIGMOD’20 paper presents TaurusDB, Huawei's disaggregated MySQL-based cloud database. TaurusDB refines the disaggregated architecture pioneered by Aurora and Socrates, and provides a simpler and cleaner separation of compute and storage. 

In my writeup on Aurora, I discussed how "log is the database" approach reduces network load, since the compute primary only sends logs and the storage nodes apply them to reconstruct pages. But Aurora did conflate durability and availability somewhat and used quorum-based replication of six replicas for both logs and pages.

In my review of Socrates, I explained how Socrates (Azure SQL Cloud) separates durability and availability by splitting the system into four layers: compute, log, page, and storage. Durability (logs) ensures data is not lost after a crash. Availability (pages/storage) ensures data can still be served while some replicas or nodes fail. Socrates stores pages separately from logs to improve performance but the excessive layering introduces significant architectural overhead.

Taurus takes this further and uses different replication and consistency schemes for logs and pages, exploiting their distinct access patterns. Logs are append-only and used for durability. Log records are independent, so they can be written to any available Log Store nodes. As long as three healthy Log Stores exist, writes can proceed without quorum coordination. Pages, however, depend on previous versions. A Page Store must reconstruct the latest version by applying logs to old pages. To leverage this asymmetry, Taurus uses synchronous, reconfigurable replication for Log Stores to ensure durability, and asynchronous replication for Page Stores to improve scalability, latency, and availability.

But hey, why do we disaggregate in the first place?

Traditional databases were designed for local disks and dedicated servers. In the cloud, this model wastes resources as shown in Figure 1. Each MySQL replica keeps its own full copy of the data, while the underlying virtual disks already store three replicas for reliability. Three database instances mean nine copies in total, and every transactional update is executed three times. This setup is clearly redundant, costly, and inefficient.

Disaggregation fixes this and also brings true elasticity! Compute and storage are separated because they behave differently. Compute is expensive and variable; storage is cheaper and grows slowly. Compute can be stateless and scaled quickly, while storage must remain durable. Separating them allows faster scaling, shared I/O at storage, better resource use, and the capability of scaling compute to zero and restarting quickly when needed.

Architecture overview

Taurus has two physical layers, compute and storage, and four logical components: Log Stores, Page Stores, the Storage Abstraction Layer (SAL), and the database front end. Keeping only two layers minimizes cross-network hops.

The database front end (a modified MySQL) handles queries, transactions, and log generation. The master handles writes; read replicas serve reads.

Log Store stores (well duh!) write-ahead-logs as fixed-size, append-only objects called PLogs. These are synchronously replicated across three nodes. Taurus favors reconfiguration-based replication: If one replicaset fails or lags, a new PLog is created. Metadata PLogs track active PLogs.

Page Store materializes/manages 10 GB slices of page data. Each page version is identified by an LSN, and the Page Store can reconstruct any version. Pages are written append-only, which is 2–5x faster than random writes and gentler on flash. Each slice maintains a lock-free Log Directory mapping (page, version) to log offset. Consolidation of logs into pages happens in memory. Taurus originally prioritized by longest chain first, but then reverted to oldest unapplied write first to prevent metadata buildup. A local buffer pool accelerates log application. For cache eviction, Taurus finds that LFU (least frequently used) performs better than LRU (least recently used), because it keeps these hot pages in cache longer, reducing I/O and improving consolidation throughput. 

Storage Abstraction Layer (SAL) hides the storage complexity from MySQL by serving as an intermediary. It coordinates between Log Stores and Page Stores, manages slice placement, and tracks the Cluster Visible LSN, the latest globally consistent point. SAL advances CV-LSN only when the logs are durable in Log Stores and at least one Page Store has acknowledged them. SAL also batches writes per slice to reduce small I/Os.

Write path and replication

Did you notice the lack of LogStore to PageStore communication in Figure 2 and Figure 3? The paper does not address this directly, but yest there is no direct LogStore-to-PageStore communication. The SAL in the master mediates this instead. SAL first writes logs to the Log Stores for durability. Once acknowledged, SAL forwards the same logs to the relevant Page Stores. This ensures that Page Stores only see durable logs and lets SAL track exactly what each replica has received. SAL monitors per-slice persistent LSNs for Page Stores, and resends missing logs from the Log Stores if it detects regressions.

I think, this choice adds coupling and complexity. A chain-replication design, where LogStores streamed logs directly to PageStores, would simplify the system. This way, SAL wouldn't need to track every PageStore’s persistent LSN. And Log truncation could be driven by LogStores once all replicas confirmed receipt, instead of being tracked by SAL again. 

Read path

Database front ends read data at page granularity. A dirty page in the buffer pool cannot be evicted until its logs have been written to at least one Page Store replica. This ensures that the latest version is always recoverable.

As mentioned above, SAL maintains the last LSN sent per slice. Reads are routed to the lowest-latency Page Store replica. If one is unavailable or behind, SAL retries with others.

Read replicas don't stream WAL directly from the master. Instead, the master publishes which PLog holds new updates. Replicas fetch logs from the Log Stores, apply them locally, and track their visible LSN. They don't advance past the Page Stores' persisted LSNs, keeping reads consistent. This design keeps replica lag below 20 ms even under high load and prevents the master from becoming a bandwidth bottleneck.

Recovery model

If a Log Store fails temporarily, writes to its PLogs pause. For long failures, the cluster re-replicates its data to healthy nodes.

Page Store recovery is more involved. After short outages, a Page Store gossips with peers to catch up. For longer failures, the system creates a new replica by copying another's data. If recent logs were lost before replication, SAL detects gaps in persistent LSNs and replays the missing records from Log Stores. Gossip runs periodically but can be triggered early when lag is detected.

If the primary fails, SAL ensures all Page Stores have every log record persisted in Log Stores. This is the redo phase (similar to ARIES). Then the database front end performs undo for in-flight transactions.

I want you to understand what it is to live in Chicago.

Every day my phone buzzes. It is a neighborhood group: four people were kidnapped at the corner drugstore. A friend a mile away sends a Slack message: she was at the scene when masked men assaulted and abducted two people on the street. A plumber working on my pipes is upset, and I find out that two of his employees were kidnapped that morning. A week later it happens again.

An email arrives. Agents with guns have chased a teacher into the school where she works. They did not have a warrant. They dragged her away, ignoring her and her colleagues’ pleas to show proof of her documentation. That evening I stand a few feet from the parents of Rayito de Sol and listen to them describe, with anguish, how good Ms. Diana was to their children. What it is like to have strangers with guns traumatize your kids. For a teacher to hide a three-year-old child for fear they might be killed. How their relatives will no longer leave the house. I hear the pain and fury in their voices, and I wonder who will be next.

Understand what it is to pray in Chicago. On September 19th, Reverend David Black, lead pastor at First Presbyterian Church of Chicago, was praying outside the ICE detention center in Broadview when a DHS agent shot him in the head with pepper balls. Pepper balls are never supposed to be fired at the head, because, as the manufacturer warns, they could seriously injure or even kill. “We could hear them laughing as they were shooting us from the roof,” Black recalled. He is not the only member of the clergy ICE has assaulted. Methodist pastor Hannah Kardon was violently arrested on October 17th, and Baptist pastor Michael Woolf was shot by pepper balls on November 1st.

Understand what it is to sleep in Chicago. On the night of September 30th, federal agents rappelled from a Black Hawk helicopter to execute a raid on an apartment building on the South Shore. Roughly three hundred agents deployed flashbangs, busted down doors, and took people indiscriminately. US citizens, including women and children, were grabbed from their beds, marched outside without even a chance to dress, zip-tied, and loaded into vans. Residents returned to find their windows and doors broken, and their belongings stolen.

Understand what is is to lead Chicago. On October 3rd, Alderperson Jesse Fuentes asked federal agents to produce a judicial warrant and allow an injured man at the hospital access to an attorney. The plainclothes agents grabbed Fuentes, handcuffed her, and took her outside the building. Her lawsuit is ongoing. On October 21st, Representative Hoan Huynh was going door-to-door to inform businesses of their immigration rights when he was attacked by six armed CBP agents, who boxed in his vehicle and pointed a gun at his face. Huynh says the agents tried to bash open his car window.

Understand what it is to live in Chicago. On October 9th, Judge Ellis issued a temporary restraining order requiring that federal agents refrain from deploying tear gas or shooting civilians without an imminent threat, and requiring two audible warnings. ICE and CBP have flaunted these court orders. On October 12th, federal agents shoved an attorney to the ground who tried to help a man being detained in Albany Park. Agents refused to identify themselves or produce a warrant, then deployed tear gas without warning. On October 14th, agents rammed a car on the East Side, then tear-gassed neighbors and police.

On October 23rd, federal agents detained seven people, including two U.S. citizens and an asylum seeker, in Little Village. Two worked for Alderperson Michael Rodriguez: his chief of staff Elianne Bahena, and police district council member Jacqueline Lopez. Again in Little Village, agents tear-gassed and pepper-sprayed protestors, seizing two high school students and a security guard, among others. Alderperson Byron Sigcho-Lopez reported that agents assaulted one of the students, who had blood on his face. On October 24th, agents in Lakeview emerged from unmarked cars, climbed a locked fence to enter a private yard, and kidnapped a construction worker. As neighbors gathered, they deployed four tear-gas canisters. That same day, a few blocks away, men with rifles jumped out of SUVs and assaulted a man standing at a bus stop.

“They were beating him,” said neighbor Hannah Safter. “His face was bleeding”.

They returned minutes later and attacked again. A man from the Laugh Factory, a local comedy club, had come outside with his mother and sister. “His mom put her body in between them, and one of the agents kicked her in the face”.

Understand what it is to raise a family in Chicago. The next day, October 25th, federal agents tear-gassed children in Old Irving Park. Again, no warnings were heard. On October 26th, agents arrested a 70-year-old man and threw a 67-year old woman to the ground in Old Irving Park, then tear-gassed neighbors in Avondale. That same day, federal agents deployed tear gas at a children’s Halloween parade in Old Irving Park.

“Kids dressed in Halloween costumes walking to a parade do not pose an immediate threat to the safety of a law enforcement officer. They just don’t. And you can’t use riot control weapons against them,” Judge Ellis said to Border Patrol chief Gregory Bovino.

Understand how the government speaks about Chicago. On November 3rd, paralegal Dayanne Figueroa, a US citizen, was driving to work when federal agents crashed into her car, drew their guns, and dragged her from the vehicle. Her car was left behind, coffee still in the cup holder, keys still in the car. The Department of Homeland Security blamed her, claiming she “violently resisted arrest, injuring two officers.” You can watch the video for yourself.

“All uses of force have been more than exemplary,” Bovino stated in a recent deposition. He is, as Judge Ellis has stated, lying. Bovino personally threw a tear-gas canister in Little Village. He claimed in a sworn deposition that he was struck in the head by a rock before throwing the canister, and when videos showed no rock, admitted that he lied about the event. When shown video of himself tackling peaceful protestor Scott Blackburn, Bovino refused to acknowledge that he tackled the man. Instead, he claimed, “That’s not a reportable use of force. The use of force was against me.”

“I find the government’s evidence to be simply not credible,” said Judge Ellis in her November 6th ruling. “The use of force shocks the conscience.”

Understand what it is to be Chicago. To carry a whistle and have the ICIRR hotline in your phone. To wake up from nightmares about shouting militiamen pointing guns at your face. To rehearse every day how to calmly refuse entry, how to identify a judicial warrant, how to film and narrate an assault. To wake to helicopters buzzing your home, to feel your heart rate spike at the car horns your neighbors use to alert each other to ICE and CBP enforcement. To know that perhaps three thousand of your fellow Chicagoans have been disappeared by the government, but no one really knows for sure. To know that many of those seized were imprisoned a few miles away, as many as a hundred and fifty people in a cell, denied access to food, water, sanitation, and legal representation. To know that many of these agents—masked, without badge numbers or body cams, and refusing to identify themselves—will never face justice. To wonder what they tell their children.

The masked thugs who attack my neighbors, who point guns at elected officials and shoot pastors with pepper balls, who tear-gas neighborhoods, terrify children, and drag teachers and alderpeople away in handcuffs are not unprecedented. We knew this was coming a year ago, when Trump promised mass deportations. We knew it was coming, and seventy-seven million of us voted for it anyway.

This weight presses upon me every day. I am flooded with stories. There are so many I cannot remember them all; cannot keep straight who was gassed, beaten, abducted, or shot. I write to leave a record, to stare at the track of the tornado which tears through our city. I write to leave a warning. I write to call for help.

I want you to understand, regardless of your politics, the historical danger of a secret police. What happens when a militia is deployed in our neighborhoods and against our own people. Left unchecked, their mandate will grow; the boundaries of acceptable identity and speech will shrink. I want you to think about elections in this future. I want you to understand that every issue you care about—any hope of participatory democracy—is downstream of this.

I want you to understand what it is to love Chicago. To see your neighbors make the heartbreaking choice between showing up for work or staying safe. To march two miles long, calling out: “This is what Chicago sounds like!” To see your representatives put their bodies on the line and their voices in the fight. To form patrols to walk kids safely to school. To join rapid-response networks to document and alert your neighbors to immigration attacks. For mutual aid networks to deliver groceries and buy out street vendors so they can go home safe. To talk to neighbor after neighbor, friend after friend, and hear yes, yes, it’s all hands on deck.

I want you to understand Chicago.

Query compilation based on the produce/consume model has a reputation for delivering high query performance, but also for being more difficult to implement than interpretation-based engines using vectorized execution. While it is true that building a production-grade query compiler with low compilation latency requires substantial infrastructure and tooling effort what is sometimes overlooked is that the vectorization paradigm is not easy to implement either, but for different reasons.

Vectorization relies on tuple-at-a-time rather than vector-at-a-time processing. High-level operations (e.g., inserting tuples into a relation) must be decomposed into per-attribute vector kernels that are then executed successively. This requires a specific way of thinking and can be quite challenging, depending on the operator. In practice, the consequence is that the range of algorithms that can be realistically implemented in a vectorized engine is limited.

In contrast, compilation is fundamentally simpler and more flexible in terms of the resulting code. It supports tuple-at-a-time processing, which makes it easier to implement complex algorithmic ideas. Developing a query processing algorithm typically involves three steps: First, implement the algorithm manually as a standalone program, i.e., outside the compiling query engine. Second, explore different algorithms, benchmark, and optimize this standalone implementation. Finally, once a good implementation has been found, modify the query compiler to generate code that closely resembles the manually developed version.

At TUM, we use a simple pedagogical query compilation framework called p2c ("plan to C") for teaching query processing. It implements a query compiler that, given a relational operator tree as input, generates the C++ code that computes the query result. The core of the compiler consists of about 600 lines of C++ code and supports the TableScan, Selection, Map, Sort, Aggregation, and Join operators.

The generated code is roughly as fast as single-threaded DuckDB and can be easily inspected and debugged, since it is simply straightforward C++ code. This makes it easy for students to optimize (e.g., by adding faster hash tables or multi-core parallelization) and to extend (e.g., with window functions).

To keep p2c simple, it does not support NULL values or variable-size data types (strings have a fixed maximum length). Compiling to template-heavy C++ results in very high compilation times, making it impractical for production use. At the same time, p2c employs the same core concepts as Hyper and Umbra, making it an excellent starting point for learning about query compilation. Note that this type of compilation to C++ can also serve as an effective prototyping platform for research that explores new query processing ideas.

Find the code for p2c on github: https://github.com/viktorleis/p2c

If this were one person, I wouldn’t write this publicly. Since there are apparently multiple people on board, and they claim to be looking for investors, I think the balance falls in favor of disclosure.

Last week Prothean Systems announced they’d surpassed AGI and called for the research community to “verify, critique, extend, and improve upon” their work. Unfortunately, Prothean Systems has not published the repository they say researchers should use to verify their claims. However, based on their posts, web site, and public GitHub repositories, I can offer a few basic critiques.

ARC-AGI-2

First: Prothean’s core claim, repeated in the white paper, the FAQ, and the press release, is that Prothean has successfully solved all 400 tasks of the ARC-AGI-2 benchmark:

We present Prothean Emergent Intelligence, a novel five-pillar computational architecture that achieved 100% accuracy on all 400 tasks of the ARC-AGI-2 challenge in 0.887 seconds.

This system achieved 100% accuracy on all 400 tasks of the ARC-AGI-2 challenge in 0.887 seconds—a result previously considered impossible for artificial intelligence systems.

ARC-AGI-2 is specifically designed to resist gaming:

• 400 unique, novel tasks
• No training data available

It is not possible to solve “all 400 tasks of the ARC-AGI-2 challenge” because ARC-AGI-2 does not have 400 evaluation tasks to solve. Nor is it true that ARC-AGI-2 has no training data. The ARC-AGI-2 web site and docs are clear about this: the workload consists of “1,000 public training tasks and 120 public evaluation tasks”.

Prothean System’s “verification protocol” instructs researchers to download these 400 tasks from a non-existent URL:

Step 3: Obtain ARC-AGI-2 Dataset

# Download official dataset
wget https://github.com/fchollet/ARC-AGI/arc-agi-2-dataset.zip

This isn’t even the right repository. ARC-AGI-2 is at https://github.com/arcprize/ARC-AGI-2.

Local Operations

Prothean Systems repeatedly claims that its operations are purely local. The announcement post states:

🧬 WE SURPASSED AGI. THIS IS EGI. NOT AI. HERE’S THE PROOF.

Welcome to Prothean EGI.
──────────
After years of research and months of physical development, I’ve been building something that shouldn’t exist yet.

After months of solo development, I put together a small strike force team—the Prothean Systems team—to bring this vision to life.

Prothean Systems — Emergent General Intelligence (EGI) that lives on your device, remembers everything, and never leaves your hands.

No servers. No uploads. No corporate AI deciding what you can know.

The demo on beprothean.org also advertises “no servers” and “zero uploads”. This is obviously false. Typing almost anything into this box, such as “what are the five pillars”, triggers two queries to Wikipedia and ten messages, including a mix of reads and writes, to the Firebase hosted database service:

Compression

Prothean’s launch video explains that Prothean uses a multi-tier compression system called “Memory DNA”. This system “actually looks at the data, understands its characteristics, and then applies the absolute best compression technique for it”. The white paper lists nine compression tiers:

Compression Cascade:
1. Semantic Extraction (40% reduction) - Core meaning preservation
2. Pattern Recognition (25% reduction) - Recurring structure identification
3. Fibonacci Sequencing (15% reduction) - φ-ratio temporal organization
4. Harmonic Resonance (10% reduction) - Frequency-domain optimization
5. Contextual Pruning (8% reduction) - Relevance-based filtering
6. Recursive Abstraction (6% reduction) - Meta-pattern extraction
7. Golden Ratio Weighting (4% reduction) - φ-importance scaling
8. Temporal Decay (3% reduction) - Age-based compression
9. Neural Synthesis (2% reduction) - Final integration

The demo on beprothean.org does nothing of the sort. Instead, it makes a single call to the open-source lz-string library, which is based on LZW, a compression algorithm from 1984.

function demoMemoryDNA(){
  const text = $('mdna-input').value.trim();
  ...
  const compressed = LZString.compressToUTF16(text);

Guardian

Prothean advertises a system called “Guardian”: an “integrity firewall that validates every operation” and “detects sensitive data, prevents drift, enforces alignment at runtime”. The demo does not appear to address drift or alignment in any way. Instead, it matches strings against three regular expressions, looking for things shaped like e-mail addresses, credit cards, and API keys.

// Guardian: Pattern detection & integrity validation
function runGuardian(text){
  ...
  if(/\b\w+@\w+\.\w+/.test(text)){ patterns.push('email'); threatLevel += 1; }
  if(/\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}/.test(text)){ patterns.push('card'); threatLevel += 3; }
  if(/(password|secret|api[_-]?key|token)/i.test(text)){ patterns.push('secret'); threatLevel += 2; }
  ...
  return { patterns, threatLevel, integrity, safe: threatLevel < 5 };
}

Semantic Bridging

Prothean’s “Universal Pattern Engine” invites users to “connect two concepts through semantic bridging”. This semantic bridging system has no awareness of semantics. Instead, it is based entirely on the number of letters in the two concepts and picking some pre-determined words out of a list:

  const a = $('upe-a').value.trim();
  const b = $('upe-b').value.trim();
  ...

  // Semantic bridge building
  const bridges = ['system', 'design', 'architecture', 'implementation', 'protection',
                   'control', 'ownership', 'agency', 'autonomy', 'dignity', 'privacy'];

  // Calculate semantic distance (simulated)
  const similarity = (a.length + b.length) % 10 / 10;
  ...
  const selectedBridges = bridges.slice(0, 3 + Math.floor(similarity * 3));

Tree Height

The white paper describes a “Radiant Data Tree” structure, which uses a "Fibonaci-branching tree structure with φ-ratio organization. The depth of this tree is given as:

Depth(n) = φ^n / √5 (rounded to nearest Fibonacci number)

Depth is generally understood to be a property of a specific node, not a tree as a whole—I assume they mean Height(n). In either case this is impossible. The height and maximum depth of a tree are bounded by the number of edges between the deepest node and the root. Trees cannot have a height which grows as φⁿ, because this implies that the depth increases faster than the number of nodes. Per this equation, a “Radiant Data Tree” containing just six nodes must have a height of eight—and must therefore contain at least nine nodes, not six.

Transcendence Score

The white paper also introduces a transcendence score T which measures “mathematical beauty and emergent capability”:

T = (0.25×C + 0.25×(1-M) + 0.3×N + 0.2×P) × φ mod 1.0

Where:

• C = Complexity handling (0 to 1)
• M = Memory compression ratio (0 to 1)
• N = Neural emergence score (0 to 1)
• P = Pattern recognition quality (0 to 1)
• φ = golden ratio (1.618…)

Speaking broadly, a quality metric should increase as its inputs improve. This score does not. As each metric improves, the transcendence score rises—until roughly halfway through it reaches one, and immediately wraps around to zero again. Consequently, small improvements in (e.g.) “pattern recognition quality” can cause T to fall dramatically. The white paper goes on to claim specific values from their work which are subject to this modular wrapping.

There’s a lot more here, but I don’t want to burn too much time on this.

Large Language Model Hazards

Based on the commit history and prose style, I believe Prothean is largely the the product of Large Language Models (LLMS). LLMs are very good at producing plausible text which is not connected to reality. People also like talking to LLMs: the architecture, training processes, and financial incentives around chatbots bias them towards emitting agreeable, engaging text. The technical term for this is “sycophancy”.

The LLM propensity for confabulation and reward hacking has led to some weird things in the past few years. People—even experts in a field—who engage heavily with LLMs can convince themselves that they have “awoken” an LLM partner, or discovered a novel form of intelligence. They may also become convinced that they have made a scientific breakthrough, especially in the field of AI.

Please be careful when talking to LLMs.