Full Text Search over Postgres: Elasticsearch vs. Alternatives
A comprehensive comparison between Elasticsearch and PostgreSQL for search workloads.
July 31, 2024
Full Text Search over Postgres: Elasticsearch vs. Alternatives
A comprehensive comparison between Elasticsearch and PostgreSQL for search workloads.
July 30, 2024
Faster backups with sharding
Sharding a database comes with many benefits: Scalability, failure isolation, write throughput, and more. However, one of the lesser-known benefits comes from improved backups and restore performance.
Delightful, production-grade replication for Postgres
This is an external post of mine. Click here if you are not redirected.
July 29, 2024
Building data pipelines with Vitess
Learn the basics of Change Data Capture (CDC) and how to leverage Vitess VStream API to build data pipelines.
Import Postgres tables into Tinybird with the PostgreSQL Table Function
Today we announce the PostgreSQL Table Function, a powerful and flexible way to import and sync data from Postgres to Tinybird with just a few lines of SQL.
A Deep Dive into German Strings
A Deep Dive into German Strings
“Strings are Everywhere”! At least according to a 2018 DBTest Paper from the Hyper team at Tableau. In fact, strings make up nearly half of the data processed at Tableau. This high prevalence undoubtedly applies to many other companies as well, as the paper’s dataset consists of data analyzed by Tableau’s users. The string-heavy nature of the data makes string processing one of the most important tasks of a database system.
Building Data Pipelines With Vitess
Vitess is a popular CNCF project that is used to scale some of the largest MySQL installations in the world — by companies like Slack, Square, Shopify, and GitHub. It provides sharding, connection pooling, and many other features that make it easy to scale MySQL horizontally.
Vitess and MySQL are ideally suited for use as an Online Transaction Processing (OLTP) system — where the end-user interacts directly with the system and fast response times are essential as they get product and service information, generating critical business records such as orders, user profiles, and more.
Import Postgres tables into Tinybird with the PostgreSQL Table Function
Today we announce the PostgreSQL Table Function, a powerful and flexible way to import and sync data from Postgres to Tinybird with just a few lines of SQL.
July 25, 2024
The Hidden Cost of Data Movement
The Hidden Cost of Data Movement
Recently, Mark Raasveldt of DuckDB wrote an excellent post about why memory management is crucial for efficient data processing. In his post, he focuses on the cost of having data on disk and moving it to memory. After all, everyone knows that having data in memory is what you want. As Jim Gray famously said in 2006:
Tape is Dead, Disk is Tape, Flash is Disk, RAM Locality is King
July 23, 2024
The State of Online Schema Migrations in MySQL
Learn about the options for running non-blocking schema changes natively to MySQL, using Vitess, or other tools
July 22, 2024
Optimizing aggregation in the Vitess query planner
The Vitess query planner takes multiple passes over a query plan to optimize it as much as possible before execution. A recent tricky bug report led to an improvement in how the optimizer functions.
An Interesting Optimization
Introduction # I recently encountered an intriguing bug. A user reported that their query was causing vtgate to fetch a large amount of data, sometimes resulting in an Out Of Memory (OOM) error. For a deeper understanding of grouping and aggregations on Vitess, I recommend reading this prior blog post.
The Query # The problematic query was:
selectsum(user.type)fromuserjoinuser_extraonuser.team_id=user_extra.idgroupbyuser_extra.idorderbyuser_extra.id;The planner was unable to delegate aggregation to MySQL, leading to the fetching of a significant amount of data for aggregation.
July 19, 2024
Behind the scenes of Tinybird's big frontend refactor
We recently refactored the Tinybird frontend. Take a look behind the scenes at how we started, the choices we made, and how we arrived at a faster and simpler app deployment.