Estimate project
← All posts

Comparing TigerBeetle and PostgreSQL Performance: Benchmark Harness and Cloud Tests

An in-depth performance benchmark that pits TigerBeetle against PostgreSQL in a multi-node Google Cloud Platform environment. It examines double-entry bookkeeping workloads at different request rates and levels of account contention (Zipfian distribution), looking at metrics such as mean throughput, latency percentiles, and the share of dropped requests.

8 min read
DatabasePostgreSQLTigerbeetle

Several months back, we tested the performance of TigerBeetle and PostgreSQL with local benchmarks. The test reproduced the typical workflow of TigerBeetle’s use case – creating transfers using double-entry bookkeeping.

Locally, TigerBeetle turned out to be nearly 3 times faster than the quickest PostgreSQL approach, but it remained an open question whether the same holds for a cluster deployment on dedicated servers. That is what we set out to answer here.

A brief recap: the PostgreSQL schema

TigerBeetle comes with a fixed schema designed solely for double-entry bookkeeping. It has three main entities: ledgers, accounts, and transfers. Every transfer involves two accounts – one gets credited and the other debited. As noted earlier, creating a transfer is the core database operation. Below is the comparable schema we use for this test in PostgreSQL:

CREATE TABLE IF NOT EXISTS accounts (
    id BIGINT PRIMARY KEY,
    balance BIGINT NOT NULL DEFAULT 0,
    CONSTRAINT balance_non_negative CHECK (balance >= 0)
);

CREATE TABLE IF NOT EXISTS transfers (
    id BIGSERIAL PRIMARY KEY,
    source_id BIGINT NOT NULL REFERENCES accounts(id),
    dest_id BIGINT NOT NULL REFERENCES accounts(id),
    amount BIGINT NOT NULL,
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    CONSTRAINT amount_positive CHECK (amount > 0),
    CONSTRAINT different_accounts CHECK (source_id != dest_id)
);

CREATE INDEX IF NOT EXISTS idx_transfers_source ON transfers(source_id);
CREATE INDEX IF NOT EXISTS idx_transfers_dest ON transfers(dest_id);
CREATE INDEX IF NOT EXISTS idx_transfers_created_at ON transfers(created_at);

We will test 2 strategies for creating transfers in PostgreSQL (implicit locking and explicit locking). As they were already described earlier, we will not go over them again.

Running the performance benchmark in the cloud

Previously, we ran everything locally with a single database instance and a single client. To get closer to real-world conditions, this time we run a cluster for each database plus 5 client instances. TigerBeetle recommends six nodes spread across three cloud providers; for this test, we used three to match the PostgreSQL setup, configured for its standard leader + 2 replicas consensus. Similarly, PostgreSQL ran with one primary and two synchronous standbys (quorum commit, meaning a transaction is acknowledged only once it is durable on the primary and on at least one standby).

The benchmarks ran on Google Cloud Platform. Besides the client and database instances mentioned above, one more node hosts our monitoring stack with Prometheus and Grafana.

Clients ran on n2-standard-2 (2 vCPUs, 8 GB of RAM) instances, and the databases on n2-highmem-4 (4 vCPUs, 32 GB of RAM) instances.

What we measure and what we vary

Before getting into the methodology and results, let’s go over what we want to measure and which variables we control. First, the variables:

  • target_rate is the overall number of transfers per second that clients on all instances attempt to execute. Requests are sent no matter how quickly the database responds.
  • max_concurrency limits how many in-flight requests a client may have at once. We noticed that clients sometimes dropped transfers because of too many in-flight requests, not because the database failed to respond. We used this setting to lift the ceiling and allow more requests.

What we measure:

  • mean throughput
  • p50, p95, p99, p999 latency
  • error rate
  • dropped requests – requests dropped because of max_concurrency, i.e. too many in-flight requests

Running and coordinating the tests

The coordinator code takes care of distributing the client code, starting the database, initializing accounts, and running the benchmark. We still deploy the database with Docker, but since the host is not shared, we use --security-opt seccomp=unconfined to enable io_uring.

As for the client code, we did not want to install the full Rust and Zig toolchains on the instances just to build the binary. Instead, we cross-compile the client binary with cargo zigbuild and copy it to the VM instances.

Test results

Now for the results. Every test runs for 5 minutes, after a 2-minute warm-up, over 3 iterations. We ran two tests for TigerBeetle and 6 for PostgreSQL, because PostgreSQL supports 2 different strategies. After comparing the two databases, we also ran several more benchmarks to find out how far TigerBeetle can be pushed. We also defined 2 kinds of account contention, described by the Zipfian exponent, which produce “hotspot” accounts.

The first test runs 5k requests per second in fixed_rate mode.

Metric TigerBeetle PostgreSQL Standard (FOR UPDATE) PostgreSQL Atomic
Fixedrate (zipfian=1.0)
Test mode fixed_rate fixed_rate fixed_rate
Mean throughput 4,097 TPS 2,890 TPS 3,068 TPS
p50 latency 32.3 ms 324.7 ms 303.7 ms
p95 latency 508 ms 581 ms 556 ms
p99 latency 748 ms 960 ms 927 ms
p999 latency 987 ms 1,492 ms 1,473 ms
Error rate 0% 0% 0%
Balance verified 3/3 3/3 3/3
Hotspot (zipfian=2.0)
Test mode fixed_rate fixed_rate fixed_rate
Mean throughput 4,461 TPS 753 TPS 977 TPS
p50 latency 32.1 ms 1,306 ms 1,001 ms
p95 latency 358 ms 1,890 ms 1,472 ms
p99 latency 560 ms 2,123 ms 1,821 ms
p999 latency 726 ms 2,916 ms 3,138 ms
Error rate 0% 0% 0%
Balance verified 3/3 3/3 3/3

All systems throughput plot

TigerBeetle achieves roughly 33% more throughput than the best PostgreSQL strategy. We can also see that as skew increases and more accounts become “hot”, TigerBeetle keeps its performance, whereas PostgreSQL’s row-locking approach slows down considerably. In TigerBeetle, a single core processes transfers sequentially, so there is no contention, while in PostgreSQL connections have to wait whenever several of them try to update the same row. TigerBeetle comes close to saturating the target rate. In the following benchmarks, we will see it actually reach the target rate. We get there by permitting more in-flight requests, so that clients stop dropping requests that could otherwise have been served.

All systems latency plot

The latency plot shows an even sharper contrast. TigerBeetle remains steady regardless of how often the same accounts are hit, while PostgreSQL has to wait on locks before updating a row, which leads to higher latency overall and occasional big spikes. Let’s compare the percentiles against FOR UPDATE in PostgreSQL under moderate skew.

Latency percentiles - moderate skew plot

Let’s also check the higher skew, where PostgreSQL begins to slow down considerably.

Latency percentiles - high skew plot

It is evident that locking is not the ideal choice – as more accounts become “hot”, latency keeps rising, whereas on TigerBeetle it actually drops.

Key takeaways

  • TigerBeetle comes out ahead on every metric in both skew scenarios. Its throughput is 40% higher under moderate skew and ~4.6x higher under heavy hotspot skew, and its latency is far lower – between 4x and 40x.
  • With higher skew, TigerBeetle actually became faster (both higher throughput and lower latency). More requests get rejected quickly (because of insufficient balance) instead of waiting on a lock. Both PostgreSQL modes performed worse under hotspot skew.
  • Correctness was preserved across the board. Every test in every scenario satisfies the double-entry invariant; the only differences are in performance.

Even though TigerBeetle does not seem to be under pressure, we still fall short of 5k throughput. The reason is the max_concurrency setting mentioned before: there were too many in-flight requests, so some were dropped. For PostgreSQL, the throughput ceiling and latency appear to stem from locking, which is not the case with TigerBeetle. Next, we will try to determine TigerBeetle’s limit in this setup under heavy hotspot skew by giving it more room with a larger max_concurrency value.

Taking TigerBeetle to its limit

We set max_concurrency to 2.5x the target requests-per-second rate. That should keep the client from dropping requests while the database is still able to handle them. These tests use hotspot accounts (Zipfian = 2.0), since that appears closer to real-world conditions.

Metric TigerBeetle PostgreSQL Standard PostgreSQL Atomic
concurrency5k
Mean throughput 5,060 TPS 683 TPS 878 TPS
Dropped requests 0 3.9M 3.8M
rate10k
Mean throughput 9,431 TPS 703 TPS 867 TPS
Dropped requests 598K (~9%) 8.5M 8.4M
rate20k
Mean throughput 20,257 TPS not tested not tested
Dropped requests 0 - -
rate40k
Mean throughput 40,388 TPS not tested not tested
Dropped requests 0 - -
rate80k
Mean throughput 81,171 TPS not tested not tested
Dropped requests 0 - -
rate90k
Mean throughput 91,165 TPS not tested not tested
Dropped requests 0 - -
rate100k
Mean throughput 101,497 TPS not tested not tested
Dropped requests 0 - -
rate120k
Mean throughput 107,112 TPS – just 89% of the offered load not tested not tested
Dropped requests ~12% of the offered load - -
rate160k
Mean throughput 107,858 TPS – just 67% of the offered load not tested not tested
Dropped requests ~34% of the offered load - -

All systems throughput corrected plot

Permitting more simultaneous in-flight requests lets us reach the target rate. PostgreSQL showed no significant change. In this setup, we managed to push TigerBeetle beyond 100k requests per second, which illustrates how big a difference a specialized architecture can make. Let’s also check latency, because the higher max_concurrency setting could mean higher latency.

TigerBeetle latency corrected plot
A remark about the rate160k figures in particular: they are the average of 2 of the 3 planned runs. The coordinator process was terminated by a hard ~24.5-minute limit on the background task running it. This happened on several attempts, each time just as balance verification for the third run was beginning – by which point the first two runs had finished and cleanly passed balance verification every time.

Hatched bars marked with an asterisk fell just under one of the client histogram’s bucket boundaries – 1.5 s for rates 40k, 80k, and 90k, and 4 s for rate 120k. Since Prometheus interpolates inside a finite bucket, and the buckets above those boundaries existed but remained empty, these values are upper bounds.

Summary

In our benchmarks, TigerBeetle appears to live up to its promise. Under heavy traffic of 40k requests per second, the database keeps latency at a reasonable 70 ms. At the same time, it reaches over 5x higher throughput than PostgreSQL in our 5k test and 10x higher in the 10k test. TigerBeetle can even provide 100x higher throughput at the cost of latency, and yet at 100k requests per second its latency is still lower than that of PostgreSQL serving only 5k requests per second. We aimed to recreate a more realistic environment and workload for testing both databases, and the differences are still clear: TigerBeetle offers lower latency, higher throughput, and steadier behavior despite the different workload.