pgBackRest Compression: How Much CPU Is a Smaller Backup Worth?

In this blog post, we’ll compare pgBackRest’s compression algorithms and levels to find where spending more CPU stops buying a meaningfully smaller backup. The short version of the answer, which we’ll build up to with real numbers, is that Zstandard at a low level is the sweet spot, and its default (zst(3)) already sits right on the shoulder of the curve.

To make the inflection points (or “knees”) in the curves more visible, we sampled each algorithm at closely spaced levels across its whole range. Everything here was measured on PostgreSQL 18.4, more precisely Percona Distribution for PostgreSQL 18.4, with pgBackRest 2.59.0.

Compression types and levels in pgBackRest

There are two options: compress-type picks the algorithm (none, gz, lz4, zst, or bz2), and compress-level picks how hard that algorithm works. The catch is that both the valid range and the default level depend on the algorithm you chose, so a “level 9” means different things in gz versus zst. Here is how they look like:

Note that zst runs all the way to 22 and that the negative levels are the “fast” modes, trading ratio for speed. The default algorithm in pgBackRest is gz, so if you never touch these settings you’re running gz(6).

You can set both on a per-backup basis on the command line:

Setting up pgBackRest and the dataset

We kept pgBackRest stock. The configuration file is simple, using only the minimal options it needs to work:

The repository is a local filesystem, not S3 or object storage. There’s also no encryption, no bundling, and no block incremental, so everything is the stock repository format. Compression was the only thing we varied, and we varied it per run on the command line. We also used --process-max=8 (the default is 1) purely to cut wall-clock time. That doesn’t distort the comparison, because our cost metric is CPU-seconds summed across all the worker children, which is independent of how many workers split the job. Only wall-clock changes with parallelism. Every backup was a fresh --type=full so each run is independent and directly comparable. The Postgres side was stock too, with archive_mode=on and wal_level=replica.

Note: One option worth a mention is block incremental (repo-block, off by default), which splits files into blocks so that incremental backups store only the blocks that changed, saving repository space and speeding up delta restores. Fully testing with it is out of the scope of this blog, but do know it exists and it can affect your results. In a quick check, turning it on left zst(3) almost unchanged but made a zst(22) backup about 22% larger, with savings falling from 74% to 68%. If you run with block incremental, the case against using the ultra high compression levels only gets stronger.

To exercise the algorithms on realistic, heterogeneous data rather than one synthetic table, we loaded the public Stack Exchange data dump for askubuntu.com, about 4 GiB once in PostgreSQL. It mixes large text bodies (post and comment HTML) with integer-and-timestamp-heavy tables like votes and badges, and it’s fully reproducible from a public source.

Measuring the trade-off

The cost metric throughout is CPU-seconds (user + system), not wall-clock, precisely because it’s stable regardless of how many cores you throw at the job or what else the host is doing. And these are numbers from our hardware. Treat the shape of the curves as the takeaway, not the absolute seconds, and measure on a representative backup of your own data before you commit to a setting.

Each backup was a fresh full backup wrapped in /usr/bin/time -v, and we took the median of three runs per setting. The zst(3) run, for example, looks like this:

The two numbers we care about are user and system seconds. Adding them gives the CPU-seconds figure we compare across settings (here, about 59), while the elapsed time shows what parallelism bought us on wall-clock (about 14 seconds in total with eight running workers).

Where each algorithm’s curve bends

The full picture is easiest to see as a chart. Each line is one algorithm and its levels, with savings on one axis and CPU cost on the other:

Check the Appendix at the bottom for a table with the full results.

Zstandard has its knee between levels 1 and 5. Efficiency declines gently across the low levels, then drops at level 5, and collapses beyond it. That gentle-then-steep shape is exactly why the default, zst(3), is so well placed! It sits on the middle, delivering 64.7% savings for 59 CPU-seconds. Push on to zst(22) and you get the smallest file in the whole test (74.0%), but it costs 5217 CPU-seconds, roughly 88 times the CPU of zst(3), for nine more points (reducing the size of the backup file by roughly one quarter). The ultra levels (19 to 22) are even worse: on the order of 45 to 90 times the cost of zst(3) for single-digit gains.

LZ4 is only useful at level 1 or below. lz4(1) and lz4(2) are identical in practice, then there’s a cliff at lz4(3): savings increase but CPU triples. From there it stays flat while the cost keeps climbing, all the way to 692 CPU-seconds at lz4(12) for basically the same gains. Above its default, LZ4 is dominated by zst. It shines on the cheap, fast end, and also uses a sane default.

gzip flattens out early, around level 5. gz(6), the pgBackRest default, is another reasonable choice, since there’s almost nothing left to gain above it: gz(9) costs 2.8 times the CPU for two-tenths of a point more. On ratio-per-CPU it’s dominated by zst at every level, so there’s no level of gz that zst doesn’t beat at the same cost.

bzip2 never makes it onto the frontier. It’s flat, sitting between 63% and 68% savings while burning 411 to 464 CPU-seconds regardless of level. zst(9) matches bz2(9)‘s 67.8% for less CPU, so there’s no case where you’d reach for it. You are already paying a high price at any level, and the cost barely grows with them, so using the max level as default is not such a bad choice.

Putting all of the data together in one graph, the efficient frontier (nothing is both smaller and cheaper) is easy to see:

LZ4 owns the cheap end (its fast, negative levels), and zst owns everything from about 46% savings on up. The one thing to avoid is zst‘s own negative levels, which are worse than lz4 at the same cost, so if you want cheap-and-fast reach for lz4 instead of fast zst.

How compressible is your data?

pgBackRest compresses the physical files on disk, and different tables hold very different bytes, so the ratio you get depends heavily on your schema. For example, here’s what each relation compressed to at zst(3). We measured this by compressing each relation’s on-disk files directly (heap and TOAST main forks) with zstd(3), the same bytes pgBackRest reads:

The numeric tables like votes and badges squeeze down more than 5x, while the big text bodies land around 2.5 to 3x. The interesting one is posts TOAST at only 1.4x, and there’s a reason for it: PostgreSQL already compresses them, so by the time the data reaches pgBackRest there’s little left to be done. You can see it in the stored sizes for the same 361 MB of raw text under different TOAST strategies. We took the same 100k large post bodies and stored them three ways, changing only the column’s TOAST compression, and measured the resulting TOAST size:

How fast does each setting work through your data?

CPU-seconds tell you what a setting costs, but they don’t tell you how long the backup takes. Here is the same data seen as a rate: how many megabytes each setting works through per second.

Here is a slice of the same numbers, with the per-GB CPU cost next to the rate:

Those are two different numbers answering two different questions. MB/s is what we measured on this host with eight workers, so it tells you how fast it went here. CPU-s/GB is the one that travels, because it’s a property of the algorithm and the data rather than of how many cores you own. Multiply it by the size of your database and you have the CPU bill for one full backup, whatever you set --process-max to. The rate curve bends where the cost curve bends, so the settings that are cheap on CPU are also the ones that finish quickly. There’s no separate trade-off to reason about for your backup window.

Note: the fast end of the chart isn’t only measuring the compressor. For instance, lz4(-5) runs at 93% of the no-compression baseline, which means it’s mostly bound by reading and writing the data. On storage that can’t sustain that throughput, the tall bars would flatten out to whatever your disks do, while zst(5) and the slower settings would barely move, so take this into account when reviewing your results. These are also rates for a single 4 GiB dataset held warm in RAM on one host, with eight workers that bought us about 4x, not 8x. Use them to compare settings against each other, then measure your own database before you plan a maintenance window around a number.

Choosing a compression setting

Ranking all 40 compression settings by efficiency presents the data in yet another way:

So what should you actually set? A few recommendations:

• Start with the default, zst(3). It’s on the shoulder of the curve and it’s hard to beat.
• If you’re short on CPU or your backup window is tight, drop to lz4(1). It’s the cheapest useful setting, still gets you 46% saving, and at 531 MB/s it runs at 80% of no-compression speed.
• Don’t go past about zst(5) unless storage or network bandwidth are at a premium. The gains above it are tiny and the CPU cost isn’t.
• Avoid zst(19) through zst(22) and all of bz2. They don’t pay for themselves.
• Keep gz only if you need it for compatibility with something else that reads the repository.

As mentioned above, measure on a representative backup of your own data before changing a production setting. Your schema and your data mix will decide where your knee falls.

Conclusion

The trade-off between backup size and CPU is non-linear, and sampling closely across each range is what makes the knees visible (if you only sample a few levels per algorithm you may lose the real inflection points). zst(3) sits right where the curve bends, which is why it’s a good default. Throughput bends at the same levels, so a tight backup window doesn’t push you toward a different setting than a busy CPU does. This is a straightforward setting to get right: pick zst, keep the level low, and you’re most of the way there. Try it on your own data and let us know the numbers you get!

Appendix: the full results

For completeness, here is every setting we measured (each value is the median of three full backups).

• size is the on-disk repository size (the uncompressed database is 4 GiB)
• saved% is the reduction against the uncompressed baseline
• CPU-s is user plus system seconds
• wall-s is elapsed time with --process-max=8
• MB/s is that elapsed time as a rate over the uncompressed database
• CPU-s/GB is the same CPU cost per GB of database
• savedMB/CPU-s is megabytes saved per CPU-second