MariaDB S3 Engine: Implementation and Benchmarking

MariaDB no longer meeting your needs?

Migrate to Percona software for MySQL – an open source, production-ready, and enterprise-grade MySQL alternative.

Learn More

MariaDB 10.5 includes an S3 storage engine plugin based on Aria. Its main feature is the ability to move tables from local storage to S3 using ALTER TABLE, while still allowing access through standard SQL.

This is useful for low-cost data archiving. The S3 engine is read-only (no INSERT/UPDATE/DELETE), but table structure changes are allowed.

In this blog, we’ll explore implementation details and compare performance between local and S3-backed tables.

S3 Engine Implementation

The S3 engine is alpha-level and not enabled by default. Enable it in the config:

Configure S3 credentials:

Note: Credentials are stored in plaintext. Use restricted IAM credentials.

Restart MariaDB and install the plugin:

Moving Tables to S3

Example table:

Convert to S3:

After conversion, only the .frm file remains locally. Data moves to S3:

Note: Data and index are stored separately in S3.

S3 Engine Operations

SELECT

INSERT / UPDATE / DELETE

Index and Schema Changes

DROP TABLE

Note: This removes data from S3 as well.

To modify data:

• Convert table back to InnoDB
• Modify data
• Convert back to S3

S3 vs Local Performance

COUNT(*)

S3:

Local:

COUNT(*) is faster on S3, likely due to metadata usage similar to MyISAM.

Full Table Scan

S3:

Local:

Primary Key Lookup

S3:

Local:

Performance Considerations

• Reduce s3_block_size
• Use COMPRESSION_ALGORITHM=zlib
• Increase s3_pagecache_buffer_size

Performance depends on:

• Fast network → S3 performs better
• Fast disk → Local performs better

Conclusion

• Good solution for low-cost archival
• Read-only tables
• Fast COUNT(*) queries
• Schema changes supported
• Some operations slower due to S3 dependency

Stay tuned for more on S3 engine compression.