From punch cards to containerization, database technologies have changed tremendously over the years. And there has been no shortage of watershed moments along the way. However, we believe the emergence of advanced AI may very well be the most significant one yet.
Percona certainly isn’t alone in that belief. But, when it comes to the question of how best to navigate this change, consensus is scarce. So, with the entire tech sector in a state of flux over AI, the team at Percona felt it important that we be as explicit and transparent as possible regarding our own vision, goals, and values surrounding this transformative new technology.
In this blog post, we’ll share Percona’s take on AI and how it’s shaping the future of database technology.
How data drives AI development & its undeniable role in determining success
You’d be hard-pressed to overstate the importance of data to the field of AI. Never before has there been an industry or area of technology more dependent upon data—and, more importantly, truly vast amounts of it. While the amount of data needed to train an AI model can vary dramatically, research suggests that the size of the datasets used to train today’s large language models (LLMs) doubles approximately once every eight months, with the largest models currently using datasets containing tens of trillions of words.
However, AI’s demanding data requirements aren’t simply a matter of volume. At a recent conference, IBM CEO Arvind Krishna said that nearly 80% of the work done on AI development projects is spent collecting, cleansing, and preparing data—highlighting the extent to which AI and data are intertwined not only materially but operationally as well.
All this highlights the fundamental importance of an optimized data layer in AI development. With so much of a project’s time dedicated to data, it should go without saying that database monitoring, management, and optimization are all mission-critical to a project’s success. Undoubtedly, those organizations with the most streamlined, efficient, scalable, and reliable data stacks will be those best positioned to thrive in this incredibly competitive industry. A couple of sub-optimized queries could be the difference between an organization being first-to-market or falling behind its competitors.
From simple storage layer to essential AI enablement tool—Open source databases in the age of AI
As a result of the above, it’s safe to say that the role of the database in a post-AI world will be much different than that of the past. What has long been seen as a largely static storage layer will instead be seen as one of (if not the) most important enablement tools in the field of AI. Already, we are seeing almost the entire database industry lean into this new role—with a flood of new, vector-focused databases rushing in to capture this demand while tried and true solutions race to develop new features and functionality dedicated to AI development.
Typically, when disruptive new technologies emerge, legacy solutions are the losers. However, in the world of databases and database management systems (DBMSs), we’re confident that the opposite is true. And the reason why is the growing prominence of open source solutions in this space.
Although the dedicated, aforementioned new vector databases represent meaningful competition, ultimately, no proprietary offering can reasonably compete with the pace of innovation that comes with crowd-sourced development and contribution. In fact, leading open source database management systems like PostgreSQL have already found themselves in pole position, with well-established, community-developed extensions such as pgvector delivering enterprise-grade vector data capabilities all within a familiar, reliable, and well-tested environment.
And this is just one example. Tools like pg_trg and MySQL’s full-text search enhance capabilities for querying, similarity matching, and preprocessing data. At the same time, JSON and semi-structured data support in PostgreSQL (JSONB) and MongoDB further enrich AI pipelines.
More generally, OS databases offer a number of inherent advantages that provide an unparalleled foundation for managing the massive data requirements of today’s AI models. From the flexibility required to remain adaptable to evolving data structures and workloads to the scalability needed to handle increasingly large and complex datasets in a cost-effective manner—OS solutions are perfectly suited to meet both AI’s immediate needs and the wide field of unknowns that still lie ahead.
The path ahead for Percona: Bringing AI enablement to the fore
With all of the above in mind, Percona is perfectly positioned to help lead the way in realizing this transformation of the data layer—helping organizations scale and optimize their AI efforts while remaining compliant and managing costs. With Percona, large and enterprise organizations can unlock the full potential of AI while ensuring their OSS databases remain cost-effective, scalable, and future-ready.
At the very same time that Percona is helping its customers succeed in their AI initiatives, we will also be hard at work developing AI solutions of our own—enhancing our product suite by introducing things like predictive insights, automated optimization features, and more. While the finer details aren’t yet ready for public consumption, rest assured that Percona is actively working on a number of promising AI-powered capabilities designed to improve the value of our products and enhance our own internal efficiencies.
Just as the field of AI is placing all-new levels of demand on matters of storage and compute, cloud and SaaS costs are soaring. To this end, Percona’s continued commitment to open source will be truly invaluable in the democratization of AI—lest we find ourselves in a world where AI is dominated by a small handful of super-massive multinationals.
Firm principles and flexible plans are necessary for a positive, fruitful AI-driven future
Finally, we at Percona feel that, amidst the mad dash to capitalize on this powerful new technology, it is of paramount importance that we do not lose sight of its impact on society. Like every major technological breakthrough before it, AI has the potential to bring untold benefits and untold harm to society.
While we aren’t eager to see progress ground to a halt, we do feel advancements in the field should always be preceded by careful consideration of their impact. In order to ensure we get the most out of this technology moving forward, organizations from across the public and private sectors must come together and advocate for sensible regulations and consensus-building industry standards. While it is inevitable that some harm will come from AI, we are confident that a healthy balance between innovation and caution will go a long way to minimizing such effects.
Finally, while we obviously gave all of these matters the careful, measured consideration that they deserve, it’s important to keep in mind that the AI space and our knowledge of what’s possible continue to evolve by the day. As such, one of the core tenets guiding our approach to AI at Percona is adaptability. We expect our vision and pursuits to change and evolve over time as new insights emerge and the frontier of possibilities extends into the unknown.
About the Author
Liz WarnerLiz Warner is Chief Technology Officer at Percona. Before joining Percona she was CTO at Weaveworks, an open source company in the Kubernetes space. She’s worked as a CTO for over a decade, with a background in software and infrastructure engineering. She’s led product/engineering organizations for companies like NatWest, Nationwide, and Toyota Connected, as well as leading teams and building systems for Mastercard and Apple. A native of California and graduate of UC Berkeley, she lives in the United Kingdom with her family.
