Open Source Data Management Software Survey

The database landscape has evolved rapidly over the past few years, and open source database software and the open source community continue to innovate to meet user and enterprise needs. Percona conducted a survey of open source database users to find out more about how they deploy open source software (OSS) databases.

There is no “one size fits all” database needs. Though software vendors have worked hard to add features, users gravitate towards the best database (and best tool) for the job, using the right database for the right application in database environments that can exist either on-premises, in the cloud, or a hybrid of the two.

“Open Source is dominating the software industry. It’s got a bright future. It offers more options for customers and developers. It’s a win-win situation.”

The current market landscape includes:

  • Multi-database usage
  • Hybrid public-private cloud deployments
  • Multi-cloud deployments
  • Security and compliance adherence
  • Fast and scalable systems
  • Portability assurance
  • Vendor lock-in resistance

Most companies now run more than one database, often on more than one platform. Another development is the rise of microservices and containerization, along with the growing popularity and awareness of Kubernetes.

Determining what database software is right for your application, customer experience, and enterprise goals has become a very complicated process.


Non-Simple: Database Deployment Is a Complex Series of Options

Percona’s open source database survey found interesting and informative results about the state and direction of enterprise database choices and deployments:

62% of respondents use open source software to avoid vendor lock-in.
Our survey revealed that users are eight times more likely to adopt an open license than the alternative.
Security and compliance concerns, along with data breaches, were flagged as key concerns in the survey results (more than half at 53%).
Over 90% of survey respondents have more than one database technology in their work environment, and 85% of respondents use more than one open source database technology.
The survey also shows growing adoption of multi-cloud strategies and hybrid environments. More than 38% of respondents say that they are using more than one cloud deployments, either with a single or multiple providers.
There is a move toward more heterogeneous environments as database and cloud vendors make their databases more accessible and easier to use. Over 50% of survey respondents already run some of their workload in the public cloud.

As a market-leader delivering unbiased, open source database expertise, Percona will continue to promote and support easy database deployments, database optimization, free software, and solutions that enable users to grow in a secure and scalable way.

Percona thanks everyone who participated in our survey. We look forward to hearing your thoughts on these results.


About This 2019 Report

Who Responded to the Survey?

The US represented the biggest base of respondents with 26%. However, the remaining 74% are spread widely across the globe, giving us a broad mix of global replies and demonstrates the diversity and reach of the open source community. It is also a testament to the power of the data architectures and technologies discussed: technology allows people to remain connected while working anywhere in the world, in any time zone.

836 TOTAL RESPONDENTS FROM 85 COUNTRIES

215 from the United States

161 from European Nations

33 from Latin America

COMPANY SIZE

We received feedback from a mix of small, medium, and large companies — demonstrating that open source software is a solution that works no matter what size your business.

INDUSTRY

Many of the respondents, unsurprisingly, were technology-focused companies. Many offer cloud and SaaS solutions to startups, growing businesses, and established enterprises.

131 were Information Technology companies

89 identify as Software as a Service (SaaS) Development

84 are Cloud-based Solutions or Services companies

“We are a non-profit, international NGO. We have limited resources for internal projects, and the clients we work with in the third world have virtually no budget to pay for database licenses/maintenance/support. We go open source whenever possible.”

Cloud-based solutions or services

84

Retail or eCommerce

67

Consulting

49

Software as a Service (SaaS) development

89

Data and analytics

51

Web development or design

55

Financial technology or services

50

Other software development

63

Healthcare technology or services

27

Prefer not to answer

26

Information technology

131

Other industry (please add in free text box)

103

Media, advertising, publishing, or entertainment

71

 

 

Download the full report or the raw data:


Multiple Databases, Multiple Locations, and Multiple Platforms

It is now the norm to have multiple databases in multiple locations over multiple platforms as companies attempt to adapt to changing business needs. Quickly responding to customer demands and shifting market pressures is a guarantee of maintaining market leadership.

Number of Database Instances In Production

Using multiple databases, in multiple locations, across multiple platforms is now a standard. As companies attempt to streamline and increase efficiency, it is essential that they retain experts to give a close and proactive overview of all database operations.

While the majority of businesses use 25 instances or less, the larger the enterprise the more often responders use up 10000 instances (or more). As businesses grow, they need exponentially more instances to hold their data. Over 92% of respondents say they use more than one database.

  • Over 92% of respondents say they use more than one database
  • 89% of respondents have more than one OSS database
  • 43% are running both PostgreSQL and some variant of MySQL
  • 54% are running some purpose-built NoSQL database (Document, key-value, big data, etc.)
  • 73% are running both a relational database as well as a NoSQL purpose-built database


Database Combinations

The survey showed some interesting numbers around what databases companies are using together in multi-database deployments.

The larger the company size, the more likely they are to have multiple databases. Larger company adoption of a multi-database environment jumps 10-15% over small companies. It was also somewhat surprising how many people use both MySQL and PostgreSQL. The overlap of these two databases in a single environment is much higher than that of MySQL and MongoDB.

These numbers show that multi-database deployments are king!



Hybrid Hosting: A Combination of Private, Public, and On-premise Cloud

Most survey respondents are well-informed about using open source technology in the cloud and do so. Interestingly, however, these passionate open source evangelists championing cost-effectiveness, flexibility, and freedom from vendor lock-in often find themselves tied to cloud vendors with a single solution and large monthly costs.

As the company size grows, it is much more likely that they are hosting their database infrastructure both on-premises and in the cloud. The larger the organization, the more complex the hosting environment. Larger organizations have a 10-15% swing in hybrid cloud, private cloud, and on-premise.



Multi-Cloud and Cloud Hosting

AWS continues to dominate the public cloud provider market, with over 50% of respondents using its cloud platform. Google Cloud and Microsoft Azure show similar numbers of respondents using their technologies and offer alternatives to companies resistant to using Amazon, or won’t use Amazon due to a competitive clash.

The multi-cloud usage for databases is about a third of our respondents, with 41% of larger companies using multi-cloud deployments (close to 10% over smaller companies).

Smaller companies are more likely to use Google than Microsoft, but larger companies prefer Microsoft to Google (which could have something to do with startup businesses need for cost-effectiveness and agility).



Cloud DBaaS Use

“I think it’s interesting that companies view cloud DBaaS as open source and not as enterprise software. We seem to throw caution to the wind there and look at it as a panacea rather than equivalent to running commercial software.”

Although AWS again dominates, Microsoft has continued to build out their DBaaS platform and are catching with AWS on features. It is a surprise that more than 60% of companies completing the survey do not use DBaaS.

But of those who did respond, 17% of large companies run two different DbaaS providers. This means half the large companies who responded to our survey have two separate DBaaS experiences. Larger companies also seem to be slightly more inclined to running DBaaS.

As the industry trend for streamlining and commoditizing services continues, the number of companies adopting DBaaS is likely to increase significantly in the coming years.


Are Containers the Next Big Thing?

Containers are the next generation of virtual machines. As containers have become more sophisticated, and with the advent of orchestration options like Kubernetes, more companies are looking at using containers to run and manage their database environments.

Our survey looked into the current popularity of containers and how this might change in the future.

Containers and Database Use

Over 25% of respondents are using containers, but not necessarily to run databases. This could be due to some early bias against running databases in containers. Many respondents aren’t aware if they use containers for their databases or not.

Our results show that container adoption in production environments increases with the company size.



Install Demographics

In our survey, the top databases in use were as expected, but with 30 total databases noted, the expanse of systems in use is quite wide.

MySQL Community Edition 58.7% Google Cloud SQL for MySQL 5.7% TiDB 1.3%
PostgreSQL 46.1% Percona Server for MongoDB 4.2% Postgres-BDR 1.3%
MariaDB Community Edition 36.1% Amazon Aurora (PostgreSQL Compatible Edition) 3.9% Alibaba Cloud: ApsaraDB RDS for MySQL 1.0%
Percona Server for MySQL 34.4% MongoDB Enterprise 3.6% CockroachDB 0.9%
MongoDB Community 34.4% MariaDB Enterprise 3.1% Tencent Cloud: Cloud Database for MySQL 0.9%
Amazon RDS for MySQL 19.8% Microsoft Azure Database for MySQL 3.1% Postgres-XL 0.8%
Percona XtraDB Cluster 17.2% MongoDB Atlas 2.6% Clustrix 0.7%
Amazon Aurora (MySQL Compatible Edition) 10.4% EDB Postgres Platform 1.9% Alibaba Cloud: ApsaraDB RDS for PostgreSQL 0.7%
MySQL Enterprise 9.4% Microsoft Azure Database for PostgreSQL 1.7% Azure Cosmos DB 0.5%
Amazon RDS for PostgreSQL 9.2% Amazon DocumentDB 1.4% FoundationDB Document Layer 0.1%

MySQL dominates in open source database adoptions. MySQL Community has always been the open source database front-runner (with less than 10% of companies opting to pay for the MySQL Enterprise edition).

Most people rely on the out-of-the-box MySQL Community installs. A the combined installs of Percona Server for MySQL and Percona XtraDB Cluster are in second place. MariaDB is in third. More than half the install base uses more than one provider for MySQL.

As the company size grows, the number of MySQL and MySQL-compatible software and the number of different types at a single company grows.

No one consolidates on a single version of MySQL, minimally opting for a split between community and enterprise (probably as a cost saving measure). Smaller companies are more likely to use AWS RDS than the more expensive Aurora. Most of MariaDB’s install base in our survey is coming from the free, non-paid version of MariaDB.

MySQL Community is the clear leader in usage, but large companies are more willing to run the enterprise versions than smaller (or even medium-sized companies). Percona’s software (most likely because of its free enterprise versions and features) spans the small and large — there is no difference in install base through any company size.


What Are the Most Popular Databases Used with My SQL?

Our survey looked at what databases are popular in combination with MySQL. Interestingly, we see Redis at only 29% in the enterprise. Enterprises still use a lot of proprietary databases. Once the size of a company drops below five thousand employees, the percentage of proprietary database drops to 47% for oracle and 59% for SQL Server. It keeps dropping from there.


What Are the Most Popular Databases Used With PostgreSQL?

84% of enterprise customers who responded are running some sort of proprietary database, only 16% of enterprises are running only OSS databases. Compare this to overall (no size) 54% said they run only OSS (or their enterprise versions) internally.


MongoDB-Compatible Database Use

The dominance of big data, real-time applications, development agility, and the rigidity of relational database models gave rise to the adoption of NoSQL databases. NoSQL databases were built with flexibility and scalability in mind, and follow the Basic Availability, Soft State, and Eventual Consistency (BASE) model.

MongoDB specifically is the most popular NoSQL database. MongoDB Community is the clear winner (much more so than MongoDB Enterprise) in our survey, with 34% of respondents using it at their companies. It is unusual to have more than one different MongoDB and MongoDB-compatible vendor installs as a single company. Large companies, however, are more likely to use MongoDB Enterprise over MongoDB Community than small or medium companies. 


Do You Pay for Support or Self Support?

Our respondents generally agree that they prefer to self-support their open source database environment rather than pay for support. This is (somewhat surprisingly) true across the board for both technical employment and management (regardless of the size of the company). For management, this could be a perceived cost issue. For technical roles, this could be an attempt to ensure job security.

In a later section, however, respondents claim that the fact that open source software doesn’t generally include support options is a limiter to OSS adoption. Perhaps “support” is being used in different ways: the standard support included with commercial software that provides limited help (installs, easy configuration problems, etc.), and the type of support that is required for deep technical assistance in a particularly complicated environment (the kind people are reluctant to pay for unless they have a specific need).



What Are Compelling Reasons to Adopt More Open Source Technologies?

The top two responses to this question are the same ones that continue to dominate discussion on the benefits of open source: cost savings (79.4%) and avoiding vendor lock-in (62%). The benefit of having a community also scored highly (over 50%).

There are some interesting differences between management and non-management answers to these questions.

There is an 8% uptick in responses from management on avoiding vendor lock-in. It looks like larger enterprise management and non-management are on the same page about vendor lock-in. In small to medium companies, however, there seems to be a disconnect.

There is a 6% uptick in those looking for additional security.

Those who list vendor lock-in as a critical reason to adopt OSS are on average 10% less likely to buy support from a vendor (potentially viewing support as another form of lock-in). Note that in medium-sized companies, the number jumps up 19% for management being more likely to pay for support versus non-management. In large companies, management “support” for support drops 16% and is 4% below non-management.


Non-Management Reponses:

Size

Cost Savings

Ease of Use

Security

Avoiding Vendor Lock-in

Community

Small

72%

44%

28%

53%

48%

Medium

68%

43%

29%

52%

46%

Large

68%

40%

28%

60%

38%

Unknown Size

20%

20%

20%

10%

20%

Total

70%

43%

30%

53%

46%


Management Responses:

Size

Savings

Ease of Use

Security

Avoiding Vendor Lock-in

Community

Small

74%

40%

38%

58%

46%

Medium

76%

48%

28%

72%

52%

Large

69%

19%

25%

56%

38%

Unknown Size

50%

50%

0%

0%

0%

Total

73%

42%

36%

61%

49%


Concerns About Adopting Open Source Database Software

Many companies are still wary of open source database technologies, with a perceived potential lack of support topping the list at 46% for management, and 39% for non-management respondents. The concern is similar amongst all company sizes.

There is a large jump in medium-size companies management respondents, 23% higher than non-management. It does make sense however that the medium-to-large size company management shows a 60% combined concern around lack of support, compared to 44% of technical users: that’s a 16% swing.

The second biggest concern is bugs, coming in at 32%. It is interesting that in large companies, management seems to be slightly less concerned (9% less). This could go hand in hand with larger companies who are willing to pay for support from vendors to get these fixed.

The third reason is security and compliance. It’s interesting that the tech users here are a bit more concerned then the management — maybe because management expects tactical employees to be proactive on security and compliance needs.

There is a 16% upswing in management’s concerns about vendor lock-in versus technical respondents in the survey (in large companies). Overall, management is 4% more concerned with vendor lock-in.

“I believe all companies (especially start-ups) cannot afford to buy support plans.”


Non-Management Responses:

Size


Costs

Lack of Support

Licensing

Bugs

Performance and Scalability

High Availability

Data Breaches

Security and Compliance Issues

Vendor Lock-in

Small

7%

42%

14%

30%

24%

20%

12%

23%

12%

Medium

8%

43%

12%

35%

27%

26%

14%

37%

10%

Large

15%

45%

15%

34%

31%

23%

12%

45%

9%

Unknown Size

30%

30%

20%

30%

20%

20%

20%

20%

10%

Total

9%

39%

15%

32%

25%

21%

14%

27%

12%


Management Responses:

Size


Costs

Lack of Support

Licensing

Bugs

Performance and Scalability

High Availability

Data Breaches

Security and Compliance Issues

Vendor Lock-in

Small

7%

47%

18%

35%

28%

25%

15%

26%

18%

Medium

7%

66%

14%

34%

21%

28%

7%

28%

10%

Large

0%

50%

13%

25%

19%

6%

13%

31%

25%

Unknown Size

0%

0%

0%

0%

50%

0%

0%

0%

0%

Total

9%

46%

12%

32%

24%

20%

13%

26%

16%


Compelling Reasons to Run Enterprise or Subscriber Only Versions of Software

For respondents to this question, support continues to be a strong theme. 67% of companies who use enterprise software or subscriber-only versions cite 24x7 support as the most compelling reason why. It is worth pointing out that expert 24x7 support is available to open source software users, and generally much cheaper than it would be from an Enterprise software vendor.

Peace of mind also rates highly at 42%, as companies take comfort in the status quo and longer-established “brand” versions. As much as companies talk about agility, change can be very difficult.

Nearly 30% are attracted by additional security features, often locked behind a paywall for Community users.

Monitoring and management tools come in second. However, there is a noticeable drop in their value in the management space.

This could be an indication that management is less interested in “how” something is done, only that the systems perform and provide the data they need to make decisions.

Non-Management Responses:


Size

Peace of Mind

Security

HA

Monitoring and Management Tools

24/7 support

Integration with Enterprise Tools

Performance and Scalability

Automation Tools

Small

32%

23%

28%

32%

57%

21%

22%

17%

Medium

34%

27%

28%

32%

66%

29%

18%

22%

Large

40%

35%

28%

40%

62%

29%

22%

22%

Unknown Size

30%

10%

20%

20%

20%

10%

10%

10%

Total

34%

25%

28%

33%

56%

22%

21%

18%



Management Responses:

Size

Peace of Mind

Security

HA

Monitoring and Management Tools

24/7 Support

Integration with Enterprise Tools

Performance and Scalability

Automation Tools

Small

42%

22%

31%

33%

52%

22%

19%

13%

Medium

38%

28%

24%

34%

62%

21%

14%

10%

Large

25%

19%

25%

13%

50%

19%

13%

0%

Unknown Size

0%

0%

0%

50%

50%

0%

0%

0%

Total

36%

22%

24%

27%

52%

19%

19%

11%


Conclusions

Percona would like to once again thank everybody who participated in this survey. It provides us and the entire open source community with interesting, timely, and useful information about how enterprises of all sizes are using, developing, and troubleshooting open source database software.

Percona is an open source company that is committed to supporting the goals and ideals of the open source community. We will continue to conduct surveys like this in the future, and continue to provide excellent data to community members.

As a champion of open source, we are committed to having data open and free to all.

Download the full report or the raw data: