Open Source Data Management Software Survey

2020 Results

This year, we set out to build upon the data we collected last year and continue to monitor the open source industry’s pulse. This year’s survey was perhaps reflective of the wider world environment, as companies indicated a desire to consolidate their database infrastructure and software, avoid risk, and manage costs.

There were some impressive pull-out figures, which revealed the underlying concerns of the market and some of the drivers for open source adoption:

  • Architects41% of buying decisions are now made by architects, giving them significant power over software adoption within a company.
  • cloudcostsAlthough promoted as a cheap and convenient alternative, cloud costs can spiral, with 22% of companies spending more on cloud hosting than planned.
  • toolsOpen source software users favor open source tools. 66% of respondents reported adoption of open source database observation, monitoring, and alerting tools.
  • footprintThe deluge of data continues. A massive 82% of respondents reported at least a 5% database footprint growth over the last year, with 62% reporting more significant growth and 12% growing over 50%.
  • upgradeTo keep pace with their competition, many companies needed to upgrade or migrate their databases and software in 2020. Only 12% of respondents made no changes to their environment in the last year, compared to 28% who made changes 2-3 times, and 21% who made changes ten times or more.
  • downtimeWhen asked ‘what keeps you up at night?,’ our respondents flagged downtime and performance as their biggest concerns. Performance issues were the most significant issue that companies experienced, impacting 74% of respondents.
  • costsavingsOpen source database software remains crucial for many companies. 81% of respondents gave cost savings as the most important reason for adoption. In this challenging economic climate, many companies are actively avoiding vendor license costs and lock-in.

Who Responded to the 2020 Survey?

The United States continues to represent the most significant base of respondents, with 24%. We also saw a considerable uptick in India’s responses, which contributed 4.8% in 2019 and shot up to just over 10% in 2020.

The remaining respondents are spread across the world, giving us a truly global perspective and showing the diversity, reach, and enthusiasm of the open source community.

693 Total Respondents from 83 Countries

169 from the United States

229 from European Nations

44 from Latin America

79 from South Asia

Company Size

429 Small Company Respondents (1-500)

130 Medium Company Respondents (500-5,000)

112 Large Company Respondents (5,000+)

Interestingly, although we received slightly fewer responses than the 2019 survey, we saw a marked increase in responses from medium and large companies. This backs-up the claim that larger organizations are increasingly utilizing open source solutions for their businesses.

Open source software is attractive for various reasons, including cost reduction, the need for agile and responsive technologies, and a desire to avoid vendor lock-in.

“The vision that we have from the company is to be able to always use free software. Although we don't have many databases, the little that we have, we always opt for free software.”


Industry

“I have been using open source software for the past 11 years. Worked with internet, telecom, eCommerce, advertising industries. It’s always a challenging and good experience to work with open source software.“

  • 116 were Information Technology Companies
  • 79 are Cloud-based Solutions or Services Companies
  • 75 Identify as Software as a Service (SaaS) Development Companies

  • As in 2019, many survey respondents were technology-focused companies, offering IT, cloud, and SaaS solutions to startups, growing businesses, and established enterprises.

    Information technology

    16.69%

    Cloud-based solutions or services

    11.37%

    Software as a service (SaaS) development

    10.79%

    Financial technology or services

    9.50%

    Retail or eCommerce

    7.77%

    Media, advertising, publishing, or entertainment

    6.19%
    Consulting 5.32%
    Education 4.89%
    Healthcare technology or services 4.89%
    Government 4.32%
    Other software development 3.88%
    Web development or design 3.17%
    Data and analytics

    2.88%




    Roles and Decision Making

    Following our 2019 survey, we wanted to get more information on our respondents’ specific roles and the decision-making process that organizations go through.

    In 2020, the largest group who responded to the survey were Database Administrators (almost 50% of the total). However, when it comes to actual decision-making, Architects were flagged as predominantly deciding which database technology should be used for new applications.

    “My role is almost entirely technical. I can say that it has certainly changed over the last few years, and management is far more open to the use of open source software in general then it used to be.”

    Which of the following best describes your role?

    Database Administrator

    49.21%

    Architect

    28.55%

    Developer/Software Engineer

    26.26%

    DevOps Engineer

    22.38%

    System Engineer

    19.51%

    Manager

    15.06%

    Site Reliability Engineer

    11.19%

    Executive

    9.33%

    Who usually chooses which database technology is used for new applications in your organization?

    Architects

    41.41%

    Developers

    26.12%

    DBAs

    16.45%

    Management

    13.13%

    Other

    5.98%

    Download the report:

    Database Demographics

    Number of Database Instances In Production

    Our data showed that over half of businesses used 25 instances or less. Unsurprisingly, the larger the company, the more often respondents had up to 10000 instances (or even more) in production.


    Database Footprint

    Another of our new questions for 2020 looked at how database footprints were growing. As we suspected, 82% of respondents reported at least a 5% growth over the last year, and 62% reported a more significant increase, with 12% growing over 50%!

    Multiple Databases, Multiple Locations, and Multiple Platforms

    In 2019 we announced that multi-database deployments were king. In 2020 the trend has continued, with even more companies choosing to run multiple databases in multiple locations over multiple platforms.

    Multi-Database Deployments

    Overall, companies that ran a combination of MongoDB and PostgreSQL showed the most significant overall jump, with a rise from 24% in 2019, to 30% in 2020.

    Medium-sized companies increased their adoption of multi-database deployments the most in 2020, with an 8% increase in the combinations of MySQL and MongoDB, and MongoDB and PostgreSQL, and a 7% increase in the mix of MySQL, MongoDB, and PostgreSQL. This growth in multi-database adoption by medium-sized companies could be a result of companies wanting to stay agile and avoid vendor lock-in by using open source alternatives.

    The most significant growth segment for larger companies, up by 4% in 2020, was the combination of MongoDB and PostgreSQL. Conversely, the combination of MySQL and PostgreSQL dropped by 6% for larger companies this year. The combination of MySQL and PostgreSQL had the least growth of the four combinations, remaining static, or falling, for small and medium companies. One explanation might be that companies are choosing a completely NoSQL alternative, rather than running multiple flavors of SQL databases.

    “I've used predominantly open source databases like MySQL and PostgreSQL for over 12 years now and I'm confident in open source technology because they've evolved well over the years.”




    Hybrid Hosting: Private, Public, or On-Premise

    Last year we identified an increasing number of companies adopting a hybrid cloud strategy to mitigate risk. This trend has flattened out, with 38% of companies responding that they utilize a hybrid approach compared to 41% last year. There was also a small drop in the number of companies hosting on-premises, from 55% in 2019 to 51% in 2020.

    “We do not use SaaS, PaaS, IaaS or anything cloud-like. All servers are physical, all applications are installed on-prem that we manage ourselves. This is how we keep costs down while achieving zero downtime.”



    Cloud Hosting

    Unsurprisingly AWS continues to dominate the public cloud provider market, with a bump from 46% of respondents using its cloud platform in 2019 to 50% in 2020. Microsoft Azure jumped to 23% from 16% in 2019, but Google Cloud failed to make an impact this year with an adoption rate that remained static year-or-year at 18%.

    “A catchphrase in the industry was ‘Nobody gets fired for buying IBM.’ Such disregard for vendor lock-in has now become ‘Nobody gets fired for buying AWS.’”

    In 2019 smaller companies were more likely to use Google than Microsoft, though this gap closed entirely in 2020. Medium and large companies indicated that they were far more likely to choose Microsoft over Google this year, with a big jump from 31% of large companies using Microsoft in 2019 to 44% in 2020. Our results showed a significant decline in popularity for Google, who lost market share to Microsoft across the board.



    Multi-Cloud Use

    This data shows that moving to the cloud is still a significant focus for all company sizes, as cloud companies make it as easy as possible to migrate. Many businesses are attracted by the possibility of ‘fully-managed services,’ lower hosting and license fees, and reduced data center and hardware costs.

    Our results showed a bump in companies now ‘running some DBaaS,’ up from 40% in 2019 to 45% in 2020. Unsurprisingly, this also meant companies that are running ‘No DBaaS’ fell from 60% in 2019 to 56% in 2020.

    The number of companies using more ‘2 or more providers’ also increased (in line with the trend of companies looking to mitigate their risk), going from just 10% in 2019 to 14% in 2020.

    The most significant change here is large companies, 28% of which now use two providers or more, compared to just 17% in 2019, far higher than small and medium companies (at 12% and 10% respectively). This might be because larger companies have more data to parcel off and more money to spend on multiple providers.


    Upgrades

    At Percona we have always advocated database optimization as a way of reducing upgrade and storage costs. The responses we received to this new question show that businesses are potentially spending vast amounts on potentially unnecessary upgrades, and making significant changes to their database infrastructure.

    Only 12% of respondents made no changes in the last 12 months, compared to a massive 28% who made changes 2-3 times, and 21% who made changes ten times or more!

    “Thank you Percona for your contribution to the open source database community. Thank you for giving awesome tools and software. It has given a lot of confidence to the community and to open source database technologies.”


    Cloud Costs

    On the other hand, it looks as if businesses estimated their cloud hosting costs fairly accurately in 2020, with 60% saying that the amount they spent was about what they had planned. 17% spent less than they planned, but nearly a quarter of respondents spent more than anticipated.



    Containers

    Last year we asked the question, ‘are containers the next big thing?’ It looks as if the answer is perhaps, ‘not quite yet!’

    In 2019, 26% of respondents used containers, but not necessarily to run databases. Many respondents were unaware of whether they were using containers for their databases. As the company size increased, so did container adoption in production environments.

    This year, just over 28% of respondents say they use containers, with large companies increasing their usage the most from 33%, up to 41% in 2020.


    Adoption of Kubernetes increased slightly in 2020, with 47% of respondents using it internally for development and testing of applications (up from 44%) and 36% using it for production (up from 32%).

    The percentage of respondents who used Kubernetes to run databases also increased this year, with 36% using it for development and testing (up from 33%) and 23% using it for production (up from 17%).



    Install Demographics

    MySQL continues to dominate in open source database adoptions, though its market share has remained static year on year. PostgreSQL, on the other hand, had a decent boost, going from 46% in 2019 to 52% in 2020.

    “We have used open source software as much as we can for our operations, saving hundreds of thousands of dollars in the process. Though some initial investment in time was needed to learn and implement the software, it is important to note that, for the most part, we have deployed and then not had to even touch the systems except for upgrades and patches.”

    MySQL Community Edition

    58%

    PostgreSQL

    52%

    Microsoft SQL Server

    44%

    Elastic Search

    43%

    Redis

    42%

    MariaDB Community Edition

    36%

    Oracle

    36%

    Percona Server for MySQL

    34%

    MongoDB Community

    36%

    Kafka

    28%

    Amazon RDS for MySQL

    24%

    SQLite

    22%

    Percona XtraDB Cluster

    17%

    Cassandra

    18%

    MySQL Enterprise

    14%

    Amazon Aurora (MySQL Compatible Edition)

    13%

    Amazon RDS for PostgreSQL

    14%

    Solr

    14%

    IBM DB2

    11%

    Splunk

    11%

    Hive

    9%

    HBase

    8%

    MongoDB Enterprise

    8%

    Percona Server for MongoDB

    8%

    Amazon Aurora (PostgreSQL Compatible Edition)

    7%

    Google Cloud SQL for MySQL

    6%

    Couchbase

    6%

    Neo4j

    6%

    Microsoft Azure Database for MySQL

    5%

    EDB Postgres Platform

    5%

    MariaDB Enterprise

    3%

    MongoDB Atlas

    4%

    Amazon DocumentDB

    4%

    Microsoft Azure Database for PostgreSQL

    3%

    Percona Distribution for PostgreSQL

    3%

    TiDB

    2%

    Alibaba Cloud: ApsaraDB RDS for MySQL

    2%

    CockroachDB

    2%

    Azure Cosmos DB

    2%

    Postgres-BDR

    1%

    Tencent Cloud: Cloud Database for MySQL

    1%

    Postgres-XL

    1%

    FoundationDB Document Layer

    0.3%


    MySQL-Compatible Database Use

    MariaDB Community Edition

    36%

    Percona Server for MySQL

    34%

    Amazon RDS for MySQL

    24%

    Percona XtraDB Cluster

    17%

    MySQL Enterprise

    14%

    Amazon Aurora (MySQL Compatible Edition)

    13%

    Google Cloud SQL for MySQL

    6%

    Microsoft Azure Database for MySQL

    5%

    MariaDB Enterprise

    3%

    TiDB

    2%

    Alibaba Cloud: ApsaraDB RDS for MySQL

    2%

    Tencent Cloud: Cloud Database for MySQL

    1%


    MongoDB-Compatible Database Use

    MongoDB Community

    36%

    MongoDB Enterprise

    8%

    Percona Server for MongoDB

    8%

    MongoDB Atlas

    4%

    Amazon DocumentDB

    4%

    Azure Cosmos DB

    2%


    PostgreSQL-Compatible Database Use

    PostgreSQL

    52%

    Amazon RDS for PostgreSQL

    14%

    Amazon Aurora (PostgreSQL Compatible Edition)

    7%

    EDB Postgres Platform

    5%

    Microsoft Azure Database for PostgreSQL

    3%

    Percona Distribution for PostgreSQL

    3%

    CockroachDB

    2%

    Postgres-BDR

    1%

    Postgres-XL

    1%



    Database Observation, Monitoring, and Alerting Tool Use

    People who prefer to use open source database options show a strong preference for open source database observation, monitoring, and alerting tools. Our survey shows that 66% of respondents use open source database tools — consistent across all business sizes.

    26% of our survey respondents use cloud-based tool options, up from 22% last year. Continued growth here is likely, as the race to host and run databases in cloud-based environments increases.

    “We only use open source unless it's a very specific tool or service. It means we can scale [and] automate and not worry about horrible things like license costs.”



    Support — How and Where to Get Help

    Last year’s data indicated that respondents generally preferred to self-support their open source database environment, rather than pay for support. The 2020 results showed the same story.

    “Community support has become the premium option. It far surpasses my past experiences with proprietary support from Oracle and Microsoft - which were both very good for many years.”



    Concerns and Obstacles

    This year, we dug deeper into the issues that companies face in their environments and their biggest database management concerns. When asking, ‘what keeps you up at night?,’ the most significant issues flagged were downtime and performance, along with the concern of fixing emergency issues. This maps closely with commentary in the real world, where performance issues and unplanned downtime were the top reported issues of the past year.

    “Since there are diverse contributors to the open source community, the rate and level of knowledge sharing need to accelerate to enable increased robustness of the products. Specifically for open source database engines, the focus needs to be on security and audit which has been neglected for long.”


    Interestingly, the number of people who actually experienced security issues in the last year was just 12%, despite it coming reasonably high on the list of concerns people had (28%). Performance issues were by far the biggest experienced issue for nearly three-quarters of respondents at 74%, with unplanned downtime impacting 45%.

    Given the negative impact that performance issues and downtime has on businesses, this reiterates how crucial it is to ensure that your databases are correctly configured and optimized.

    What Keeps You Up At Night?

    Downtime/High Availability

    59%

    Performance issues

    51%

    Fixing emergencies

    35%

    Security issues

    29%

    Bad queries

    23%

    Lack of resources

    16%

    Staffing issues

    15%

    Cost concerns

    13%

    What Issues Have You Experienced in the Past 12 Months?

    Performance issues

    74%

    Unplanned downtime

    45%

    Bad code rolled into production

    38%

    Overworked staff

    28%

    Unexpected costs

    14%

    Security issues

    12%



    The Continuing Importance of Open Source

    Percona was founded on a belief in honesty and transparency, and as a leading provider of unbiased open source database solutions, we have a strong interest in the health of the open source market.

    When comparing 2019 results to this year, we saw some interesting trends. This survey was conducted at the beginning of the COVID-19 pandemic and concerns over cost savings and avoiding vendor lock-in were both flagged, each one a compelling reason to consider open source.

    “Without open source software, most of the companies I have worked at or owned would not have existed.”



    Concerns About Adopting Open Source Database Software

    Many companies remain wary of open source database technologies, and overall the weighting given to concerns about adopting open source remained relatively static in 2020.

    Concerns around performance, scalability, high availability, and security rose slightly, unsurprising given the number of stories in the press around data breaches and security issues. Companies are increasingly aware of the importance of their applications and websites being highly available to support online learning and working during the COVID outbreak.

    “Personally, I feel open-source software is the future, the community is amazing and features [are] added constantly, but I have seen most of our clients ask for enterprise edition mainly due to security and support concerns.”

    Cost concerns grew, which is a little strange given that open source software doesn’t charge license fees. This might be confusion around the ‘open source’ versus ‘open core’ model, which has caught some companies out in the past.



    Compelling Reasons to Run Enterprise or Subscriber-Only Versions of Software

    There was a bump in these numbers across the board in 2020, which might partly be explained by the significant increase in companies moving to the cloud and now finding themselves under contract to new vendors.

    The appeal of 24/7 support increased by over 10% in 2020 to 67%, which is interesting given that earlier in the survey, few companies indicated a willingness to pay for support.

    “Our company has supported open source initiatives since the 1990s and we are very familiar with the major players in the open source database space, we only use proprietary databases when the software we need requires it.”



    Conclusion

    Percona would like to thank everybody who participated in this survey. It provides the entire open source community with engaging, timely, and useful information about how enterprises of all sizes use, develop, and troubleshoot open source database software.

    Percona is committed to supporting the goals and ideals of the open source community. We will continue to conduct our annual

    Open Source Data Management Software Survey and provide the final data to community members.

    Download the report: