Berlin

November 4 & 5, 2024

New York

September 4 & 5, 2024

PostgreSQL: The database that quietly ate the world

How the open source database slowly but surely infiltrated the enterprise as a go-to, developer-friendly choice.
April 29, 2024

You have 1 article left to read this month before you need to register a free LeadDev.com account.

The open source database has slowly but surely become ubiquitous, and has infiltrated the enterprise as a go-to, developer-friendly choice.

PostgreSQL (or just Postgres) is a hugely popular object-relational database. Now while that statement should seem obvious, not everyone will realize just how ubiquitous this community-driven project has become since its humble beginnings all the way back in 1986. “We can’t think of any other technology that has the same level of maturity, while also attracting new developers,” says Ajay Kulkarni, CEO and cofounder of Timescale, a time series database built on Postgres.

Recent studies show the extent of the PostgreSQL takeover. For instance, the 2023 StackOverflow survey saw PostgreSQL eclipse MySQL for the first time as the top database of choice, with 49% of professional developers citing they have done extensive development work with it. According to the latest State of PostgreSQL report, 51% of respondents said that it’s being used more today in their organizations than a year ago.

“PostgreSQL has been quietly developed and improved by a dedicated and constantly growing group of developers,” says Ryan Booz, PostgreSQL advocate at Redgate Software. Due to its simple and permissive license, community backing, and versatility, PostgreSQL has silently found itself at the center of modern software development.

Slow and steady wins the day

PostgreSQL development has gone on behind the scenes for a while, where maintainers have consistently released a major version for almost 20 years straight. “Of the other major database platforms, only PostgreSQL can claim such consistency, despite the challenge of global volunteer developer coordination,” says Booz.

That slow and steady growth has seen PostgreSQL slowly consume market share, more like a black hole than a supernova. But is it really eating the world

“Absolutely it is,” says Craig Kerstiens, Chief Product Officer of Crunchy Data and curator of the Postgres Weekly newsletter. While MySQL initially gained popularity due to its use in large off-the-shelf applications like the content management systems WordPress and Drupal, Kerstiens credits his old employer, Heroku, for helping usher PostgreSQL into the massive Ruby and Python communities.

Christos Karamanolis, a fellow at Broadcom’s VMware Cloud Foundation, says that PostgreSQL is the second most popular database running on VMware virtual machines, after Microsoft’s SQL Server, and its usage keeps growing. Based on production data and conversations with users, VMWare anticipates PostgreSQL adoption to continue to accelerate as many databases have already become PostgreSQL-compatible, but also due to its high maturity, low cost of ownership, and extensibility.

The manifold benefits of using PostgreSQL

There are a handful of benefits to using PostgreSQL compared to other database styles. Reliability, for one. “PostgreSQL has always emphasized data reliability and safety first and foremost,” he says. When it comes to database management, boring is good.

A solid foundation has enabled PostgreSQL to layer on more advanced features over the years, such as JSON support, full-text search, advanced geospatial capabilities, and more. “PostgreSQL has been developed over time to address the needs of the larger community, resulting in many features that are not available elsewhere,” says Booz. PostgreSQL sports hundreds of standalone extensions to enhance the core database engine. He also highlights its support for built-in datatypes, indexes, and solid programming language support.

The maturity of PostgreSQL also means that many security holes have been patched. “It has been battle-tested for decades,” says Kulkarni. To Kulkarni’s point, PostgreSQL has introduced various security features over the years, such as role-based access control (RBAC), secure socket layer support (SSL), row-level security (RLS), and a secure method to extend the database. The maintainers have also proven themselves at responding to new vulnerabilities within regular patches.

That doesn’t mean it’s a perfect solution, however. “PostgreSQL is the jack of all trades in the databases world,” says Karamanolis. It’s not a data warehouse solution, but can support specialized capabilities like data intelligence and vector indexing decently well. Therefore, for him, defaulting everything to PostgreSQL outweighs the operational overhead of supporting multiple niche database styles simultaneously, helping to reduce total cost of ownership.

Through native features and extensions, PostgreSQL supports transactional, analytical, time-series, vector, graph, search, document, and other use cases. In addition to some of the benefits mentioned above, support for ACID transactional semantics and full serializability are plusses for developers.

The real charm is in its openness

However, one of the biggest benefits comes from PostgresSQL’s open, non-proprietary nature. And since PostgreSQL is not supported by a single company, you avoid vendor lock-in. “Due to its permissive license, many data companies start with Postgres and evolve it to other use cases,” says Kerstiens, making it a good core layer for various database-as-a-services.

According to Tanmai Gopal, CEO of Hasura, the open protocol underpinning PostgreSQL allows innovations and ideas from outside of core Postgres to flow into the community. Its open extension ecosystem and commitment to reliability allow PostgreSQL to rival some of the most expensive and incumbent enterprise databases, he says.

On the other end of the spectrum is MySQL, owned and maintained by Oracle. ”Oracle is starving the development of the MySQL Community Edition and focusing on HeatWave,” says Dave Stokes, technology evangelist at the database specialists Percona. In contrast, the Postgres community is large and thriving.

Understandability within PostgreSQL is also helping it reach new audiences. “Technology leaders love PostgreSQL because their teams can start writing code on it almost immediately due to its familiar API,” says Karthik Ranganathan, founder and co-CEO of Yugabyte. As such, he sees PostgreSQL becoming the default API for cloud-native transactional databases – this is no surprise since most relational databases already offer some sort of compatibility with PostgreSQL.

Where does PostgreSQL stand out? Nearly everywhere

PostgreSQL is flexible, and its use cases span industries and verticals. “It is really used all over the place,” says Kerstiens, who has seen PostgreSQL implementations across enterprise banking, healthcare, startups, and even the education space. Kulkarni also attests that PostgreSQL knows no bounds, having seen it in production from industrial conglomerates to crypto startups, AI games, and beyond. “We literally see PostgreSQL everywhere,” he says.

Although PostgreSQL is primarily a relational database management system, its configurability and extensions make it quite versatile. Booz says PostgreSQL is malleable for various workloads, spanning “from weekend projects to petabyte-scale analytical workloads.” As he describes it, PostgreSQL isn’t the answer for every project, but it’s a good starting point.

It’s tough to find areas where PostgreSQL does not already excel, partially because ongoing developments continue to cover new use cases. “Currently, core Postgres is perhaps the best overall general-purpose transactional relational database,” says Gopal. “Each release increases its ability to handle more scale and an increasing variety of workloads,” he says.

That said, PostgreSQL is not the only tool for the job. For example, Gopal notices a general-purpose database war playing out between PostgreSQL and MongoDB. The latter is preferred by some developers because of its modern, NoSQL database document-oriented model, as opposed to PostgreSQL’s traditional relational model using ACID constraints. 

Some developers also desire niche databases catered to specific workloads for performance and cost savings. Therefore, organizations will likely continue to grapple with a polyglot database ecosystem, in some form or another.

The future of Postgres

PostgreSQL may not be the shiny new tech of the day, but in an age of AI fanfare, expensive cloud-native tech, and increasingly commercially-bent open source, stable open software of this maturity and scale is a rare gem. “You want a safe and reliable database, kind of like my bank,” says Kerstiens.

The PostgreSQL community is strong and still growing, receiving new contributions in the form of extensions and tooling, with some companies even hiring developers to actively support the open source project, notes Booz. These efforts will keep the project relevant far into the future. “There are no signs that PostgreSQL’s growth is slowing down, and I believe we will continue to see the first-of-their-kind features and enhancements for many years to come,” says Booz. “Don’t bet against Postgres.”

This wager comes at a time when application developers are having a more influential role in technical decisions within enterprises. “There is no doubt that the market is moving towards open source databases and other data services,” says Karamanolis. However, he admits that operationalizing vanilla open source databases will require technical expertise – a skill that is hard to come by.

Keep on Postgres’ing

It’s taken years, but Postgres has emerged as the ubiquitous lingua franca of the relational database world. “Nearly everything speaks PostgreSQL or has a PostgreSQL connector,” says Kulkarni. “Everything works with it, out-of-the-box.”

That said, there is always room for development. And the maintainers seem well aware. In fact, one analyst who reviewed ten years of major PostgreSQL updates found drastic improvements in latency from v8 to v16, with each new release bringing a 15% performance improvement.

Other commentators anticipate the tried-and-true database continuing to enhance performance, expand capabilities, and, in essence, march at the same steady pace it has been for the last twenty years. In short, says Kerstiens, “Postgres is mostly going to keep Postgres’ing.”