Serverless architecture has been gaining popularity in the world of technology. With its promise of reduced infrastructure costs and increased scalability, serverless architecture has become a go-to solution for many companies, big and small. This popular architecture makes developers not to consider issues such as server management, scalability, backup, failover or anything infrastructure-related issues. Instead, they can focus on application development, improving productivity and CI/CD performance.
Serverless can also have cost benefits, because users only pay for the computing resources they actually use, instead of buying expensive servers and storage devices or renting virtual private servers. It offers scale-to-zero when there is no traffic and scale up when there is traffic.
Pros and Cons of serverless architecture
- Enterprises do not need to manage backend infrastructure
- Save team time, no need for a full server management team
- Low operating costs and better development productivity
- Optimized pay-by-use architecture to have cost savings
- Lack of control, higher dependency
- Cold start affects performance
- Vendor lock-in may be a risk
- Security concerns
Neon Serverless Database
Neon Serverless database is one of the current players in the serverless database space. It is built on top of PostgreSQL and achive the separation of
storage, which essentially turns PostgreSQL from a stateful to a stateless machine. The core components that make this possible are written in
rust. This fundamental change is the main building block of bringing PostgreSQL serverless.
The basic architecture of Neon looks like this:
where the green and purple blocks form the
storage part and the blue forms the
compute part. The PostgreSQL nodes in the blue box are the
compute nodes that can be started or stopped by an orchastrator like kubernetes based on load. It also supports multi-tenancy, which means we can have multiple compute nodes up to serve different tenants, each have a totally isolated and separated database spaces.
Changes done on the compute nodes generate WAL files, which are sent to safekeepers. Paxos consensus protocol is used to ensure majority of the safekeepers ackowledged the receipt of a WAL segment, before it passes the segment to pageserver where the actual
WAL REDO takes place. Safekeeper is also responsible for do WAL backups on cloud storage (currently support minio and AWS S3).
Page server is responsible for serving
page data to requesting compute nodes. It produces a page based on the LSN requested by compute nodes. Based on this, it could derive a new page from a historical page by apply WAL (REDO) on the page. It is also responsible for backing up layer files (page and WAL files needed to derive any page) to cloud storage.
Serverless architecture is changing the game in database hosting. It offers a cost-effective and scalable solution for companies of all sizes. With Neon achieving separation of
compute and have them open-sourced; it could potentially attract other players to build more innovations around this innovation. I am sure, very soon, we will see more new PostgreSQL-based serverless solutions emerging in the market.
Cary is a Senior Software Developer in HighGo Software Canada with 8 years of industrial experience developing innovative software solutions in C/C++ in the field of smart grid & metering prior to joining HighGo. He holds a bachelor degree in Electrical Engineering from University of British Columnbia (UBC) in Vancouver in 2012 and has extensive hands-on experience in technologies such as: Advanced Networking, Network & Data security, Smart Metering Innovations, deployment management with Docker, Software Engineering Lifecycle, scalability, authentication, cryptography, PostgreSQL & non-relational database, web services, firewalls, embedded systems, RTOS, ARM, PKI, Cisco equipment, functional and Architecture Design.