Enterprise PostgreSQL Solutions

Parallel execution of postgres_fdw scan’s in PG-14 (Important step forward for horizontal scaling)

Back in August 2019, I wrote about the Horizontal scaling / Sharding in PostgreSQL and where it is going https://www.highgo.ca/2019/08/08/horizontal-scalability-with-sharding-in-postgresql-where-it-is-going-part-3-of-3/. One of the key feature that i talked about in this blog is parallel foreign scan i.e. asynchronous append of FDW nodes. Finally the first patch of this feature was committed to PG-14, while they…
Read more

HG-PGSQL 1.7 and HG-PGSQL 2.3 Released

HighGo CA has released an update for its supported database systems on all supported platforms. HighGo Software Canada (a subsidiary of HighGo Software Inc) has released an update for HighGo Postgres Sever (HG-PGSQL 1.7) and HighGo Postgres Sever (HG-PGSQL 2.3).  The HG-PGSQL 1.7 update contains all the changes included in PostgreSQL community release version 12.7…
Read more

Getting started with pg_bulkload

  pg_bulkload is a high-speed data loading tool for PostgreSQL, compared with the copy command. The biggest advantage is speed. In the direct mode of pg_bulkload, it will skip shared buffer and WAL buffer, writing files directly. It also includes the data recovery function to recover in case of import failure. GitHub: https://github.com/ossc-db/pg_bulkload pg_bulkload mainly…
Read more

Avoiding, Identifying, and dealing with PostgreSQL database corruption – Part 2

This is the second of the two-part series of blogs about PostgreSQL data corruption. Part-1 discussed identifying the data corruption in PostgreSQL and lists some DO’s and DON’Ts for avoiding that. In this part, I will attempt to walk you through possible ways to recover from the corruption and ways to salvage data in case of…
Read more

Hash Index Internals

Indexes are key database server features that enhance its performance to retrieve data faster than a complete table scan (unless the index is vastly bloated). They generally work by maintaining a smaller set of data in a more “searchable” structure as compared to the complete table data. This data organization comes at the cost of…
Read more

Calling Stored Procedure from JAVA – Part 2

Back in Nov 2021, I wrote about calling Stored procedure from Java and demonstrated how we can call a simple stored procedure from Java or a one that contains a IN parameter. I have done some more playing around with calling Stored procedure from Java so this blog is about calling a stored procedure with…
Read more

How does Postgres handle external parameters?

There are many advantages to using bind variables in SQL statements, especially in conditional statements. Using bind variables can save the trouble to parse and optimize the query in every execution. There are also disadvantages such as bind variable will make optimizer lost its prediction accuracy and cause performance decrease. In PostgreSQL are represented by…
Read more

How to run Hierarchical Queries with PostgreSQL

A hierarchical query is built upon parent-child relationship, the relationship exist in the same table or view. The relationship dictates that each child can have one parent while a parent can have many children. Hierarchical query is a SQL query that handles data of hierarchical model i.e. an organisation structure where every employee has one…
Read more

How to check and resolve Bloat in PostgreSQL

Bloating in database is created when tables or indexes are updated, an update is essentially a delete and insert operation. The diskspace used by the delete is available for reuse but it is not reclaimed hence creating the bloat. Same is the case with PostgreSQL database, frequent UPDATE and DELETE operations can leave a lot…
Read more

Monitoring PostgreSQL with Nagios and Checkmk

A key aspect of maintaining a high performance database is continuous monitoring of system resources, as any issues anywhere on the system can potentially lead to downtime. There are options aplenty for monitoring, so reinventing the wheel doesn’t make much sense. For that purpose, I’ve selected a couple of options that are simple to deploy,…
Read more

How to create a system information function in PostgreSQL

1. Overview PostgreSQL supports many System Information Functions, such as Session Information Functions, Access Privilege Inquiry Functions, Schema Visibility Inquiry Functions, System Catalog Information Functions, Transaction ID and Snapshot Information Functions, etc. However, you may want build some special functions and integrate them into the PostgreSQL. This blog is going to walk through the steps…
Read more

How PostgreSQL Handles Sub Transaction Visibility In Streaming Replication

1. Introduction As an experienced PostgreSQL user, you may have a lot of experience in setting up streaming replication in your database clusters to make multiple backups of your data. But have you wondered how the standby is able to correctly determine if a tuple sent from the primary should be visible to the user…
Read more

PostgreSQL Logical Replication Advantage and Step by Step Setup

PostgreSQL is one of the most advanced open source databases globally with a lot of great features. One of them is Streaming Replication which was introduced in PostgreSQL 9.0—known as Physical Replication. It works by sending the WAL segments generated by the primary PostgreSQL to another standby server also running PostgreSQL. However, if we only…
Read more

Exploring SQL command changes in PG-14

I have spent sometime exploring all the SQL command changes in PostgreSQL 14, this blog will provide a brief summary about the SQL changes along with some examples. There are exciting new features that in progress for PG-14, this blog will only focus on the SQL changes that are committed and will be released with…
Read more

Troubleshooting Performance Issues Due to Disk and RAM

Troubleshooting Performance Issues Due to Disk and RAM

Quite a few managers or clients expect DBAs to type away a few commands to fix a sluggish PostgreSQL instance. If only things were that simple. Unfortunately, debugging a sluggish instance that was previously running optimally requires analysis at many levels. In this blog, I will attempt to help that debugging process on Linux by…
Read more

How PostgreSQL Inserts A New Record With The help of Table Access Method API and Buffer Manager

1. Introduction This blog talks about a high level description of the mechanism behind PostgreSQL to execute an INSERT query. This process involves many steps of processing before the data is put in the right place. These process normally involves several catalog cache lookup to determine if the destination table exists or several checking on…
Read more

How to setup Postgres 13 WAL streaming replication on Ubuntu 18.04

1. Overview PostgreSQL supports different type of replications, i.e. logical and physical, and there are many tutorials discussed about the replications. This blog is a simply walk-through of the WAL Streaming replication using the latest Postgresql-13 on Ubuntu 18.04. 2. Install Postgresql-13 from source code In this blog, we will install Postgresql from the source…
Read more

Avoiding, Identifying, and dealing with PostgreSQL database corruption – Part 1

All the things that can happen to databases, corruption is not the one that happens frequently, but when it does it can result in major downtime and data loss. Probably it is one of the most terrifying things for a DBA to come across and the best way to deal with it is to not…
Read more

A preliminary exploration of dynamic hashing in PostgreSQL

In computing, a hash table (hash map) is a data structure that implements an associative array abstract data type, a structure that can map keys to values. A hash table uses a hash function to compute an index, also called a hash code, into an array of buckets or slots, from which the desired value…
Read more

How PostgreSQL Executes Sequential Scans with the Help of Table Access Methods APIs

1. Introduction There are many approaches for PostgreSQL to retrieve the data back to the user. Depending on the user’s input query, the planner module is responsible for selecting the most optimum approach to retrieve the requested data. Sequential scan is one of these approaches that is mostly selected when the user requests a large…
Read more