Setting up a debugging environment in PostgreSQL to better understand OpenSSL APIs

Enterprise PostgreSQL Solutions

Comments are off

Setting up a debugging environment in PostgreSQL to better understand OpenSSL APIs

1. Overview

In my previous blog, I discussed how to enhance PostgreSQL TLS security with OCSP Stapling. In this blog, I will share a simple procedure for setting up a gdb debugging environment to dive into TLS connections and gain a better understanding of the OpenSSL APIs used in PostgreSQL.

2. Build OpenSSL with debugging symbols

First, check out the OpenSSL source code and switch to the release you want to use. In this example, I want to compile the PostgreSQL with the OpenSSL 3.0.2 release, so I switch to release tag openssl-3.0.2.

$ git clone https://github.com/openssl/openssl.git
$ cd openssl
$ git checkout openssl-3.0.2

Second, compile OpenSSL with gdb enabled using the following options:

$ ./config --prefix=/tmp/ssl --openssldir=/tmp/ssl -d shared -g3 -ggdb -fno-inline -O0 -fno-omit-frame-pointer
$ make -j
$ make install

Compiling OpenSSL with debugging symbols can take a while, and you’d better set the prefix to somewhere so that this particular debug version can be persisted and reused.

3. Build PostgreSQL with debugging OpenSSL libraries

Again, first, check out the PostgreSQL source code and switch to the branch you are interested in. In this blog, I’ll just use the master branch for demo purposes:

$ git clone https://github.com/postgres/postgres.git

Second, compile PostgreSQL with debugging options enabled and specifically specify the OpenSSL source code and debugging libraries you want to link to:

$ ./configure --prefix=/tmp/pgapp --enable-tap-tests --enable-debug CC="gcc -std=gnu99" CFLAGS="-O0 -fno-omit-frame-pointer -I/tmp/openssl/include" LDFLAGS="-Wl,-rpath=/tmp/openssl" --with-openssl
make -j
make install

Here, you must specify LDFLAGS=”-Wl,-rpath=/tmp/openssl” so that PostgreSQL will link to the OpenSSL libraries as you specified; otherwise, it will link to the system ones. To double confirm the OpenSSL libraries linked with PostgreSQL are the ones you wanted, I typically run the following commands:

$ which postgres
/tmp/pgapp/bin/postgres
$ ldd /tmp/pgapp/bin/postgres
    linux-vdso.so.1 (0x00007ffd8a3c1000)
    libssl.so.3 => /tmp/openssl/libssl.so.3 (0x00007f6878dc6000)
    libcrypto.so.3 => /tmp/openssl/libcrypto.so.3 (0x00007f6877c00000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f68781cf000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f68780e8000)
    libicui18n.so.70 => /lib/x86_64-linux-gnu/libicui18n.so.70 (0x00007f6877800000)
    libicuuc.so.70 => /lib/x86_64-linux-gnu/libicuuc.so.70 (0x00007f6877605000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6877200000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f6878e75000)
    libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f6876e00000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f68780c8000)
    libicudata.so.70 => /lib/x86_64-linux-gnu/libicudata.so.70 (0x00007f6875000000)

Here, check the libraries libssl.so.3 and libcrypto.so.3 to make sure PostgreSQL and psql link to the OpenSSL libraries you expected. In the above example, my PostgreSQL links to the new OpenSSL libraries just compiled and installed with debugging symbols.

4. Enjoy debugging

To debug the OpenSSL APIs during a TLS connection setup or init, you need to have some certificates. You can either follow the PostgreSQL documentation or my blog to set up your own certificates or follow below steps to use the pre-built certificates under the ssl regression test folder. Here are the steps:

$ initdb -D /tmp/pgdata
$ cd postgres/src/test/ssl/ssl
$ cp -pr root_ca.crt server-cn-only+server_ca.crt server-cn-only.key /tmp/pgdata/
$ chmod 600 /tmp/pgdata/server-cn-only.key

Then edit the postgresql.conf file to enable SSL and specify the certificate-related files. For example:

$ vim pgdata/postgresql.conf
ssl = on
ssl_ca_file = 'root_ca.crt'
ssl_cert_file = 'server-cn-only+server_ca.crt'
ssl_key_file = 'server-cn-only.key'

Here, the ssl has been enabled, the root CA certificate pointed to root_ca.crt, and the entity certificate server-cn-only with the intermediate CA server_ca has been used as the PostgreSQL server’s certificate with the corresponding private key file.

Now, we are ready for gdb debugging the OpenSSL APIs.

$ gdb -args postgres -D pgdata
...

(gdb) b be_tls_init
Breakpoint 1 at 0x419bf2: file be-secure-openssl.c, line 95.
(gdb) b SSL_CTX_use_certificate_chain_file
Breakpoint 2 at 0xd9970

Breakpoint 1, be_tls_init (isServerStart=true) at be-secure-openssl.c:95
95      int         ssl_ver_min = -1;
(gdb) c
Continuing.

Breakpoint 2, SSL_CTX_use_certificate_chain_file (ctx=0x5555561e9ba0, file=0x5555561ddc08 "server-cn-only+server_ca.crt") at ssl/ssl_rsa.c:534
534     return use_certificate_chain_file(ctx, NULL, file);
(gdb) s
use_certificate_chain_file (ctx=0x5555561e9ba0, ssl=0x0, file=0x5555561ddc08 "server-cn-only+server_ca.crt") at ssl/ssl_rsa.c:419
419 {
(gdb) l
414  * Read a file that contains our certificate in "PEM" format, possibly
415  * followed by a sequence of CA certificates that should be sent to the peer
416  * in the Certificate message.
417  */
418 static int use_certificate_chain_file(SSL_CTX *ctx, SSL *ssl, const char *file)
419 {
420     BIO *in;
421     int ret = 0;
422     X509 *x = NULL;
423     pem_password_cb *passwd_callback;

(gdb) p file
$2 = 0x5555561ddc08 "server-cn-only+server_ca.crt"

(gdb) pt ctx
type = struct ssl_ctx_st {
    OSSL_LIB_CTX *libctx;
    const SSL_METHOD *method;
    struct stack_st_SSL_CIPHER *cipher_list;
    struct stack_st_SSL_CIPHER *cipher_list_by_id;
    struct stack_st_SSL_CIPHER *tls13_ciphersuites;
    struct x509_store_st *cert_store;

In the above example, I enabled two breakpoints: be_tls_init and SSL_CTX_use_certificate_chain_file. be_tls_init is used to ensure that PostgreSQL has been built with debugging symbols enabled, while SSL_CTX_use_certificate_chain_file is used to dive into OpenSSL. Here, we can print out this complex object ctx as one example”

With this debugging environment setup, you should be able to debug the psql client side to better understand the entire TLS process.

5. Summary

In this blog post, I have explained a simple process for setting up a gdb debugging environment to dig into OpenSSL, and I hope it can be helpful.