Encryption and Authentication using TLS


TLS Overview

With TLS authentication, the server authenticates the client (also called “2-way authentication”). Since TLS authentication requires TLS encryption, this page shows you how to configure both at the same time.

By default, Apache Pulsar communicates in plain text service url, which means that all data is sent in the clear. To encrypt communication, it is recommended to configure all the Apache Pulsar components in your deployment to use TLS encryption.

TLS can be configured for encryption or authentication. You may configure just TLS encryption (by default TLS encryption includes certificate authentication of the server) and independently choose a separate mechanism for client authentication, e.g. TLS, Athenz, etc. Note that TLS encryption, technically speaking, already enables 1-way authentication in which the client authenticates the server certificate. So when referring to TLS authentication, it is really referring to 2-way authentication in which the broker also authenticates the client certificate.

Note that enabling TLS may have a performance impact due to encryption overhead.

Creating TLS Certificates

Creating TLS certificates for Pulsar involves creating a certificate authority (CA), broker certificate, and client certificate.

Certificate authority

The first step is to create the certificate for the CA. The CA will be used to sign both the broker and client certificates, in order to ensure that each party will trust the others.

Linux

$ CA.pl -newca

macOS

$ /System/Library/OpenSSL/misc/CA.pl -newca

After answering the question prompts, this will store CA-related files in the ./demoCA directory. Within that directory:

  • demoCA/cacert.pem is the public certificate. It is meant to be distributed to all parties involved.
  • demoCA/private/cakey.pem is the private key. This is only needed when signing a new certificate for either broker or clients and it must be safely guarded.

Broker certificate

Once a CA certificate has been created, you can create certificate requests and sign them with the CA.

The following commands will ask you a few questions and then create the certificates. When asked for the common name, you need to match the hostname of the broker. You could also use a wildcard to match a group of broker hostnames, for example *.broker.usw.example.com. This ensures that the same certificate can be reused on multiple machines.

$ openssl req \
  -newkey rsa:2048 \
  -sha256 \
  -nodes \
  -out broker-cert.csr \
  -outform PEM

Convert the key to PKCS 8 format:

$ openssl pkcs8 \
  -topk8 \
  -inform PEM \
  -outform PEM \
  -in privkey.pem \
  -out broker-key.pem \
  -nocrypt

This will create two broker certificate files named broker-cert.csr and broker-key.pem. Now you can create the signed certificate:

$ openssl ca \
  -out broker-cert.pem \
  -infiles broker-cert.csr

At this point, you should have a broker-cert.pem and broker-key.pem file. These will be needed for the broker.

Client certificate

To create a client certificate, repeat the steps in the previous section, but did create client-cert.pem and client-key.pem files instead.

For the client common name, you need to use a string that you intend to use as the role token for this client, though it doesn’t need to match the client hostname.

Configure the broker for TLS

To configure a Pulsar broker to use TLS authentication, you’ll need to make some changes to the broker.conf configuration file, which is located in the conf directory of your Pulsar installation.

Add these values to the configuration file (substituting the appropriate certificate paths where necessary):

# Enable TLS and point the broker to the right certs
tlsEnabled=true
tlsCertificateFilePath=/path/to/broker-cert.pem
tlsKeyFilePath=/path/to/broker-key.pem
tlsTrustCertsFilePath=/path/to/cacert.pem

# Enable the TLS auth provider
authenticationEnabled=true
authorizationEnabled=true
authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderTls

A full listing of parameters available in the conf/broker.conf file, as well as the default values for those parameters, can be found in Broker Configuration.

Configure the discovery service

The discovery service used by Pulsar brokers needs to redirect all HTTPS requests, which means that it needs to be trusted by the client as well. Add this configuration in conf/discovery.conf in your Pulsar installation:

tlsEnabled=true
tlsCertificateFilePath=/path/to/broker-cert.pem
tlsKeyFilePath=/path/to/broker-key.pem

Configure clients

For more information on Pulsar client authentication using TLS, see the following language-specific docs:

Configure CLI tools

Command-line tools like pulsar-admin, pulsar-perf, and pulsar-client use the conf/client.conf config file in a Pulsar installation.

You’ll need to add the following authentication parameters to that file to use TLS with Pulsar’s CLI tools:

serviceUrl=https://broker.example.com:8443/
authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationTls
authParams=tlsCertFile:/path/to/client-cert.pem,tlsKeyFile:/path/to/client-key.pem
useTls=true
tlsAllowInsecureConnection=false
tlsTrustCertsFilePath=/path/to/cacert.pem