Skip to main content

Prepare Kubernetes resources

For a fully functional Pulsar cluster, you need a few resources before deploying the Apache Pulsar Helm chart. The following provides instructions to prepare the Kubernetes cluster before deploying the Pulsar Helm chart.

Google Kubernetes Engine

To get started easier, a script is provided to create the cluster automatically. Alternatively, a cluster can be created manually as well.

Manual cluster creation

To provision a Kubernetes cluster manually, follow the GKE instructions.

Scripted cluster creation

A bootstrap script has been created to automate much of the setup process for users on GCP/GKE.

The script can:

  1. Create a new GKE cluster.
  2. Allow the cluster to modify DNS (Domain Name Server) records.
  3. Setup kubectl, and connect it to the cluster.

Google Cloud SDK is a dependency of this script, so ensure it is set up correctly for the script to work.

The script reads various parameters from environment variables and an argument up or down for bootstrap and clean-up respectively.

The following table describes all variables.

VariableDescriptionDefault value
PROJECTID of your GCP projectNo default value. It requires to be set.
CLUSTER_NAMEName of the GKE clusterpulsar-dev
CONFDIRConfiguration directory to store Kubernetes configuration${HOME}/.config/streamnative
INT_NETWORKIP space to use within this clusterdefault
LOCAL_SSD_COUNTNumber of local SSD counts4
MACHINE_TYPEType of machine to use for nodesn1-standard-4
NUM_NODESNumber of nodes to be created in each of the cluster's zones4
PREEMPTIBLECreate nodes using preemptible VM instances in the new cluster.false
REGIONCompute region for the clusterus-east1
USE_LOCAL_SSDFlag to create a cluster with local SSDsfalse
ZONECompute zone for the clusterus-east1-b
ZONE_EXTENSIONThe extension (a, b, c) of the zone name of the clusterb
EXTRA_CREATE_ARGSExtra arguments passed to create command

Run the script, by passing in your desired parameters. It can work with the default parameters except for PROJECT which is required:

PROJECT=<gcloud project id> scripts/pulsar/ up

The script can also be used to clean up the created GKE resources.

PROJECT=<gcloud project id> scripts/pulsar/ down

Create cluster with local SSDs

To install the Pulsar Helm chart using local persistent volumes, you need to create a GKE cluster with local SSDs. You can do so by specifying USE_LOCAL_SSD to be true in the following command to create a Pulsar cluster with local SSDs.

PROJECT=<gcloud project id> USE_LOCAL_SSD=true LOCAL_SSD_COUNT=<local-ssd-count> scripts/pulsar/ up