Prepare Kubernetes resources
For a fully functional Pulsar cluster, you need a few resources before deploying the Apache Pulsar Helm chart. The following provides instructions to prepare the Kubernetes cluster before deploying the Pulsar Helm chart.
Google Kubernetes Engine
To get started easier, a script is provided to create the cluster automatically. Alternatively, a cluster can be created manually as well.
Manual cluster creation
To provision a Kubernetes cluster manually, follow the GKE instructions.
Alternatively, you can use the instructions below to provision a GKE cluster as needed.
Scripted cluster creation
A bootstrap script has been created to automate much of the setup process for users on GCP/GKE.
The script can:
- Create a new GKE cluster.
- Allow the cluster to modify DNS (Domain Name Server) records.
- Setup
kubectl, and connect it to the cluster.
Google Cloud SDK is a dependency of this script, so ensure it is set up correctly for the script to work.
The script reads various parameters from environment variables and an argument up or down for bootstrap and clean-up respectively.
The following table describes all variables.
| Variable | Description | Default value |
|---|---|---|
| PROJECT | ID of your GCP project | No default value. It requires to be set. |
| CLUSTER_NAME | Name of the GKE cluster | pulsar-dev |
| CONFDIR | Configuration directory to store Kubernetes configuration | ${HOME}/.config/streamnative |
| INT_NETWORK | IP space to use within this cluster | default |
| LOCAL_SSD_COUNT | Number of local SSD counts | 4 |
| MACHINE_TYPE | Type of machine to use for nodes | n1-standard-4 |
| NUM_NODES | Number of nodes to be created in each of the cluster's zones | 4 |
| PREEMPTIBLE | Create nodes using preemptible VM instances in the new cluster. | false |
| REGION | Compute region for the cluster | us-east1 |
| USE_LOCAL_SSD | Flag to create a cluster with local SSDs | false |
| ZONE | Compute zone for the cluster | us-east1-b |
| ZONE_EXTENSION | The extension (a, b, c) of the zone name of the cluster | b |
| EXTRA_CREATE_ARGS | Extra arguments passed to create command |
Run the script, by passing in your desired parameters. It can work with the default parameters except for PROJECT which is required:
PROJECT=<gcloud project id> scripts/pulsar/gke_bootstrap_script.sh up
The script can also be used to clean up the created GKE resources.
PROJECT=<gcloud project id> scripts/pulsar/gke_bootstrap_script.sh down
Create cluster with local SSDs
To install the Pulsar Helm chart using local persistent volumes, you need to create a GKE cluster with local SSDs. You can do so by specifying USE_LOCAL_SSD to be true in the following command to create a Pulsar cluster with local SSDs.
PROJECT=<gcloud project id> USE_LOCAL_SSD=true LOCAL_SSD_COUNT=<local-ssd-count> scripts/pulsar/gke_bootstrap_script.sh up
Next Steps
Continue with the installation of the chart once you have the cluster up and running.