Deploying Apache Pulsar

Docker

Apache Pulsar is a cloud-native distributed messaging and streaming platform originally created by Yahoo!. It is now a top-level project of the Apache Software Foundation.

pulsar-manager is a web tool for managing Pulsar.

https://github.com/apache/pulsar-manager

Docker Deployment of Pulsar and Pulsar-manager

docker pull apachepulsar/pulsar:latest
docker run -d -it \
    -p 6650:6650 \
    -p 8080:8080 \
    --name pulsar-standalone \
    apachepulsar/pulsar:latest \
    bin/pulsar standalone

docker pull apachepulsar/pulsar-manager:v0.2.0
docker run -it \
    -p 9527:9527 -p 7750:7750 \
    -e SPRING_CONFIGURATION_FILE=/pulsar-manager/pulsar-manager/application.properties \
    --link pulsar-standalone \
    apachepulsar/pulsar-manager:v0.2.0

If you want to persist data:

docker run -d -it \
    -p 6650:6650 \
    -p 8080:8080 \
    -v pulsardata:/pulsar/data \
    -v pulsarconf:/pulsar/conf \
    --name pulsar-standalone \
    apachepulsar/pulsar:latest \
    bin/pulsar standalone

If you encounter errors after starting, try removing the mapping of /pulsar/conf.

Then open port 9527 to access pulsar-manager.

However, there is no default username and password, so you will need to operate inside the container.

1. Set the domestic mirror source

sed -i 's/dl-cdn.alpinelinux.org/mirrors.ustc.edu.cn/g' /etc/apk/repositories

2. Install curl

apk add curl

Execute the following command to create an administrator user.

CSRF_TOKEN=$(curl http://localhost:7750/pulsar-manager/csrf-token)

CSRF_TOKEN=$(curl http://localhost:7750/pulsar-manager/csrf-token)
curl \
   -H 'X-XSRF-TOKEN: $CSRF_TOKEN' \
   -H 'Cookie: XSRF-TOKEN=$CSRF_TOKEN;' \
   -H "Content-Type: application/json" \
   -X PUT http://localhost:7750/pulsar-manager/users/superuser \
   -d '{"name": "pulsar", "password": "pulsar123456", "description": "test", "email": "username@test.org"}'

If the password is too short, it may cause issues.
You don’t have to enter the container, you can also operate from the outside, for example:

CSRF_TOKEN=$(curl http://10.43.30.135:7750/pulsar-manager/csrf-token)
curl \
    -H "X-XSRF-TOKEN: $CSRF_TOKEN" \
    -H "Cookie: XSRF-TOKEN=$CSRF_TOKEN;" \
    -H 'Content-Type: application/json' \
    -X PUT http://10.43.30.135:7750/pulsar-manager/users/superuser \
    -d '{"name": "pulsar", "password": "pulsar123456", "description": "whuanle", "email": "yzf@whuanle.cn"}'

Because the two containers are linked together using docker link, you can access Pulsar from the pulsar-manager container, but you cannot access it directly via 127.0.0.1.

/pulsar-manager # cat /etc/hosts
127.0.0.1	localhost
::1	localhost ip6-localhost ip6-loopback
fe00::0	ip6-localnet
ff00::0	ip6-mcastprefix
ff02::1	ip6-allnodes
ff02::2	ip6-allrouters
172.17.0.2	pulsar-standalone d11df558ff42
172.17.0.3	1f9c6cf31c8b

You can see that the IP of the other container is 172.17.0.2, while the IP of the pulsar-manager is 172.17.0.3.

You can access Pulsar using either 172.17.0.2 or pulsar-standalone.

Configure the connection on the pulsar-manager page:

Then clicking in you can see:

Kubernetes

A kubernetes deployment template without helm for quick startup:

kind: Deployment
apiVersion: apps/v1
metadata:
  name: pulsar
  namespace: pulsar
  labels:
    app: pulsar
spec:
  replicas: 1
  selector:
    matchLabels:
      app: pulsar
  template:
    metadata:
      labels:
        app: pulsar
    spec:
      volumes:
        - name: pulsardata
          hostPath:
            path: /pulsar/data
            type: ''
      containers:
        - name: pulsar
          image: 'apachepulsar/pulsar:2.10.0'
          command:
            - bin/pulsar
            - standalone
          ports:
            - name: tcp-6650
              containerPort: 6650
              protocol: TCP
            - name: tcp-8080
              containerPort: 8080
              protocol: TCP
          resources: {}
          volumeMounts:
            - name: pulsardata
              mountPath: /pulsar/data
          imagePullPolicy: IfNotPresent
        - name: pulsar-manager
          image: 'apachepulsar/pulsar-manager:v0.2.0'
          ports:
            - name: tcp-9527
              containerPort: 9527
              protocol: TCP
            - name: tcp-7750
              containerPort: 7750
              protocol: TCP
          env:
            - name: SPRING_CONFIGURATION_FILE
              value: /pulsar-manager/pulsar-manager/application.properties
          resources: {}
          imagePullPolicy: IfNotPresent
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
      schedulerName: default-scheduler
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 25%
      maxSurge: 25%
  revisionHistoryLimit: 10
  progressDeadlineSeconds: 600

The above uses local storage. If you need to use volume mapping:

      volumes:
        - name: pulsar
          persistentVolumeClaim:
            claimName: pulsar

kind: Service
apiVersion: v1
metadata:
  name: pulsar-svc
  namespace: pulsar
  labels:
    app: pulsar-svc
spec:
  ports:
    - name: pulsar-tcp-6650
      protocol: TCP
      port: 6650
      targetPort: 6650
      nodePort: 36650
    - name: pulsar-tcp-8080
      protocol: TCP
      port: 8080
      targetPort: 8080
      nodePort: 38080
    - name: pulsar-mana-tcp-9527
      protocol: TCP
      port: 9527
      targetPort: 9527
      nodePort: 39527
    - name: pulsar-mana-tcp-7750
      protocol: TCP
      port: 7750
      targetPort: 7750
      nodePort: 37750
  selector:
    app: pulsar
  type: NodePort