Using headless services to better handle nsqlookupd

[regner]

Hey @antihax,

I was just looking through your kube configs and noticed you could clean up the nsq definitions a bit if you want to. Currently you duplicate the kube definitions for nsqlookupd so that you can have unique DNS entries to point at. A simpler solution is using a headless service in front of a stateful deployment.

Example stateful set:

apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  labels:
    app: nsq
    component: nsqlookupd
  name: nsqlookupd
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nsq
      component: nsqlookupd
  serviceName: nsqlookupd
  template:
    metadata:
      labels:
        app: nsq
        component: nsqlookupd
    spec:
      containers:
      - command:
        - /nsqlookupd
        image: nsqio/nsq:v1.0.0-compat
        imagePullPolicy: Always
        name: nsqlookupd
        ports:
        - containerPort: 4170
          name: tcp
        - containerPort: 4171
          name: http
      terminationGracePeriodSeconds: 5

Example service:

apiVersion: v1
kind: Service
metadata:
  labels:
    app: nsq
    component: nsqlookupd
  name: nsqlookupd
spec:
  clusterIP: None
  ports:
  - name: tcp
    port: 4160
    targetPort: 4160
  - name: http
    port: 4161
    targetPort: 4161
  selector:
    app: nsq
    component: nsqlookupd

Specifically by specifying clusterIP: None we now get what kube calls a "headless service".

This is useful because you can then reference each instance via a DNS entry:

Example nsq deployment:

apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  labels:
    app: nsq
    component: nsqd
  name: nsqd
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nsq
      component: nsqd
  serviceName: nsqd
  template:
    metadata:
      labels:
        app: nsq
        component: nsqd
    spec:
      containers:
      - command:
        - /nsqd
        - -max-msg-size=9388608
        - -mem-queue-size=1000
        - -data-path
        - /data
        - -lookupd-tcp-address
        - nsqlookupd-0.nsqlookupd:4160
        - -lookupd-tcp-address
        - nsqlookupd-1.nsqlookupd:4160
        - -lookupd-tcp-address
        - nsqlookupd-2.nsqlookupd:4160
        image: nsqio/nsq:v1.0.0-compat
        imagePullPolicy: Always
        name: nsqd
        ports:
        - containerPort: 4150
          name: tcp
        - containerPort: 4151
          name: http

The result is you don't have to repeat the nsqlookupd declarations, you can drop the broadcast IP command line attribute, and things are just a bit nicer.

I use basically the same config above for a few different applications.

Hope that helps! Sorry I don't have time to submit a PR.