• Published:

After covering what etcd is, how to use it from the command line, and some of the basic ideas behind it, it makes sense to move on to the more practical part: deploying a cluster and talking to it from Go.

In real use, etcd nodes are typically deployed in a 2N+1 layout, so cluster setup is not something you can skip over. The good news is that for a basic working cluster, the configuration is much simpler than many tutorials make it seem.

A minimal etcd cluster

There are plenty of cluster deployment guides around, and some of them look much more complicated than what you actually need for day-to-day usage. A straightforward static cluster is enough in many cases.

Downloading etcd

You can grab a release from:

https://github.com/etcd-io/etcd/releases

Pick the version you need and download it however you prefer. The example here uses:

etcd-v3.3.13-linux-amd64.tar.gz

You can fetch it directly onto a Linux machine with wget, or download it locally first and upload it afterward.

Deploying three nodes

For this example, the cluster runs on three machines with these IP addresses:

  • 192.168.4.224
  • 192.168.4.225
  • 192.168.4.226

Before starting, make sure the required firewall ports are open.

Unpack the downloaded archive, enter the extracted directory, and start etcd on each machine with the matching command below. These are three separate commands for three different hosts, so you need to replace the IPs with your own environment if necessary.

$ ./etcd --name infra0 --initial-advertise-peer-urls http://192.168.4.224:2380 \
  --listen-peer-urls http://192.168.4.224:2380 \
  --listen-client-urls http://192.168.4.224:2379,http://127.0.0.1:2379 \
  --advertise-client-urls http://192.168.4.224:2379 \
  --initial-cluster-token etcd-cluster-1 \
  --initial-cluster infra0=http://192.168.4.224:2380,infra1=http://192.168.4.225:2380,infra2=http://192.168.4.226:2380 \
  --initial-cluster-state new

$ ./etcd --name infra1 --initial-advertise-peer-urls http://192.168.4.225:2380 \
  --listen-peer-urls http://192.168.4.225:2380 \
  --listen-client-urls http://192.168.4.225:2379,http://127.0.0.1:2379 \
  --advertise-client-urls http://192.168.4.225:2379 \
  --initial-cluster-token etcd-cluster-1 \
  --initial-cluster infra0=http://192.168.4.224:2380,infra1=http://192.168.4.225:2380,infra2=http://192.168.4.226:2380 \
  --initial-cluster-state new

$ ./etcd --name infra2 --initial-advertise-peer-urls http://192.168.4.226:2380 \
  --listen-peer-urls http://192.168.4.226:2380 \
  --listen-client-urls http://192.168.4.226:2379,http://127.0.0.1:2379 \
  --advertise-client-urls http://192.168.4.226:2379 \
  --initial-cluster-token etcd-cluster-1 \
  --initial-cluster infra0=http://192.168.4.224:2380,infra1=http://192.168.4.225:2380,infra2=http://192.168.4.226:2380 \
  --initial-cluster-state new

That is enough to bring up a three-node cluster. It really can be this direct.

Using a config file instead of a long command

If you do not want to type a long startup command every time, write the same settings into a configuration file.

Example:

# 当前节点名称
name: infra1
# etcd数据保存目录
data-dir: /usr/local/etcd
# 供外部客户端使用的url
listen-client-urls: http://192.168.4.225:2379,http://127.0.0.1:2379
# 广播给外部客户端使用的url
advertise-client-urls: http://192.168.4.225:2379
# 集群内部通信使用的URL
listen-peer-urls: http://192.168.4.225:2380
# 广播给集群内其他成员访问的URL
initial-advertise-peer-urls: http://192.168.4.225:2380
# 集群的名称
initial-cluster-token: etcd-cluster-1
# 初始集群成员列表
initial-cluster: infra0=http://192.168.4.224:2380,infra1=http://192.168.4.225:2380,infra2=http://192.168.4.226:2380
#初始集群状态
initial-cluster-state: new

Start etcd by pointing it to the file:

./etcd --config-file=conf.yml

What this simple setup leaves out

This static three-node deployment is about as simple as it gets. There are even easier installation paths, such as using yum to install etcd directly, but startup strategy is only one part of deployment.

This basic setup also comes with some obvious limitations:

  • There is no authentication enabled, which means anyone who knows the IP and port can connect to the cluster. In practice, this kind of setup is much more suitable for an internal network unless you add users and access control.
  • Communication is not encrypted.
  • The cluster is configured with static IP addresses. That is still one of the most common real-world approaches, but etcd also supports discovery-based deployment for more flexible cluster configuration.

These concerns matter much more in production environments. For advanced online deployment strategies, the official documentation goes into much greater detail.

https://doczhcn.gitbook.io/etcd/index/index-1/clustering

Working with etcd from Go

Once the cluster is up, the next step is using it from code. A basic Go client can cover the same core operations you would normally try from the command line: get, put, del, watch, and lease.

For additional APIs, check the client documentation:

https://godoc.org/github.com/coreos/etcd/clientv3

Getting the client code

Using go get is not always the fastest option. Downloading the repository directly can be quicker. The source can be obtained from:

https://github.com/etcd-io/etcd

After downloading and extracting it, place it under the corresponding path in GOPATH:

go/src/go.etcd.io/etcd

Example Go program

The following program connects to the three endpoints, starts a watcher on key aaa, performs a put, reads the key back, deletes it, creates a lease, writes the key again attached to that lease, and finally waits for the lease to expire automatically.

package main
import (
    "context"
    "fmt"
    "go.etcd.io/etcd/clientv3"
    "go.etcd.io/etcd/mvcc/mvccpb"
    "time"
)
func main() {
    // 配置客户端连接
    client, err := clientv3.New(clientv3.Config{
        // Endpoints: []string{"127.0.0.1:2379"},
        Endpoints: []string{"192.168.4.224:2379", "192.168.4.225:2379", "192.168.4.226:2379"},
        DialTimeout: 5 * time.Second,
    })
    if err != nil {
        panic(err)
    }
    defer client.Close()
    // 启动watch监听
    watch := client.Watch(context.TODO(), "aaa")
    go func() {
        for {
            watchResponse := <- watch
            for _, ev := range watchResponse.Events {
                switch ev.Type {
                case mvccpb.DELETE:
                    fmt.Printf("监听到del：%s\n", ev.Kv.Key)
                case mvccpb.PUT:
                    fmt.Printf("监听到put：%s, %s\n", ev.Kv.Key, ev.Kv.Value)
                }
            }
        }
    }()
    // 新增
    putResponse, err := client.Put(context.TODO(), "aaa", "xxx")
    if err != nil {
        fmt.Println(err)
        return
    }
    fmt.Println(putResponse.Header.String())
    // 查询
    getResponse, err := client.Get(context.TODO(), "aaa")
    if err != nil {
        fmt.Println(err)
        return
    }
    fmt.Println(getResponse.Kvs)
    // 删除
    deleteResponse, err := client.Delete(context.TODO(), "aaa")
    if err != nil {
        fmt.Println(err)
        return
    }
    fmt.Println(deleteResponse.Header.String())
    // 申请租约
    grantResponse, err := client.Grant(context.TODO(), 10)
    if err != nil {
        fmt.Println(err)
        return
    }
    // 使用租约
    response, err := client.Put(context.TODO(), "aaa", "xxx", clientv3.WithLease(grantResponse.ID))
    if err != nil {
        fmt.Println(err)
        return
    }
    fmt.Println(response.Header.String())
    // 等待租约自动过期
    time.Sleep(time.Second * 20)
}

A typical output would look roughly like this:

监听到put：aaa, xxx
cluster_id:14841639068965178418 member_id:10276657743932975437 revision:53 raft_term:4
[key:”aaa” create_revision:53 mod_revision:53 version:1 value:”xxx” ]
监听到del：aaa
cluster_id:14841639068965178418 member_id:10276657743932975437 revision:54 raft_term:4
监听到put：aaa, xxx
cluster_id:14841639068965178418 member_id:10276657743932975437 revision:55 raft_term:4
监听到del：aaa

There is not much mystery in the day-to-day usage. Once the cluster is reachable and the client is configured with the right endpoints, the basic etcd operations are very direct.

Related Posts