This articles introduces how Contrail services use ephemeral nodes in Zookeeper. 

When device-manager, svc-monitor and schema connect to Zookeeper, they create an ephemeral node for itself to represent these services. One important property of ephemeral node is that it is removed automatically once the session is down.

Example:

At the beginning, we enter Zookeeper CLI and see one ephemeral znode created for svc-monitor:

# /usr/share/zookeeper/bin/zkCli.sh
Connecting to localhost:2181
<snipped>
[zk: localhost:2181(CONNECTED) 1] ls /svc-monitor
[833db70d0d674dc2b2c05d7e9016e2c8__lock__0000000008]
[zk: localhost:2181(CONNECTED) 2] get /svc-monitor/833db70d0d674dc2b2c05d7e9016e2c8__lock__0000000008
10515
cZxid = 0x59b
ctime = Tue Oct 29 13:45:59 PDT 2019
mZxid = 0x59b
mtime = Tue Oct 29 13:45:59 PDT 2019
pZxid = 0x59b
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x16e18e9152f0031
dataLength = 5
numChildren = 0

If we stop svc-monitor service, this node is removed automatically:

# service contrail-svc-monitor stop
contrail-svc-monitor: stopped

[zk: localhost:2181(CONNECTED) 3] ls /svc-monitor
[]
[zk: localhost:2181(CONNECTED) 4] get /svc-monitor/833db70d0d674dc2b2c05d7e9016e2c8__lock__0000000008
Node does not exist: /svc-monitor/833db70d0d674dc2b2c05d7e9016e2c8__lock__0000000008

This property is very useful when electing active svc-monitor/schema/device-manager because these services can only have one instance running as an active one. The others should stay as backup. To coordinate these instances, they need to register an ephemeral znode. After one of them acquires the lock, it becomes active. Once the active one fails for any reason, it will lose the connection to Zookeeper. Thus, Zookeeper knows the session has timed out and removes the ephemeral node. Other sessions which set watch on ephemeral znode will get notification and retry acquiring the lock. So ephemeral znode is very important for election and failover of specific Contrail services.  

For more on ZooKeeper refer to KB31144 - Contrail Getting Started - Administration, Configuration & Troubleshooting (JumpStation) [juniper.net]