-
Notifications
You must be signed in to change notification settings - Fork 7
Install As Daemon
Download releases from Releases and unzip the archive, you will have main executable
user:~/barrel$ ls
eru-barrel
README.md
If you have minions installed before, you should check Environment first. Otherwise skip this stage. Find out on which network your containers are running, 'calico-pool' is the network on which containers are running by the following example
user:~/barrel$ docker network ls | grep calico
923afd0245df76 calico-pool calico global
Use the following script to check calico namespace, run with './check_namespace.py ${calico_network}'
user:~/barrel$ ./check_namespace.py calico-pool
created container 2b2af5552e0effa3d3b55877ced0930254f2c77c910a231568b1228989148d32
container address = 10.11.120.2
namespace = cloud-test-10-22-11-8
We will use the namespace later, so do remember it
#!/usr/bin/env python3
import sys
import subprocess
from string import Template
import json
import re
from typing import List, Tuple, Any
white_spaces_regex = re.compile("\s+")
def run(args: List[str]) -> Tuple[Any, Any]:
process = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
return process.communicate()
def create_container() -> str:
stdout, stderr = run(['docker', 'container', 'create', '--network', sys.argv[1], 'nginx'])
if stdout == b'':
raise Exception('create container error')
return stdout[:len(stdout) - 1].decode()
def start_container(id: str):
stdout, stderr = run(['docker', 'container', 'start', id])
if stdout == b'':
raise Exception(
Template('start container $id failed, cause = $cause')
.substitute({'id': id, 'cause': stderr[:len(stderr) - 1].decode()})
)
def inspect_container_addr(id: str) -> str:
stdout, stderr = run(['docker', 'container', 'inspect', id])
if stdout == b'':
raise Exception(
Template('inspect container $id failed, cause = $cause')
.substitute({'id': id, 'cause': stderr[:len(stderr) - 1].decode()})
)
result = stdout[:len(stdout) - 1].decode()
container_infos = json.loads(result)
if len(container_infos) != 1:
raise Exception(
Template('inspect container $id failed, cause = $cause')
.substitute({'id': id, 'cause': 'less then or more then one container is inspected'})
)
return container_infos[0]['NetworkSettings']['Networks'][sys.argv[1]]['IPAddress']
def look_up_namespace(ip: str) -> str:
stdout, stderr = run(['calicoctl', 'get', '--all-namespaces', 'weps'])
if stdout == b'':
raise Exception(
Template('look up namespace by ip $ip failed, cause = $cause')
.substitute({'ip': ip, 'cause': stderr[:len(stderr) - 1].decode()})
)
table = stdout[:len(stdout) - 1].decode().split('\n')
if len(table) <= 1:
return ""
for line in table:
values = white_spaces_regex.split(line)
if len(values) >= 3 and values[3].startswith(ip):
return values[0]
return ""
def remove_container(id: str):
stdout, stderr = run(['docker', 'container', 'rm', id, '--force'])
if stdout == b'':
raise Exception(
Template('remove container $id failed, cause = $cause')
.substitute({'id': id, 'cause': stderr[:len(stderr) - 1].decode()})
)
def main():
containerID = create_container()
print("created container " + containerID)
start_container(containerID)
address = inspect_container_addr(containerID)
print("container address = " + address)
print("namespace = " + look_up_namespace(address))
remove_container(containerID)
if __name__ == '__main__':
sys.exit(main())Create barrel.conf, this is the enviroment variable barrel will use
You may edit it according to Config
# part of barrel.conf
# the name of your calico node, check it by calicoctl get no, if it's the same with your hostname, you may leave it blank
HOSTNAME=calico0
# your etcd endpoints on which calico meta is saved
ETCD_ENDPOINTS=http://127.0.0.1:2379
# docker daemon unix socket path, by default it is the value below
BARREL_DOCKERD_PATH=unix:///var/run/docker.sock
# barrel host, working as docker endpoints, you may use schema starts with unix/http/https
BARREL_HOSTS=unix:///var/run/barrel.sock,http://0.0.0.0:2377
If you are trying to use https to encrypt your connection, you should also filling the ca cert and key files
# part of barrel.conf
BARREL_TLS_CERT_FILE_PATH=/etc/docker/tls/server.crt
BARREL_TLS_KEY_FILE_PATH=/etc/docker/tls/server.key
If the calico namespace you got a few moment before is not the value of your host name, add it to barrel.conf
# part of barrel.conf
CALICO_LIBNETWORK_NAMESPACE=${namespace}
Create eru-barrel.service, This is the systemd config of barrel
You may edit it according to Config
[Unit]
Description=Eru Docker Proxy
After=network.target
After=network-online.target
Wants=network-online.target
Before=docker.service
[Service]
Type=simple
# set GOMAXPROCS to number of processors
EnvironmentFile=/etc/eru/barrel.conf
ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/bin/eru-barrel"
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
Find out which container is the calico-node on the current machine
user:~/barrel$ docker ps | grep calico
3da67adi8bfd3f calico/node:release-v3.10 "start_runit" 1 months ago Up 1 hours
Check whether the calico-node is running with libnetwork
user:~/barrel$ docker container inspect ${calico_container_id} | grep CALICO_LIBNETWORK_ENABLED
"CALICO_LIBNETWORK_ENABLED=false",
If CALICO_LIBNETWORK_ENABLED is true, disable it
user:~/barrel$ calicoctl node run --node-image="${calico_image}" --disable-docker-networking
If CALICO_LIBNETWORK_ENABLED is false, check whether you have minions installed, otherwise UNINSTALL it
Now we have the following files
user:~/barrel$ ls
eru-barrel
README.md
barrel.conf
eru-barrel.service
The following script is install.sh, execute it, and barrel will be running
You may manually execute these commands
#!/bin/bash
if [ "$EUID" -ne 0 ]
then echo "Please run as root"
exit
fi
declare -a FILES=(
"barrel.conf" "/etc/eru/"
"eru-barrel" "/usr/bin/"
"eru-barrel.service" "/usr/lib/systemd/system/"
)
copy_file () {
echo "copy $1 to $2"
cp "$1" "$2"
}
if [ -d "/etc/eru" ];
then
echo "/etc/eru exists"
else
echo "create /etc/eru"
mkdir -p /etc/eru
fi
echo "===copy files==="
for i in $(eval echo "{0..$((${#FILES[@]} - 1))..2}")
do
copy_file "${FILES[${i}]}" "${FILES[$((i + 1))]}"
done
echo "===start service==="
systemctl enable eru-barrel.service
systemctl start eru-barrel.service
echo "===barrel install success==="
You may create new pods and nodes to deploy fixed ip containers, how ever you can use the existed nodes to deploy also.
To deploy fixed ip container on existed nodes, you should modify these node, force them to connect barrel endpoints.
You may find the prefix of eru etcd metadata in '/etc/eru/core.yaml', this should be done at host machine of eru-core
user:~/barrel$ cat /etc/eru/core.yaml
# ...
etcd:
machines:
- "http://127.0.0.1:2379"
prefix: "/eru"
lock_prefix: "eru/_lock"
# ...
Get the node metadata by following method, you should modify both of the values under different key mentioned below
etcdctl get /${eru_prefix}/node/${pod_name}:pod/${node_name}
etcdctl get /${eru_prefix}/node/${node_name}
You may search the full key by 'etcdctl get /${eru_prefix}/node --prefix'
user:~/barrel$ etcdctl get /eru/node/test:pod/test-node-10-10-10-10
{"name":"test-node-10-10-10-10","endpoint":"tcp://10.10.10.10:2376","podname":"spp", ...}
user:~/barrel$ etcdctl get /eru/node/test-node-10-10-10-10
{"name":"test-node-10-10-10-10","endpoint":"tcp://10.10.10.10:2376","podname":"spp", ...}
Update the metadata using barrel endpoint by
etcdctl put /${eru_prefix}/node/${pod_name}:pod/${node_name} ${new_node_meta_data}
etcdctl put /${eru_prefix}/node/${node_name} ${new_node_meta_data}
- It's risky so test it on dev environment before operating on production environment
user:~/barrel$ etcdctl put /eru/node/test:pod/test-node-10-10-10-10 '{"name":"test-node-10-10-10-10","endpoint":"tcp://10.10.10.10:2377","podname":"spp", ...}'
user:~/barrel$ etcdctl put /eru/node/test-node-10-10-10-10 '{"name":"test-node-10-10-10-10","endpoint":"tcp://10.10.10.10:2377","podname":"spp", ...}'
You MUST RESTART eru-core to make node endpoint modification to take effect
You can using 'docker -H unix:///var/run/barrel.sock' to manage containers via barrel proxy.
You can try an alias to make sure you are operating via barrel proxy.
user:~/barrel$ alias docker='docker -H unix:///var/run/barrel.sock'
Create a container with fixed-ip label, and check whether the ipam config is in network config
You MUST do it via barrel socket 'docker -H unix:///var/run/barrel.sock ...'
user:~/barrel$ docker -H unix:///var/run/barrel.sock container create --network ${your_calico_network_name} --label fixed-ip nginx
4271a67a8b79
user:~/barrel$ docker -H unix:///var/run/barrel.sock container inspect 4271a67a8b79
[
{
...
"Networks": {
"${your_calico_network_name}": {
IPAMConfig": {
"IPv4Address": "10.10.10.10"
},
...
}
]
You can also restart it to observe whether its address changes
After the validation, remember to remove the container VIA BARREL SOCKET
user:~/barrel$ docker -H unix:///var/run/barrel.sock container rm 4271a67a8b79 --force
You may try alias docker in case you forget to connect via barrel socket
user:~/barrel$ alias docker="docker -H unix:///var/run/barrel.sock"
After you have modified eru nodes metadata or created new eru nodes, you can validate fixed ip function on eru
The following script describes an http application which will only response with http 200 ok
user:~/barrel$ cat << EOF > ok-fixed.yaml
appname: echo
entrypoints:
nc8080:
cmd: '/bin/ash -c "while true; do { echo -ne \"HTTP/1.0 200 OK\r\nContent-Length: 0\r\n\r\n\"; } | nc -l -p 8080; done"'
publish:
- 8080
healthcheck:
http_port: 8080
code: 200
nc9080:
cmd: '/bin/ash -c "while true; do { echo -ne \"HTTP/1.0 200 OK\r\nContent-Length: 0\r\n\r\n\"; } | nc -l -p 9080; done"'
publish:
- 9080
healthcheck:
http_port: 9080
code: 200
labels:
fixed-ip: true
EOF
You can deploy it via eru-cli
user:~/barrel$ eru-cli container deploy --image busybox:latest --pod ${your_pod} --entry nc8080 --node ${the_node_you_want_to_varify} --network ${network_you_want_to_deploy_on} ok-fixed.yaml
Then you can check on whether the address of the container is fixed the same way as check on each machine
user:~/barrel$ docker -H unix:///var/run/barrel.sock container inspect 4271a67a8b79