Service Discovery

A discovery provider reads proxy definitions from an external system and adds them to the proxy list. When a definition changes, r3v3rs3 updates the routes without a restart.

This page is the reference. To build a working setup with Docker labels, follow Proxies from Docker.

Discovered Proxies

• A discovered proxy is read-only. The admin API returns proxy_read_only for an update or a delete. The WebUI shows the proxy with its source and without the edit actions. To change the proxy, change its definition in the source.
• r3v3rs3 does not save discovered proxies to proxies.toml. The provider sends them again after a restart.
• The id of a discovered proxy comes from the provider and the key of the definition. The proxy keeps its id after a restart, so the logs and the status keep the same id.
• A provider does not open ports. A definition names existing ports in ports, by port name or by port id. A port name must select exactly one port, and the port must accept the protocol of the proxy. When you rename or delete a port, r3v3rs3 resolves the port names again.
• A discovered TCP proxy cannot use a TCP port that another TCP proxy uses.
• When a provider loses its connection, the proxies of its last read stay active.

Provider Status

The proxy list shows the state of each provider, the number of proxies that it added and the definitions that did not become a proxy. The admin API returns the same data at GET /api/discovery.

When the deployment platform runs, the list also holds the provider platform, which publishes the proxies of the deployed apps.

An issue names the resource and the reason, for example web-1: http.app: port not found: https.

Labels

Docker labels, Consul tags and key-value entries use the same keys. Each key has this form:

r3v3rs3.<protocol>.<name>.<field>=<value>

• <protocol> is http, tcp or udp.
• <name> identifies the proxy in the resource. One resource can define more than one proxy.
• <field> is a field of the proxy model of the admin API. A dot separates nested fields.
• r3v3rs3 skips labels without the r3v3rs3. prefix. An unknown r3v3rs3. label or an unknown field is an issue, and the proxy is not added.

Proxy Keys

Values

• A boolean is true or false. A number is a decimal number.
• A duration uses units, for example 500ms, 30s or 5m.
• A list is a comma-separated value: vhosts=app.example.com,www.example.com.
• A list of groups uses the keys 0, 1, 2 and so on: routes.0.path=/, routes.1.path=/api. The numbers set the order.
• An empty value is no value.
• A comma-separated value is not available inside auth and header rules, because their fields are text. Use numbered keys there: auth.users.0.username=alice.

HTTP Proxies

port and scheme define one route to / with one server. They cannot be used together with routes.

r3v3rs3.http.app.ports=https
r3v3rs3.http.app.vhosts=app.example.com
r3v3rs3.http.app.port=8080

A route can also use port and scheme. They cannot be used together with servers or response of the same route.

r3v3rs3.http.app.ports=http,https
r3v3rs3.http.app.vhosts=app.example.com
r3v3rs3.http.app.routes.0.path=/
r3v3rs3.http.app.routes.0.port=8080
r3v3rs3.http.app.routes.1.path=/api
r3v3rs3.http.app.routes.1.port=9443
r3v3rs3.http.app.routes.1.scheme=https

A server can have an explicit URL and a weight:

r3v3rs3.http.app.routes.0.servers.0.url=http://10.0.0.5:8080
r3v3rs3.http.app.routes.0.servers.1.url=http://10.0.0.6:8080
r3v3rs3.http.app.routes.0.servers.1.weight=3
r3v3rs3.http.app.load_balancing=round_robin

The other fields use the names of the admin API:

r3v3rs3.http.app.upgrade_insecure=true
r3v3rs3.http.app.ip_filter.allow=10.0.0.0/8,192.168.0.0/16
r3v3rs3.http.app.rate_limit.requests=100
r3v3rs3.http.app.rate_limit.per=minute
r3v3rs3.http.app.rate_limit.burst=20
r3v3rs3.http.app.timeouts.connect=3s
r3v3rs3.http.app.timeouts.request=30s
r3v3rs3.http.app.health_check.interval=10s
r3v3rs3.http.app.health_check.path=/healthz
r3v3rs3.http.app.compression.algorithms=br,gzip
r3v3rs3.http.app.cache.enabled=true
r3v3rs3.http.app.cache.default_ttl=1m
r3v3rs3.http.app.headers.response.0.action=set
r3v3rs3.http.app.headers.response.0.name=X-Frame-Options
r3v3rs3.http.app.headers.response.0.value=DENY
r3v3rs3.http.app.auth.type=basic
r3v3rs3.http.app.auth.realm=Staff
r3v3rs3.http.app.auth.users.0.username=alice
r3v3rs3.http.app.auth.users.0.password=change-me

r3v3rs3 replaces plain text passwords and tokens with hashes before it uses the proxy. The source still holds the plain text value, so prefer password_hash and token_hash in labels.

TCP and UDP Proxies

port defines one upstream server on the address of the resource. It cannot be used together with upstream_servers.

r3v3rs3.tcp.db.ports=postgres
r3v3rs3.tcp.db.port=5432

r3v3rs3.udp.dns.ports=dns
r3v3rs3.udp.dns.upstream_servers.0.addr=/ip4/10.0.0.53/udp/53
r3v3rs3.udp.dns.session_idle_timeout=30s

ACME Certificates

acme names an existing ACME entry of the certificate list. r3v3rs3 orders a certificate for the virtual hosts of the proxy with the account, the challenge and the DNS provider of the entry.

r3v3rs3.http.app.ports=https
r3v3rs3.http.app.vhosts=app.example.com,www.example.com
r3v3rs3.http.app.port=8080
r3v3rs3.http.app.acme=abc-def

• The certificate has the virtual hosts as its domain names. It is a separate order of the entry, so it renews on its own schedule after the renewal_days of the entry. The certificates of the entry keep their schedule.
• r3v3rs3 does not order a certificate while a valid server certificate with a private key has every virtual host, for example a wildcard certificate of the entry. After the first certificate of the proxy, the renewal follows the proxy certificate.
• The proxies with the same entry and the same virtual hosts share one certificate.
• A missing or inactive entry, a proxy without virtual hosts, a regular expression virtual host and a name that the challenge of the entry cannot validate are issues, for example a wildcard virtual host with http-01. The proxy is added without a certificate.
• An inactive proxy does not order a certificate. A TCP or UDP proxy with acme is an issue.
• After a failed order, r3v3rs3 waits one hour before the next order.
• When the label is removed, the certificate is not renewed. r3v3rs3 removes it from the certificate list after it expires.
• An Ingress uses the r3v3rs3.io/acme annotation, and an R3v3rs3Proxy uses spec.acme.

Docker

The Docker provider reads the labels of the running containers through the Docker Engine API. It follows the container events, so a started, stopped or changed container updates the proxies within about one second.

Settings

Fill in "Docker Service Discovery" in "Settings", or edit config.toml:

[discovery.docker]
enabled = true
endpoint = "unix:///var/run/docker.sock"
network = "proxy"
exposed_by_default = false

A change of the settings restarts the provider without a server restart. The proxies and certificates of the provider are removed at once and return with its first read. Turning the provider off removes them. A certificate that the provider uses cannot be deleted.

Containers

• The upstream address of a container is its IP address on the selected network. A container with the host network mode uses 127.0.0.1.
• A container whose health check fails is skipped and shown as an issue. A container whose health check has not passed yet is skipped.
• The replicas of a Compose service share their proxies. Each replica adds its servers to the routes of the proxy, so the proxy balances the load across the replicas. The replicas must define the same number of routes.
• The key of a proxy is the Compose project, the Compose service and the proxy name. A container outside Compose uses its container name.
• r3v3rs3 reads the container list again every five minutes without events.

Compose Example

r3v3rs3 runs on the host and reads /var/run/docker.sock. The Compose file holds the labeled containers:

services:
  whoami:
    image: traefik/whoami
    networks: [proxy]
    labels:
      r3v3rs3.enable: "true"
      r3v3rs3.http.whoami.ports: http
      r3v3rs3.http.whoami.vhosts: whoami.example.com
      r3v3rs3.http.whoami.port: "80"

networks:
  proxy:
    name: proxy

Add a port with the name http in "Ports", enable the provider with the network proxy, and start the stack. Compose adds the project name to a network name unless the network sets name.

etcd

The etcd provider reads the keys under a prefix through the etcd v3 HTTP API. It follows the changes with a watch stream, so a changed key updates the proxies within about one second.

Settings

Fill in "etcd Service Discovery" in "Settings", or edit config.toml:

[discovery.etcd]
enabled = true
endpoints = ["http://10.0.0.1:2379", "http://10.0.0.2:2379"]
username = "r3v3rs3"
password = "<password>"
prefix = "r3v3rs3"

endpoints needs at least one address, and prefix cannot be empty.

The admin API does not return the password. GET /api/config returns password_set: true when a password is saved. A PUT /api/config without password keeps the saved password, and "password": "" removes it. config.toml holds the password as plain text, so allow only the r3v3rs3 user to read the config directory.

With authentication, r3v3rs3 requests a token with the user name and the password. When etcd rejects an expired token, r3v3rs3 requests a new token and sends the request again. The user needs a role with the read permission on the prefix:

etcdctl role add r3v3rs3-reader
etcdctl role grant-permission r3v3rs3-reader --prefix=true read r3v3rs3/
etcdctl user add r3v3rs3
etcdctl user grant-role r3v3rs3 r3v3rs3-reader

Keys

• The keys use the layout of the Consul key-value store. With the prefix r3v3rs3, the key r3v3rs3/http/app/ports is the label r3v3rs3.http.app.ports.
• r3v3rs3 reads every proxy under the prefix, so the keys do not need r3v3rs3.enable.
• A key has no address, so port is not available. Use routes.<n>.servers.<n>.url or upstream_servers.<n>.addr.
• A part of a key cannot contain a .. Such a key is an issue.
• When etcd compacts the revision that the watch stream needs, r3v3rs3 reads every key again.

etcdctl put r3v3rs3/http/app/ports https
etcdctl put r3v3rs3/http/app/vhosts app.example.com
etcdctl put r3v3rs3/http/app/routes/0/servers/0/url http://10.0.0.5:8080