The cluster's own DNS

Beta

Delegate a domain to a cluster once, and its servers answer its DNS themselves, naming only the healthy ones. No record to edit when a server stops.

Beta

This feature is currently beta/experimental and subject to change. It is not recommended for production. Try it out and share your feedback on our GitHub.

Experiment

An instance admin turns them on for every team in Settings -> Deplo -> Experiments. Until then, clusters stay hidden.

A domain that answers on every server needs DNS to name every healthy server. Kept by hand, that means editing records each time a server stops or comes back.

The cluster can answer that DNS itself. A zone is delegated to it once. From then on every server of the cluster answers for the zone, and names only the servers that are up. Nothing to edit when one stops, with Deplo up or down.

Your domainWhat you do
Connected to Deplo for Automatic DNSAdd it to the app and keep Let the cluster answer. Deplo makes the zone and writes every record, see A domain connected to Deplo
Anywhere elseDelegate a zone once and add one record per domain, as below
You wantDo this
shop.example.com to follow the appDelegate a zone such as go.example.com to the cluster, then add one CNAME for shop.example.com
A whole domain for the cluster's appsDelegate the domain itself, such as example-apps.com
The bare domain example.comDelegate example.com itself, or use an ALIAS record, see A bare domain

Delegate the zone

Add the zone to the cluster

An instance admin opens the cluster in Settings -> Servers. Under Settings, DNS zone (shown once a domain of the cluster answers on every server), type the zone and click Add:

go.example.com

A zone belongs to one cluster. A zone inside another cluster's zone is refused. Under a domain connected to Deplo, Deplo writes its records itself and the next step is done for you.

Add the records where you bought the domain

The zone's row says Add its records where you bought the domain: open Records, then Check, and Records opens on its own once the zone is added. It lists the records to add at your registrar or DNS provider, each with a copy button. For a cluster of three servers:

go.example.com.      NS    ns1.go.example.com.
go.example.com.      NS    ns2.go.example.com.
go.example.com.      NS    ns3.go.example.com.
ns1.go.example.com.  A     203.0.113.10
ns2.go.example.com.  A     203.0.113.11
ns3.go.example.com.  A     203.0.113.12

The NS records hand the zone to the cluster. The A (and AAAA) records are the glue: the addresses of those name servers, which live inside the zone they serve.

Deplo picks at most four name servers, the servers that joined first and can answer. The list stays as it is: it changes only when one of them leaves the cluster or can no longer answer, and Activity then says to update the records.

Check

In the same dialog, click Check. Deplo asks the domain's parent (example.com here) for the delegation, and each name server for the zone. It says delegated, or exactly which record to add or remove:

Add the NS record go.example.com -> ns3.go.example.com.

A registrar can take a few minutes to publish new records.

Port 53 must be open on every server that answers, UDP and TCP, from the internet. A server whose port 53 is taken by another program, or blocked, is named as no name server, and Records says why.

Point a domain at it

In the app's Domains, a domain that answers on every server shows Let the cluster answer (with more than one zone, pick one). It needs the Manage domains capability; only adding the zone itself needs an instance admin.

The domainWhat to add
Inside the zone: shop.go.example.comNothing. The zone answers it
Outside it: shop.example.comOne CNAME, shown on the row with a copy button

For shop.example.com the row says:

Add a CNAME record for shop.example.com -> shop-example-com.go.example.com

shop.example.com.  3600  IN  CNAME  shop-example-com.go.example.com.

The name in the zone is the domain with its dots turned into dashes, so no two domains share one. The CNAME never changes: what changes is what the cluster answers for the name inside the zone.

Use my own records on the row goes back to records you write yourself.

A domain connected to Deplo needs none of this: Deplo writes its CNAME in Cloudflare and the row says Deplo keeps its records in Cloudflare, pointed at the cluster's zone go.example.com.

Rename the zone

Rename, on the zone's row in the cluster's settings, gives it another name. Its domains follow:

  • Records Deplo keeps in Cloudflare are rewritten for you, the zone's own included.
  • Records you typed must point at the new name. Activity lists them, and each row shows its new CNAME.

A name inside the old zone (shop.go.example.com) cannot follow: remove it first. Remove waits for the domains whose records Deplo keeps: remove them first, or rename the zone instead.

A bare domain

A CNAME cannot point a bare domain (example.com, without anything in front) anywhere. DNS forbids a CNAME next to the NS and SOA records every bare domain has. Deplo refuses it:

example.com is a bare domain, and DNS gives no way to point one at another name. Add example.com itself as a zone of the cluster, or use an ALIAS record at a DNS provider that offers one.

Two ways out:

  • Delegate the bare domain itself to the cluster: add example.com as the zone, and its NS records at the registrar. The cluster then answers for all of example.com, www included. Records the cluster does not hold (mail MX, TXT verifications) cannot live there, so pick a domain used only for the cluster's apps.
  • Use an ALIAS or ANAME record, at a DNS provider that offers one, pointed at the name in the zone, such as example-com.go.example.com. The provider resolves it and answers with the addresses itself.

DNS at Cloudflare

A domain connected to Deplo

With example.com connected for Automatic DNS, Add Domain on an app of the cluster offers two choices for app.example.com:

ChoiceWho answers
Let the cluster answer (default)Every server of the cluster, nearest first. Nothing to type
Cloudflare TunnelCloudflare, through the app's own tunnel

With Let the cluster answer, the first domain sets everything up:

Deplo makes the cluster's zone

The first name under example.com that nothing in Cloudflare uses: cluster.example.com, else cluster-2.example.com, and so on. Its records go in Cloudflare, DNS only, and the name servers start:

cluster.example.com.      NS  ns1.cluster.example.com.
ns1.cluster.example.com.  A   203.0.113.10

The zone shows in the cluster's settings, where Rename changes it.

Deplo checks that the cluster answers

The same check as Check above, a few times over about ten seconds while Cloudflare publishes the records.

Deplo points the domain at it

Two records in Cloudflare, DNS only, and Answer on every server turned on:

app.example.com.                  CNAME  app-example-com.cluster.example.com.
_acme-challenge.app.example.com.  CNAME  _acme-challenge.app-example-com.cluster.example.com.

The second one gets its certificate through the zone, so every server holds it.

Every later domain reuses the zone: only the last step.

  • Making the zone takes an instance admin. For anyone else, until it exists, the default is the tunnel, and Let the cluster answer says: "an instance admin has to add the cluster's DNS zone first."

  • A domain is never left unanswered. When the check fails, the domain goes on the tunnel and its row says why:

    On the Cloudflare Tunnel for now: server-c (203.0.113.12) does not answer for cluster.example.com: open port 53, UDP and TCP, on that server. Once fixed, let the cluster answer it.

    Once fixed, Let the cluster answer on that row moves it. The same button moves any tunnel domain of the app. Visitors whose DNS still points at Cloudflare may see errors for up to five minutes.

  • A name inside the zone, like shop.cluster.example.com, is always answered by the zone, never by a tunnel. Nothing is written for it in Cloudflare.

  • When a name server changes, Deplo rewrites the zone's records in Cloudflare on its own.

  • An instance admin adding a zone in the cluster's settings under example.com gets the same: Deplo writes its records in Cloudflare.

A domain not connected to Deplo

When the domain's DNS is at Cloudflare but not connected to Deplo, add the records yourself:

  • Delegate a subdomain inside the Cloudflare zone. In Cloudflare's DNS for example.com, add the NS records and the A (and AAAA) records Records lists for go.example.com, each one DNS only. Nothing changes at the registrar.

  • Set the CNAME to DNS only (grey cloud). Cloudflare proxies a new record by default. A proxied CNAME puts Cloudflare in the cluster's place: visitors reach Cloudflare, never the nearest server, and the domain stays misconfigured. The row says:

    Cloudflare answers for shop.example.com in the cluster's place: in Cloudflare, set its record to DNS only (grey cloud).

    shop.example.com.  CNAME  shop-example-com.go.example.com.   DNS only
  • A bare domain works through Cloudflare's CNAME flattening: a CNAME on example.com to example-com.go.example.com, DNS only. Cloudflare asks the cluster and answers with the addresses itself, so the answer still names only healthy servers. It loses the geography: Cloudflare asks in the visitor's place, so the servers are the nearest to Cloudflare, not to the visitor.

What the servers answer

Every server of the cluster answers for the zone, on its own addresses. Each name answers with the healthy servers, one A record (and AAAA for IPv6) each. Once every server has a location, a visitor gets the ones nearest it first; until then, every healthy server, in a shuffled order:

$ dig +short shop-example-com.go.example.com @203.0.113.10
203.0.113.12
203.0.113.10
203.0.113.11
  • Healthy means the answering server heard it over the cluster's private network a moment ago. Every server judges from where it stands, so the answers keep working with Deplo or the leader down.
  • A server being emptied leaves the answers at once.
  • Only the zone's own names are answered: a name of no domain gets "no such name", and any other domain is refused. The servers never look anything up on someone's behalf.
  • One public address names one server. When two servers use the same public address, the usual case for virtual machines behind one router, the cluster names it once: for the server that joined first, in its answers, its glue and its name servers. A router sends that address to one machine only, so naming it twice would point visitors at the same machine twice. The cluster's page says so: pve-1 and pve-2 both use the public address 203.0.113.5.

Its app's server first

An app that runs on one server at a time (it keeps data, or the cluster does not load-balance it) is answered with that server first, then the other healthy servers nearest first. A visitor then reaches the app directly, not through another server.

  • It follows the app. When the app moves to another server, Deplo tells every server at once.
  • Health still wins. A stopped server is never answered, its app's server included.
  • Deplo down during a failover leaves the old server first for a while. It still answers and passes visitors on to the new one: one more hop, never an error.
  • A load-balanced app runs on every server, so it is answered nearest first, as below.

Nearest servers first

Each visitor is answered with the healthy servers nearest it, the nearest first, and never fewer than two when two are up: a server that stops between two answers still leaves the browser another to try.

The visitorAnswered with
Its resolver says which network it asks for (EDNS Client Subnet; most public resolvers do)The servers nearest that network
Its resolver says nothingThe servers nearest the resolver
Neither has a place in the location tableEvery healthy server

For a cluster with 203.0.113.10 in Milan, 203.0.113.11 in New York and 203.0.113.12 in Tokyo, a visitor in Rome is answered:

203.0.113.10
203.0.113.11
  • Health always wins. A stopped server is never answered, however near: the visitor gets the next nearest ones.
  • Servers in about the same place share the visitors. Every server within 500 km of the nearest one is answered too.

Where each server is

Deplo places each server by its own address. To correct one, an instance admin opens the cluster in Settings -> Servers, then Server locations -> Change, and types a city:

Frankfurt am Main, Hesse, Germany
  • Advanced takes a latitude and a longitude instead, for a place no city names.
  • Use its address goes back to the place Deplo found.

While one server has no location at all, every healthy server is answered to everyone.

The location table

Deplo places visitors and servers with a location table that ships with each release and changes only when Deplo is updated. No server and no instance ever downloads it, so an install with no internet access has it too. Server locations shows the table's month, and says so once it is more than three months old.

The table is built from IP geolocation by DB-IP, its IP to City Lite database, under the CC BY 4.0 licence.

A server whose agent is too old to place visitors keeps answering every healthy server, and places them once its agent is updated.

When servers cannot see each other

Each server counts the servers of its cluster it hears, itself included.

It hearsIt answers
More than halfThe healthy servers
Exactly halfThe last set it answered while it heard more than half
Fewer than halfAn error (SERVFAIL), so resolvers ask another server

So a server cut off from the rest stops naming itself, and the others keep answering without it. It never answers with itself alone: it could not pass a visitor on to the server that runs the app.

Two consequences to know:

  • If more than half of the servers stop at once, the ones left answer with an error until the others are back, or until you remove the stopped servers from the cluster, which gives the smaller cluster a majority again.
  • A cluster of two servers keeps naming a stopped one. One server alone is exactly half, so it repeats the last answer. Three servers is the smallest cluster whose answers drop a stopped one.

How fast a stopped server leaves

DefaultRange
A stopped server leaves the answers after10 s5 to 60 s
Resolvers keep an answer for30 s10 to 300 s

So a server that stops leaves every answer within about 40 s. Meanwhile a browser that tries it moves on to the next address after a few seconds.

A server that comes back is named again after it has been up for the same 10 s, by the other servers and by itself: its proxy needs a moment to serve the domains after it starts. Until then the other servers answer for it, so no visitor meets it while it is still starting.

Both are under DNS zone -> Advanced in the cluster's settings. A longer time means fewer queries, and a stopped server named for longer.

Certificates

The zone also proves the cluster owns its names, so HTTPS certificates are checked over DNS (DNS-01) instead of over HTTP. No DNS provider token is involved: the check is a TXT record the cluster's own servers answer.

One certificate for the zone

When the first domain inside the zone needs HTTPS, the cluster gets one certificate for the zone and every name one label below it: go.example.com and *.go.example.com.

DomainCertificate
go.example.comThe zone's
shop.go.example.com, blog.go.example.comThe zone's, nothing ordered for them
a.b.go.example.com (two labels below)Its own, checked over DNS
  • Ordered once, never per server. One server of the cluster orders and renews it. Every other server gets it from Deplo and serves it.
  • A server that stops does not stop renewals. After 10 minutes without it, another server takes renewals over, with the certificate it already holds. Nothing new is ordered while it is valid.
  • A server back from a power cut, or after weeks away, orders nothing. Before its proxy starts, the certificates the cluster may have handed to another server are set aside: it keeps serving them, renews none, and takes back only those Deplo still gives it.
  • An empty zone orders nothing.

The cluster's settings show it under DNS zone: the names it serves, valid until, which server renews it, and why the last order failed when one did:

The certificate authority could not look the zone up. Check that the records above are added where the domain was bought.

DNS zone -> Advanced -> One certificate per zone turns it off: each name in the zone then gets a certificate of its own, still checked over DNS.

A name's own certificate

A name two labels down, every name when One certificate per zone is off, and a name outside the zone with its record (below) get a certificate of their own, the same way as the zone's:

  • One server renews it, at first the server its app runs on. Every other server gets it from Deplo and serves it.
  • After 10 minutes without that server, another one takes renewals over, with the certificate it already holds.
  • Activity says it: Every server of eu now holds the certificate of shop.example.com, valid until 2027-02-19.

So every server can serve the app, and load balancing can run it on all of them.

A name outside the zone

A domain outside the zone, such as shop.example.com, keeps its HTTP check. The row in Domains offers one optional record that moves it to DNS too:

_acme-challenge.shop.example.com.  3600  IN  CNAME  _acme-challenge.shop-example-com.go.example.com.

Add it where shop.example.com lives, then Check. Without it, nothing changes.

A certificate the name already has keeps working, and its next renewal is checked over DNS. To switch it, that server's proxy restarts once, at its quietest hour before the renewal is due, and Activity says so. Removing the record moves it back to the HTTP check the same way, and so does turning One certificate per zone off or on for a name in the zone.

Who can set a check

Only the server's own proxy can ask its server agent for a TXT record, and only for an _acme-challenge name of a domain of that cluster, inside its zone. No app, and nothing else on the network, can set one. The certificate's key is never shown in the panel or written to a log.

Safety

  • The name server runs with none of the agent's rights. It is a separate process, as the nobody user, locked into an empty directory, with no way to open a connection. It cannot reach Docker, the server agent's keys or anything else on the server. It only answers, and reads the location table it is handed, which it cannot change.
  • No amplification. No ANY answers, a rate limit per network of visitors, and an answer too big for UDP is cut short so the resolver asks again over TCP.
  • No DNSSEC. The zone is not signed, so the delegation must have no DS record; Check says so if one is there.

Did this page help you?

On this page