These posts are primarily intended for a technical audience who are trying to deploy a Docker setup on a minimal budget, and for those who typically do not have the luxury of a large enterprise setup — it is ideal for someone who is setting up their own development environment at home or in an isolated capacity.
To set the scene, I am running off a non-commercial internet connection which only has a single IP address — this means that I have to be somewhat creative with regard to running multiple services. I have multiple Docker hosts on my local network, running different services and configurations, including at least one Swarm with multiple nodes.
My decision to use both Nginx and Traefik are based on the fact that Nginx can supply the .htpasswd support for basic authentication, while Traefik can supply the automated Lets Encrypt certificate management for HTTPS support, as well as being easy to configure dynamically.
In this post we will be exploring the deployment of a secured Docker Registry, with authentication and HTTPS support. In order to do that we will be using both Nginx and Traefik, as they supply different parts of the puzzle.
Prerequisites
- You must have a Docker host installed and configured to properly run Docker
- You must have a public domain name or subdomain which points at your internet public IP address
- Your internet public IP address should ideally be static, or you should be using a dynamic DNS service
- Your internet router must forward port 80 and port 443 to the IP of the host that you will be running Traefik on
Deploying Docker Registry
This is probably the most straightforward part of the process, as its basically just pulling the image from the Docker Store and telling it what port to listen on and where to put the images.
Run the command:
docker create -d \
--restart=always \
--name registry \
-v /dockerstorage/registry:/var/lib/registry \
registry:2
Lets break that down a little:
-d— detach this container, don’t run it interactively--restart=always— always attempt a restart of this container--name registry— give it a useful name-v /storage/registry:/var/lib/registry— give it a volume for persistent storage of images, I’m using a local directoryregistry:2— use the official registry image, version 2
This will run an insecure registry, where insecure means that anyone who can access the address can push, pull and update images. Thats not a great situation to be in, as anyone can update your images!
We do mitigate this a little in the above command by only binding the published port to the Docker hosts localhost address — but this still means that anyone who can access that address (anyone on the Docker host) can access it — and its not that useful if you have more than one Docker host on your network that you want to access the registry.
Adding Basic Authentication with Nginx
Docker itself understands the concept of basic access authentication in order to access a secured registry, so lets set that up.
In order to do this, we will need to proxy all traffic to the registry container, and its that proxy which will handle the basic access authentication.
Run the command:
docker create --name registry-proxy \
-d \
--restart=always \
-p 192.168.1.1:5000:8080 \
-v /storage/registry-nginx/nginx.conf:/etc/nginx/nginx.conf:ro \
-v /dockerstorage/registry-nginx/htpasswd:/etc/nginx/.htpasswd:ro \
--link registry:registry
nginx:latest
Again, lets break that down
-p 192.168.1.1:5000:8080— publish port 5000 to the Docker hosts public IP address (yup, you will have to insert yours in the right place)-v /storage/registry-nginx/nginx.conf:/etc/nginx/nginx.conf:ro— add a volume which provides the custom nginx configuration file. Theromeans ‘read only’-v /storage/registry-nginx/htpasswd:/etc/nginx/.htpasswd:ro— add a volume which provides thehtpasswdfile, again make it read only--link registry:registry— this is the important part, as it allows this container to talk to our registry containernginx— use the official Nginx image
Here we are creating an Nginx container which will act as a proxy specifically for our registry — it will listen on a public port, and the configuration we give it will make it pass all traffic back to the registry, but only if the requests are authenticated.
The two critical parts of the puzzle are the Nginx configuration file and the basic authentication password file.
The Nginx configuration file is as follows:
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
keepalive_timeout 65;
server {
listen 8080;
auth_basic "Private Docker Registry";
auth_basic_user_file /etc/nginx/.htpasswd;
location / {
proxy_pass http://registry:5000/;
proxy_redirect off;
}
}
}
All this configuration file does is tell Nginx to listen on port 8080 (Docker handles the forwarding from the published port of 5000 to the containers internal port of 8080), send all traffic to the registry container on port 5000, and lastly it tells Nginx to use basic authentication and what password file to use. Don’t worry about the path to the .htpasswd file here, its the containers internal path — we have injected our own file within that path by supplying a volume.
So, if you haven’t already, please add the above Nginx configuration to the nginx.conf file that you pointed to in the following part of the docker run command:
-v /storage/registry-nginx/nginx.conf:/etc/nginx/nginx.conf:ro
The second Nginx puzzle piece is the basic authentication password file, which we are calling htpasswd.
A htpasswd file consists of entries similar to this:
joeblogs:$apr1$wI1/T0nB$jEKuTJHkTOOWkopnXqC1d1
and its as simple as it looks — username, a separator of «:» and then the encrypted password. So how do we create one?
There are two methods, firstly using openssl to generate the password, and secondly using the htpasswd command from the apache2-utils bundle. We will use the first one, but please do look into the second.
Firstly, pick a username. Remember it. You don’t need it to generate the password, but you will need to add it to the htpasswd file.
The username I have chosen is yomama.
Secondly, generate the password by running the following command:
openssl passwd -apr1
This will ask you for the password, and then it will ask you for the password again, and then if both match, it will output the encrypted form of that password, similar to this (passwords are not echoed):
$ openssl passwd -apr1
Password:
Verifying - Password:
$apr1$zTW.JbHQ$qvIyBxxOl586K0COTF34Y1
That last line is the one you need to copy into your htpasswd file, so go ahead and do it, following the convention I mention above, like this:
yomama:$apr1$zTW.JbHQ$qvIyBxxOl586K0COTF34Y1
You can repeat this for any number of users, and removing a user is as simple as removing their entry from the file.
Save that file out in the location that you used in the following part of the docker run command:
-v /storage/registry-nginx/htpasswd:/etc/nginx/.htpasswd:ro
Quick recap
So, thus far we have created two Docker containers, one for the registry itself and one for the nginx proxy that will supply the basic access authentication.
At this point, neither of the containers are running and we haven’t added HTTPS support — however, what we have done so far will actually work fine, as the HTTPS support simply adds transport encryption.
The caveat to that is that Docker itself doesn’t support insecure registries out of the box — you have to do extra configuration on each and every Docker host in order to allow that Docker host to pull images from the insecure registry.
So lets start up these two containers and move on to adding HTTPS support with Traefik.
Start your containers:
docker start registry registry-proxy
Adding HTTPS with Traefik
In the final part of this example I am going to run Traefik as a single node on the same host as the two previous containers — if you want to see how I actually ended up running it on my Swarm cluster, then there will be another post on that side of things shortly…
In order to run Traefik in the manner which I prefer, I am going to cheat a little and introduce another component to the mix: Consul by Hashicorp.
Ok, now that I’ve blindsided you, I should explain a little about what Consul is and what it provides in this situation.
Traefik is fantastic at doing what it does, but it really comes into its own when you allow it to dynamically discover the services it needs to provide — it has various back ends to do this, and it can even listen to Docker directly to discover containers that it can provide access to, but the configuration back end I prefer is consul.
Consul is a service discovery and configuration store — you have individual services register with it and also get information about other services from it. Its like DNS on steroids, which is an ironic description since it can also do DNS…
Consul…
Run the command:
docker run \
-d \
--restart=always \
--name consul \
--net=host \
-e 'CONSUL_LOCAL_CONFIG={"skip_leave_on_interrupt": true}' \
-v /usr/data/consul:/consul/data \
consul agent -server \
-client=<IP OF HOST>\
-bind=<IP OF HOST>\
-ui
Lets break that down (I shall skip the ones we have already covered):
--net=host— don’t use a private Docker network, bind this container directly to the hosts network-e 'CONSUL_LOCAL_CONFIG={"skip_leave_on_interrupt": true}'— give some configuration to consul via an environment variable-v /storage/consul:/consul/data— lets persist the data that consul storesconsul agent -server— run consul in production mode as a server-client=<IP OF HOST>— give it the IP address that the client aspects need to bind to-bind=<IP OF HOST>— give it the IP address that the server aspects need to bind to-ui— run the consul UI, so you can inspect the data
Once the container is running, you can browse to http://<IP>:8500/ to access the UI.
You can run one Consul service or you can run many in a cluster — here we will stick to one, but in production I would recommend at least three for redundancy. There is also no security on Consul as setup here, you should look into setting up its ACL security if you are going to use this in production or on a insecure network.
…and Traefik
Ok, lets setup Traefik!
Traefik is what is going to act as our ingress load balancer, and in other blog posts I will be covering what other things you can do with it — here we will just be using it to handle HTTPS SSL termination for our registry.
Run the command:
docker run \
--name ingress-lb \
--restart=always \
-v /storage/traefik/acme:/etc/traefik/acme \
-p 80:80 -p 443:443 -p 8089:8089 \
-d \
traefik:1.1.2 \
--consulcatalog=true \
--consulcatalog.endpoint="<IP_OF_CONSUL_SVC>:8500" \
--consulcatalog.constraints="tag==public" \
--entryPoints='Name:https Address::443 TLS' \
--entryPoints='Name:http Address::80' \
--acme.entrypoint=https \
--acme=true \
--acme.ondemand=true \
--acme.onhostrule=true \
--acme.email="[email protected]" \
--acme.storage=/etc/traefik/acme/acme.json \
--web \
--web.address=":8089"
As ever, lets break that down:
-v /storage/traefik/acme:/etc/traefik/acme— lets give it a volume for persistent storage of Lets Encrypt certificatestraefik:1.1.2— lets use a specific version of the official Traefik image--consulcatalog=true— enable use of the consul catalog as a configuration store (note that Traefik also offers the consul key-value store as a separate configuration store, and that uses the argument--consul, do not confuse them)--consulcatalog.endpoint="<IP_OF_CONSUL_SVC>:8500"— point Traefik at the consul service, fill in the IP address here--consulcatalog.constraints="tag==public"— only use services which have this constraint (allows you to only make public what you want to be made public, but still use consul for lots of other things)--entryPoints='Name:https Address::443 TLS'— tell Traefik to use HTTPS as an entry point on port 443--entryPoints='Name:http Address::80'— tell Traefik to use standard HTTP as an entry point on port 80 (not strictly needed for our example, but included for completeness)--acme.entrypoint=https— tell Traefik to do Lets Encrypt certification requests on HTTPS--acme=true— turn Lets Encrypt support on--acme.ondemand=true— tell Traefik to do Lets Encrypt when it needs to--acme.onhostrule=true— tell Traefik to do Lets Encrypt on the basis of information supplied in the host rule--acme.email="[email protected]"— your email address to submit to Lets Encrypt--acme.storage=/etc/traefik/acme/acme.json— where to store the Lets Encrypt certificates, relative to the container root (the volume we supply persists this outside the container)-web— starts Traefiks web UI so you can see whats going on--web.address=":8089"— tells Traefik what port to put its UI on
Once the container is running, you can browse to http://<IP>:8089/ to access the UI.
So, here we have told Traefik to talk to Consul, specifically its service catalog, and also to use both HTTP and HTTPS, and to generate SSL certificates using Lets Encrypt when needed to (it also handles renewals, which is neat).
What we haven’t told Traefik is how to talk to our registry. Or more accurately, our registry proxy (Nginx).
Registering with Consul
Create a text file consisting of the following:
{
"Name": "dockerregistry",
"Address": "<IP_ADDRESS_OF_NGINX_HOST>",
"Port": 5000,
"Tags": [
"traefik.tags=public",
"traefik.frontend.rule=Host:<PUBLIC_DOMAIN_HOST>",
"traefik.frontend.entryPoints=http,https"
]
}
Make sure you use proper values for the <IP_ADDRESS_OF_NGINX_HOST> placeholder (which should be the IP of the Docker host that the Nginx proxy is listening on) and the <PUBLIC_DOMAIN_HOST> placeholder (which should be your publicly reachable domain name, eg registry.example.com).
Save it out as docker-registry.json.
You then simply send this file to Consul to register it as a service, using the following command:
curl --upload-file docker-registry.json http://<IP_OF_CONSUL_SVC>:8500/v1/agent/service/register
Again, replace the <IP_OF_CONSUL_SVC> placeholder with the correct value.
You can browse to http://<IP_OF_CONSUL_SVC>:8500 to see that the service has actually been registered with Consul.
Traefik will automatically pick this registration up and create its own internal configuration to handle it, and it will also talk to Lets Encrypt and create valid SSL certificates, so now we have a secure registry that talks over HTTPS and also has access control via Nginx and htpasswd.
You can browse to https://<PUBLIC_DOMAIN_HOST>/v2/_catalog and you should be asked for a username and password (use the credentials you created earlier), and if you successfully authenticate, you will get a json list of images in your registry!
Authenticating a Docker host
The last step is to tell your Docker host about the credentials needed to interact with your registry.
On your Docker host, simply run the following command:
docker login registry.example.com
You will then be asked for your credentials, and if you enter them correctly then that Docker host will be authenticated against that registry, and you can now push and pull from it.
There you have it, a quick guide to setting up a secure public Docker registry with Nginx and Traefik (and a little help from Consul).