Posted on

Exploring Named Volumes in Docker Compose

We’re continuing our examination of Docker data storage options. If you’d just like to dive into the code for adding storage volumes to the WordPress app from the Part 3, you can skip ahead to Part 4d – Working with Data Storage in Docker Compose.

Introduction

This is Part 4c in a series describing a project to create a local WordPress development environment using Docker-Compose. The series is structured as follows:

In Part 4a we saw that Docker created anonymous volumes to store our data absent our specifying a storage option in our Compose file. In Part 4b we examined the bind mount option for persistent data storage. In Part 4c, we’ll continue discussing data storage within Docker, focusing on the named volume option for persistent data storage. We’ll also look at how to start up a project with multiple Compose files.

If you need an overview of Docker data storage options before we move on you can read Docker’s storage overview. For the remainder of this post we will continue working with our project from Part 4b.

Exploring Named Volumes with Our WordPress App

Named volumes provide a second means to persist an app’s data between sessions. They also make it easy to share data between services in a project. As with the bind mount, we use the volumes sub-topic key in our Compose file to specify a named volume, with the difference that we use a name:container format. Another difference is that we must also specify a top-level volumes key with each named volume in our project listed as a sub-topic.

Let’s explore named volumes further by adding one to the MariaDB and WordPress services in our WordPress app. The code needed to add a named volume to our service configurations is very similar to that used for bind mounts. Let’s name our volumes db and wp. Then the code we need to add to the MariaDB service is

    volumes:
      - db:/var/lib/mysql

and the code we need to add to the WordPress service is

    volumes:
      - wp:/var/www/html

We also need to add a top-level volumes key

volumes:
  db:
  wp:

As with bind mounts in Part 4b, we can simply add this code to our Compose file to put our named volumes into action. However, let’s use another Compose feature that allows us to start up a project using more than one configuration file. This gives the ability to startup a project with different configurations depending on the situation, for example using one configuration during development and another for production. Again, this is a bit of overkill for our purposes here, but it does provide a nice example of the capability.

Starting a Project from Multiple Compose Configuration Files

When we start a Compose project with the docker-compose up command, by default Compose looks for a docker-compose.yml file in our project directory which it will use to start the project. If a docker-compose.yml file isn’t found, Compose will look for it in the parent directories. As we saw in Part 4b, Compose will also look for an optional docker-compose.override.yml file. If it is present, Compose will combine the files prior to starting the project.

Optionally, we can use the -f option to specify the configuration file or files we want to use to start the project. Let’s create a file, called docker-compose-named-volume.yml, containing the above code for our named volumes, to demonstrate this feature.

docker-compose-named-volume.yml

version: '3.7'

services:
  db:
    volumes:
      - db:/var/lib/mysql

  wp:
    volumes:
      - wp:/var/www/html

volumes:
  db:
  wp:

Again, Compose versions must be the same in all the configuration files we use for starting a project. We can start up the project with the following command.

docker-compose -f docker-compose.yml -f docker-compose-named-volume.yml up -d

We need to add a -f option for each configuration file we want to include in our project, so above we have one for the docker-compose.yml file and another for the docker-compose-named-volume.yml file. If you ran this command without first shutting down your project from the Part 4b, no problem. You’ll note that Compose indicates that the Adminer and nginx containers already exist and that it is recreating the MariaDB and WordPress containers. In fact your project is recreated with our new named volumes but without the bind mounts from Part 4b. I’ll leave it as an exercise to show what happens if you did the same thing in Part 4b, that is, start up the project with the new override file before closing down the project from Part 4a. Spoiler alert: things aren’t as successful.

You can use the docker ps command to confirm your project’s containers are up and running. The list should be similar to those we’ve seen before since we haven’t changed the services our project uses. You will see changes with the docker volume ls command though, with something like the following.

DRIVER              VOLUME NAME
local               part-4_db
local               part-4_wp

You can see that Compose has created two named volumes by appending the db and wp volume names from our Compose file to our directory name. Let’s do a quick inspection of our part-4_wp volume with the following command.

docker inspect part-4_wp
[
    {
        "CreatedAt": "2020-01-23T11:06:03-08:00",
        "Driver": "local",
        "Labels": {
            "com.docker.compose.project": "part-4",
            "com.docker.compose.version": "1.25.0",
            "com.docker.compose.volume": "wp"
        },
        "Mountpoint": "/var/lib/docker/volumes/part-4_wp/_data",
        "Name": "part-4_wp",
        "Options": null,
        "Scope": "local"
    }
]

You’ll see something similar for the part-4_db volume. You can see that Compose has created volumes for us in the Docker storage area just as it had with the anonymous volumes in Part 4a. Only this time the volumes are named. You can navigate to the Docker storage area at /var/lib/docker/volumes to verify this for yourself.

:/var/lib/docker/volumes# ls -l
total 68
-rw------- 1 root root 65536 Apr 29 15:50 metadata.db
drwxr-xr-x 3 root root  4096 Apr 29 15:50 part-4_db
drwxr-xr-x 3 root root  4096 Apr 29 15:50 part-4_wp

If you’d like, go ahead and repeat the testing we did above to verify that our data persists between sessions as follows:

  • Access your site, complete the WordPress installation and log into your site
  • Make some changes to your WordPress site
  • Shutdown your app with docker-compose down
  • Verify your volumes are still there with docker volume ls
  • Restart your app with docker-compose -f docker-compose.yml -f docker-compose-named-volume.yml up -d
  • Verify any changes you made are still there

Let’s shut our app down again, but this time using the remove volumes option

docker-compose down -v

and verify that our volumes are gone using the docker volume ls command. You may be surprised to find that your volumes are still there. This sheds some light on how Compose shuts down projects. In this case it used the docker-compose.yml file to shut down our project. Since our named volumes were specified in the docker-compose-named-volume.yml file, and we didn’t specify this file in our shutdown command, our db and wp volumes were not removed. (I’ll leave it as an experiment for you to see what happens if you start your project with the docker-compose up -d command at this point).

We could remove the volumes properly by reissuing the shutdown command with our startup configuration files

docker-compose -f docker-compose.yml -f docker-compose-named-volume.yml down -v

but let’s examine other ways to remove unwanted volumes.

Removing Unwanted Docker Volumes with the Volume Command

We’ve seen that unwanted project volumes can be removed when shutting down a project using the docker-compose down -v command. Docker also provides two other ways to remove volumes with the docker volume prune or docker volume rm commands.

docker volume prune

and

docker volume rm <name of volume to be removed>

The first command will remove all volumes not associated with a running container. Be careful when using the prune option. It will remove all such volumes regardless of the project they were created with. Docker will give a warning and ask for confirmation prior to deleting any volumes, but it is easy to blow right through this. I’ve done it. I think this is a weakness of the named volume. It is a bit too easy to delete volumes from other shutdown projects if you’re not paying close attention to what you’re doing when working on another, unrelated project.

The second command overcomes this problem and is my go to command for removing unwanted volumes. With this command, you can remove any volume by name if it is not associated with a running container. Or, you can use the -f option to force the removal of a volume associated with a running container. You can remove more than one volume at a time by providing a list of names separated by spaces such as

docker volume rm <name of first volume to be removed> <name of second volume to be removed> <etc.>

In our case we can remove our volumes with the following command.

docker volume rm part-4_db part-4_wp

Wrapping Up

In Part 4d – Working with data storage in Docker Compose, we’re going to turn to the more practical matter of working with data storage in our WordPress app. Specifically, we’ll create a child WordPress theme, a simple plugin and look at various ways to backup and restore our data.