Introduction
One of the most common questions for people using Docker is the use of volumes. In order to understand what is a docker volume? First, you will need to know how docker filesystem works. Docker images are stored as a series of read-only layers. When we start a container, Docker takes the read-only image and adds a read-write layer on top. By default docker filesystems are temporary union filesystems. You can pull a docker image, start a container, create, modify and delete files in docker system. But if you stop container and start it up again, all your changes will be lost. In this situation a docker volume will be used to retain your data in volumes.
When a Docker container is deleted, relaunching the image will start a fresh container without any of the changes made in the previously running container and those changes are lost. Docker calls this combination of read-only layers with a read-write layer on top a Union File System.
The Docker volume is a directory that is located outside of the root filesystem of your container. This allows you to import this directory in other container’s and use volumes to mount directories from your host machine inside a container.
It also permits data to be stored in a container outside of the boot volume.
There are two ways to manage data in Docker –
- Docker Data Volumes
A data volume is a directory within a container that persists beyond the life cycle of the container. All volumes are stored on the Docker host in a system path. It can be shared and reused among containers.
- Docker Data Volume Containers
A data volume container is a container that houses a volume and used to store data in a persistent way. Because volumes can be shared with other containers and used as a centralised data store across multiple containers on the same Docker host. You can also mount other containers volume inside a data volume container and save their data to it.
In this tutorial, we will learn the concept of the docker volume, types of volume and how effectively data volumes can be used to share data between the Docker host and the containers as well as between containers.
Requirements
Ubuntu server 14.04 with docker installed on your system.
Features of Data Volume
A data volume is a specially designated directory within one or more containers that bypasses the Union File System.
Docker data volumes provide several useful features, some of them are listed below:
- Data volumes can be shared and reused among containers.
- Data volumes can be used to share data between the host filesystem and the Docker container.
- You can directly change a data volume, changes to a data volume will not be included when you update an image.
- Data volumes persist, even if container is restarted or deleted.
- Data can be backed up, restored, and migrated more easily.
- You don’t need to manually specify host mount points and handle the additional concerns of file permissions and security restrictions.
Creating and Mounting Data Volume
You can not create data volumes directly in Docker. First, you will need to create a data volume container with a volume attached to it.
You can use docker create command to create the data volume container.
Now, first create a new data volume container to store your volume by using the following command:
sudo docker create -v /mnt --name data_volume_container ubuntu
This will create a new data volume container named data_volume_container from the ubuntu base image in the directory /mnt
You can use docker ps -a command to list data volume container created above:
sudo docker ps -a
You should see the following output:
4522bd3ec0da ubuntu "/bin/bash" 33 minutes ago Created data_volume_container
534544f2e924 centos "/bin/bash" 3 weeks ago Exited (137) 3 weeks ago boring_hamilton
6c31f7ad39a7 dockerui/dockerui "/dockerui /bin/bash" 5 weeks ago Exited (2) 5 weeks ago amazing_meitner
0925473d7dd2 dockerui/dockerui "/dockerui" 5 weeks ago Exited (2) 5 weeks ago tender_boyd
d25b0a968294 dockerui/dockerui "/dockerui" 5 weeks ago Exited (2) 5 weeks ago suspicious_ptolemy
e919b224e52d ubuntu "/bin/bash" 8 weeks ago Exited (0) 2 weeks ago lonely_poincare
573197038ed1 apache "/usr/sbin/apache2ctl" 8 weeks ago Exited (143) 8 weeks ago Apache-Instance
f49877210e7d ubuntu:Apache_Server "/usr/sbin/apache2ctl" 9 weeks ago Exited (143) 9 weeks ago Apache_Instance
Next, you will need to run Ubuntu container with the --volumes-from flag. After this anything you write to the /mnt directory will get saved to
the /mnt volume of your data_volume_container container.
Let’s run the following command with --volumes-from flag:
sudo docker run -t -i --volumes-from data_volume_container ubuntu /bin/bash
Now, check the mount details by runningng the following command in container shell:
mount
You should see the following output:
none on / type aufs (rw,relatime,si=8b6ea26c8dbb142d,dio)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev type tmpfs (rw,nosuid,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=666)
sysfs on /sys type sysfs (ro,nosuid,nodev,noexec,relatime)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,relatime,mode=755)
cgroup on /sys/fs/cgroup/cpuset type cgroup (ro,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/cpu type cgroup (ro,nosuid,nodev,noexec,relatime,cpu)
cgroup on /sys/fs/cgroup/cpuacct type cgroup (ro,nosuid,nodev,noexec,relatime,cpuacct)
cgroup on /sys/fs/cgroup/memory type cgroup (ro,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/devices type cgroup (ro,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/freezer type cgroup (ro,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/blkio type cgroup (ro,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/perf_event type cgroup (ro,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (ro,nosuid,nodev,noexec,relatime,hugetlb)
systemd on /sys/fs/cgroup/systemd type cgroup (ro,nosuid,nodev,noexec,relatime,name=systemd)
/dev/disk/by-uuid/6425dd49-3030-4430-b695-8fffb6779f62 on /mnt type ext4 (rw,relatime,errors=remount-ro,data=ordered)
/dev/disk/by-uuid/6425dd49-3030-4430-b695-8fffb6779f62 on /etc/resolv.conf type ext4 (rw,relatime,errors=remount-ro,data=ordered)
/dev/disk/by-uuid/6425dd49-3030-4430-b695-8fffb6779f62 on /etc/hostname type ext4 (rw,relatime,errors=remount-ro,data=ordered)
/dev/disk/by-uuid/6425dd49-3030-4430-b695-8fffb6779f62 on /etc/hosts type ext4 (rw,relatime,errors=remount-ro,data=ordered)
shm on /dev/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=65536k)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
devpts on /dev/console type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
proc on /proc/asound type proc (ro,nosuid,nodev,noexec,relatime)
proc on /proc/bus type proc (ro,nosuid,nodev,noexec,relatime)
proc on /proc/fs type proc (ro,nosuid,nodev,noexec,relatime)
proc on /proc/irq type proc (ro,nosuid,nodev,noexec,relatime)
proc on /proc/sys type proc (ro,nosuid,nodev,noexec,relatime)
proc on /proc/sysrq-trigger type proc (ro,nosuid,nodev,noexec,relatime)
tmpfs on /proc/kcore type tmpfs (rw,nosuid,mode=755)
tmpfs on /proc/latency_stats type tmpfs (rw,nosuid,mode=755)
tmpfs on /proc/timer_stats type tmpfs (rw,nosuid,mode=755)
You should see the mountpoint /mnt in above output.
Next, create a file named volume.txt in /mnt directory of Ubuntu container to test whether data volume working or not:
echo "Testing Docker volume" > /mnt/volume.txt
Next, run the exit command to return to your host machine’s shell:
exit
Next, run the same command again:
sudo docker run -t -i --volumes-from data_volume_container ubuntu /bin/bash
You should see the volume.txt file already there.
cat /mnt/volume.txt
You should see:
Testing Docker volume
You can use the -v multiple times to mount multiple data volumes and also use the VOLUME instruction in a Dockerfile to add one or more new volumes to any container created from that image.
Locating a Data Volume
You can locate the volume on the host by running the docker inspect command. This command provides details on the container configurations, including the volumes.
sudo docker inspect data_volume_container
The output should look something similar to the following:
"Mounts": [
{
"Name": "ae26fc614c8dbf12c55a56ab0a0c1d2c2c0c4dcc9d070832969e9761b46d7330",
"Source": "/var/lib/docker/volumes/ae26fc614c8dbf12c55a56ab0a0c1d2c2c0c4dcc9d070832969e9761b46d7330/_data",
"Destination": "/mnt",
"Driver": "local",
"Mode": "",
"RW": true,
"Propagation": ""
}
],
Here, the data volume is mapped to a directory in the Docker host, and the directory is mounted in read-write mode. This directory is created by the
Docker engine automatically during the container launch time. In the above output, source specifies the location on the docker host system, destination specifies the volume location inside the container and RW shows if the volume is read/write.
Sharing Data Between Containers
In this section, we will create a data volume container. This is very useful if you want to share data between containers or you want to use the data from non-persistent containers.
First, create a container named volume_container1 and mount a volume named data inside it.
You can do this by running the following command:
sudo docker run -it -v /data --name volume_container1 ubuntu
Now, go into the /data volume and create some files inside it for testing purposes.
cd /data
echo "Testing Data Volume Container" > file1
echo "Testing Data Volume Container" > file2
Now press Ctrl-P-Q to come back to the Docker host system without exiting the container.
Next, run docker ps command on the Docker host system:
sudo docker ps
You should see your running container:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d0d7d4e3d466 ubuntu "/bin/bash" 6 minutes ago Up 6 minutes volume_container1
Now, execute a command on the running volume_container1 and see the contents of /data volume:
sudo docker exec volume_container1 ls /data
file1
file2
You can see that the two files are present.
Now, launch another container volume_container2 and mount the data volume from volume_container1.
You can do this by running the following command:
sudo docker run -it --volumes-from volume_container1 --name volume_container2 ubuntu
Next, run ls command inside volume_container2, you should see that data folder is present with two files file1 and file2 inside the data folder.
ls
bin boot data dev etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var
`cd data`
`ls`
file1
file2
Mount a Host file as a Data Volume
You can also mount a single file instead of directories from the host system:
For example, to mount /var/log/dmesg file as a data volume from the Docker host system, run the following command:
sudo docker run --rm -it -v /var/log/dmesg:/var/log/dmesg ubuntu /bin/-bash
This will drop you into a bash shell in a new container, you will have your dmesg log from the host system and when you exit the container, the host will have the dmesg log of the container.
Backup and Restore Data Volumes
Another important function you can perform with docker volumes is to use them for backup and restore. To backup a data volume you can run a new container using the volume you want to backup and executing the tar command to produce an archive of the volume content .
You can do this by using the --volumes-from flag to create a new container that mounts that volume:
To backup data_volume_container run the following command:
sudo docker run --rm --volumes-from data_volume_container -v $(pwd):/backup ubuntu tar cvf /backup/backup.tar /mnt
--volumes-from flag creates a new nameless container that mounts the data volume inside data_volume_container you wish to backup.
A localhost directory is mounted as /backup . Then tar archives the contents of the /mnt volume to a backup.tar file inside the local /backup directory.
The container will be --rm removed once it ends and exits
You can also restore the volume in whatever way you wish.
To restore the volume into new container data_volume_container1, run the following command on Docker host system:
sudo docker run -v /mnt --name data_volume_container1 ubuntu /bin/bash
Next, extract the backup.tar file into the new container’s data volume by running the following command:
sudo docker run --rm --volumes-from data_volume_container1 -v $(pwd):/backup ubuntu bash -c "cd /mnt && tar -zxvf /backup/backup.tar"
Now the new container is up and running with the files from the original /mnt volume.
Removing Volumes
A Docker data volume persists after a container is deleted. When a container is removed using the docker rm subcommand, the Docker engine does not remove the directory that was automatically created during the container launch time. This behavior is designed to preserve the state of the containers application that was stored in the directory. If you want to remove the directory that was automatically created by the Docker engine, you can do so while removing the
container by providing a -v option to the docker rm subcommand, on an already stopped container:
Run the following command to delete container with volume:
sudo docker rm -v "container_id"
If the container is still running, then you can remove the container as well as the
auto-generated directory by adding a -f option to the previous command:
sudo docker rm -fv "container_id"
Conclusion
In this tutorial you have learned how docker volumes work, sharing data between container to container as well as between the Docker host to container. Data sharing using data volumes is turning out to be a very powerful and essential tool in the Docker.
Share this Article!
