Javier Ramírez

  • Technology Advocate
  • Docker Captain
  • Docker Certificate Associate
  • Docker Certified Instructor
  • Microsoft Certified Systems Engineer
  • HP Openview Solutions Certified

    • Docker 1.13 Features 1/3

    15 Jan 2017 » docker, swarm

    I am quite interested in some of the new features announced in new Docker Engine 1.13. We will try to make a small review of the most important ones, or at least the ones that will make my life easier :).

    At the time of writing this article, some of these features are only available in “experimental mode”. We will use 1.13.0-rc7 with daemon running with experimental support enabled (see references to learn how to enable it).

    We start this series of articles with the new “docker system” command.

    ubuntu@swarmnode1:~$ docker system --help

Usage: docker system COMMAND

Manage Docker

Options:
 --help Print usage

Commands:
 df Show docker disk usage
 events Get real time events from the server
 info Display system-wide information
 prune Remove unused data</pre>
As we can see in the help page, there are for commands available.
<h2>docker system df</h2>
Docker system df will show information about storage used in our host related to images, containers and volumes. This information is available for running and stopped containers.

Let's see some examples. We will use an empty host for all examples for easy understand of data usage (you can use our vagrant environment docker-swarmmode, see references).
<pre>ubuntu@swarmnode1:~$ docker system df
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 0 0 0 B 0 B
Containers 0 0 0 B 0 B
Local Volumes 0 0 0 B 0 B

ubuntu@swarmnode1:~$ docker run -d busybox ping www.google.es
Unable to find image 'busybox:latest' locally
latest: Pulling from library/busybox
4b0bc1c4050b: Pull complete 
Digest: sha256:817a12c32a39bbe394944ba49de563e085f1d3c5266eb8e9723256bc4448680e
Status: Downloaded newer image for busybox:latest
aa0327042ba1065e0638f6d4d5ddfe241b029ba7d743d00a4a81bc7fec422006

ubuntu@swarmnode1:~$ docker system df
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 1 1 1.11 MB 0 B (0%)
Containers 1 1 0 B 0 B
Local Volumes 0 0 0 B 0 B</pre>
Latter we will see a more interesting example.

docker system df will show us the disk usage for downloaded and build images, stopped and running containers and all local volumes created.
<h2>docker system events</h2>
Docker system events show information about all docker engine events (it is the same as docker events command) .

As we already know, docker engine will generate events for almost all actions. Everytime a container is created, started/stopped or destroyed, events related to these actions are generated. Same happens when pulling or building images, volumes, networks, etc...

Here is a quick example
<pre>ubuntu@swarmnode1:~$ docker system events &amp;
[1] 18176

ubuntu@swarmnode1:~$ docker run -d busybox ping www.google.es
856b75a829f5cc63c0a7aef919d1f91e60f054b5e0f9236ad98a9c40ae1f6f01

2017-01-15T12:16:31.354759539+01:00 container create 856b75a829f5cc63c0a7aef919d1f91e60f054b5e0f9236ad98a9c40ae1f6f01 (image=busybox, name=trusting_bell)
2017-01-15T12:16:31.363098524+01:00 network connect 13a0e22fb628078c032ab4732d5eed687fde7c62ee7720cdee9bd87fbb7dcbf9 (container=856b75a829f5cc63c0a7aef919d1f91e60f054b5e0f9236ad98a9c40ae1f6f01, name=bridge, type=bridge)
2017-01-15T12:16:31.447693344+01:00 container start 856b75a829f5cc63c0a7aef919d1f91e60f054b5e0f9236ad98a9c40ae1f6f01 (image=busybox, name=trusting_bell)

ubuntu@swarmnode1:~$ docker rm -fv 856b75a829f5cc63c0a7aef919d1f91e60f054b5e0f9236ad98a9c40ae1f6f01
2017-01-15T12:16:48.077769551+01:00 container kill 856b75a829f5cc63c0a7aef919d1f91e60f054b5e0f9236ad98a9c40ae1f6f01 (image=busybox, name=trusting_bell, signal=9)
2017-01-15T12:16:48.089993793+01:00 container die 856b75a829f5cc63c0a7aef919d1f91e60f054b5e0f9236ad98a9c40ae1f6f01 (exitCode=137, image=busybox, name=trusting_bell)
2017-01-15T12:16:48.119506809+01:00 network disconnect 13a0e22fb628078c032ab4732d5eed687fde7c62ee7720cdee9bd87fbb7dcbf9 (container=856b75a829f5cc63c0a7aef919d1f91e60f054b5e0f9236ad98a9c40ae1f6f01, name=bridge, type=bridge)
856b75a829f5cc63c0a7aef919d1f91e60f054b5e0f9236ad98a9c40ae1f6f01
2017-01-15T12:16:48.127383429+01:00 container destroy 856b75a829f5cc63c0a7aef919d1f91e60f054b5e0f9236ad98a9c40ae1f6f01 (image=busybox, name=trusting_bell)

ubuntu@swarmnode1:~$ kill %1</pre>

    As a resume, we can say that docker system events works the same as docker events. I couldn’t find any difference between them :|

    docker system info

    Docker system events show information about the docker engine and  (it is the same as docker info command) .

    With docker system info we can review all configuration information related to docker engine. We will have different sections with information about different aspects of docker engine:

    • Number of containers created on this host engine.
    • Number of images available (remember tagging doesn't count as they are the same).
    • Engine version.
    • Enabled drivers for graph-storage, logging and cgroups.
    • Enabled plugins (volume-storage, networking, etc...).
    • Swarm related information about its role in the cluster, the managers addresses, raft clustering consensus algorithm and internal certificates configuration.
    • Information about container runtime and versions.
    • Security options enabled for the engine.
    • Host OS information used for labeling this engine.
    • Docker Root location.
    • Debug (for client and engine), Experimental modes status.
    • Registries Associated.
    • Live Restore feature status.

    This is an example ouput.

    ubuntu@swarmnode1:~$ docker info 
Containers: 4
 Running: 3
 Paused: 0
 Stopped: 1
Images: 2
Server Version: 1.13.0-rc7
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 12
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins: 
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
Swarm: active
 NodeID: ufhgrbz5br9lqtfjfukxi2kym
 Is Manager: true
 ClusterID: osklt6hzcrzp92ycg6hmcxaou
 Managers: 3
 Nodes: 4
 Orchestration:
 Task History Retention Limit: 5
 Raft:
 Snapshot Interval: 10000
 Number of Old Snapshots to Retain: 0
 Heartbeat Tick: 1
 Election Tick: 3
 Dispatcher:
 Heartbeat Period: 5 seconds
 CA Configuration:
 Expiry Duration: 3 months
 Node Address: 10.10.10.11
 Manager Addresses:
 10.10.10.11:2377
 10.10.10.12:2377
 10.10.10.13:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 03e5862ec0d8d3b3f750e19fca3ee367e13c090e
runc version: 2f7393a47307a16f8cee44a37b262e8b81021e3e
init version: 949e6fa
Security Options:
 apparmor
 seccomp
 Profile: default
Kernel Version: 4.4.0-59-generic
Operating System: Ubuntu 16.04.1 LTS
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 1.45 GiB
Name: swarmnode1
ID: UGRA:FCQA:6RKE:L4PS:Z5BM:C3G4:6KHI:KECB:U6LY:NRF7:MV2B:TK5N
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Experimental: true
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

    You can notice that Experimental Mode is enabled and swarmnode1 is a manager on swarm cluster with ID osklt6hzcrzp92ycg6hmcxaou.

     

    We found that docker system info and docker system events work the same as docker info and docker events, they could be aliases for same information. Sorry but I haven’t reviewed source code yet :|

    docker system prune

    Finishing the review for docker system commands, we have docker system prune, which is quite interesting.

    Docker system prune will clean all stopped containers, all volumes not used by at least one container, all networks not used by at least one container and all dangling images. We can even force to clean all unused images. This way, docker system prune will help docker engine maintenance. We have being doing this clean up works manually looping or filtering for docker rm, docker rmi or docker networkvolume rm, but this command will do everything needed for us. Be careful because as uncle Ben said ”With Great Power comes Great Responsibility”, and take a look at your docker images, containers, volumes and networks before executing docker prune because it will remove everything not needed …

    A quick example.

    ubuntu@swarmnode1:~$ docker run -d busybox ping www.google.es
ebe01b4a69e4e023892b500b9f2a7d2028c2674b0df9907537d8799aab0e7a1e

ubuntu@swarmnode1:~$ docker run -d busybox ping www.google.es
1048304de0e8be7279778a1509fdf31591e9f140e2355e6030c5720290417229

ubuntu@swarmnode1:~$ docker run -d busybox ping www.google.es
cb03d9bb00151029be99f7abff147969d773922114291d13521fd2c98324ae44

ubuntu@swarmnode1:~$ docker run -d busybox ping www.google.es
83c18ce521ba273f63f6532d1141ae6fb9d08f39b36ac6643bfa9e2c9662f632

ubuntu@swarmnode1:~$ docker stop ebe01b4a69e4e023892b500b9f2a7d2028c2674b0df9907537d8799aab0e7a1e 1048304de0e8be7279778a1509fdf31591e9f140e2355e6030c5720290417229 cb03d9bb00151029be99f7abff147969d773922114291d13521fd2c98324ae44 83c18ce521ba273f63f6532d1141ae6fb9d08f39b36ac6643bfa9e2c9662f632
ebe01b4a69e4e023892b500b9f2a7d2028c2674b0df9907537d8799aab0e7a1e
1048304de0e8be7279778a1509fdf31591e9f140e2355e6030c5720290417229
cb03d9bb00151029be99f7abff147969d773922114291d13521fd2c98324ae44
83c18ce521ba273f63f6532d1141ae6fb9d08f39b36ac6643bfa9e2c9662f632

ubuntu@swarmnode1:~$ docker system prune
WARNING! This will remove:
 - all stopped containers
 - all volumes not used by at least one container
 - all networks not used by at least one container
 - all dangling images
Are you sure you want to continue? [y/N] y
Deleted Containers:
83c18ce521ba273f63f6532d1141ae6fb9d08f39b36ac6643bfa9e2c9662f632
cb03d9bb00151029be99f7abff147969d773922114291d13521fd2c98324ae44
1048304de0e8be7279778a1509fdf31591e9f140e2355e6030c5720290417229
ebe01b4a69e4e023892b500b9f2a7d2028c2674b0df9907537d8799aab0e7a1e

Total reclaimed space: 0 B

    Docker prune will report reclaimed space after executing for getting happy with the results of this maintenance task. You will get happier if your system is storing a large amount of big unused or dangling images and containers are writing a lot of data to its runtime or volume and you can remove those stopped containers.

    Let’s see a better example using docker system df and docker system prune.

    ubuntu@swarmnode1:~$ docker volume ls
DRIVER VOLUME NAME

ubuntu@swarmnode1:~$ docker run -d -ti -v TEST:/DATA busybox dd if=/dev/zero of=/DATA/100MB_file bs=104857600 count=1
9401cdc50ae56f67a78534f4bfdc593e37734c3ce9450930e19608ea99762244

ubuntu@swarmnode1:~$ docker system df 
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 1 1 1.11 MB 0 B (0%)
Containers 1 0 0 B 0 B
Local Volumes 1 1 104.9 MB 0 B (0%)

ubuntu@swarmnode1:~$ docker system df -v
Images space usage:

REPOSITORY TAG IMAGE ID CREATED SIZE SHARED SIZE UNIQUE SIZE CONTAINERS
busybox latest 7968321274dc 37 hours ago 1.11 MB 0 B 1.11 MB 1

Containers space usage:

CONTAINER ID IMAGE COMMAND LOCAL VOLUMES SIZE CREATED STATUS NAMES
9401cdc50ae5 busybox "dd if=/dev/zero o..." 1 0 B 7 seconds ago Exited (0) 6 seconds ago peaceful_dijkstra

Local Volumes space usage:

VOLUME NAME LINKS SIZE
TEST 1 104.9 MB

ubuntu@swarmnode1:~$ docker system prune -f
Deleted Containers:
9401cdc50ae56f67a78534f4bfdc593e37734c3ce9450930e19608ea99762244

Deleted Volumes:
TEST

Total reclaimed space: 104.9 MB

    In this example we used a busybox image for creating a container that creates a 100MB file on /DATA, mounted on runtime as volume. Docker system df have shown us that our container haven’t stored anything big enough for showing statistics on ouput, but we have used a new volume where we created a 100MB file (/DATA/ 100MB_file). Executing docker system prune in this example reclaimed 104.9 MB of storage in /var/lib/docker (default mount point for docker root).

     

    We have reviewed some basic new features in new docker version 1.13 and hope this will help anyone who had read this post :) I will review other quite interesting new features in next posts.

     

    REFERENCES:

Docker Releases
https://github.com/docker/docker/releases

Enabling Experimental Mode
https://github.com/docker/docker/tree/master/experimental#use-docker-experimental

Vagrant Docker Swarm Mode Testing Environment 
https://github.com/frjaraur/docker-swarmmode

     

     

    Related Posts