In the first and second post of this series a brief introduction to control groups was made, the basic concepts and a little tinkering with the real thing, now I just want to see for myself that docker actually uses control groups to limit the amount of memory assigned to a real container, and if so, how does it do it.

If you find yourself in trouble trying to understand what I say, for a reason other than my ability to communicate in English, I suggest that you go back to the two posts I just mentioned in the previous paragraph.

The Containerized Memory Sink

FROM ubuntu

ADD memory-sink /memory-sink
RUN apt-get -y update; apt-get -y install stress;

CMD ["/memory-sink"]

This is a docker file that will create a docker image as follows,

• Extends the official latest tag ubuntu has gently made available on the docker hub.
• Adds the memory-sink script to the build context
• Updates the list of packages
• Installs stress
• Sets the memory-sink script as the entry point to my container, that is to say, the executable docker will run when I run my container

#!/bin/bash

memory_limit="128M"

stress --quiet --vm 4 --vm-bytes ${memory_limit} --vm-keep &>/dev/null &
while true; do
    total=0;
    for p in $(pgrep -P $!); do
        rss=$(grep "^VmRSS" "/proc/$p/status" | awk '{print $2}');
        process=$(ps -q ${p} -eo ppid,pid | grep ${p})
        total=$(($total + $rss));
        echo "$process ==> $(($rss/1024))MB"
    done;
    echo "Total ==> $(($total/1024))MB";
done

And this is memory-sink, it’s a bash script that will fork four processes, each one of them will reserve a maximum of 128MB of memory, that means a total of 512MB (128MB x 4), then it will loop indefinitely printing for each forked process, its parent process id, its process id and the amount of memory it has reserved. It’s The memory sink.

Ok, let’s build the image,

and there it is, it’s a pretty little image called the-memory-sink, now let’s run it,

hmmm, interesting, the container created a process with id 7, which in turn forked four processes as expected, the processes 9, 10, 11 and 12, and after a few loops every process reserved 128MB, as expected. Now let’s see what docker did to the memory hierarchy in the control groups file-system while my container was running,

look, there’s a new control group in the memory hierarchy, it’s called docker, let’s see what’s in there,

those are the usual input/output files of the docker control group, but there’s yet another control group within this one, it has a mysteriously long name which looks awfully familiar to a container id, so in the last command I queried info about my running container and I was rather delighted to find out that its id is actually a short version of the name of that mysterious control group, so that control group must have been created by docker exclusively for my container.

Now I want to check if this control group, created exclusively for my container, has any limit on the amount of memory it can use,

the file memory.limit-in-bytes is set to 9223372036854771712 = 2^63 meaning that my container has no practical limit on how much memory it can use, as it is now, if there was a bug in my memory-sink script it could drain all of the memory available on my system condemning me to an unwanted reboot, and I don’t want that, so let’s try to fix it. Now I will run my container, but this time asking docker to set a limit on how much memory the container can use, the docker documentation explains how to limit different resources, but this time let us focus only on memory,

by suppling the option –memory=256m docker is instructed to set a limit of 256MB on the container,

after a while my memory sink stabilizes somewhere a little below 256MB, it’s not allowed to reserve 512MB as it did before. Now let’s see what’s in the file memory.limit-in-bytes in the control group created for my new container,

it’s set on 268435456 = 2^28 = 256MB!. So docker is obviously using control groups to limit the resources my container can use, nothing like good old evidence.

Wrapping Up

What we’ve seen in this post regarding memory can be extended to the other resources that can be restrained by control groups, docker creates a control group called docker on every hierarchy and for every running container it creates a control group within (children of) that docker control group, it is so in order to allow the resources for a single container to be controlled independently from other containers while at the same time allowing docker to control the resources used by all the containers running on a single host.

Just like the two previous posts this was a well-intentioned attempt to serve as an introduction to the concepts behind control groups, I strongly recommend to go to the kernel and RHEL docs.

Here’s the source code for this post. See you around.