Latest Docker level available for Linux on z Systems

The Docker on z anchor page on developerWorks has been updated for the latest Docker binaries: version 1.9.1 is available now for Linux on z.
Please note you could run into issue 16256, at least we have seen some occurrences. A workaround for that is to start the daemon with the parameter "--exec-opt native.cgroupdriver=cgroupfs".

Fedora now comes with Docker

Earlier this month, Fedora 23 has been announced for z Systems. Note it comes with Docker packages (e.g. docker-1.7.0-22.gitdcff4e1.fc23.s390x.rpm in the download directory).

DockerCon EU

Should you be at DockerCon Europe 2015 in Barcelona, don't miss to stop by at the IBM booth. I will be there, too (for a cool 10,000 containers demo that the Linux on Power folks prepared, but if you start talking about z Systems, I will of course not stop the discussion).
See you there...

Another Base Image for Docker on z Systems: ClefOS

Neale Ferguson from Sine Nomine Associates has uploaded another base image: ClefOS. ClefOS is technically a CentOS port to z Systems, but since it is not a part of the CentOS organization, it's called ClefOS. The image is brunswickheads/clefos71-base-s390x (available tags listed here), so

docker pull brunswickheads/clefos71-base-s390x

will get you the image. It is yum-enabled, so you can easily install additional packages.

The user page https://hub.docker.com/u/brunswickheads/ provides a pile of nice images on top of ClefOS, so there is more software to play with... worth while checking out!

Docker 1.8.3 on developerWorks

The Linux on z Systems Docker page on developerWorks now carry links to new binaries -- version 1.8.3 is available for SLES 12 and RHEL 7.1.

docker-machine on z Systems

Recent posts have shown how to get compose, swarm, and the registry working. Of the prominent docker tools, the last one missing is docker-machine. machine is a tool that turns a Linux server into a Docker host. It can even create the server in the first place, e.g. in various public Cloud environments or through OpenStack. https://docs.docker.com/machine/ is the starting point of the documentation.

Let's see how docker-machine can be adapted for Linux on z Systems. Caution: long post.
We will use the generic driver which uses an existing Linux image as starting point, and uses its ssh access to make it a Docker host. First step -- let's setup the build environment for machine.

Container Patterns

Containers can be deployed in several ways, depending on goals and usage scenarios. Let's have a look at typical patterns and view this in light of z Systems:

New Docker Binaries on developerWorks

The Docker page on developerWorks provides links to new Docker binaries. They are now at version 1.8.2 and provided for SLES 12 and RHEL 7.1.
Building this took a while: for proper operation, two fixes from glibc (1 and 2) would be required. However, installing a modified glibc on your system is not required, since the Docker binaries come with code working around the defects in glibc.

Docker and z/VM or KVM?

At LinuxCon, KVM for z has been announced. From a Docker perspective, what does KVM change over a z/VM environment? There's a pretty short answer: nothing. KVM is cool for a number of things, z/VM is still king of the road, PR/SM still has got some bragging rights, and Docker doesn't care, as long as a Linux runs in it. Docker under both z/VM and KVM benefits from complementing technologies.

Closing Another Base Image Gap

Most recently, I mentioned (here and here) how to best deal with the fact that Ubuntu is widely used as base image, but does not exist on z Systems. A sufficiently close Debian is a decent solution for that.

Today's announcement at LinuxCon shows how this will get even easier: Ubuntu will be made available on z Systems. This will make all "FROM ubuntu" images no-brainers.

Image Decomposition

Looking at the swarm image we built previously, it's a couple of hundred MB in size -- for a small application. This image size will not slow down execution, since only files are touched which are needed when the application starts, but it will still be offending an aesthete's eye. Specifically for mainframers who have tried to save on every bit, because their core memory only had a couple of them (my knowledge is entirely anecdotal...). How can we kill that pain?

Portability Series: A Go Environment

Previously we looked at the portability of container setups. One of the observations is that a lot of containers are based on a Go environment -- take swarm or the registry. On x86, that usually means golang, but we have gccgo on s390. Let's see, how we can deal with this difference.

Portability Series: Usage of a Debian Base Image: compose

So we have created a Debian base image and are looking at some of the steps to run containers which may have platform dependencies. An example which brings this together is one of the Docker ecosystem components: compose. Docker compose lets you define applications which consist of several containers. You can the manage your entire multi-container application with single commands.
Let's get the code for compose:

Portability Series: Another Base Image

In the previous post, we looked at various challenges when moving existing (read: x86) container setups onto z Systems. One challenge is the widespread use of Ubuntu as Docker base images. The closest thing on z is Debian having roughly the same packaging structure and naming.

Let's have a look how we can easily get to such an environment.

A Look at Docker and Portability

Often enough, Docker is mentioned in one breath with portability. However, there are two dimensions of portability: first, running a container on any system of the same platform. Here, Docker really simplifies deploying applications on any system (of the same instruction set architecture). From a Linux on z Systems perspective, the second aspect of cross platform portability is more interesting: portability across platforms. Often enough, getting a container to run on z is not a big deal, but sometimes it is. Let's decompose this:

The Mainframes of Real Containers

In some time off, I was able to study some serious container setups. Here are some impressions of the mainframes of real world containers (so you might be able to figure out where I have been to).
IMHO it becomes apparent that large scale deployments provide the best economics for container service providers, and the entire world's economy relies on it.

No posts in June

Will be back with new postings in July. Have fun with containers running Linux on z Systems.

Public base images

https://registry.hub.docker.com/search?q=s390x shows base images available on Docker's public registry. Simply "docker pull <image>" to download them.
If you need to use a proxy, you can set the environment variable http_proxy (set it to http://proxy-address:port/) when starting the docker daemon.

Docker 1.6.0 builds for Linux on z Systems

http://www.ibm.com/developerworks/linux/linux390/docker.html has been updated and contains links to binaries of the new 1.6.0 version of Docker.
Installation/updates are simple: copy the binary into /usr/local/bin or a preferred location in your $PATH.

Drilling Holes Into Containers

Container management software is often deployed in containers itself. But how can code running inside a container look at other containers? Let's have a look how components like cAdvisor and datadog agents can gather that sort of data.

Windows Containers in Linux on s390x

Some experimentation really made it work:

New Docker binaries available for Linux on z Systems

There are new Docker binaries available for Linux on z Systems, fixing a segmentation fault. Check out http://www.ibm.com/developerworks/linux/linux390/docker.html. Replacing is simple: extract the archive and copy the binary over your old one. Please make sure you are on a recent devicemapper level.

Creating Base Images

In First Steps with Docker, we have used a fairly crude way to create initial base images. A minimalistic approach to come to a base image environment is to have only the necessary rpms installed through on-board tools. This assumes the system is enabled for accessing all packages online.

Inspired from https://github.com/docker/docker/blob/master/contrib/mkimage-yum.sh, this script works both for SLES 12 and RHEL 7:

Virtualization vs. Containerization

Containers provide isolated runtime environments for applications: the entire user space environment is exclusively presented to the container, and any changes to it do not impact other containers' environments. To provide this isolation, a combination of OS-based mechanisms is used: Linux name spaces are used for isolation and scoping mechanism. File system mounts define what files are accessible to the container. cgroups define resource consumption of containers. Still all containers share the same OS kernel which can realize memory footprint efficiencies when identical libraries are used by multiple containers.

With system virtualization, the hypervisor provides a full virtual machine to a guest: the entire OS image including the kernel is now dedicated to the virtual machine. CPU virtualization is used to provide each guest with an exclusive view of a full system environment, and these mechanisms also ensure isolation from other guests. Hypervisor-based management of virtual CPUs, memory and I/O devices is used to define resource consumption of guests.

Which one is better?

Dockerfiles

Starting a container, doing manual changes to it and committing into a new image is error-prone and does not scale for handling many images. In this post, I will show a powerful and more precise way of shaping new images.

This method uses a file called Dockerfile to control how new images are built. It is placed in a directory and consists of keyword-parameter lines. Let's take a look:

[root@r1745042 ~]# cat apache/Dockerfile
FROM rhel7-yum
RUN yum -y install httpd
COPY index.html /var/www/html/index.html

First Steps with Docker

Docker is platform-agnostic and relies on Linux kernel functionality which is present on most platforms, including s390x (IBM z Systems). Its commands are identical on every platform, so there are lots of resources on the Internet which work for z Systems z.
However, Docker only executes native z code and does not emulate anything: you need to work with containers which contain s390(x) binaries, not e.g. x86 binaries.

This post shows how containers are created, used, and modified. For all of these commands, an online help is available: docker --help shows an overview of all commands, and docker COMMAND --help shows more specifics on the specified command.

Docker

Docker provides a comprehensive tool to create, deploy, manage and run containers. In a normal setup, it bases on container technology as well as on Linux control groups and copy-on-write file systems to achieve good manageability and efficiency during execution of containers. Docker distinguishes between images and containers: images are a set of files used to run a container instance. A container is a runnable instance of an image.

What Are Containers?

A container provides an envelope for running an application. Only resources assigned to the container are visible inside the container. All containers running in a system use the same OS kernel, but they don't have visibility to each other. A containerized application usually comes as a set of self-contained files, eliminating all dependencies into the host OS environment.

Welcome

Welcome to my new blog. The goal of this site is to regularly share thoughts on container technology and what it means to IT infrastructure, specifically with Linux on z Systems. Please visit regularly or subscribe, share this page and feel free to contact me at utz.bacher@de.ibm.com.