Using pre-packaged software is the quickest and most robust way to provision instances.
However, building software packages can be tedious as 1) it must be done on a clone of the instances to be provisioned and 2) it requires specific knowledge on the type of package/platform that has to be created. Also, 3) a repeatable/automatic workflow might be difficult to create.
The idea is to create a base Docker image for each OS we need packages for. Additionally, the Effing Package Management (fpm) tool, used to build packages, is included in the base image.
In effing’s own words…
... The goal of fpm is to make it easy and quick to build packages such as rpms, debs, OSX packages, etc. ... It should be easy to say "here's my install dir and here's some dependencies; please make a package" ...
With some practice, it certainly achieves its goal.
You start from a public image of the chosen target OS/architecture. In our case, we use the public image for the AWS Linux AMI and we season it with all the dependencies we require to build software.
As more software packages are built with this method, the number of dependencies in the docker base image will of course increase, but it is likely you will not have to rebuild the base image very often.
FROM amazonlinux MAINTAINER Giovanni Giupponi "email@example.com" RUN yum -y update && \ yum -y install ruby23.x86_64 ruby-devel.noarch \ git wget tar bzip2 \ perl-App-cpanminus.noarch \ openssl-devel libpcap-devel \ pcre-devel zlib-devel RUN yum -y group install "Development Tools" RUN gem install fpm
You can rearrange the docker layers creation in order to optimize the docker build process. In any case, after having empowered the base instance
to compile the target software, the effing package management tool is installed (last line).
The packages directory
We create a directory where to store the base dockerfiles and one subdirectory for each software package, containing accessory files/scripts.
$ ls packages README.md Dockerfile.packetizer_amazonlinux_base Dockerfile.packetizer_ubuntu_14.04_base collectd ossec rubies snort
This directory can be versioned in git, for example in your infrastructure repository.
For each package, we typically define a Dockerfile, an entrypoint and a build script.
$ ls packages/collectd/amazonlinux Dockerfile.packetizer_amazonlinux_collectd build.sh entrypoint.sh
The Dockerfile is responsible for the tailored compilation and installation of target software
cat Dockerfile.packetizer_amazonlinux_collectd FROM packetizer_amazonlinux_base MAINTAINER Giovanni Giupponi "firstname.lastname@example.org" ENV COLLECTD collectd-5.6.1 ENV PACKAGE_DIR /opt/package RUN wget https://storage.googleapis.com/collectd-tarballs/$COLLECTD.tar.bz2 -O- | tar -xj WORKDIR $COLLECTD RUN ./configure --sysconfdir=/etc --exec-prefix=/usr RUN make DESTDIR=$PACKAGE_DIR all install WORKDIR $PACKAGE_DIR COPY ./entrypoint.sh ./entrypoint.sh RUN chmod 755 ./entrypoint.sh ENTRYPOINT ["./entrypoint.sh"]
The entrypoint, given a docker image built from the Dockerfile, is responsible to package the software using fpm. Creating the package while the container is running permits to mount local filesystem to ….
#!/bin/bash PACKAGE_NAME=seq_collectd_5.6.1_aws_linux.rpm fpm -s dir -t rpm -C . \ -p /tmp/$PACKAGE_NAME \ --name sequra_collectd_5.6.1_aws_linux \ etc/ opt/ usr/
Each package is then created launching the build.sh script, which creates the image and then runs it.
#!/bin/bash docker build -t packetizer_amazonlinux_collectd -f Dockerfile.packetizer_amazonlinux_collectd . docker run --rm -v $PWD:/tmp packetizer_amazonlinux_collectd
When build.sh finishes, you can find the package in the current directory.
$ ls Dockerfile.packetizer_amazonlinux_collectd build.sh entrypoint.sh seq_collectd_5.6.1_aws_linux.rpm
In this post, we show how we can avoid some headaches and save time building packages with Docker the Effing Package Management.
Packages assembly is simplified by fpm, and docker allows to build them locally, without the need to spin up a remote clone instance, saving time. Also, versioning workflow files in git increases accountability and flexibility.
Generated packages can then be put into a private mirror repo to be consumed by configuration management systems. As an alternative, they can also be uploaded to S3 to be wget-ed and localinstall-ed during provisioning.