Skip to content

Latest commit

 

History

History
157 lines (112 loc) · 7.19 KB

File metadata and controls

157 lines (112 loc) · 7.19 KB

OpenShift Online Operations Docker Containers

This directory contains the OpenShift Online Operations docker containers.

These containers are used for various things, some are specific just to OpenShift Online, but in an effort to be as open as possible, we’re sharing these here.

We hope you find them useful.

Setup

The RHEL-based containers in this directory expect a secrets directory to be setup with a structure like:

/etc/pki/entitlement
└── yum
    ├── certs
    │   ├── client-cert.pem
    │   └── client-key.pem
    ├── repos
    │   ├── oo-ops-rpm.repo
    │   ├── rhui-rhel-7-extras.repo
    │   └── rhui-rhel-7-server.repo
    └── rpm-gpg
        └── RPM-GPG-KEY-redhat-release

Structure

All containers in this directory should have the following files:
  • README.adoc - Information specific to the container

  • build.sh - a script to make local building of the container(s) easy

  • run.sh - a script to make local running of the container(s) easy

Directory Structure

openshift-tools/docker
├── oso-host-monitoring/(rhel7|centos7)    # A superprivileged container that runs on the host system (includes pcp and zagg client)
├── oso-ops-base/(rhel7|centos7)           # Base RHEL/CENTOS7 container that the other containers are based on
├── oso-zabbix-server/(rhel7|centos7)      # Zabbix server container (doesn't include web, db, etc)
├── oso-zabbix-web/(rhel7|centos7)         # Zabbix webui container
├── oso-zagg-web/(rhel7|centos7)           # Exposes REST interface, collects metrics and sends them to zabbix (or another zagg)
├── oso-rhel7-zaio/(rhel7|centos7)         # Zabbix All In One container (zabbix server, webui and db). Very nice for local development/testing

The RHEL and CentOS versions of the containers are extremely similar, and the shared information in stored in the 'src' directory (ie openshift-tools/docker/oso-host-monitoring/src/*). When making changes to the files in 'src' be sure to run the appropriate 'generate_containers.yml' Ansible script so that the changes in 'src' propogate out. If you make a change to openshift-tools/docker/oso-ops-base/src/Dockerfile.j2 in the oso-ops-base container, then be sure to run openshift-tools/docker/oso-ops-base/generate-containers.yml to get the changes into the rhel7 and centos7 directories.

Note
Why are these generated files commited?

The generated files are commited to the git repo because the BuildConfigs in https://github.com/openshift/openshift-tools/tree/prod/openshift/templates (which are used to bring up and maintain a new monitoring cluster/project) need to be able to point to an already built directory structure with the Dockerfile and all its dependencies.

Here’s an example:

- apiVersion: v1
  kind: BuildConfig
  metadata:
    labels:
      name: oso-rhel7-ops-base
    name: oso-rhel7-ops-base
  spec:
    output:
      to:
        kind: ImageStreamTag
        name: oso-rhel7-ops-base:latest
    resources: {}
    source:
      contextDir: docker/oso-ops-base/rhel7
      git:
        uri: https://github.com/openshift/openshift-tools
      type: Git

Building the monitoring containers

For the containers to build properly, they should be built in a certain order.

Build order:
  1. oso-ops-base

  2. oso-zabbix-server

  3. oso-zabbix-web

  4. oso-host-monitoring

Example of building the oso-ops-base container (other containers are similar):
> cd openshift-tools/docker/oso-ops-base/centos7
> ./build.sh
[sudo] password for user: *********

Testing sudo works...

[... snip ...]

Building oso-centos7-ops-base...
Sending build context to Docker daemon 27.14 kB
Step 1 : FROM centos:centos7
 ---> 28e524afdd05

[... snip ...]

Successfully built e4f0219b978a
0.06user 0.02system 0:00.55elapsed 15%CPU (0avgtext+0avgdata 27908maxresident)k
0inputs+0outputs (0major+2358minor)pagefaults 0swaps

[... snip ...]

Caveats

AUTOGENERATED content used to build centos7 and rhel7 images

In order to be able to maintain our containers for multiple distros (namely CentOS 7 and RHEL 7), we use a templated build system. The Dockerfile and the contents of the image are autogenerated from the src directory to the centos7 and rhel7 directories.

Any changes made directly in the centos7 or rhel7 directories WILL BE OVERWRITTEN the next time the generate-containers.yml script is run. This is intentional.

If you want to make persistent changes, they need to be made in the src directory inside each container direcotry (e.g. inside the oso-ops-base directory). That is what the generate-containers.yml script uses to generate the centos7 and rhel7 directories.

When submitting a PR, please make sure to have generated the latest source files, otherwise your PR will be rejected.

Build time secrets for RHEL images (DOES NOT APPLY TO CENTOS)

Since the containers are built on RHEL, they expect to have yum credentials setup such that they can yum install packages.

The expected directory structure is outlined here.

The RHEL7 version of docker contains a secrets patch which copies in the contents of the host’s directory /etc/pki/entitlements into the container at the location /run/secrets/etc-pki-entitlement. This copy is done in such a way that it does NOT become part of the final image (which is great for build time secrets).

Unfortunately Fedora (and other distros) don’t have the secrets patch, so we have a workaround in the build script for oso-rhel7-ops-base that copies the contents of that directory into the image. This means that this docker image contains secrets and thus should NOT be shared externally. It should ONLY be used for local development.

This also means that any containers built from oso-rhel7-ops-base image should also NOT be shared, as they contain the same credentials.

Build dependencies

Docker doesn’t have container build dependencies, so it is the responsibility of the end user to build the containers in the prescribed order.

Stale builds

Docker doesn’t automatically build dependent containers, so it is the responsibility of the end user to re-build containers as they get updated in git. It is very difficult to know if a docker file, or any RPMs that a docker file installs have been updated. So, it is recommended that the end user does rebuilds on a regular basis.

All of the build scripts pass additional options down to docker, so normal docker options work.

It is recommended when doing the first build of a container, to pass the --no-cache option so that all layers are re-built. This will ensure the container is fully up to date.

Example build without using the cache
> ./build.sh --no-cache