Docker and IBM Domino (part 5)

Setting up Domino to run for the first time

Now we have Domino 9.0.1 with FP 10 installed, it's about time we ran the server for the first time and put it in listen mode. We can then connect to it with Remote Server Setup to finish off the installation.

docker build -t timsterc/domino:domino_9_0_1_FP_10 -f dom_on_docker/setup_domino/Dockerfile .

The Dockerfile this time is slightly different to the ones before, so lets go through it to make sure we understand what's going on.

FROM timsterc/domino:9_0_1_FP_10

Here we go again from the latest FixPack 10 image. Hopefully you are getting the idea now, that this is giving us lots of flexibility.

ENV DOM_CONF=dom_on_docker/setup_domino

Environment variable to define where our setup files are.

EXPOSE 25 80 443 1352

This line opens up the tcp ports 25 (smtp), 80 (http), 443 (https) & 1352 (nrpc). Normally here, as this server is not configured yet, you might expect us to open up port 8585 for when we open the server in listen mode. To stop having another port open at run time and minimize the config files we are going to make this 1352 in the docker-entrypoint.sh. See later in this blog post for that.

COPY ${DOM_CONF}/resources/docker-entrypoint.sh /
RUN chmod 775 /docker-entrypoint.sh
USER notes
WORKDIR /local/notesdata
ENV LOGNAME=notes
ENV PATH=$PATH:/opt/ibm/domino/

Here we copy in a bash script file 'docker-entrypoint.sh' that is used each time we start the image. We will cover that later.

NTRYPOINT ["/docker-entrypoint.sh"]

The ENTRYPOINT command tells Docker what to run at startup. E.g. where to start.

docker-entrypoint.sh

#!/bin/bash
serverID=/local/notesdata/server.id
if [ ! -f "$serverID" ]; then
    /opt/ibm/domino/bin/server -listen 1352
else
    /opt/ibm/domino/rc_domino_script start
    /bin/bash
fi

This is the batch file with the magic dust in it. (joking)

First we set the environment variable for where the server.id 'should' be. If it's not there we start the server in listen mode otherwise we start it normally.

So let's try this out.

STOP!!!! We don't have a data directory that will stay when we stop the image. Docker by default is not persistent, so we need to create somewhere for the domino date to live. Docker does allow you to attach a virtual disk to the image, so lets create that now.

docker volume create –name=domino_data

Now that we have the disk image, lets stick the keys in the ignition and fire this thing up.

docker run -it -p 1352:1352 -p 8888:80 -p 8443:443 –name 901FP10 -v domino_data:/local/notesdata timsterc/domino:domino_9_0_1_FP_10

Now we are creating a runtime 'container' from an 'image'.

Want to know what you just typed in? I thought so, and here comes the breakdown.

-i keeps the STDIN open even if you are not connected to the container. Think of it like 'headless mode'.
-t "allocates a pseudo TTY" according to the Docker run reference. The TTY is expected by most programs and by Domino. This also means that we can't redirect the output via a pipe, as the -t is not allowed in that scenario.
-p allocates a tcp port to the image. As you can see, it's possible to create a redirect at this point, from incoming port 8888 to internal port 80
-name is kinda obvious, but note that the container name doesn't have to be the same as the image it's created from
-v this is the data volume we created being attached. Syntax = volumename:mountpoint
imagename is the last option we specify here, e.g. what we are creating our container from

There is another option that you might want to consider here,

--hostname [hostname] will allow you to force the hostname in your container, as it's not specified in the image it gets a runtime hostname. Not so clever if you want to actually connect to your container from anywhere else.

When that's running you can now launch the Domino Remote Server Setup and connect it to the ip address of the Docker image. Ah, hang on, we have not set the ip address, so how do we know what it's going to be? I'm glad you asked as this took me a while to get my head around. It actually bridges to the ip address of the host machine. So what ever that is set to, is how you connect to it.

Good luck.

Before we leave this, you'll need just a couple more bits of information.

From the command line you can connect to your running image using this command

docker attach [imagename]

You can also start and stop the image using the same format but replacing the 'attach' option.

One last thing you need to know is that starting and stopping a container will not restart the OS and therefore not restart your Domino server. For that you either need to attach and issue a 'sudo init 6' to restart the OS or use '/opt/ibm/domino/rc_domino_script restart' to reboot the Domino server. You could always the Domino server from the console or Remote Console if you wish.

And with that, I wish you good luck in your adventures with IBM Domino on Docker.

Buy me a coffeeBuy me a coffee

Docker and IBM Domino (part 4)

Domno on Docker part 4.png

Installing FP 10 into the Domino 9.0.1 image

Into the good stuff now. Using an image we have created ourselves, timsterc/domino:9_0_1 and making a new image from it with FP 10 in it.

Why this is useful

This is the part of this whole process that I have found to be probably the most exciting. Why? I hear you scoff into your coffee. Is it just because I'm a sad geek that has finally gotten on board with the whole Docker thing? Or maybe its because I've finally found a use for something I'm playing with? It could be that I wish this had been around about 20 years ago when I was working in support. Let me explain.

You now have a base Domino 9.0.1 image to work with. That's right, you can WORK with it. Make it do what you want. And because we are adding to it into a new image, this image stays the same. Meaning we can build other stuff from it, time and time again.

Imagine having a 'store' of small, fast Domino builds with each of the Domino 9.0.1 Fix Packs in them. Sitting on the shelf and ready to go. Need to test something in Domino 9.0.1 FP6 IF3? Well even though it's not supported, what's to stop you from creating a FP6 image from the 9.0.1 image and then adding IF3 to it. Run it up, do your testing and then dump it in the trash. Total time to set up? I think, about 15-30 minutes. That's why I'm excited about this. I've clearly worked in support for too many years, but this would have saved me weeks in my previous careers.

"Ok, enough! Get to the good bit already."

Install the FP into the Domino image

So a similar process to before, build a new image using a Dockerfile.

docker build -t timsterc/domino:9_0_1_FP_10 -f dom_on_docker/domino901FP10/Dockerfile .

No need to explain this again, but note the new tag. 9_0_1_FP_10 helps us find the right image when we have loads of them to chose from.

Dockerfile explanation

FROM timsterc/domino:9_0_1

Using our existing image. ;o)

ENV DOM_CONF=dom_on_docker/domino901FP10/resources/serverconfig
ENV DOM_SRC=Dom/
ENV NUI_NOTESDIR /opt/ibm/domino/
COPY ${DOM_CONF}/ /tmp/sw-repo/serverconfig
RUN mkdir -p /tmp/sw-repo/
RUN cd /tmp/sw-repo/    
COPY ${DOM_SRC}/domino901FP10_linux64_x86.tar /tmp/sw-repo/
RUN cd /tmp/sw-repo && \
    tar -xf /tmp/sw-repo/domino901FP10_linux64_x86.tar && \
    cd /tmp/sw-repo/linux64/domino && \
    /bin/bash -c "./install -script /tmp/sw-repo/serverconfig/domino901_fp10_response.dat" && \
    cd / && \
    rm /tmp/* -R && \
    rm /opt/ibm/domino/notes/90010/linux/90010/* -R

Almost the same as before, but as you can see we are running the command to install FixPack 10 and not Domino.

When you’ve run this, and again it takes a while, you should end up with a new image in your list when you run "docker images"

REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
timsterc/domino     9_0_1_FP_10         e8dcc1a238fd        12 days ago         5.51GB
timsterc/domino     9_0_1               cbe3262a559d        12 days ago         2.73GB
centos              latest              49f7960eb7e4        3 weeks ago         200MB
hello-world         latest              e38bc07ac18e        2 months ago        1.85kB

In the final step we will get the image ready to have the remote admin client connect and then finalise the setup.

Buy me a coffeeBuy me a coffee

Docker and IBM Domino (part 3)

Domno on Docker part 3.png

Installing Domino in Centos

The first bit of this that we need to understand is the command that we are using to do the initial install. Don't run this yet.

docker build -t timsterc/domino:9_0_1 -f dom_on_docker/domino901/Dockerfile .

docker - well we've seen this already and it's how we tell docker we want to work with it
build - this tells Docker that we want to create an image from a Dockerfile
-t timsterc/domino:9_0_1 - tag list, the name that we want to call something. I have timsterc here as that is my name in docker, so if I upload it then I can find it again.
-f Dockerfile - the name of the script file that you want to run during the build
. - Don't miss this full stop as the command will not work without it. This is placing the file in the same place that Docker uses for everything.

You should change the 'timsterc' part to your own name. Keep in mind, that if you do, you will need to edit the Dockerfiles that follow to use your name and not mine. ;o)

Now let us take a look at the Dockerfile and what that is going to do during the build. I've broken it up into sections to make it easier to explain.

This is "dom_on_docker/domino901/Dockerfile"

FROM centos

This command tells Docker to take all of the contents of the 'centos' image that we downloaded earlier and use that as a basis for what we are about to do here. Really useful for adding to an existing image to make a new one. We will use this technique later to create a new image from the base Domino image and add the FixPack to it.

ENV DOM_SCR=dom_on_docker/domino901/resources/initscripts
ENV DOM_CONF=dom_on_docker/domino901/resources/serverconfig
ENV DOM_SRC=Dom/
ENV NUI_NOTESDIR /opt/ibm/domino/

These are just environment variables that we use later in the script. An easy way to config the script file without having to trawl through each line of code.
DOM_SCR and DOM_CONF are the locations of the scripts to run Domino (thanks to Daniel Nashed for those) and the server response file for the silent install.
DOM_SRC is the local directory where you copied the Domino 9.0.1 for 64bit Linux image from Passport Advantage.
NUI_NOTESDIR defines the location you want to install Domino into in the CentOS image.

RUN yum update -y && \
    yum install -y which && \
    yum install -y nano && \
    yum install -y wget && \
    yum install -y perl && \
    useradd -ms /bin/bash notes && \
    usermod -aG notes notes && \
    usermod -d /local/notesdata notes && \
    sed -i '$d' /etc/security/limits.conf && \
    echo 'notes soft nofile 60000' >> /etc/security/limits.conf && \
    echo 'notes hard nofile 80000' >> /etc/security/limits.conf && \
    echo '# End of file' >> /etc/security/limits.conf

This next section updates CentOS to the latest updates, adds the Notes user and sets the soft and hard file limits for Domino

COPY ${DOM_CONF}/ /tmp/sw-repo/serverconfig
RUN mkdir -p /tmp/sw-repo/ && \
    cd /tmp/sw-repo/
COPY ${DOM_SRC}/DOMINO_9.0.1_64_BIT_LIN_XS_EN.tar /tmp/sw-repo/

The section above creates a temporary directory and copies over the Domino tar ball to it. The && \ concatenates the lines together and is removed at runtime. Great way of splitting up really long command lines into readable parts.

RUN cd /tmp/sw-repo && \
    tar -xf /tmp/sw-repo/DOMINO_9.0.1_64_BIT_LIN_XS_EN.tar && \
    cd /tmp/sw-repo/linux64/domino && \
    /bin/bash -c "./install -silent -options /tmp/sw-repo/serverconfig/domino901_response.dat" && \
    cd / && \
    rm /tmp/* -R

I found while messing about with this script file to use a local file and not one from a web server, that every command runs in its own environmental space. This is why we have to change the directory to our temp directory again in the next command, otherwise, the command runs with a new space and is in the wrong directory.
Next, we extract the install code inside the image, then run the Domino install with the server response while we copied in earlier. Once we are done with the install (which does look like it's hanging while it works) we remove all the temporary files to keep the image clean and as small as possible.

RUN mkdir -p /etc/sysconfig/
COPY ${DOM_SCR}/rc_domino /etc/init.d/
RUN chmod u+x /etc/init.d/rc_domino && \
    chown root.root /etc/init.d/rc_domino
COPY ${DOM_SCR}/rc_domino_script /opt/ibm/domino/
RUN chmod u+x /opt/ibm/domino/rc_domino_script && \
    chown notes.notes /opt/ibm/domino/rc_domino_script
COPY ${DOM_SCR}/rc_domino_config_notes /etc/sysconfig/

Lastly, we set up the scripts that run Domino in the CentOS image, making sure that the Notes user can run them.

So now we understand what's happening, go ahead and make sure you are in your docker folder and then run this command.

docker build -t timsterc/domino:9_0_1 -f dom_on_docker/domin901/Dockerfile .

Don't forget that if you change the tag names, you'll need to change them in the follow-on scripts. I'll highlight these when they come up.

This can take a while and it will look like it's hung when it gets to the Domino install bit. Don't Panic, just have a little faith. If you have a slow machine or it goes past 30 minutes, you could have a problem. You should be fine though, it just takes a while.

Once this has run, you can try the follow Docker command to list out your images.

docker images

The results should look something like the below.

REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
timsterc/domino     9_0_1               cbe3262a559d        12 days ago         2.73GB
centos              latest              49f7960eb7e4        3 weeks ago         200MB
hello-world         latest              e38bc07ac18e        2 months ago        1.85kB

Next time we will follow a similar procedure to install FixPack 10.

Buy me a coffeeBuy me a coffee

Docker and IBM Domino (part 2)

Domno on Docker part 2.png

So how do we get started now that we've been tempted by the sales pitch?

Install Docker

First off, if you have not already, go and get Docker from the Docker website. https://www.docker.com/community-edition#/download

Test Docker

Once you have it installed, go to a command line for your platform and try the following.

docker run hello-world

You should get a response something like this.

Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
9bb5a5d4561a: Already exists 
Digest: sha256:f5233545e43561214ca4891fd1157e1c3c563316ed8e237750d59bde73361e77
Status: Downloaded newer image for hello-world:latest
Hello from Docker!
This message shows that your installation appears to be working correctly.
To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub. (amd64)
 3. The Docker daemon created a new container from that image which runs the executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it to your terminal.
To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash
Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/
For more examples and ideas, visit:
 https://docs.docker.com/engine/userguide/

Yay! It works.

Now you can do something crazy, like download the whole of latest CentOS image with this command.

docker pull centos

That will save us about 10-30 seconds later on. ;o)

So we know that we have Docker running and that we can get to the Internet with Docker, pull and store images and run them as containers. That wasn't so hard. Was it?

Domino in Docker

Let's start thinking about how this will work with Domino. Most people that have done this already have gone down a route that looks a lot like partitioned servers on a single machine. That's the closest analogy I can come up with, in the Domino world. The Domino server code is in one place and the data would be stored in a separate disk volume. You can have multiple volumes that could be for the same container of Domino or they could be for different servers/customers.

First, we will need the base version of Domino 9.0.1 for 64 bit Linux. This is available from the IBM download sources with part number (CIQ7ZEN)

Second, you want to get Feature/Fix Pack 10 (CNQ2HEN) (https://www.ibm.com/support/fixcentral/swg/selectFixes?product=ibm%2FLotus%2FLotus+Domino&fixids=DominoServer_901FP10_Linux64&function=fixId&parent=ibm/Lotus)

Just place them in a directory that you can get to. For my machine (Mac) I have created a directory called 'docker' from my user directory. Under that, I have two subdirectories; "dom" and "dom_on_docker"

"dom" contains the server code and FixPack that we downloaded, still compressed.
"dom_on_docker" has the scripts in it that I downloaded from Ulrich Krause's blog post about this very subject.

Whilst you can download Ulrich's scripts I would recommend you take my updated scripts.
a, Because I have updated them to work with FixPack 10, and
b, They also don't rely on you having the code on a web server, which I don't.
My scripts can be downloaded from here --> TimsterC Scripts

In the next blog post we'll go through the script and explain the command you need to run it in Docker. 

Buy me a coffeeBuy me a coffee

Docker and IBM Domino (part 1)

Domno on Docker part 1.png

Steep learning curve

On day one I find myself in the Docker departure lounge and I'm still trying to work out why I'm travelling by ship when I could be flying on a Virtual Machine. As my journey into Docker land continues I find lots of good reasons to be here and wonder why it's taken me so long to travel this way.

What is Docker?

It's a question I have asked myself for the past couple of years and never had the time or inclination to find out. Now I have a potential customer using Docker to run IBM Domino and one of BCC's products, I figured I should see if this is going to work before I try and talk knowledgeably to the customer about it.

Here is my take on what Docker is and why it's not a Virtual Machine that we’ve been used to in the past.

  • As any reference material that you search for on <<your fav search engine>> you don't need the whole operating system for most things you want to do in a VM, but you have to run the whole lot. Not with Docker! You only end up with what you need in the final 'thing' you run. (I'll cover what things are called later)
  • It's a lot faster than a whole VM. "Yeah Duh! Thanks for that blinding inspiration Einstein." The images below show a comparison between a Docker "Container" on the left and a traditional VM on the left. The container for my resulting CentOS 7 with Domino 9.0.1 FP 10 server is about 5.5GB. Compare that with a full blown VM for the same of at least 20-30GB. 
Container@2x.png
VM@2x.png
  • Names can cause confusion until you 'get the concept'. So here is a little story to explain the concept of Docker, Containers and Images.
    I wanted to move to a new house from one town to another but I had a lot of stuff to move. When I went to the moving company they asked me what I needed to move, and could they provide me with the right sized box for the stuff I have to move. I asked for 10 large boxes, 25 medium, 50 small and 10 wardrobe boxes. We also arranged for 3 large wooden crates, 2 for my motorcycles and 1 for my pedal cycles. Two weeks before the move I got delivery of all the boxes, but they were flat packed to save space. I got a box full of packing tape and marker pens to set everything up with.
    We merrily went about the house taking flat boxes to all the rooms, unfolding them and taping the bottom to make sure it was secure. We'd write the name of the room and what was going in that box with the marker pen. When we had filled the box with 'stuff' it was taped shut for the movers to come and collect and load into the truck the day before we moved.
    • Docker is like the moving company that can provide the right stuff when you need it.
    • Images are the boxes that were supplied in flat pack form. This is like downloading a image from the Docker store. Unfolding the box is like uncompressing the image on your local system and then you fill it with the right code to do your task. 
    • Containers are like the boxes that you have filled, labelled and sealed to be moved. They are specific to the task, the right size and only used for what you have packed inside them.
  • Another reason I'm smitten with this setup is because I've effectively been given a 'snapshot' of each step of the process of creating an Image that I can turn into a container.
    Let me expand this theory a little.
    • Create a base CentOS image. Now I can base any further images or containers on that one and I never have to 'build' it again. (200MB)
    • Create a new image from the CentOS image and install Domino 9.0.1 into it. (2.73GB)
    • Create another new image from the Domino 9.0.1 image and install FP10. (5.5GB)
    • Lastly, create an image from your Domino 9.0.1 FP10 image that is configured to start up or wait in setup mode if it's not configured.
      This now allows me to create a new Domino server container in about 5 minutes apply whatever fixes I want to it and cookie cut that out into as many running images as I need. 

This is great if you want to run up multiple servers quickly, in a small environment and tweak them all at run time. I can maintain one 'server code base' and all instances running from it will be the same as soon as I refresh them with a single command for each one.

What about if I want to have one image running the http task for my Domino environment and one image just running the indexer or database server parts of Domino? Well that may well be coming in the future, we'll have to wait an see.

Next time I'll cover how I got started, 3 times, and why Docker on Mac seems to be pointless. (I hope someone can prove me wrong)

Buy me a coffeeBuy me a coffee