Deployment Challenges
Microservices have become very popular mainly due to the introduction of Docker containers and cloud resource schedulers such as Mesos. With all the advantages of modularity and distribution that they bring, microservices migrate considerable complexity from development to deployment:
- An SSO is expected for uniform security access to services and for transferring the security context in service-to-service calls.
- Services need to be packed into Docker images. Deployment instructions are essential to launch the composite application through a cloud resource scheduler.
- Persistent state through disk volumes needs to be provisioned and services have to be launched physically close to their volumes and migrated along with the volumes.
- Client-side or server-side service discovery is needed. Server-side discovery such as a dynamic DNS service (e.g. Mesos-DNS) can be used to map service instances to their symbolic name.
- Centralized and dynamic configuration is required to avoid hard-coding sensitive data into Docker images and provide a context to the composite application that depends on the deployment profile.
Central & Dynamic Configuration
We will employ Zookeeper and confd to provide context-based and deployment-time configuration to Spring Boot – based microservices.
confd Templates
A typical Spring Boot application is configured through its application.properties located in the working directory or the classpath. E.g.
# HTTP service service.instances=4 # Zookeeper zookeeper.connection.timeout=10000
This can be converted to a confd template where the base path of configuration keys depends on the activated profile. The profile name is pulled from an environment variable which renders the template friendly to Docker image parameterization:
# {{$base:= print "/profiles/" (getenv "PROFILE")}}
# HTTP service
service.instances={{getv (print $base "/http/instances")}}
# Zookeeper
zookeeper.connection.timeout={{getv (print $base "/http/zk.connection.timeout")}}
The configuration template is accompanied by a template resource config:
[template] src = "application.properties.tmpl" dest = "/opt/service/application.properties" keys = [ "/profiles/" ]
Docker Image
The Dockerfile installs confd and pulls the configuration templates and the Spring Boot jar into the image:
FROM java:8
ENV APP_HOME /opt/service
ENV APP_DATA = /var/lib/service
RUN apt-get update && apt-get install -y wget && rm -rf /var/lib/apt/lists/*
# Create user & group
RUN /usr/sbin/groupadd -g 1000 srv
&& /usr/sbin/useradd -M -d /opt/service -g srv -s /sbin/nologin -u 1000 srv
# Add confd
RUN wget "https://github.com/kelseyhightower/confd/releases/download/v0.10.0/confd-0.10.0-linux-amd64" -O /usr/bin/confd
&& chmod +x /usr/bin/confd && mkdir -p /etc/confd/conf.d
&& mkdir -p /etc/confd/templates
ADD *.toml /etc/confd/conf.d/
ADD *.tmpl /etc/confd/templates/
ADD start.sh /
RUN chmod +x /start.sh
# Create directories
RUN mkdir -p $APP_HOME $APP_DATA
ADD app.jar $APP_HOME/
RUN chown -R srv:srv $APP_HOME $APP_DATA
USER srv
CMD /start.sh
EXPOSE 8080
The start.sh script will accept a Zookeeper URL and pass it to confd which runs once to create the final application.properties before launching the executable JAR:
#!/bin/bash
APP_PATH=/opt/service
if [ -z $ZK ]; then
echo "Zookeeper URI is required."
exit 1
fi
# Convert ZK node list
IFS=',' read -ra ADDR <<< "$ZK"
for node in "${ADDR[@]}"; do
nodes=$nodes" -node="$node
done
confd -onetime -backend zookeeper $nodes
cd $APP_PATH && java -jar app.jar
All that is left is to setup the actual values in Zookeeper for a ‘test’ profile:
[zk: localhost:2181(CONNECTED) 1] create /profiles "" [zk: localhost:2181(CONNECTED) 2] create /profiles/test "" [zk: localhost:2181(CONNECTED) 3] create /profiles/test/http "" [zk: localhost:2181(CONNECTED) 4] create /profiles/test/instances "4" [zk: localhost:2181(CONNECTED) 5] create /profiles/test/zk.connection.timeout "10000"
When the Docker instance is created, we pass 2 environment variables ZK=zookeeper_ip:2181 and PROFILE=test:
docker create -t -i
-p 8080:8080
--name service
-e "ZK=$1"
-e "PROFILE=$2"
registry.modio.io/service:latest
Any of the other backend storage supported by confd, such as etcd and consul, could have been used as well. In addition confd can be deployed as a daemon that reacts to configuration updates, regenerates the configuration file and optionally restarts the Spring Boot process.
The above approach allows decoupling the Docker image from context-specific deployments and it is compatible with multi-container deployment tools such as docker-compose and distributed deployment platforms such as Kubernetes and Mesos.