Product, Use Cases

Doing continuous delivery on the DCOS with CircleCI

Nov 06, 2015

D2iQ

D2iQ

3 min read

 
This post was written by Abhishek Amralkar of Talentica Software.
 
In this post, we explain how to use the CI/CD environment CircleCI with Mesosphere's Datacenter Operating System (DCOS). We are using DCOS for our testing environment and it works seamlessly. Our aim is to deploy our containers as Marathon apps in DCOS, and we use CircleCI for continuous integration. CircleCI creates the Docker images and pushes them into a private Docker registry.
 
One of the manual tasks that we were doing was to update Marathon apps manually after code changes. However, it turns out that we can automate this process via CircleCI as well.
 
So, to start off, we assume that you are running your DCOS on Amazon Web Services, as we will use Amazon S3 to store our JSON and shell script. In turn, CircleCI will download those at runtime and execute it.
 
Setup
 
The first step is to append the following lines in the dependencies section of your circleci.yml file:
 
Dependencies:  Pre:    - pip install awscli    - aws s3 cp s3://aaa-tools/scripts/marathon.json /home/ubuntu/dcos/;    - aws s3 cp s3://aaa-tools/scripts/put.sh /home/ubuntu/dcos/; chmod a+x /home/ubuntu/dcos/put.sh    - mkdir -p /home/ubuntu/dcos
 
Make sure to replace the S3 bucket path with your own S3 bucket name and file name. We use the following conventions:
 
  • aaa-tools is the S3 bucket
  • marathon.json is the Marathon app spec file
  • put.sh is the shell script that updates Marathon via its HTTP API
 
Our marathon.json file looks as follows:
 
{    "id": "aaa-notifier",    "cpus": 0.4,    "mem": 256,    "uris": ["hdfs://namenode1.hdfs.mesos:50071/artifact_store/.dockercfg"],    "ports": [],    "instances": 1,    "container": {        "type": "DOCKER",        "docker": {            "image": "docker-registry-000-staging:443/aaa/aaa-notifier:latest",            "network": "BRIDGE",            "portMappings": [{                "containerPort": 0,                "hostPort": 0,                "servicePort": 0,                "protocol": "tcp"            }],            "forcePullImage": true        }    }}
 
… and put.sh would contain:
 
#!/usr/bin/env shMARATHON=http://master-001-dcos4.aaa.net:8080MYAPP=aaa-notifier
 
cd /home/ubuntu/dcosfor file in *.jsondo  echo "Installing $file..."  curl -X PUT "$MARATHON/v2/apps/$MYAPP" -d @"$file" -H "Content-type: application/json"  echo ""
 
Now, copy the above two files (marathon.json and put.sh) to the S3 bucket. Then, append the below code snippet at the very end of circle.yml file in the docker section:
 
- bash /home/ubuntu/dcos/post.sh
 
After CircleCI triggers the build, it automatically updates the Marathon app and you're done!
 
How to verify?
 
We can verify by using the version number; after the build succeeds, we will get the following message in CircleCI:
 
Installing marathon.json.{"version":"2015-11-04T05:56:03.996Z","deploymentId":"9772b7a4-8870-479d-bfac-a25c93da90a5"}
 
Then, in the Marathon UI check for the app version:
 
2015-11-04T05:56:03.996Z
 
This should correspond with what you get from CircleCI.
 
In this post I walked you through setting up CircleCI with the DCOS and automating the entire CI/CD process. I hope it is useful for you and if you have any suggestions or questions, please leave a comment, below.

Ready to get started?