Open CI/CD Systems Gaining Traction

Open Dev CI/CD conference calls for increased collaboration in developing and integrating new developer tools.
Open Dev CI/CD conference calls for increased collaboration in developing and integrating new developer tools.

At the tail end of May I had the pleasure of being invited to participate in Open Dev CI/CD. OpenDev was co-located with the OpenStack Summit. Much has been written about how the OpenStack Foundation has been changing, and this event is part of that.

Previously firmly focused on the technology stack of OpenStack itself, the OpenStack Foundation has shifted focus to instead be around open infrastructure more generally. OpenStack remains at the core of the push for companies to forego, or at least augment, proprietary cloud platforms, but they’re also now host to the Kata Containers and Zuul CI, both of which can exist independently of a strictly OpenStack ecosystem to further the adoption of open infrastructure.

OpenDev CI/CD 2018 Programming Committee

OpenDev itself was a meeting of open source projects and organizations with two specific goals:

  1. Increase collaboration between open source communities running independent CI systems.
  2. Increase adoption of open source CI/CD tooling across the industry.

With this in mind, there was a keynote from Fatih Degirmenci and Daniel Farrell of OpenCI who shared details about how the OPNFV and OpenDaylight communities and others came together to test changes against each other. Another keynote focused on tooling, with James E. Blair taking the stage to talk about Zuul CI.

CI/CD of Microservices with Containers

I also gave a keynote (slides) to put a spotlight on using containers for CI/CD pipelines. In both of these talks we didn’t focus on open source communities, but instead made a case for using these tools more generally in CI/CD ecosystems. one of the things I touched upon was how I had embarked on a project four years ago while I was on the OpenStack Infrastructure team to use containers in testing. At the time, the container ecosystem was not sophisticated enough to meet the challenges of our testing needs, most notably due to issues around container management and networking that made the tests we sought to run a poor reflection of how OpenStack was used in reality. The container ecosystem has matured considerably since then, so I was excited to show how a full CI/CD pipeline with testing could be deployed in about 10 minutes, including installation of Jenkins and GitLab on DC/OS. The demo I gave is available here.

Free Software Needs Free Tools

The keynotes concluded with a very important one from Benjamin Mako Hill titled “Free Software Needs Free Tools” where he shared the story of the Linux kernel and the proprietary BitKeeper software that was used for a time, but ended badly when a legal problem arose from the use of the software, showing that even companies that are friendly to open source communities can change over time. His answer to this was that the only way to guarantee the health and safety of your project is by using free software as a basis for all of your work, rather than relying upon proprietary software. This message is particularly valuable in light of concerns over the recent acquisition of GitHub.

Next Steps for the Open CI/CD Community

The rest of the event was a mix of talks and tutorials covering technologies like Spinnaker, security in the CI/CD pipeline, Aptomi for Kubernetes, and the general trend toward immutable infrastructure which doesn’t require “moving parts” like configuration management once the application is deployed. The event also had a series of open collaboration sessions where community members got together to talk through common problems around CI/CD. Directly following the keynotes I ran one about Continuous Delivery/Deployment Fundamentals, and there was a similar Continuous Integration Fundamentals session. In another session we discussed the use of CI to turn open source projects into actual products, something companies have traditionally been responsible for. These sessions were valuable for getting a lot of like-minded people in the same room, which is often also accomplished at DevOps events, but it was refreshing to have an event that was entirely focused on CI/CD itself.

At the conclusion of the two-day event we held a wrap-up session where we mapped out next steps for this open CI/CD community. Collaboration between open source projects via OpenCI was agreed upon, and there was desire to do more focused events co-located with communities we’ll find value in collaborating with, such as the Apache Foundation. As we move forward, I hope to see this community strengthened and the desire for companies and projects to run their own open source systems grow.