In this episode of Semaphore Uncut, Alexis Richardson, CEO and co-founder of Weaveworks, shares insights on how GitOps is reshaping the industry. We talk about why Weaveworks was founded, the problems it solves, and how GitOps best practices emerged from their work. We follow the evolutionary path all the way to the company’s new GitOps Toolkit. Alexis also touches on ways one can contribute to the community and ongoing open-source projects.
Prior to Weaveworks, Alexis co-founded RabbitMQ and was also the Technical Oversight Committee (TOC) chair at Cloud Native Computing Foundation (CNCF) for three years.
We talked about:
- Weaveworks: minimizing Kubernetes complexity through automation
- The history and essence of GitOps
- The GitOps conversation kit
- The GitOps toolkit
- GitOps Days: call for contributors
Listen to our entire conversation above, and check out my favorite parts in the episode highlights!
You can also get Semaphore Uncut on Apple Podcasts, Spotify, Google Podcasts, Stitcher, and more.
Like this episode? Be sure to leave a ⭐️⭐️⭐️⭐️⭐️ review on the podcast player of your choice and share it with your friends.
Highlights from this Episode
Darko (00:00): Hello, and welcome to Semaphore Uncut, a podcast for developers about building great products. Today, I’m excited to welcome Alexis Richardson. Alexis, thank you so much for joining us. Please go ahead and introduce yourself.
Alexis (00:18): My name is Alexis Richardson. I am the CEO and co-founder of Weaveworks, a company specializing in Kubernetes and GitOps, the operating model for Kubernetes applications. We are an international company hiring today, remote or London, SF, Colorado, Berlin. Previously, I was the CEO and founder of RabbitMQ, and I led the spring team for a while at Pivotal. I also was the TOC chair for CNCF for three years.
What is Weaveworks?
Darko (00:51): Can you tell us a bit more about Weaveworks? What was your initial mission, how did GitOps develop over time, and what are you most known for?
Alexis: Today’s software and technology are at the heart of the business. We’ve seen how companies like Google and Facebook, but also Amazon, Twilio, and Airbnb, have introduced online ways of doing business or interacting with friends and family. Traditional businesses like big banks or travel companies or entertainment companies have then copied them. Technology is at the heart of everything that people are doing. And I see that just continuing to grow.
There’s a big change happening, which is that developers are becoming more empowered. One of the most exciting jobs in the industry is working out what are the tools to make those developers incredibly productive building applications. Myself and my co-founder Matthias, who I co-founded RabbitMQ with, saw that people were using containers to build Platform-as-a- Service, and we could see that the containers were very useful for this, but difficult to use for developers. We decided that there was a gap, that developers needed help to build applications with containers.
We started Weaveworks with the vision of enabling application developers to build applications and manage them with containers. That was a great vision, but back in 2014, everything was even more of a mess than it is now, with so many companies, technologies, it was very confusing, not many things worked. Some applications on multiple machines didn’t work because you couldn’t network containers easily. Therefore, we decided first to solve that problem. We wrote a networking solution and a software-defined network called Weave Net for Docker containers.
We also decided to create some other useful tools like something for monitoring and managing your apps, regardless of which orchestrater that we’re using, that was called Weave Scope. And we started to think about how we were going to engage with customers commercially using these technologies. And we wanted to write a SaaS, Weave Cloud, which people are still using today.
The Weave Cloud SaaS is a combination of management, monitoring, and deployment. And we thought those would all be things that people needed when they were running an application.
How Did GitOps Emerge?
Alexis (6:42): But here’s the problem that we had, how are we going to run the SaaS on the cloud? So, we started to write our Docker-based system for running it. And we very soon discovered that writing your own orchestration tool is not necessarily a productive use of a small startup’s time. Thus, we decided to switch to a third-party tool: Kubernetes, that came from Google, which we reached because of a process of elimination.
And so we started using Kubernetes, and we found it initially to be very difficult to use. It was inherently complicated in ways that I think could still be simplified. We found that it was useful to work in a very specific way where you had a complete description of your staff, your whole system, the cluster, the apps, the services, the networks, the monitoring screens, the dashboards, the alerts, everything was described in one place, which was Git and the associated image repos, but Git was really where the configuration was held. And so we could have a YAML based description of our entire system, and this meant that we could shut it down and reproduce it at will. And when we had a disaster, we could recover from our complete snapshot that was in Git. We managed upgrades this way, application rollouts, rollbacks, the lot.
What’s the Essence of GitOps?
Alexis (10:16): The essence was to have a complete description of your system from day zero, make everything work in this way and bake security and monitoring into that, too. We came up with the name, GitOps, because fundamentally we are doing operations. There is this concept of reconciliation, this convergence orchestrated loop that is powered by Kubernetes and agents and operators running in Kubernetes that drives the system always to the correct state, regardless of whether it’s a cluster or an app or a fleet as well.
Yes, there are better ways of having user interfaces and other methods of triggering changes than through Git. Still, fundamentally the idea of Git as a system of record for the true state of your system or the true desired state, the one that you want it to be in, is incredibly powerful because Git has this really strong security apparatus around it. We think it’s a remarkable tool for powering operations and a big step forward from just using it for development, and we couldn’t be more excited about it.
GitOps Workflows and Tools
Darko (12:29): Many open-source tools are supporting GitOps workflows, and Flux is one of those. Can you maybe share a little bit about the journey of GitOps, the tools that you created, and what’s perhaps the future for some of them?
Alexis: First of all, let me say that if you want to do GitOps, please get in touch with us. Feel free to drop me an email at alexis@weave.works, or just send an email to help@weave.works. If you go to our website, look for the GitOps conversation kit. That was something that we created after doing GitOps Days. If you can’t find that, email me, I’ll send you a link or go on our Slack: slack.weave.works. You’ll find people who can help you.
Now, to your question, let’s talk a little bit about the open source tools. We have a bunch of open-source tools that we have created, but we are not alone. There are other GitOps open source tools in the industry, some of which overlap with the tools that we’ve created and some of which are different. I think of Terraform as being a GitOps type tool, although it’s organized around provisioning machines. Then you have Kubernetes, which I think is the first modern GitOps tool in that it uses immutable infrastructure, containers, to orchestrate to a convergent state.
We wrote Flux (Flux CD), our deployment tool for our SaaS stack, and then Flagger, a GitOps style tool for managing canary rollouts based on a configured description of how that management should occur. There’s a tool called Flux Helm, which lets you take a Helm chart as your configuration and use that to drive operations. So you can be a Helm person and do GitOps very easily.
GitOps Toolkit – Easy to Get Started and Powerful for Enterprises
Alexis (16:16): The one that we’ve recently launched is very exciting, though. That’s called the GitOps Toolkit, and it was created in response to customers and users asking for things that Flux couldn’t quite do. One of the biggest asks to Flux was, “I want to do Flux, but I need to change one thing. Can you make it more composable?” Another really big ask was, “I want to do complex workflows where things need to happen in a specific order.”
Today, if you want to install a cluster and a mesh and an application and do it securely, you have to make the mesh be up and running before you start deploying the application. Otherwise, application security won’t work with the mesh. So, GitOps Toolkit introduces dependency management. Also, there are other obvious requests like role-based access control, multi-tenancy, fleet management, cluster management. All these are supported in the GitOps Toolkit. Plus, we’re thinking of some integrations with Flagger and with third-party tools like Spinnaker, Argo CD, and the GitOps Engine that we helped write. Also Jenkins X and Jenkins, GitLab, GitHub, Bitbucket.
You can read about GitOps Toolkit on the Flux CD website. There is also a presentation on Weaveworks’ SlideShare – it was presented at Cloud Native Nordics. That was the open-source piece that we’ve done most recently that I’m really excited about. This is going to change GitOps in terms of making it much easier for people to get started with and extremely powerful for enterprises.
The Future of Flux and the GitOps Engine
Darko (18:34): In regards to Flux and Argo potentially merging – it seems that something changed and that Flux is continuing its life on its own. Is that right?
Alexis: That’s sort of 75% correct. So situation there is that we worked with Intuit and Amazon to come up with a joint roadmap around Argo CD and Flux in which we would create something called a GitOps Engine, which would be something that could be hooked into Flux and also sit under the Argo CD, but we didn’t say that Argo CD and Flux would merge into a single user experience.
We learned from the GitOps Engine that although it was good for integrating Argo and Flux generation one, we still needed fundamental new features that were missing, which we put into the GitOps Toolkit.
The next step for us is to talk to the GitOps Engine team and the Argo team and see if we can come up with the right set of integration points so that the GitOps Toolkit can drive Argo CD, just as it will drive Flux and other things like Spinnaker and GitLab, GitHub etc, to give people a GitOps pipeline. There’s a lot of work to do. If you’re listening to this podcast and want to contribute either as an open-source contributor or as an employee, please get in touch.
GitOps Days – What Happened and What’s Next?
Darko (20:24): One last question. Recently, there was a conference – GitOps Days. Can you maybe share some experiences? Is it also happening next year or potentially another edition later this year?
Alexis: GitOps Days was a two-day online conference. We had 2000 people, I think. All of the content is now online in YouTube videos, SlideShares, and the GitOps community site that we’ve created with the conversation pack on it, which I highly recommend people look up. The plan is to do a GitOps Days Europe. I can’t give you a date yet, but I think it’s something that we’re working towards.
And if you want to be involved, this is a good time to follow up on that. We want interesting stories from real-world users; it is about the community. We want to hear from you. It’s an open community forum for people to talk about what they’re doing.
Darko: Thank you for sharing all this with us, and good luck!
