GitOps Primer

Originally posted on SogetiLabs:
https://labs.sogeti.com/gitops-primer/

With the rise of cloud, cloud-native and a lot of other buzzwords, we also see the rise of all kinds of Dev(fill here)Ops practices. While I’m an avid advocate of the DevOps mindset, all the buzz around pipelines sounds old-school to me.

What is GitOps?

GitOps evolved from DevOps. The specific state of deployment configuration is version-controlled (using Git for example). Changes to configuration can be managed using code review practices, and can be rolled back using version-controlling.

https://en.wikipedia.org/wiki/DevOps#GitOps

A quick look at Wikipedia brings you the above description. It is stuffed away somewhere in the DevOps article. Like with DevOps, there is a whole religion around what tool is the best to use, and people even claim to be ‘DevOps Engineers’ whatever that may be. In most cases they mean ‘cloud infrastructure’ because DevOps is a mindset, and not a function.

It doesn’t help that some tools like Azure DevOps are actually called this way, but hey, VSTS is also not really a snappy name IMHO.

To explain GitOps in my own words and in a single sentence: Instead of pushing your code and deploy, your system pulls the software from the version control system.

A nice one-liner by WeaveWorks is: GitOps is Continuous Delivery meets Cloud Native

https://www.weave.works/technologies/gitops/

GitOps versus DevOps Pipelines

So, it is the same as DevOps Pipelines right? No it is not. From a distance it can seem to be the same. In a way your code, if configured correctly, will be deployed automatically with CICD.

The fundamental difference is that your system (for example Kubernetes) and not your DevOps CICD pipeline, is in the lead. Remember, the settings of the k8s system itself also need a repository.

Let’s look at a typical CICD setup:

DevOps workflow with containers
Source: https://cloudblogs.microsoft.com/opensource/2018/11/27/tutorial-azure-devops-setup-cicd-pipeline-kubernetes-docker-helm/

As you can see in the example above, the Azure DevOps pipelines are in the lead. The Build step pulls in the code, the Release step creates the artifact and pushes it towards a registry, and also installs it on AKS. But this does not include the state of your cluster.

Let’s say your DevOps pipeline trigger failed, or someone made changes manually to the AKS applications or configuration. AKS has no idea where the code is located so it cannot check if there are any changes. This is where GitOps can be a solution. Let’s look at the following example:

GitOps CI/CD architecture
Source: https://docs.microsoft.com/en-us/azure/azure-arc/kubernetes/conceptual-gitops-ci-cd

In the example above you can see that (in this case Flux) is used to pull the desired state towards your Kubernetes cluster. This pulling makes sure that you are always up-to-date. And it can also detect changes so if anything or anyone changes the configuration manually, this ‘drift’ will be resolved by the desired state pulling with Flux!

Yes, this complicates your setup, but you always know that your clusters are exactly how you described them in the desired state. Especially in situations with Kubernetes, deployment and configuration can be really complex, so these tools make sure you are in control of your systems.

Example of GitOps tools

So I need Kubernetes? No. In theory any system can use the GitOps setup, but for Kubernetes a lot of tools and so called operators are readily available.

In the previous paragraph, Flux is drawn in the picture. There are many more tools out there, but we see convergence on two major tools: Flux andArgo. The resources section at the end of the blog shows more tools and resources if you want to dive in the deep end of GitOps.

There is a nice writeup (sponsored by Red Hat) on deciding between Flux and Argo. Depending on your situation it can fall either way. GitOps on Kubernetes: Deciding Between Argo CD and Flux – The New Stack.

OpenGitOps

The emergence of GitOps led to the OpenGitOps project.

OpenGitOps is a set of open-source standards, best practices, and community-focused education to help organizations adopt a structured, standardized approach to implementing GitOps.

https://opengitops.dev/

The principles of GitOps are four-fold:

  1. Declarative
    system managed by GitOps must have its desired state expressed declaratively.
  2. Versioned and Immutable
    Desired state is stored in a way that enforces immutability, versioning and retains a complete version history.
  3. Pulled Automatically
    Software agents automatically pull the desired state declarations from the source.
  4. Continuously Reconciled
    Software agents continuously observe actual system state and attempt to apply the desired state.

“The GitOps Working Group is a WG under the CNCF App Delivery SIG.

The focus of the GitOps WG is to clearly define a vendor-neutral, principle-led meaning of GitOps. This will establish a foundation for interoperability between tools, conformance, and certification. Lasting programs, documents, and code are planned to live within the OpenGitOps project.”

The main goal of the OpenGitOps CNCF Sandbox project is to define a vendor-neutral, principle-led meaning of GitOps. This will establish a foundation for interoperability between tools, conformance, and certification through lasting programs, documents, and code.

Concluding

GitOps has been around for several years, and continues to evolve. CNCF has OpenGitOps as Sandbox project and embraced Flux and Argo as Incubating projects. The shift in mindset can be difficult, as some people still try to embrace DevOps as a principle.

Watch this space for more information and an upcoming eBook on our take and practices surrounding GitOps!

Resources and further reading

OpenGitOpshttps://opengitops.dev/
GitOps eBookhttps://www.gitops.tech/
Weaveworks Guide to GitOpshttps://www.weave.works/technologies/gitops/
Harnesshttps://harness.io/blog/devops/what-is-gitops/
Fluxhttps://fluxcd.io/
Argo CDhttps://argo-cd.readthedocs.io/en/stable/
Jenkins Xhttps://jenkins-x.io/
CNCFhttps://www.cncf.io/

Does language matter?

Originally posted on SogetiLabs: https://labs.sogeti.com/does-language-matter/

In my previous blog I stated my learning goal for 2020: Learning (or getting a better understanding) of the Go language. Currently I’m studying various courses, and have to give huge credits to Todd McLeod for his excellent work on courses on this subject.

Working with what you were given

In many cases I have to work what is given. If I am helping a client running Java with Docker, I cannot simply move them to another language or framework, just because that seems a better technical solution. I have to keep in mind that people and companies have invested in a language or framework, and sometimes it seems almost a religious feud (Windows vs Linux) instead of an objective view on the subject. So in a way, a language does not matter in my work as a consultant or architect. I have my preferences, but cannot force them onto a group just for sake of my own reasons.

Let’s Go! (pun intended)

Go is a very elegant and fast language. Testing is built right into the language, as is documentation. It pretty much runs on any OS, and does that really fast.

The authors (Robert Griesemer, Rob Pike, and Ken Thompson) wanted to address criticism of other languages in use at Google, but keep their useful characteristics (Wikipedia):

  • static typing and run-time efficiency (like C++)
  • readability and usability (like Python or JavaScript)
  • high-performance networking and multiprocessing

In Go everything revolves around ease of programming and frankly, it is just plain fun coding in Go! I’m amazed with the speed of coding and simplicity of concurrency. Numerous IDE’s are available, making it easy to start with the language.

What language do you choose?

If you are in a greenfield environment, or want to create services that have the need for speed; think beyond your comfort zone. Experiment with languagues and frameworks. See what fits for your purpose.

In any case, take a look at Go with your team, and let me know what you think of it!

2020: Keep on learning

This blog was originally posted on SogetiLabs: https://labs.sogeti.com/2020-keep-on-learning/

As the new year starts, many of us have New Year’s resolutions, and many of those will eventually perish within a month or two.

New Year, New Technology

I don’t have any resolutions. The only thing I try to do each year is to learn a new technique or language. Please note that new means new to me and not necessarily a brand new technique.

Why? In my day-job I focus on designing cloud native systems and architecture, and most of my ‘programming’ is done in Visio and PowerPoint. As my roots are in Software Engineering, I keep myself up-to-date by learning new languages and techniques.

For the upcoming year I’ve already made my choice. I started out with the following short-list.

  1. Scala
  2. Rust
  3. Go

Creating the short-list

The reason I chose these techniques is not random. In my work as an external examiner for the University of Applied Sciences Avans and Fontys in the Netherlands, I see the work of many students each year. They inspire me to look at specific techniques that normally do not cross my path. In my day-to-day work, the most used languages are C#, TypeScript, Java, and JavaScript languages and frameworks like Angular.

1. Scala

Scala is a general-purpose programming language providing support for functional programming and a strong static type system. Designed to be concise, many of Scala’s design decisions aimed to address criticisms of Java.

Wikipedia

Functional programming is something that I do not see often in my day-to-day job, so I was intrigued by the capabilities of this language. Also, some very fast and popular software is written in Scala. Examples are: Apache Kafka, Apache Spark and Akka.

2. Rust

Rust is a multi-paradigm system programming language focused on safety, especially safe concurrency. Rust is syntactically similar to C++, but is designed to provide better memory safety while maintaining high performance.

Wikipedia

Originally invented by Mozilla and used within Firefox and Dropbox. Rust has been the “most loved programming language” in the Stack Overflow Developer Survey every year since 2016, so that drew my attention.

3. Go

Go, also known as Golang, is a statically typed, compiled programming language designed at Google by Robert Griesemer, Rob Pike, and Ken Thompson. Go is syntactically similar to C, but with memory safety, garbage collection, structural typing, and CSP-style concurrency.

Wikipedia

Go has been around for quite some time, and has an impressive list of applications that were built with the language. Kubernetes, OpenShift, Docker and the list goes on.

Choosing my 2020 technique

For an internal project, we were looking for a tool to provides us programmability against a multitude of APIs. Instead of grabbing an off-the-shelve product, I investigated tools and frameworks that could help us build an MVP, fast and reliably. This eventually made me look into Terraform Custom Providers.

In Terraform, a Provider is the logical abstraction of an upstream API. This guide details how to build a custom provider for Terraform.

https://www.terraform.io/docs/extend/writing-custom-providers.html

Terraform will let you wrap any API, so it will enable us to wrap our ITSM tooling, Monitoring tooling and what have you not.

As Terraform and the custom providers are written in Go, that was my main reason to dive into this language. I’ve created a Github repo with an example custom provider, find it here on Github.

Keep on learning

I’m curious if any of you are also keen on ‘staying relevant’ and want to keep up with new techniques and languages. And, what techniques and languages do you try out and investigate? Please feel free to let me know, and contact me on LinkedIn or Twitter!