Why you should stop spinning up VMs

Originally posted on SogetiLabs: https://labs.sogeti.com/why-you-should-stop-spinning-up-vms

If I would get paid a euro for each time someone says ‘Let’s spin up a VM for that!’ or ‘We have this VM running anyway’, I would have been a rich man.

Although I do advocate reuse, stuffing many services on a machine ‘because it runs there anyway’ will cause a huge problem if there is a shift in architecture or platform. In the end all services need to be carved out and that can be a real pain in the behind. Also it does not really scale, for obvious reasons.

Yes let’s put everything in a pod/container!

A big cluster of VMs is still just infrastructure chewing away your money in the cloud. Do not forget that in many cases, a k8s cluster, Service Fabric cluster or whatever-cluster-cloud-native-you-like in the end is mostly a bunch of VMs running 24/7 for a lot of money.

We underestimate the power of simplicity

Let’s take an example. If you want to run a simple process, for example ingest some data and put it in a store, you could go full ETL/ELT. Alternatively you can leverage simple things, like Azure Functions or LogicApps. With connectors to almost all services, it is really easy and simple to ingest data using these services.

And it’s cheap too!

Azure Functions (ran in Azure) run really cheap, as the first million(!) executions are free anyway.

What about portability?

If you start out with Azure Functions on the Azure Platform, and later on need to run on a cluster like Kubernetes, you’re in luck. Azure Functions run basically anywhere so hosting them in a container or pod, should be a piece of cake. Development is also easy exactly because of the portability.

How to start?

Microsoft has done really great work in the training area. For Azure Functions, you can simply go to: Create serverless logic with Azure Functions – Learn | Microsoft Docs. This module will get you started easily.

Want to kickstart your development? Reach out and we’ll help!

Everything is Code. Why and how you should test your Terraform

Originally posted on SogetiLabs: https://labs.sogeti.com/everything-is-code-why-and-how-you-should-test-your-terraform/

’ve written in a previous blog that for writing maintainable code, you really should automate your tests. Currently we see Terraform really taking off as the standard to provision and deploy almost everything in cloud like Azure and AWS. In this short blogpost I will show how you can start testing your Terraform code using the Open Policy Agent.

Infrastructure-as-Code

With Terraform, engineers write declarative statements that, when applied, result in for example a network, of a database:

Write Declarative Configuration Files

This obviously is great because you can automate and repeat your steps minimizing the manual labour of setting up services. Next to that, you can reuse components so all provisioned services will adhere to the same standard.

What about standards and compliancy?

In the previous example, a network will be created with some IP range, in a specific location from Google.

If you use any cloud, you should have some policies in place. It is not a free-for-all, do whatever you want, scenario. It could be very important where your data resides. In that case you don’t want to enable teams to deploy in non-compliant regions.

With code however, especially in the sample shown above, it is not directly clear what is allowed. Some examples of questions DevOps team may have are:

  • Am I allowed to create a public IP address on a machine or database?
  • Am I allowed to deploy services in the US or only in Europe?

Worst case, the team finds out when they deploy their services onto their cloud environment. In case of Azure, policies can block a certain location, blocking the entire deployment of their stack.

This means they have to go back to trace why something is wrong, adjust the code, go through all steps again and hope they did not miss anything.

Shift left

One solution to this problem is to test as early as possible. If a team gets feedback on their code, this will save a lot of time in the stages later on. It will also give the teams the confidence they are doing things right!

Open Policy Agent (OPA)

OPA states the following on their website:

Stop using a different policy language, policy model, and policy API for every product and service you use. Use OPA for a unified toolset and framework for policy across the cloud native stack.

Whether for one service or for all your services, use OPA to decouple policy from the service’s code so you can release, analyze, and review policies (which security and compliance teams love) without sacrificing availability or performance.

Source: Open Policy Agent website

Using OPA for testing your Terraform plans

You can use Open Policy Agent for a variety of use cases. Testing Terraform plans is only one of them, but really interesting for DevOps team.

The language for writing the OPA policy is called rego. It is easy to understand and should be relatively simple to learn. To get a grasp, take a look at the explanation of the language on the OPA site: https://www.openpolicyagent.org/docs/latest/policy-language/.

A very complete manual is located at https://www.openpolicyagent.org/docs/latest/terraform/ and helps you understand writing a test.

The great thing about the tool, is that you can use it in your pipelines, running the eval command in for example Azure DevOps. There is even an installertask for Azure Pipelines.

Getting started

If you are using Terraform, just download OPA, and go to the getting started page. Use the given example to test against your own Terraform plan outputs.

Just want to play around with the rego language? The site https://play.openpolicyagent.org/ contains several examples, and makes exploring easy, as it is an online playground without the need for installing anything!

Is low code replacing traditional development?

Originally posted on SogetiLabs: https://labs.sogeti.com/is-low-code-replacing-traditional-development/

Spoiler alert; no.

Low code vs Traditional Dev

My colleague, friend and SogetiLabs Fellow Daniel Laskewitz and I frequently talk about this topic. His field of expertise as Microsoft MVP covers the Microsoft PowerPlatform including low code systems like Power Automate (formerly Flow).

All too often people see a division between low code and traditional development using languages like C#, Java, TypeScript and Go.
In the real world however, these systems work together perfectly.

Most of the times, you cannot solve a problem with only low code. Think about scenario’s where you should link to old legacy systems or complex API calls. In those cases low code without any enhancement cannot natively connect to those systems.

Behold custom connectors

In the Microsoft ecosystems, custom connectors allow you to bridge this gap. This way, the low code system can interact with any system you write a connector for. This may be common knowledge, but the fact is that most developers do not see how big this really is.

This means you can link any PowerApp, Microsoft Flow, or LogicApps to your custom connector, and reuse those within your entire organisation.
You could even publicly publish these if you have a service you want to expose. So if you are an ISV, this can help you get more traction on your product.

Bridging the gap

In the end it all comes down to developers of any system and language understanding the capabilities of the platforms they and their companies are using. For low code developers this means sometimes calling in the help of traditional developers. And more importantly, this also means traditional developers should learn that these low code systems can help you simplify (and thus speed up!) your development by using ready-to-roll systems and connectors available to you.

As there are over 325 connectors available, that should really speed making connections up!

Get started!

Want to explore custom connectors? Look at these resources or feel free to contact me or Daniel, we strongly believe bridging this gap between low code and traditional dev is key for succes in the future of development!

Education in a 5 feet economy

Originally posted on SogetiLabs: https://labs.sogeti.com/education-in-a-5-feet-economy/

Our Prime Minister Mark Rutte prepared us; the 5 feet economy could be here for a while. How does that work in education? And what problems do we see around us?

Technology Leaders

I (try to) teach coworkers and clients on a regular basis, and do this mostly on-site to get the interaction going between everyone in the room, and also to get a sense of how people react to my presentation or my talk. I personally find it hard to get the same feedback while working remotely.

Mind you; we are ‘Technology Leaders‘ and are capable of doing our job and all sessions remotely. We have the tech, but do we really have the same impact? At this moment, we do not have a choice, and have to do things online.

Education Online

As a father-of-three, I see these issues also on the side of education. Kids in pre-school really learn a lot in the classroom, and also from their friends in class. Want to be engaging and keep the attention of your coworkers? Try doing that for 6 or 7-year-olds…

My belief is that, especially for children, learning from each other is key. This cannot be replaced by an online tool. Still, in these ‘5 feet times,’ you have to make concessions. Digital skills are important, so make sure your kids and your friends can work online.

I see schools struggle with Microsoft Teams, Skype, with MOOC environments and with technology in general. I see this as a fail from IT companies; We always talk about inclusion, but it is still difficult to get everyone online working together. We have done a very good job making sure ‘we’ the IT people can do everything online, but we see educators or parents in general struggling with technology.

Help the educators, help each other

Big things start small; My door is always open, and I (time permitting) am always available for a quick tip or some help. Keep in mind that using Teams or Skype or Zoom is second-nature to ‘us’, for some parents or teachers this can be challenging. By helping another parent, simply by explaining how a tool works, you already have one more person that uses the tool. Maybe you can lend out a spare laptop for someone not having those resources. Maybe you can fix their internet, or setup their environment.

Small things count, we can make a difference together.

Does language matter?

Originally posted on SogetiLabs: https://labs.sogeti.com/does-language-matter/

In my previous blog I stated my learning goal for 2020: Learning (or getting a better understanding) of the Go language. Currently I’m studying various courses, and have to give huge credits to Todd McLeod for his excellent work on courses on this subject.

Working with what you were given

In many cases I have to work what is given. If I am helping a client running Java with Docker, I cannot simply move them to another language or framework, just because that seems a better technical solution. I have to keep in mind that people and companies have invested in a language or framework, and sometimes it seems almost a religious feud (Windows vs Linux) instead of an objective view on the subject. So in a way, a language does not matter in my work as a consultant or architect. I have my preferences, but cannot force them onto a group just for sake of my own reasons.

Let’s Go! (pun intended)

Go is a very elegant and fast language. Testing is built right into the language, as is documentation. It pretty much runs on any OS, and does that really fast.

The authors (Robert Griesemer, Rob Pike, and Ken Thompson) wanted to address criticism of other languages in use at Google, but keep their useful characteristics (Wikipedia):

  • static typing and run-time efficiency (like C++)
  • readability and usability (like Python or JavaScript)
  • high-performance networking and multiprocessing

In Go everything revolves around ease of programming and frankly, it is just plain fun coding in Go! I’m amazed with the speed of coding and simplicity of concurrency. Numerous IDE’s are available, making it easy to start with the language.

What language do you choose?

If you are in a greenfield environment, or want to create services that have the need for speed; think beyond your comfort zone. Experiment with languagues and frameworks. See what fits for your purpose.

In any case, take a look at Go with your team, and let me know what you think of it!

2020: Keep on learning

This blog was originally posted on SogetiLabs: https://labs.sogeti.com/2020-keep-on-learning/

As the new year starts, many of us have New Year’s resolutions, and many of those will eventually perish within a month or two.

New Year, New Technology

I don’t have any resolutions. The only thing I try to do each year is to learn a new technique or language. Please note that new means new to me and not necessarily a brand new technique.

Why? In my day-job I focus on designing cloud native systems and architecture, and most of my ‘programming’ is done in Visio and PowerPoint. As my roots are in Software Engineering, I keep myself up-to-date by learning new languages and techniques.

For the upcoming year I’ve already made my choice. I started out with the following short-list.

  1. Scala
  2. Rust
  3. Go

Creating the short-list

The reason I chose these techniques is not random. In my work as an external examiner for the University of Applied Sciences Avans and Fontys in the Netherlands, I see the work of many students each year. They inspire me to look at specific techniques that normally do not cross my path. In my day-to-day work, the most used languages are C#, TypeScript, Java, and JavaScript languages and frameworks like Angular.

1. Scala

Scala is a general-purpose programming language providing support for functional programming and a strong static type system. Designed to be concise, many of Scala’s design decisions aimed to address criticisms of Java.

Wikipedia

Functional programming is something that I do not see often in my day-to-day job, so I was intrigued by the capabilities of this language. Also, some very fast and popular software is written in Scala. Examples are: Apache Kafka, Apache Spark and Akka.

2. Rust

Rust is a multi-paradigm system programming language focused on safety, especially safe concurrency. Rust is syntactically similar to C++, but is designed to provide better memory safety while maintaining high performance.

Wikipedia

Originally invented by Mozilla and used within Firefox and Dropbox. Rust has been the “most loved programming language” in the Stack Overflow Developer Survey every year since 2016, so that drew my attention.

3. Go

Go, also known as Golang, is a statically typed, compiled programming language designed at Google by Robert Griesemer, Rob Pike, and Ken Thompson. Go is syntactically similar to C, but with memory safety, garbage collection, structural typing, and CSP-style concurrency.

Wikipedia

Go has been around for quite some time, and has an impressive list of applications that were built with the language. Kubernetes, OpenShift, Docker and the list goes on.

Choosing my 2020 technique

For an internal project, we were looking for a tool to provides us programmability against a multitude of APIs. Instead of grabbing an off-the-shelve product, I investigated tools and frameworks that could help us build an MVP, fast and reliably. This eventually made me look into Terraform Custom Providers.

In Terraform, a Provider is the logical abstraction of an upstream API. This guide details how to build a custom provider for Terraform.

https://www.terraform.io/docs/extend/writing-custom-providers.html

Terraform will let you wrap any API, so it will enable us to wrap our ITSM tooling, Monitoring tooling and what have you not.

As Terraform and the custom providers are written in Go, that was my main reason to dive into this language. I’ve created a Github repo with an example custom provider, find it here on Github.

Keep on learning

I’m curious if any of you are also keen on ‘staying relevant’ and want to keep up with new techniques and languages. And, what techniques and languages do you try out and investigate? Please feel free to let me know, and contact me on LinkedIn or Twitter!

Infrastructure-as-Code with Pulumi

Using a public cloud like Microsoft Azure or Amazon AWS? Chances are you’ve been using templates like ARM or CloudFormation.

Hashicorp has done a terrific job making Terraform with the ability to template any API, and deploy and manage stacks in Azure, AWS and many, many more.

I’ve been following Pulumi for a while, and their approach is a bit different than the previous stated tools.

Continue reading “Infrastructure-as-Code with Pulumi”

Sync your Outlook 365 calendar to Google Calendar with LogicApps

Sometimes it is hard to keep your spouse up-to-date on all the sessions and events work-related. Most of the times I’ve been able to duplicate calendar items that are relevant for my ‘better half’.

Obviously manual work makes me sad and I forget to sync some events causing a collision in the calendars. If, like me, you have kids you’ll understand this results in hoping your babysitter can make time because you forgot to add some items in your calendar manually.

Continue reading “Sync your Outlook 365 calendar to Google Calendar with LogicApps”