Day Two Cloud 108: Go Learn Something New

I’ve said many times that I’m not a developer, and that’s true in the sense that I don’t create applications with code. If that’s your definition of a developer, then it I am not. But if we extend the moniker of developer to include those who write code on a regular basis as part of their job, then developer is I.

Still, I feel a sense of imposter syndrome creeping in. That’s because the “code” I am writing is in Terraform, and not a general purpose programming language (GPPL) like Python or Go. The fact that I’m not writing my own method and functions, importing libraries, and producing binaries makes me feel lesser than. It’s just HCL after all. It’s not like I am writing the providers that power Terraform or interacting with the APIs behind the public clouds. Nope, I am just creating a slightly more advanced template. That feels like programming lite to me.

Of course I know that’s not really true. HCL and other domain specific languages like it are Turing complete. That means they have logical operators embedded in the language from which you could develop a full blown program. If that piques your interest, stay tuned for the upcoming Day Two Cloud episode with Kris Nova. We get real nerdy about the conceptual framework behind DSLs, infrastructure as code, and infra as software.

One of the main things missing from something like Terraform is all the conveniences of a modern, general purpose programming language (GPPL). Something like Python has a rich ecosystem around it: libraries, testing harnesses, dedicated IDEs, and a vibrant community. Access to those tools can make your life as an infrastructure engineer far easier.

There’s also the problem of customization when it comes to a DSL. If something doesn’t exist in Terraform or CloudFormation, you have to write code to make it happen. And usually you cannot write that code in HCL or YAML. Instead you will need to create a function in Lambda with JavaScript or invoke a custom provider written in Go. When you hit the limit of your IaC tool of choice, you are going to end up in a general purpose programming language anyway. What if there was something that served as a shim between all those amazing cloud providers you wanted to use and the flexibility of a GPPL?

Michael Levan was our guest on Day Two Cloud episode 108, and he introduced us to the idea of a Cloud Development Kit (CDK). At it’s core, a CDK is a collection of libraries for one or more GPPLs that allow you to interact with one or more cloud environments. To give a more specific example, the AWS CDK has a collection of libraries for Python that allow you to write code that provisions AWS infrastructure. On the backend it is creating CloudFormation templates to deploy the actual infrastructure, but you don’t need to worry about that.

Using a CDK gives you an GPPL environment to work it, along with the convenience of writing Infrastructure as Code. That means no more weird loops or logic trees in your HCL configuration. It also means you can do unit and integration testing with the GPPL, instead of using something like TerraTest. If there’s a custom resource or method you need to define, you can just do it directly in the code. It also means that other folks on your team do not need to learn yet another DSL. Assuming they know the same GPPL as you, they can easily read your code, make contributions, or help with problems. That seems like a win.

The downside is that you actually need to know how to use a GPPL. If you’ve come from the infrastructure side where you’re used to scripting with Bash and PowerShell, formal programming may seem like a daunting proposition. But I’m hear to tell you that if you’ve already mastered something like ARM templates or Terraform, then moving to a GPPL is not a bridge too far. In fact, I’d say you are already familiar with many of the concepts in programming. Loops, logic, and functions are all things you’ve used before. It’s not that there isn’t a learning curve; it just isn’t as steep as you might think.

I’ve been wanting to really dig into programming for a while now, but the main blocker was a lack of need in my day-to-day duties. In general, I am so busy with projects that I don’t have time or inclination to try and learn something that doesn’t directly relate to those projects. If I want to get deep into programming with a GPPL, I need it to be part of the projects I work on. Learning how to use a CDK is a perfect mesh of infrastructure and programming, and it may get me to finally learn Go. That’s right, my plan is to learn to use Go with both Pulumi and the Terraform CDK to provision infrastructure in AWS and Azure.

I’ve blocked out dedicated time on my calendar in the next couple of months to focus on learning Go in this context. A couple years ago, I tried learning Go from a Getting Started course on Pluralsight. I got through about 50% of the course, but I didn’t have a daily project to apply it to. While some of the core concepts stuck, the rest of it vanished like a rainbow. I believe that apply Go to something I care about will finally get me over the hump and into Go fluency. And if that works, Python might be next. I’ll keep y’all up to date on my progress. In fact, you can expect a video or two about it in the autumn.