Three Weeks with AWS EC2 Image Builder — First Impressions
System Administrators get accused of being lazy, now I don’t know WHY anyone would ever accuse us of that, but it is an accusation we don’t take lightly… As a reformed sys admin myself, I guess there’s a grain of truth to those accusations, but in reality, automating repeatable work not only allows us to focus on other work, but makes repeatable work consistent. As a sys admin in the past, I’ve configured servers manually, developed scripts to be run after initial build and used configuration management tools like Puppet, Chef and Ansible to configure recently launched servers. Today I’m going to talk about some recent work I’ve been doing on building Amazon Machine Images (AMI) with AWS’ new(ish) EC2 Image Builder.
Oddly enough, as someone with a background as a Linux administrator, this is actually my first experience using automated tools to build and package my AMI’s. Up until this work, the projects I’ve worked on have all had another team that built, managed and distributed our AMI’s. Having been away from pure Linux sys admin work for some time now, its been nice getting back in and “getting my hands dirty” again.
Most of us have heard of Packer (I hope), Hashicorp’s tool for automating the build of AMI’s, Azure VM Images and more (its compatible with near everything). EC2 Image Builder is AWS’ in house response to that for building AMI’s in a repeatable, and consistent way. The product was announced at re:Invent in 2019, and currently is compatible with the following operating systems; Amazon Linux 2, CentOS 7 and 8, Red Hat Enterprise Linux 7 and 8, SUSE Linux 15 and Windows Server 2012 R2, 2016 and 2019. Image Builder uses a Pipeline based approach that combines components and recipes to build and deploy your images for you. We’ll get into each of this in depth below.
As I mentioned, AMI creation in Image Builder follows a Pipeline based approach. A pipeline in this case is simply a configuration of an image recipe and source AMI that can be run to create a “golden” AMI, which can then be distributed to one or multiple accounts under your organization. These pipelines can be configured to run based on any schedule you see fit, and out of the box Amazon offers three options; manual, schedule builder and cron expression. With manual builds, you go in and run the pipeline yourself without any sort of automated scheduling configured on your Pipeline. Schedule builder allows you to run your builds daily, weekly or monthly based on your preferred schedule while the cron expression allows you to do the same, only using a cron format to schedule it.
A pipeline consists of an image recipe. An image recipe defines the source image you’re looking to build, and the components you want to run on that image in your pipeline. That source AMI can be either Amazon managed, or something you manage. If you decide to provide it your own managed AMI to build off of, be aware that the Systems Manager (SSM) service agent must be installed and running on the source image as all build steps on an image are orchestrated through SSM.
Components are the building blocks that turn a base image, into a finished AMI. Components are documents, in YAML format that define what will be done to the base image such as operating system updates, download files from S3, or run a bash script or Powershell script. For more on what can be defined in a component, see the EC2 Image Builder documentation here. For a beginner who’s never used YAML before, it should be a pretty easy pickup, for those of you with prior CloudFormation or YAML experience its just a matter of picking up Instance Builder’s supported action modules and input types and you’re off to the races. You may apply one component to a recipe, or you may apply multiple components to a recipe. Amazon also provides a number of components they’ve created and shared with the community which can help if you’re just getting started, or frankly in a hurry to get something in place.
This article isn’t going to walk you through creating your first Image Builder pipeline, if you are looking to get started, I recommend checking out the documentation and this article here. Both were great resources to me as I developed my first pipelines. What I’m going to focus on here are my thoughts, the general maturity of Image Builder and how it compares to similar services.
When it comes to managing your golden images, Packer by and large is the tool preferred for this work. With Packer, you can use either JSON or Hashicorp Configuration Language (HCL) to define your image templates. Templates are broken down into sections; variables, builders, provisioners and post-processors in which your AMI or template is defined. The power (at least to me) in Packer is in its ability to build images for multiple providers from one template. For example, if you manage a web application and you run work loads in AWS and Azure, you can build exact copies for each provider from one template and have them deployed to the proper provider. Which brings us to Packer’s next great feature, it is vendor agnostic. In addition to AWS and Azure, Packer will build templates for Google Cloud Platform (GCP), VMWare and Digital Ocean. Packer also works with all the popular configuration management tools, Chef, Puppet, Ansible, etc. If you use any of these services to manage your infrastructure, you can use them in Packer as part of your image build process. If I had to point to anything that’s a downside to Packer, its that HCL has more of a learning curve to it than YAML does in my opinion. But, with a little effort its still not hard to pick up. If you go with JSON you just need to keep track of your braces.
It’s also worth noting that Azure recently has create a similar service, Azure Image Builder which is currently in Beta. For transparency sake, I did not use Azure Image Builder as a part of my work, so I can only speak to it having reviewed the documentation. That said, under the hood it runs Packer, and like Packer it is compatible with the same popular configuration management tools mentioned above. Like EC2 Image Builder, it will only build images for Azure or your local on-prem VSphere infrastructure, it is not vendor agnostic as Packer is. Please note, at this time Azure is NOT recommending this tool for Production workloads.
Getting to the star of the show though, how does EC2 Image Builder compare? Being a little under a year old, its not quite as mature as Packer. When compared to the other two services, the lack of compatibility with a configuration management tool is the biggest feature gap between it and Packer and Azure Image Builder. While its easy enough to write a script to get around this, being able to tie in directly to our Ansible infrastructure would be wonderfully helpful. As a team that is running workloads in AWS and Azure, we typically look for solutions that are vendor agnostic, although in my case (at least for now) we’ve gone the Image Builder route. My last gripe (its not really a gripe) is despite CentOS and RHEL getting compatibility recently, not everything works on them. Specifically, the AWS managed DISA STIG components aren’t compatible yet (CIS Benchmarks would be pretty awesome too). While I love that we can now manage our CentOS and RHEL AMI pipelines from Image Builder, having that compatibility at launch for them would have been really nice and helpful. Despite a few odds and ends here and there (which could very well be me misunderstanding the documentation) the service works, and it seamlessly integrates with Organizations to build, and distribute AMI’s to all our managed accounts. Put simply, the service works. Given Amazon’s track record for continually updating and improving their service offerings, I have no doubts this service will continually be upgraded with new features (maybe work on those first two request first though please?).
So, would I recommend the service for your use? Yes, I would. I would probably at this time recommend Packer over it, as it is the more mature and feature-rich offering, but if you’re hell bent on using this and none of the above limitations apply to you, Image Builder will do the job for you. I certainly hope the Image Builder team will continue improving on this product and adding features that truly make this a tool with the capabilities that Packer has. When we as developers and sys admins have multiple tools to choose from in our tool-set, our world is a better place. I’m really excited to see what the future has in store for Image Builder here!
