Amazon EKS AMI Build using EC2 Image Builder

Amazon Elastic Kubernetes Service (Amazon EKS) is a managed Kubernetes service of AWS that you can use to run Kubernetes without needing to install, operate, and maintain your own control plane or nodes.

For nodes, Amazon EKS provides a specialized Amazon Machine Images (AMI) called Amazon EKS optimized AMI. These AMIs are configured to work with Amazon EKS (auto-magically join your cluster) and include some required packages such as Docker, kubelet, and the AWS IAM Authenticator. You can find all types on versions on the Amazon EKS optimized AMIs documentation page.

AWS has also open-sourced bootstrap and the build scripts that are used to build the Amazon EKS optimized AMI. This enables us to create custom Amazon Linux AMI for our Amazon EKS cluster nodes. This AMI build process is accomplished by using HashiCorp Packer.

At the end of 2019, AWS announced a new service called, EC2 Image Builder as a native AMI baking tool. Compared to Packer, EC2 Image Builder takes a more pipeline approach to building AMI’s and is meant to be more of a continuous, complete solution for building and distributing AMI’s. This repository is inspired by Amazon EKS AMI configuration scripts that AWS shared with users to prepare custom AMI's using EC2 Image Builder service.

How does it work?

Step 1

Create an S3 bucket and upload the folders called scripts and files at the root of this project. These files contain some bootstrap and setup script to setup the EC2 instance as a node for your Amazon EKS cluster.

Step 2

All you need to do is creating a new stack on Cloudformation using this, amazon-eks-imagebuilder.yaml script. Parameters that you need to enter,

• Name: Stack name

• KubernetesVersion: Select the Kubernetes version

• ScriptBucket: Enter the name of the bucket that you created at Step 1

This stack will create following resources on your AWS account:

• EKSAMIComponent: AWS::ImageBuilder::Component, image builder component that runs scripts to create Amazon EKS-optimized AMI.

• EKSAMIRecipe: AWS::ImageBuilder::ImageRecipe, image recipe of the pipeline.

• InstanceRole: AWS::IAM::Role, IAM role that is used during the image building process.

• InstanceRoleBucketPolicy: AWS::IAM::Policy, IAM policy that enables EC2 instances to access to S3 bucket created by you.

• InstanceProfile: AWS::IAM::InstanceProfile, IAM instance profile that is used during the image building process.

• InstanceSecurityGroup: AWS::EC2::SecurityGroup, security group only allows outbound traffic, used during image building process.

• EKSAMIInfrastructureConfiguration: AWS::ImageBuilder::InfrastructureConfiguration, infrastructure configuration of the pipeline.

• EKSAMIPipeline: AWS::ImageBuilder::ImagePipeline, image pipeline configured to run on manual triggers.

Step 3: Customize

You can create new components on EC2 Image Builder and add these components additionally to EKSAMIRecipe.

  EKSAMIRecipe:
    Type: AWS::ImageBuilder::ImageRecipe
    Properties:
      Name: 'EKSAMIRecipe'
      Version: '1.0.0'
      # ${AWS::Partition} returns the partition where you are running the CloudFormation template. For standard AWS regions, the
      # partition is aws. For resources elsewhere, the partition is aws-partitionname. For example, China (Beijing and Ningxia)
      # regions use aws-cn and AWS GovCloud (US) regions are aws-us-gov.
      # https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/pseudo-parameter-reference.html
      ParentImage: !Join
        - ''
        - - 'arn:'
          - !Ref AWS::Partition
          - ':imagebuilder:'
          - !Ref AWS::Region
          - ':aws:image/'
          - !FindInMap [ Instance, !Ref Arch, source ]
          - '/x.x.x'
      Description: 'Recipe to create EKS Optimized AMI'
      Components:
        # Add new components here if you want to customize the build process
        - ComponentArn: !Ref EKSAMIComponent

Troubleshooting Tips

If you have any issue on customization and the build process fails, you can check details about the Image Pipeline steps from the AWS Systems Manager > Change Management > Automation page.

Advanced Troubleshooting

If you need to check detailed logs from instance:

• Change TerminateInstanceOnFailure to false on EKSAMIInfrastructureConfiguration. With this configuration, you will be able to access the instance even after image building failed.

• AmazonSSMManagedInstanceCore policy is attached to the EC2 build instance; this will allow you to connect the EC2 instance using AWS SSM through AWS Management Console. If you don't know how, please check: https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-working-with-sessions-start.html#start-ec2-console

• You can find all image builder logs under /var/lib/amazon/toe/ folder.

How to use the AMI?

If you are just getting started with Amazon EKS, we recommend that you follow the Getting Started chapter in the Amazon EKS User Guide first.

If you already have a cluster, and you want to launch a node group with your new AMI, see Launching Amazon EKS Worker Nodes in the Amazon EKS User Guide.

The amazon-eks-nodegroup.yaml AWS CloudFormation template in this repository is provided to launch a node group by asking the new AMI ID; you can enter the freshly-baked AMI ID here, that is prepared by your EC2 Image Builder pipeline.