Using AWS CodeArtifact with Python and Github Actions

AWS CodeArtifact allows you to reliably store, publish, and share the latest version of packages. There is no need to worry about infrastructure as it is a fully managed service. It can work with package managers and build tools such as Maven, Gradle, npm, yarn, twine, pip.

In this post, we are going to create a sample Python package which we are going to host on AWS CodeArtifact. We will add this package as a dependency to our Lambda function. Let's start with creating two separate git repositories and publish the package we created to AWS CodeArtifact by using Github Actions. In the second repository, we will download the sample Python package from CodeArtifact with Github actions and use the package within our Lambda function.

If you wish to learn detailed information about how to use GitHub Actions, you can check our previous blog post from the link below;

Why do we need to use CodeArtifact?

Let's imagine you're working on a software project and you're using open source packages, but you also have some custom packages that you'd like to keep private within the company or you don’t want to share. At this point, AWS CodeArtifact makes it very simple to store the packages and enable developers to access these packages.

Let’s start with creating an empty repository on Github and call it sample-pip-package, and after creating the repository, develop our simple function that returns "Hello world, AWS CodeArtifact is very easy."

Then we will create a repository where we can store our packages on AWS CodeArtifact with the AWS CLI. A domain must be created for the repository.

$ aws codeartifact create-domain --domain my-domain

$ aws codeartifact create-repository --domain my-domain --domain-owner $AWS_ACCOUNT_ID --repository my-private-repo

There are two options offered by AWS for pip and twine. It can be used as a login or necessary configuration can be made without login. In this blog, we will set it up without the pip and twine login command.

We will create a .github/workflows directory in the project folder and create the main.yml file in it.

name: Python application
on:
  push:
    branches:
      - main
jobs:
  deploy-lambda:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-python@v2
        with:
          python-version: '3.8'
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install setuptools wheel twine
      - uses: aws-actions/setup-sam@v1
      - uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: eu-west-1
      - name: Build and publish
        run: |
          export TWINE_USERNAME=aws
          export TWINE_PASSWORD=`aws codeartifact get-authorization-token --domain my-domain --domain-owner ${{ secrets.AWS_ACCOUNT_ID }} --query authorizationToken --output text`
          export TWINE_REPOSITORY_URL=`aws codeartifact get-repository-endpoint --domain my-domain --domain-owner ${{ secrets.AWS_ACCOUNT_ID }} --repository my-private-repo --format pypi --query repositoryEndpoint --output text`
          python setup.py sdist bdist_wheel
          twine upload dist/*

Package has been successfully added to the repository.

Now, we will create another empty repository on Github and call it aws-codeartifact-lambda-example. In this repository we will download the package from AWS CodeArtifact and call it within the Lambda function.

We will create a .github/workflows directory in the project folder and create the main.yml file in it.

name: Python application
on:
  push:
    branches:
      - main
jobs:
  deploy-lambda:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-python@v2
        with:
          python-version: '3.8'
      - uses: aws-actions/setup-sam@v1
      - uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: eu-west-1
      - run: echo CODEARTIFACT_AUTH_TOKEN=$(aws codeartifact get-authorization-token --domain my-domain --domain-owner ${{ secrets.AWS_ACCOUNT_ID }} --query authorizationToken --output text --region eu-west-1 ) >> $GITHUB_ENV
      - run: echo "$(echo '--extra-index-url https://aws:'"$CODEARTIFACT_AUTH_TOKEN"'@my-domain-${{ secrets.AWS_ACCOUNT_ID }}.d.codeartifact.eu-west-1.amazonaws.com/pypi/my-private-repo/simple/' | cat - sam-app/hello_world/requirements.txt)" > sam-app/hello_world/requirements.txt
      # Build using SAM
      - run: sam build --use-container --template-file sam-app/template.yaml
      # Deploy on AWS
      - run: sam deploy --config-file sam-app/samconfig.toml

There are 2 things to be aware of in this part;

• GITHUB_ENV: You can make an environment variable available to any subsequent steps in a workflow job by defining or updating the environment variable and writing this to the GITHUB_ENV environment file. ¹

• Thanks to the –extra-index-url added to the beginning of the requirement.txt file, you can download your packages both from the PyPI or the private repository. The point that should not be forgotten here is that, if there is a package with the same name on the private repository and PyPI, a conflict occurs.

You can reach source code for the projects from the links below:

Congratulations!

We built a private AWS CodeArtifact repository, using Github Actions to submit our package to the repository on CodeArtifact, and downloaded the package.
I hope this article helped you to understand how to use AWS CodeArtifact. Thank you for taking the time to read this article; stay tuned for more!

1. https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-an-environment-variable