Build lambda with a custom docker image

Introduction

In this tutorial, we will build lambda with a custom docker image.

Prerequisites

Before starting, you should have the following prerequisites configured

• An AWS account
• AWS CLI on your computer

Walkthrough

• Create a Python virtual environment
• Create a Python app
• Create a lambda with a custom docker image of ECR
• Create ECR repositories and push an image
• Create a lamba from the ECR image
• Test lambda function on local
• Test lambda function on AWS

Create a Python virtual environment

Create a Python virtual environment with the name py_virtual_env

python3 -m venv py_virtual_env

Create a Python app

This Python source code will pull a JSON file from https://data.gharchive.org and put it into the S3 bucket.
Then, transform the uploaded file to parquet format.

Download the source code from here and put it into the py_virtual_env folder.

Create a lambda with a custom docker image of ECR

#build docker image
source ./py_virtual_env/bin/activate
cd py_virtual_env
docker build -t test-image .
docker images

Create ECR repositories and push an image

#create an ecr repository
aws ecr create-repository --repository-name lambda-python-lab \
--query 'repository.repositoryUri' --output text

#authen 
aws ecr get-login-password --region ap-northeast-1 | docker login --username AWS --password-stdin xxxxxxxxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com

#tag image
docker tag test-image:latest xxxxxxxxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/lambda-python-lab:latest
docker images

#push the image to ecr repository
docker push xxxxxxxxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/lambda-python-lab:latest

#check image
aws ecr list-images --repository-name lambda-python-lab

Create a lamba from the ECR image

Create a lambda function from the ECR image

General configuration

Environment variables

Test lambda function on local

To test the lambda function locally, you can follow

docker run --name test-lambda -v /Users/hieudang/.aws:/root/.aws \
  -e BUCKET_NAME=hieu320231129 \
  -e TARGET_FOLDER=TRANSFORMED \
  -e FILENAME=2022-06-05-0.json.gz \
  -d test-image
docker container list
docker exec -it test-lambda bash
python -c "import S3app;S3app.lambda_ingest(None, None)"

Test lambda function on AWS

Conclusion

These steps provide an example of creating a lambda function and running it on docker, then, we put the docker image in the ECR repository, and create a lambda function from the ECR repository. The specific configuration details may vary depending on your environment and setup. It’s recommended to consult the relevant documentation from AWS for detailed instructions on setting up. I hope will this be helpful. Thank you for reading the DevopsRoles page!

Refer

https://docs.aws.amazon.com/lambda/latest/dg/python-package.html#python-package-create-no-dependencies

App source code: https://github.com/itversity/ghactivity-aws