Hello,
I am currently trying to create an AWS Lambda function using a custom runtime built in Elixir. I had been able to replicate the following Haskell guide AWS lambda in any language using custom runtimes | Tooploox and wanted to do something similar in Elixir.
To start off I am very new to Elixir so I am likely to miss some obvious things. I was able to get a very basic hello world function running, however the zip package I send its around 25mb vs 4-7mb in haskell, the structure of my zip is as follows. I have seen suggestions of removing things like iex that might not be used.
- /bin
- /erts-15.1.2
- /lib
- /releases
- bootstrap
When I run invoke the function it it uses a max memory of around 150mb vs 40mb for the Haskell approach. I am perfectly fine if this is the way it should be, however as a new elixir programmer I was just worried a mistake on my end has increased requirements.
In my attempt to get an approach similar to the haskell one working I ran into issues with releases and dependencies like burrito/distillery etc. The common issue I ran into was similar to the following.
/var/task/erts-15.1.2/bin/epmd: /lib64/libc.so.6: version `GLIBC_2.38' not found (required by /var/task/erts-15.1.2/bin/epmd)
/var/task/erts-15.1.2/bin/beam.smp: /lib64/libgcc_s.so.1: version `GCC_12.0.0' not found (required by /var/task/erts-15.1.2/bin/beam.smp)
I fixed this by using docker and building from the amazonlinux:2023 base image so that everything had the correct version and then just copying the function.zip to my machine to verify its correct. I had also considered some hacky approach of just patching these into the zip file.
DOCKERFILE:
# Use Amazon Linux 2023 as the base image
FROM amazonlinux:2023
# Install system dependencies with --allowerasing to handle conflicts
RUN yum update -y && \
yum install -y \
git \
curl \
tar \
gzip \
zip \
gcc \
gcc-c++ \
make \
autoconf \
openssl \
openssl-devel \
ncurses-devel \
wget \
gnupg \
nano \
glibc-langpack-en --allowerasing
# Install ASDF
RUN git clone https://github.com/asdf-vm/asdf.git ~/.asdf && \
echo '. "$HOME/.asdf/asdf.sh"' >> ~/.bashrc && \
echo '. "$HOME/.asdf/completions/asdf.bash"' >> ~/.bashrc
# Source ASDF scripts
SHELL ["/bin/bash", "-c"]
RUN source ~/.asdf/asdf.sh && \
source ~/.asdf/completions/asdf.bash
# Install Erlang plugin and latest version
RUN source ~/.asdf/asdf.sh && \
asdf plugin-add erlang https://github.com/asdf-vm/asdf-erlang.git && \
ERLANG_VERSION=27.1.2 && \
asdf install erlang $ERLANG_VERSION && \
asdf global erlang $ERLANG_VERSION
# Install Elixir plugin and latest version
RUN source ~/.asdf/asdf.sh && \
asdf plugin-add elixir https://github.com/asdf-vm/asdf-elixir.git && \
ELIXIR_VERSION=1.17.3 && \
asdf install elixir $ELIXIR_VERSION && \
asdf global elixir $ELIXIR_VERSION
# Set the working directory inside the container
WORKDIR /app
# Copy the current directory into the container
COPY . /app
# Install project dependencies
RUN source ~/.asdf/asdf.sh && \
mix local.hex --force && \
mix local.rebar --force && \
mix deps.get
# Clean up any previous build artifacts
RUN rm -rf /app/_build/prod/rel/hello_lambda
# Build the Elixir release
RUN source ~/.asdf/asdf.sh && \
MIX_ENV=prod mix release
# Package the release into function.zip after moving the bootstrap file
RUN cd _build/prod/rel/hello_lambda && \
mv /app/bootstrap . && \
zip -r /app/function.zip *
# Default entry point to start shell for testing
CMD ["/bin/bash"]
I am mainly just wondering if anyone can see some obviously superior approach within the bounds I set for myself, perhaps configuring a dependency like burrito correctly. Im sure I could, and probably should, use a different approach if I was trying to accomplish something important, but for this I just want to experiment with AWS and custom runtimes.
I will attach the files from my project here for anyone to examine/replicate if they see fit.
mix.exs:
defmodule HelloLambda.MixProject do
use Mix.Project
def project do
[
app: :hello_lambda,
version: "0.1.0",
elixir: "~> 1.17",
start_permanent: Mix.env() == :prod,
deps: deps(),
releases: [
hello_lambda: [
include_erts: true,
include_executables_for: [:unix],
applications: [runtime_tools: :permanent]
]
]
]
end
# Run "mix help compile.app" to learn about applications.
def application do
[
extra_applications: [:logger, :inets, :crypto, :public_key, :ssl],
mod: {HelloLambda.Application, []}
]
end
# Run "mix help deps" to learn about dependencies.
defp deps do
[
{:jason, "~> 1.2"}
# {:dep_from_hexpm, "~> 0.3.0"},
# {:dep_from_git, git: "https://github.com/elixir-lang/my_dep.git", tag: "0.1.0"}
]
end
end
application.ex:
defmodule HelloLambda.Application do
use Application
def start(_type, _args) do
children = [
{Task, fn -> HelloLambda.Runtime.start() end}
]
opts = [strategy: :one_for_one, name: HelloLambda.Supervisor]
Supervisor.start_link(children, opts)
end
end
runtime.ex:
defmodule HelloLambda.Runtime do
require Logger
def start do
{:ok, _} = Application.ensure_all_started(:inets)
loop()
end
defp loop do
case get_invocation() do
{:ok, request_id, event} ->
response = HelloLambda.Handler.handle(event)
send_response(request_id, response)
loop()
{:error, reason} ->
Logger.error("Failed to get invocation: #{inspect(reason)}")
:timer.sleep(1000)
loop()
end
end
defp get_invocation do
runtime_api = System.get_env("AWS_LAMBDA_RUNTIME_API")
url = "http://#{runtime_api}/2018-06-01/runtime/invocation/next"
case :httpc.request(:get, {String.to_charlist(url), []}, [], []) do
{:ok, {{'HTTP/1.1', 200, 'OK'}, headers, body}} ->
request_id = get_header_value(headers, 'lambda-runtime-aws-request-id')
event = Jason.decode!(body)
{:ok, request_id, event}
{:error, reason} ->
{:error, reason}
end
end
defp send_response(request_id, response) do
runtime_api = System.get_env("AWS_LAMBDA_RUNTIME_API")
url = "http://#{runtime_api}/2018-06-01/runtime/invocation/#{request_id}/response"
body = Jason.encode!(response)
:httpc.request(
:post,
{String.to_charlist(url), [], 'application/json', body},
[],
[]
)
end
defp get_header_value(headers, key) do
headers
|> Enum.find(fn {k, _} -> String.downcase(to_string(k)) == String.downcase(to_string(key)) end)
|> elem(1)
|> to_string()
end
end
handler:
defmodule HelloLambda.Handler do
def handle(event) do
%{"message" => "Hello, World!", "event" => event}
end
end
