Deployment of NX.Explorer to fly.io

You will need to customize the build file depending on your particular needs. For instance since you need to grab a hex dependency via git, you’ll need to add git as part of the docker build dependencies, i.e.

# install build dependencies
RUN apk add --no-cache build-base npm git

Hopefully that will get you further along, unfortunately there is probably no avoiding learning at least some docker, there are a number of examples of phoenix specific docker files floating around if you search on this forum or hit up the google machine.

Thanks, now that part works.

docker build . still fails, because the package Explorer depends on an installed Rust.
@mrkurt Can you help me modify the dockerfile for the case that a dep of a Phoenix app depends on Rust?

Well, now that the app depends on rust, you do the same that you did with git, add rust to the RUN apk line

(maybe you’ll also need to add cargo, the rust package manager, to the line)

I tried… I added

RUN apk add --no-cache build-base npm rust
RUN apk add --no-cache build-base npm cargo

to my dockerfile.

but unfortunately it isn’t so easy:

...
   Compiling mimalloc v0.1.26
   Compiling thiserror v1.0.25
   Compiling flatbuffers v2.0.0
error[E0658]: const generics are unstable
  --> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/flatbuffers-2.0.0/src/array.rs:25:35
   |
25 | pub struct Array<'a, T: 'a, const N: usize>(&'a [u8], PhantomData<T>);
   |                                   ^
   |
   = note: see issue #74878 <https://github.com/rust-lang/rust/issues/74878> for more information

error[E0658]: const generics are unstable
  --> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/flatbuffers-2.0.0/src/array.rs:27:23
   |
27 | impl<'a, T: 'a, const N: usize> Debug for Array<'a, T, N>
   |                       ^
   |
   = note: see issue #74878 <https://github.com/rust-lang/rust/issues/74878> for more information

error[E0658]: const generics are unstable
  --> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/flatbuffers-2.0.0/src/array.rs:39:23
   |
39 | impl<'a, T: 'a, const N: usize> Array<'a, T, N> {
   |                       ^
   |
   = note: see issue #74878 <https://github.com/rust-lang/rust/issues/74878> for more information

error[E0658]: const generics are unstable
  --> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/flatbuffers-2.0.0/src/array.rs:59:36
   |
59 | impl<'a, T: Follow<'a> + 'a, const N: usize> Array<'a, T, N> {
   |                                    ^
   |
   = note: see issue #74878 <https://github.com/rust-lang/rust/issues/74878> for more information

error[E0658]: const generics are unstable
  --> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/flatbuffers-2.0.0/src/array.rs:73:39
   |
73 | impl<'a, T: Follow<'a> + Debug, const N: usize> Into<[T::Inner; N]> for Array<'a, T, N> {
   |                                       ^
   |
   = note: see issue #74878 <https://github.com/rust-lang/rust/issues/74878> for more information

error[E0658]: const generics are unstable
  --> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/flatbuffers-2.0.0/src/array.rs:83:36
   |
83 | impl<'a, T: Follow<'a> + 'a, const N: usize> Follow<'a> for Array<'a, T, N> {
   |                                    ^
   |
   = note: see issue #74878 <https://github.com/rust-lang/rust/issues/74878> for more information

error[E0658]: const generics are unstable
  --> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/flatbuffers-2.0.0/src/array.rs:91:52
   |
91 | pub fn emplace_scalar_array<T: EndianScalar, const N: usize>(
   |                                                    ^
   |
   = note: see issue #74878 <https://github.com/rust-lang/rust/issues/74878> for more information

error[E0658]: const generics are unstable
   --> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/flatbuffers-2.0.0/src/array.rs:110:36
    |
110 | impl<'a, T: Follow<'a> + 'a, const N: usize> IntoIterator for Array<'a, T, N> {
    |                                    ^
    |
    = note: see issue #74878 <https://github.com/rust-lang/rust/issues/74878> for more information

error[E0658]: const generics are unstable
   --> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/flatbuffers-2.0.0/src/array.rs:120:31
    |
120 | pub fn array_init<F, T, const N: usize>(mut initializer: F) -> [T; N]
    |                               ^
    |
    = note: see issue #74878 <https://github.com/rust-lang/rust/issues/74878> for more information

error: aborting due to 9 previous errors

For more information about this error, try `rustc --explain E0658`.
error: could not compile `flatbuffers`.

To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: build failed

== Compilation error in file lib/explorer/polars_backend/native.ex ==
** (RuntimeError) Rust NIF compile error (rustc exit code 101)
    (rustler 0.22.2) lib/rustler/compiler.ex:36: Rustler.Compiler.compile_crate/2
    lib/explorer/polars_backend/native.ex:4: (module)
    (stdlib 3.15) erl_eval.erl:685: :erl_eval.do_apply/6
could not compile dependency :explorer, "mix compile" failed. You can recompile this dependency with "mix deps.compile explorer", update it with "mix deps.update explorer" or clean it with "mix deps.clean explorer"
The command '/bin/sh -c mix deps.get --only prod &&     mix deps.compile' returned a non-zero code: 1

The const generics MVP was stabilized in Rust 1.51.

You need to update the image that you’re using, the tutorial on fly.io uses an old version of alpine that has an old rust version than the 1.51 that @cschmatzler mentioned.

Change the first line from FROM hexpm/elixir:1.12.1-erlang-24.0.1-alpine-3.13.3 AS build to FROM hexpm/elixir:1.12.3-erlang-24.1.4-alpine-3.14.2 AS build to use a more recent one.

Also, you don’t need to insert new lines to add each new package, apk add works like apt, dnf, pacman and every other linux package manager, you can do it all in one line.

RUN apk add --no-cache build-base npm git rust cargo

Thank you a lot.
I am a little proud on myself that I have figured that out myself, too.

Now I run into problems that seem not to be connected to Docker. This appears during compiling:

18:54:00.654 [warn]  The on_load function for module Elixir.Explorer.PolarsBackend.Native returned:
{:error,
 {:load_failed,
  'Failed to load NIF library: \'Error relocating /app/_build/prod/lib/explorer/priv/native/libexplorer.so: (null): initial-exec TLS resolves to dynamic definition in /app/_build/prod/lib/explorer/priv/native/libexplorer.so\''}}

and the fly.io logs show:

Recent Logs
2021-11-02T19:25:43.000 [info]     exception exit: {{shutdown,
2021-11-02T19:25:43.000 [info]                          {failed_to_start_child,kernel_safe_sup,
2021-11-02T19:25:43.000 [info]                              {on_load_function_failed,
2021-11-02T19:25:43.000 [info]                                  'Elixir.Explorer.PolarsBackend.Native'}}},
2021-11-02T19:25:43.000 [info]                      {kernel,start,[normal,[]]}}
2021-11-02T19:25:43.000 [info]       in function  application_master:init/4 (application_master.erl, line 142)
2021-11-02T19:25:43.000 [info]     ancestors: [<0.1879.0>]
2021-11-02T19:25:43.000 [info]     message_queue_len: 1
2021-11-02T19:25:43.000 [info]     messages: [{'EXIT',<0.1881.0>,normal}]
2021-11-02T19:25:43.000 [info]     links: [<0.1879.0>,<0.1878.0>]
2021-11-02T19:25:43.000 [info]     dictionary: []
2021-11-02T19:25:43.000 [info]     trap_exit: true
2021-11-02T19:25:43.000 [info]     status: running
2021-11-02T19:25:43.000 [info]     heap_size: 376
2021-11-02T19:25:43.000 [info]     stack_size: 28
2021-11-02T19:25:43.000 [info]     reductions: 165
2021-11-02T19:25:43.000 [info]   neighbours:
2021-11-02T19:25:43.000 [info] =INFO REPORT==== 2-Nov-2021::19:25:43.758109 ===
2021-11-02T19:25:43.000 [info]     exited: {{shutdrnel
2021-11-02T19:25:43.000 [info]     exited: {{shutdown,
2021-11-02T19:25:43.000 [info]                  {failed_to_start_child,kernel_safe_sup,
2021-11-02T19:25:43.000 [info]                      {on_load_function_failed,
2021-11-02T19:25:43.000 [info]                          'Elixir.Explorer.PolarsBackend.Native'}}},
2021-11-02T19:25:43.000 [info]              {kernel,start,[normal,[]]}}
2021-11-02T19:25:43.000 [info]     type: permanent
2021-11-02T19:25:44.000 [info] {"Kernel pid terminated",application_controller,"{application_start_failure,kernel,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,'Elixir.Explorer.PolarsBackend.Native'}}},{kernel,start,[normal,[]]}}}"}
2021-11-02T19:25:44.000 [info] Kernel pid terminated (application_controller) ({application_start_failure,kernel,{{shutdown,{failed_to_start_child,kernel_safe_sup,{on_load_function_failed,'Elixir.Explorer.PolarsBackend.Native'}}},{
2021-11-02T19:25:44.000 [info] Crash dump is being written to: erl_crash.dump...done
2021-11-02T19:25:44.000 [info] Main child exited normally with code: 1
2021-11-02T19:25:44.000 [info] Starting clean up.
***v1 failed - Failed due to unhealthy allocations - no stable job version to auto revert to and deploying as v2 

Is anyone able to give me a hint whether that could be an elixir, explorer, rust or docker issue?

My interpretation of the following error message is that the elixir package Explorer cannot “access” Rust in my docker image:

18:54:00.654 [warn]  The on_load function for module Elixir.Explorer.PolarsBackend.Native returned:
{:error,
 {:load_failed,
  'Failed to load NIF library: \'Error relocating /app/_build/prod/lib/explorer/priv/native/libexplorer.so: (null): initial-exec TLS resolves to dynamic definition in /app/_build/prod/lib/explorer/priv/native/libexplorer.so\''}}

I have a hard time understanding the error message initial-exec TLS resolves to dynamic definition . From my googling TLS could stand for “Transport Layer Security” or “Thread Local Storage”. I guess the later, because it knows a access mode “initial-exec” (IBM Documentation). But still no glue why that fails.

btw: Is it possible to test whether Rust works in my docker image?

Any help is very appreciated.

alpine can provide it’s challenges using rustler.. try following these docker steps GitHub - avencera/fast_rss: Fast Elixir RSS feed parser, a NIF wrapper around the Rust RSS crate · GitHub

Thanks, seems similar to what I do, the error remains

My dockerfile:

###
### First Stage - Building the Release
###
FROM hexpm/elixir:1.12.3-erlang-24.1.4-alpine-3.14.2 AS build

# This step installs all the build tools we'll need
RUN apk update && \
    apk upgrade --no-cache && \
    apk add --no-cache \
    git \
    curl \
    build-base \
    libgcc  && \
    mix local.rebar --force && \
    mix local.hex --force

# install rustup
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
ENV RUSTUP_HOME=/root/.rustup \
    RUSTFLAGS="-C target-feature=-crt-static" \
    CARGO_HOME=/root/.cargo  \
    PATH="/root/.cargo/bin:$PATH"

# prepare build dir
WORKDIR /app

# extend hex timeout
ENV HEX_HTTP_TIMEOUT=20

# set build ENV as prod
ENV MIX_ENV=prod
ENV SECRET_KEY_BASE=nokey

# Copy over the mix.exs and mix.lock files to load the dependencies. If those
# files don't change, then we don't keep re-fetching and rebuilding the deps.
COPY mix.exs mix.lock ./
COPY config config


# install build dependencies
RUN apk add --no-cache build-base npm

RUN mix deps.get --only prod && \
    mix deps.compile

# install npm dependencies
COPY assets/package.json assets/package-lock.json ./assets/
RUN npm --prefix ./assets ci --progress=false --no-audit --loglevel=error

COPY priv priv
COPY assets assets

# NOTE: If using TailwindCSS, it uses a special "purge" step and that requires
# the code in `lib` to see what is being used. Uncomment that here before
# running the npm deploy script if that's the case.
COPY lib lib

# build assets
RUN npm run --prefix ./assets deploy
RUN mix phx.digest

# copy source here if not using TailwindCSS
# COPY lib lib

# compile and build release
COPY rel rel
RUN mix do compile, release

###
### Second Stage - Setup the Runtime Environment
###

# prepare release docker image
FROM alpine:3.13.3 AS app
RUN apk add --no-cache libstdc++ openssl ncurses-libs

RUN apk update && \
    apk add --no-cache \
    bash \
    libgcc \
    openssl-dev

WORKDIR /app

RUN chown nobody:nobody /app

USER nobody:nobody

COPY --from=build --chown=nobody:nobody /app/_build/prod/rel/petal ./

ENV HOME=/app
ENV MIX_ENV=prod
ENV SECRET_KEY_BASE=nokey
ENV PORT=4000

CMD ["bin/petal", "start"]
´´´