Opening the source files from iex in the editor with `open` fails to find the file

Exadra37 · May 2, 2020, 11:07am

From watching this Jose video https://youtu.be/suOzNeMJXl0?t=466 at minute 7:46 I see him opening the source file for Tuple.to_list from an iex session, but if a try to the same I get an error saying the file is not available:

docker run --rm -it hexpm/elixir:1.10.1-erlang-22.3.3-debian-buster-20200224 iex                                                           130 ↵
Erlang/OTP 22 [erts-10.7.1] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Interactive Elixir (1.10.1) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> open Tuple.to_list
Could not open: "/usr/local/lib/elixir/bin/../lib/elixir/lib/tuple.ex". File is not available.
iex(2)>

or with the docker official image:

docker run --rm -it elixir                                                 
Erlang/OTP 22 [erts-10.7.1] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Interactive Elixir (1.10.3) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> open Tuple.to_list
Could not open: "/usr/local/lib/elixir/bin/../lib/elixir/lib/tuple.ex". File is not available.

If check the path in iex:

docker run --rm -it hexpm/elixir:1.10.1-erlang-22.3.3-debian-buster-20200224 iex
Erlang/OTP 22 [erts-10.7.1] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Interactive Elixir (1.10.1) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> :code.which Tuple
'/usr/local/lib/elixir/bin/../lib/elixir/ebin/Elixir.Tuple.beam'
iex(2)>

We get a different path and file name. Why is this happening?

If we drop to bash shell the file is indeed in same path reported by :code.which Tuple

docker run --rm -it hexpm/elixir:1.10.1-erlang-22.3.3-debian-buster-20200224 bash
root@ff317b8171cc:/# ls -al /usr/local/lib/elixir/lib/elixir/ebin/Elixir.Tuple.beam
-rw-r--r-- 1 root root 3564 Apr 27 12:14 /usr/local/lib/elixir/lib/elixir/ebin/Elixir.Tuple.beam

So for some reason the function open doesn’t work in the same way as seen in the Jose video for the Elixir core, but if I try to open a file im my project it just works. I am wondering if this is anything related to be running Elixir installed in a MAC, that may use different folder structures to install Elixir when compared with Linux?

I also have the same issue when building docker images with Erlang and Elixir directly from Github releases.

To note that I don’t have Elixir installed in my host machine, because I never install any programming language or toolling in my host. Everything I can, runs from docker containers.

So I am curious if someone with Elixir installed in the Linux host is able to run open Tuple.to_list sucessefully?

gshaw · May 16, 2020, 5:19am

I watched the same video and have installed Elixir to my Mac using Homebrew and an’t get the open or any other command related to the source file location to work.

Would love to have this ability. It looks really useful.

% iex
Erlang/OTP 22 [erts-10.7.1] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe] [dtrace]

Interactive Elixir (1.10.2) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> open String.downcase
Could not open: "/usr/local/Cellar/elixir/1.10.2/bin/../lib/elixir/lib/string.ex". File is not available.
iex(2)>

Exadra37 · May 16, 2020, 8:24am

Oh thanks to let me know that also doesn’t work on Mac.

Maybe @chrismccord can elucidate us in what is different in his setup?

outlog · May 16, 2020, 8:32am

on mac with asdf I get:

iex(2)> open String.downcase

Could not open: “/Users/myuser/.asdf/installs/elixir/1.10.3-otp-22/bin/…/lib/elixir/lib/string.ex”. Please set the ELIXIR_EDITOR or EDITOR environment variables with the command line invocation of your favorite EDITOR.

which gives us enough to find Using Elixir 1.5's open command with terminal Emacs - DockYard

so if I do export ELIXIR_EDITOR="subl" and then iex - open String.downcase works

Exadra37 · May 16, 2020, 8:36am

Nice to know that for you is only missing the editor variable, but for us the error is another… doesn’t find the file

Something really different across setups. Would love to know what…

outlog · May 16, 2020, 8:45am

docker is the difference… are you remote iex’ing into the docker? (sorry don’t know/understand docker cmds - docker is horrible on mac… fs/high cpu usage issues)

NobbZ · May 16, 2020, 8:56am

The sources need to exist at the location that is referred to in the debug symbols of s module.

Does brew provide those? Elixir is complaining about not beeing able to find those files.

Exadra37 · May 16, 2020, 8:57am

No Docker is not the difference per se:

So @gshaw installed in in a Mac with Homebrew and still have the same problem as me.

No, I am running in my laptop only.

Docker commands are to build and get you inside the container, that in the end of the day a container is just another Linux server, thus it’s OS file system is the same, but doesn’t have all the packages you may find in a Linux desktop.

The problem is that when using the official installers of Linux distributions its common to end up with half backed Erlang installations and what is there tends to be located in different paths then the expected ones.

My docker images builds Elixir and Erlang from the source, therefore everything should be located in the expected paths, but if not I would love to know what are the options that I need to use when building from source.

gshaw · May 16, 2020, 4:13pm

At least for me I think it must be how Homebrew installs Elixir. The contents of the folder IEx is trying to find the source only contains .beam files and a single elixir.app file.

% pwd
/usr/local/Cellar/elixir/1.10.2/lib/elixir

% find .
.
./ebin
./ebin/Elixir.String.Tokenizer.beam
./ebin/Elixir.MatchError.beam
./ebin/elixir_locals.beam
./ebin/Elixir.Enumerable.HashSet.beam
...

hassan · May 16, 2020, 4:32pm

FWIW it works out of the box on my Mac with Elixir installed via asdf and ELIXIR_EDITOR=/usr/local/bin/atom

Exadra37 · May 16, 2020, 6:05pm

Huuumm… It looks that asdf installs Elixir and Erlang in the correct paths, but not homebrew or my install from source.

Need to figure out if this is really the case

Exadra37 · May 16, 2020, 6:07pm

Where can I find this debug symbols or can customize the path on them?

NobbZ · May 16, 2020, 7:16pm

I’m not sure if there is tooling to change those in a BEAM file.

Brew seems to not keep the source code, or never installs it.

If you want to use that functionality use a means of installation that keeps the source code in corresponding location.

Many system package managers do indeed remove the source code to save you some bytes of Harddisk.

Exadra37 · May 16, 2020, 8:29pm

I don’t use package managers, instead I install from the Github repo branch/tag:

install-erlang-from-git-branch.bash:

#!/bin/bash

set -eu

Main()
{
  ##############################################################################
  # INPUT
  ##############################################################################

    local git_branch="${1? Missing git branch or tag from where we want to install Erlang !!!}"


  ##############################################################################
  # VARS
  ##############################################################################

    local branch="${git_branch#OTP-*}"

    local first_character="${git_branch:0:1}"

    printf "\nGIT BRANCH FIRST CHARACTER: ${first_character}\n"

    case ${first_character} in
      [0-9] )
        local git_branch="OTP-${git_branch}"
      ;;
    esac


  ##############################################################################
  # CONSTANTS
  ##############################################################################

    # TODO:
    #   * some of the dependencies are in the build-essential package, thus we
    #     need to check the ones that already exist, so that we will only remove
    #     the ones we installed.

    # $ apt install build-essential
    # Reading package lists... Done
    # Building dependency tree
    # Reading state information... Done
    # The following additional packages will be installed:
    #   binutils cpp cpp-6 dpkg-dev fakeroot g++ g++-6 gcc gcc-6 libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl libasan3
    #   libatomic1 libcc1-0 libcilkrts5 libdpkg-perl libfakeroot libfile-fcntllock-perl libgcc-6-dev libgomp1 libisl15 libitm1
    #   liblocale-gettext-perl liblsan0 libmpc3 libmpfr4 libmpx2 libquadmath0 libstdc++-6-dev libtsan0 libubsan0 make
    # Suggested packages:
    #   binutils-doc cpp-doc gcc-6-locales debian-keyring g++-multilib g++-6-multilib gcc-6-doc libstdc++6-6-dbg gcc-multilib manpages-dev
    #   autoconf automake libtool flex bison gdb gcc-doc gcc-6-multilib libgcc1-dbg libgomp1-dbg libitm1-dbg libatomic1-dbg libasan3-dbg
    #   liblsan0-dbg libtsan0-dbg libubsan0-dbg libcilkrts5-dbg libmpx2-dbg libquadmath0-dbg libstdc++-6-doc make-doc
    # The following NEW packages will be installed:
    #   binutils build-essential cpp cpp-6 dpkg-dev fakeroot g++ g++-6 gcc gcc-6 libalgorithm-diff-perl libalgorithm-diff-xs-perl
    #   libalgorithm-merge-perl libasan3 libatomic1 libcc1-0 libcilkrts5 libdpkg-perl libfakeroot libfile-fcntllock-perl libgcc-6-dev libgomp1
    #   libisl15 libitm1 liblocale-gettext-perl liblsan0 libmpc3 libmpfr4 libmpx2 libquadmath0 libstdc++-6-dev libtsan0 libubsan0 make
    # 0 upgraded, 34 newly installed, 0 to remove and 0 not upgraded.
    # Need to get 33.8 MB of archives.
    # After this operation, 139 MB of additional disk space will be used.

    local INSTALL_BUILD_DEPENDENCIES="
      autoconf
      dpkg-dev
      gcc
      g++
      make
      libncurses5-dev
      unixodbc-dev
      libssl-dev
      libsctp-dev
      libwxgtk3.0-dev
      default-jdk
      fop
      libxml2-utils
      xsltproc
    "

    local REMOVE_BUILD_DEPENDENCIES="
      autoconf
      libncurses5-dev
      unixodbc-dev
      libssl-dev
      libsctp-dev
      libwxgtk3.0-dev
      default-jdk
      fop
      libxml2-utils
      xsltproc
    "

    local RUNTIME_DEPENDENCIES="
      libcanberra-gtk-module
      procps
      libncurses5
      libwxbase3.0-0v5
      libwxgtk3.0-0v5
      libodbc1
      libssl1.1
      libsctp1
      man
    "

  ##############################################################################
  # EXECUTION
  ##############################################################################

    printf "\nGIT BRANCH: ${git_branch}\n"

    # INSTALL DEPENDENCIES
    apt update
    apt -y -q install --no-install-recommends ${INSTALL_BUILD_DEPENDENCIES} ${RUNTIME_DEPENDENCIES}

    # DOWNLOAD ERLANG FROM A GITHUB BRANCH OR TAG
    git clone --depth 1 --branch "${git_branch}" https://github.com/erlang/otp.git OTP

    # INSTALL ERLANG
    cd OTP
    export ERL_TOP="${PWD}"
    ./otp_build autoconf
    gnuArch="$(dpkg-architecture --query DEB_BUILD_GNU_TYPE)"
    ./configure --build="$gnuArch"
    make -j$(nproc)
    make install
    cd -
    rm -rf OTP

    # SYSTEM CLEANUP
    # we don't need them examples and doc folders, and to see docs just use like:
    #   $ erl -man ets
    find /usr/local -name examples | xargs rm -rf
    find /usr/local -name doc | xargs rm -rf
    apt -y auto-remove ${REMOVE_BUILD_DEPENDENCIES}

    # SMOKE TEST
    erl -eval 'erlang:display(erlang:system_info(otp_release)), halt().'  -noshell
    erl -eval 'io:fwrite("~1p~n", [lists:sort(erlang:loaded())]), halt().'  -noshell

    # To list all erlang modules installed:
    # erl> rp(lists:sort(erlang:loaded())).
}

Main "${@}"

install-from-git-branch.bash

#!/bin/bash

set -eu

Main()
{
  ##############################################################################
  # INPUT
  ##############################################################################

    local git_branch="${1? Missing git branch from where we want to install Elixir !!!}"


  ##############################################################################
  # VARS
  ##############################################################################

    local remove_make=false

    local first_character="${git_branch:0:1}"

    printf "\nFIRST CHARACTER: ${first_character}\n"

    case ${first_character} in
      [0-9] )
        local git_branch="v${git_branch}"
      ;;
    esac


  ##############################################################################
  # EXECUTION
  ##############################################################################

    printf "\nGIT BRANCH: ${git_branch}\n"

    # INSTALL DEPENDENCIES
    if ! which make; then
      # just making sure that make is installed
      apt update
      apt -y install make
      local remove_make=true
    fi

    # DOWNLOAD ELIXIR FROM A GITHUB BRANCH OR TAG
    git clone --depth 1 --branch "${git_branch}" https://github.com/elixir-lang/elixir.git

    # INSTALL ELIXIR
    cd elixir
    make clean test
    make install clean
    cd -

    # CLEANUP
    rm -rf elixir

    # Do not remove make if installed
    if [ "${remove_make}" = "true" ]; then
      apt -y auto-remove make
    fi

    # SMOKE TEST
    elixir --version
}

Main "${@}"

I remove some stuff, like the repo I clone to install from source or it’s this source code in the repo that I need to keep in the expected paths?

NobbZ · May 16, 2020, 8:43pm

Yes, exactly that is the sourcecode which’s location gets stored in the BEAM files…

Exadra37 · May 16, 2020, 8:48pm

Aaah daaammm… I though that the make script was doing all it should

I will fix my bash script and let you know

gshaw · May 19, 2020, 6:44am

I needed to make two changes:

Uninstall Erlang and Elixir from Homebrew and install using asdf
Tell IEx how to use VS Code to jump to a line number:
export ELIXIR_EDITOR="code --goto __FILE__:__LINE__"