Compiling and loading modules dynamically

I have a bit of an unconventional situation.

I have an app that generates files at runtime. Those files contain valid elixir code (modules basically). I need to compile them and load them at runtime.

I am able to do most of that with Code.compile_file, Code.require_file, and friends.

However, I need to be able to check what new modules have been compiled (or all the modules currently loaded in my app).

I’m using :application.get_key(:my_app, :modules) which lists all modules correctly when the app starts, but this doesn’t include the newly compiled modules at runtime.

Any idea how to achieve that?

All I care about is, I need to compile random modules at runtime and I need to know about all the newly generated modules that were compiled randomly at runtime.

Thanks!

I seem to remember the core team saying that there are compiler callbacks or some such now but I am not able to immediately find anything on it.

1 Like

Your comment made me dip deeper into erlang and I could :code.all_loaded() which provides a list of all loaded modules from all apps including dependencies. It’s not ideal, but it’ll do for now, but I’m still looking for a better way since this can potentially be an extremely large list.

You can add them to the list manually, however I am not sure if Mix will not overwrite it.

If you only care about newly compiled modules, and your app is the one that compiles and loads each module, you can store each module name or the file name into an Agent or your own GenServer.

1 Like

You might be able to glean something from https://github.com/falood/exsync

It watches the beam folder and loads new beam files it sees. It’s meant for development but that approach could be adapted to prod

1 Like

This might just work. I’ll try it and report the result.

Thanks for the suggestion. This is definitely a workaround, but I would feel more comfortable if there was something built-in that could handle this. I believe there’s a way with the help of erlang modules but it’s not documented clearly.

Interesting package. I’m going to take a few ideas from this. Thanks!

1 Like

Personally I’d have the files register themselves, you might have tools and such you don’t want loaded via your normal way as well after all. :slight_smile:

1 Like

hello @aesmail, any luck with finding a way to load the newly compiled modules?
I am facing the same case where I need to compile different modules dynamically. I need to make sure that the modules are loaded and accessible. If you found any result please share thanks.