@aseigo: no problem, I don’t described this library yet
This library will allow you to declare api/implementation/feature similarly to protocol/implementation/function. My implementations are identified by atom and all will be fully dynamically. For example you could register api or implementation when you want (dynamically loaded files), so you load Elixir code on yourself and call register specified module … and there is much more …
I’m using GenServer that in init function reads configuration and automatically initializes apis and their implementations saving them to state. I added extra possibility to register api and implementation at runtime, because you wrote that it could be useful. As you said scanning beam files is not so easy as it originally looks, so what you need to do is just load something (beam file json, or somethimg else) and fetch/generate from them api and implementations definitions. Finally you need only to register them. I have also one extra feature: when some async calls (like registration api or implementation) are executed asynchronously then ExApi dispatch event using ^Register.dispatch/2.
I’d been meaning to make a plugin API for elixir, I was thinking of doing a registration and just letting plugins that get loaded into the system register themselves into it.
@OvermindDL1: As soon as you can compile file with my library as dependency (especially macros) then you can register them. Of course name your api how you would like. API could be named MyApp.Plugin and then you can write MyApp.Loader that simply compiles file, gets file name (as plugin identificator) and register that plugin as a MyApp.Plugin implementation.
I’m making this firstly because I don’t want to implement protocol for empty structs that does not look good - at least for me.
@OvermindDL1: I saw it, but my is little different. I need to write one mix task and add some tests then I release it and you will see my idea. It could be great if you can review my code especially my work on macros, because it’s first time I’m working on them so much.
@OvermindDL1: I’m writing tests for it now.
You said that working on *.beam files is not good way in some situations (edge-cases).
I’m using: &Code.load_file/2 to load .ex file from test/fixtures/ project directory. This file contains only import call (for calling macro to define api) + of course call that macro - nothing more.
Then I’m using my register method passing only module name. This calls only one auto-generated API function and only work on data that this method returns - no beam or other files are used here.
Is it what you want when you said about dynamically load files or should I test it in another way?
The question becomes: “Will it work if you have no ex files, no exs files, no beam files, but the code is, say, hot-loaded into the running ERTS from, say, across a network while the server had already been up for days?”
That above is a classic use-case for plugins into, say, a Forum like this one, you can load plugins on the fly or disable them again.
@OvermindDL1: hmm, as far as I know (understand how it works):
I think that any file that is valid for Elixir compiler (again, see that or similar function) is good. Server needs to have only compiled my library as dependency (to compile macros used in such file). Nothing stops you to use that method after any time.
I don’t yet read about Erlang Run-Time System Application (ERTS), so I can say nothing about it now.
Here are the steps to make it work:
Add my library as dependency
Compile your project with all dependencies
Run project
On specified action (like form submit) call your plug-in loader mode for example:
4.0 Download source file if needed
4.1 Call function that compiles it
4.2 Call my library registration method to initialize API and/or it’s implementations.
My registration method requires only module name + of course registering API before its implementations. It does not checks if module is loaded or not.
Also I currently don’t have implemented a way to restore it’s state (in case restart application), but you can add into database boolean field and simply for each plug-in that is enabled - enable it at start-up.
However I implemented ability to auto-register already compiled (in project) APIs and implementations, (by standard Elixir configuration), so in normal (not dynamically loading code) way you do not need to call register function for each your module.
If you want I can also add ability to restore state (for example save module names of APIs and implementations to two files and at start-up read that files and try to auto-register modules).
The question is: What should I do if I can restore state and have specified modules in configuration?.
I can merge them (so I will have defaults + dynamically added) or load only from that two files skipping configuration (in case some of defaults were unregistered).
We talked also about security for plug-ins and I don’t believe that this is good way to load plug-ins from untrusted source.
Note: I’m using GenServer to keep my state.
I don’t written yet code that sync any data across all nodes. I only created setup of 3 nodes that automatically talked with each other, but I only specified them invm.args` and don’t used any function to send message. Do you have an example to dispatch message for other nodes and add handler for that message?
@OvermindDL1: My macros simply generate modules with extra functions + info that I read when registering it giving module name to functions in my library. I’m storing all data in memory using GenServer state.
If you say that it would be helpful to sync that state between multiple nodes then give me please an example that I can modify to dispatch my message that I will create - currently I don’t send any except event (my struct) that I’m sending via &Registry.dispatch/4, because all functions to change state are called asynchronously (to handle_cast) and I’m sending this event for catching it. Maybe I can modify this part to send that event to other nodes and then update their states.
Note: I will not send any file with code to compile and load it, because it’s not a target of this library. You can optionally use it with your code loader, so you need to implement sharing code and then register it, so state could be updated in all nodes calling register function in any of them.
@OvermindDL1: It took more time than I originally expected. I took lots of time for tests, coverage, docs and specs, catching errors and edge cases. I had also serious problems with last big storm in Poland in last days. This could be released in previous week.
I also added more features than originally planned such as grouping implementations. That could be useful in case two teams (possibly working on two libraries) are working on same api. Without groups work of one team could be overwritten while registering.
Note: This release candidate does not share GenServer state across nodes. Currently I do not need this, but I could add this if you can link or write small example, so I can catch my notifications (please see notifications code at bottom of main ex_api.ex file).
Feel free to request more documentation in case something is not clear.
I have updated README.md file.
Unfortunately repository does not exists. This is issue a GitLab issue. You can follow @gitlabstatus at Twitter for more informations. Update: GitLab team have problem with cache and now repository is available.
)
