Special announcement of my plans due to becoming MOTM

I was surprised when I saw notification that I become MOTM so I was inspired to write my special announcement about my plans.

I posted on this forum lots of threads and now I would like to describe what I’m going to achieve.

Firstly this is a list of libraries that I’m creating / I would like to create:

  1. ExUtils (in progress) - lots of helpful methods for Elixir and Phoenix apps
  2. ExApi (in progress) - register and deregister API implementations - like protocol, but allows to add implementations at runtime and implementations are identified by atoms
  3. ExMedia - process media files: video, images and sound files - it will allow to convert formats, add watermark, manipulate, add effects and more …
  4. ExEverywhere - sending files everywhere you want (local and network devices, 3rd party storages, ftp and more) with unique API - it will support mirroring, syncing, find duplicates and more …
  5. Countriex - this library will share info about countries and their languages + language based functions like formatting numbers
  6. ExDB - PostgreSQL framework - Ecto like framework with all PostgreSQL special features and some extra features like caching. Version targets:
    a) up to version 1.0 I will use PostgreSQL
    b) if I will have enough knowledge then I will experiment with version 2.0 with ~same API, but with new pure Elixir database

and more … :slight_smile:

After I will make libraries I will focus on these projects:

  1. WebEcto - Ecto models visualisation page - you will have a preview of all models (including relations) and model them without interacting with code and finally generate an ready project or just model files
  2. Some games in Elixir
  3. Pluginex - secure plug-in manager with communication via WebSocket and privileges API - I will also make first plug-ins for it:
    a) CMS
    b) forum
    c) shop plug-in - basic seller <-> buyer plug-in that will be helpful for lots of apps
    d) invoice manager
    e) web desktop plug-in
    Finally you will able to create a site like blog only using web creator + optionally custom front-end / styles / icons etc.
  4. Fumetsu - my site based on Pluginex
    I already ordered new icons for them. Fumetsu will support as much devices as possible including HiDPI, so I will have 1024x1024 icons and up to 8k images - all in SVG + CSS3 animations.

and more … :slight_smile:

All of that work will have same license as Elixir, but don’t expect them “tomorrow” :smiley:

After Fumetsu I have other plans for future:

  1. 3D Book Reader (JavaScript + WebAssembly + OpenGL) - this is currently too complicated for me and I need to pay someone for 3D part of that plug-in project
  2. BigData algorithms in Elixir
  3. trying to at least learn about 3D Modelling and AI

and more … :slight_smile:

Currently I’m mostly focused on Elixir normal jobs, because I need to collect lots of moneys for expensive hardware that I need. One of them will be workstation focused on Elixir, BigData and 3D modelling.

Feel free to ask about any library/project that I have planned :slight_smile:

13 Likes

Just a tiny bit of curiosity. Do you have thoughts on how to implement the ExMedia parts? For example, are you looking at integrating with things like gstreamer and imagemagick (like exmagick?), or is this on another level?

1 Like

Good question!
I’m not going to reinvent the wheel, so I will look for cross-platform tools and then I will write an API for them in Elixir similarly to how ruby gems do it. I will need to look if ffmpeg have all features that my API will require. It’s well supported in lots of platforms. If it have all we need to manipulate media files then it will be easy to generate a ffmpeg shell command with generated arguments and execute it.

Nice one Tomasz :023: take your time and don’t burn yourself out - remember to have a life outside of coding as well :slight_smile:

Re ExMedia, have you considered Vips integration? It is meant to be quite a bit more performant than ImageMagic.

4 Likes

Nope I don’t know about Vips, but it looks interesting. I will look at combine this with ffmpeg (like process 10 images then join them to video file) :smiley:

1 Like

Oh I very much look forward to seeing all those! :smiley:

1 Like

For what it’s worth re: vips. It has performance with less features. The best libraries that I’ve found around media handling in other languages would aim to provide functionality available from all libraries by using the best available option on the system. In some cases you could utilize vips but offload other options to imagemagick for example.

Doing this type of thing with command line call outs and keeping it away from your application RAM is tremendously more stable as well (think Minimagick vs RMajick in Ruby).

Keep in mind, if you use the thumbnail option with imagemagick it’s tremendously faster and on part with vips. People get confused about that option thinking that it’s only for small images, but it actually just strips out all of the extra data stored within the file. Benchmarks: https://github.com/fawick/speedtest-resize

2 Likes

That sounds like a good strategy :023:

1 Like

For your Countriex plan, feel free to use/extract/copy/modify/hack the locale specific number formatting in a lib I wrote called ex_cldr

One idea I’m experimenting with is using Rustler to wrap the Rust crate image. Since I’m just getting into Rust this is taking longer than it should :slight_smile: There should be some good performance for basic operations (resize, filter, transform) on images (not other media types).

2 Likes

This one looks really nice, but I also see lots of things to do. I will include lots of code from there, but I I will also support adding custom languages and language extending, so will be able to add “English pirates language”. I see also some other things to implement.

Can you say why did you add Cldr.Math and others? I don’t see how it’s related to locales …

btw.

atomize_keys(not_a_map)
Convert map string keys to :atom keys

:smiley:

What is MOTM?

1 Like

This:

1 Like

Good call, I need to revisit some of that. Cldr.Math started life as a common library for basic maths that would work for integer, float and Decimal. Given that number formatting is rules based and those rules require math. And then I got a bit sidetracked onto some Calendar work including solar, lunar and lunisolar calendars which requires more math. And then sunrise and sunset calculations … :slight_smile:

Your message is a good prompt for a refactor of this.

BTW, since a locale is just a data definition in CLDR you can definitely add additional languages very easily.

2 Likes

Will this integrate with Phoenix contexts? I don’t know if you canget the mapping between context and Repo from introspection but it would be useful for visualization.

Alternatively, as you say, you could have a visual editor where you’d create schemas and contexts and which could call ecto/phoenix generators to generate the elixir files for you. Going from GUI to elixir code seems certainly easier than the other way around (because a GUI is nevessarily a subset of what the code can represent), but some degree of bidirectional compatibility might be possible.

For visualization, this library might be useful. It is a port of GraphViz to javascript and it generates some of the best visualizations I’ve seen.

If you’re using GravViz for visualization, my advice is to represent ecto schemas, BD tables and contexts as components with a node per field (it’s easy to rearrange such nodes into a table) and then use labeled edges for relations between schemas/tables/contexts and for the mapping between different levels (mapping from context to schema and from schema to DB table). Ideally you can generate SVG and sprinkle some JS to make ir more interactive.

Or you can go full 3D with webgl or something, but from a usability pont of view I’m not sure that’s such a good idea.

This sounds a lot like visual programming, the “trend” that no one has ever done well, but for things that can be naturally modelled as graphs (like sketching a DB schema) I find that visual approaches are much more intuitive than text most of the time.

Why would you want to use elixir for such computationally intensive purposes as BigData and 3D modelling? Maybe as a scripting language on top of something more lower level? For that case, I wonder if python wouldn’t be better. Slow as hell, but these use cases don’t seem to need much concurrency or distributed computing.

For AI it does make sense, because you might be able to distill the raw data into a high level representation that can be used with pattern matching. In that case elixir would be usefull, especially if you need multi-node communication.