dorgan

dorgan

Sourceror - Utilities to work with Elixir source code

Now that the next Elixir version will add Code.quoted_to_algebra/2 and Code.string_to_quoted_with_comments/2, we’re able to take some source code, parse it, change it and turn it back to formatted text. There are a couple gotchas if you change the ast in certain ways: since quoted_to_algebra requires the ast and comments to be given as separate arguments, we need to reconcile the line numbers of ast nodes and comments if we want the comments to be correctly placed.

So I wrote Sourceror, an (experimental) library that provides utilities to perform manipulations of the source code. I’m still working on more docs and examples(and tests), but this is an example of a function that expands multi alias syntax(ie: Foo.{Bar, Baz}) into their own lines:
https://github.com/doorgan/formatter/blob/main/lib/examples/multi_alias.ex#L32-L50

Or a function to add a dependency to mix.exs ala npm install:
https://github.com/doorgan/formatter/blob/main/lib/examples/deps_add.ex#L34-L76

Since the new functions are only available in Elixir master, Sourceror depends on Elixir 1.13.0-dev and can only be installed via git dependency.

Most Liked

dorgan

dorgan

Sourceror is now available on hex.pm and supports Elixir versions down to 1.10 :slight_smile:

dorgan

dorgan

I finally got some time work on Sourceror, and there have been a bunch of bug fixes, and a new 0.9 version! Thanks to the folks that helped to test and iron out issues in both my comments syncer and Elixir 1.13 release candidate :slight_smile:

There are some slight changes to the API, and more functionality is exposed. Also, the notebooks are now available as guide pages in hexdocs.

v0.9.0

1. Enhancements

  • [Sourceror] to_string/2 now supports options for Code.quoted_to_algebra, like locals_without_parens
  • [Sourceror] get_range/2 no longer considers comments when calculating the range. This can be enabled by passing the include_comments: true option
  • [Sourceror.Patch] Introduced Sourceror.Patch with utilities to generate patches for the most common rewriting operations
  • [Sourceror.Identifier] Sourceror.Identifier is now public

v0.8.x summary of bug fixes

  • [Sourceror] Fixed comment spacing on binary operators
  • [Sourceror] Take comment end of line counts into account to preserve spacing
  • [Sourceror] Fixed an issue that caused comments in lists to be misplaced
  • [Sourceror] Fixed issues that caused comments to be misplaced.
  • [Sourceror] Updated internal normalizer to match latest Elixir 1.13 version.
  • [Sourceror] Fixed an issue that caused newlines to be wrongly removed.
  • [Sourceror] Fixed an issue that caused comments in pipelines to be misplaced.
  • [Sourceror] Fixed issue that prevented keyword lists from preserving their
    original format in tuples.
  • [Sourceror] get_range/1 now properly handles naked AST lists, like the ones
    coming from partial keyword lists, or stabs like a -> b.
  • [Sourceror] get_range/1 now handles partial keyword list syntax instead of
    crashing.
  • [Sourceror.Zipper] down/1 now correctly uses nil as the right siblings if
    the branch node has a single child.
  • [Sourceror] Sourceror.get_range/1 now correctly calculates the range when
    there is a comment in the same line as the node.
dorgan

dorgan

The target audience is primarily tool authors, like elixir-ls or credo.

Yes, those are the kind of use cases I had in mind :slight_smile: The Sourceror.to_string/2 function has an option to set the indentation level of the resulting code for that particular use case. I will probably add functions to know how many lines an ast node uses, so one could replace a line range instead of the whole file.

This started while exploring ways to allow credo to autofix some of the issues it finds, the multi alias expansion example derived from that.

Mostly finding what people find most cumbersome or confusing to do, I think the most important thing right now is to start experimenting. There may be some bugs in Code.quoted_to_algebra/2 too, some experiments in that front would be nice as well so we can add more regression tests to core Elixir :slight_smile:

Where Next?

Popular in Announcing Top

OvermindDL1
I created a new library (rather I pulled out a couple files from my big project), it manages an operating system PID file for the BEAM. ...
New
deadtrickster
I’ve just released stable versions of my Prometheus Elixir libs: Elixir client [docs]; Ecto collector [docs]; Plugs instrumenter/Export...
New
riverrun
I’ve just released version 3 of Comeonin, a password hashing library. The following small changes have been made: changes to the NIF c...
New
tmbb
I’ve been working on two packages (not on hex.pm yet) to build admin interfaces for phoenix apps: bureaucrat - which contains a bunch ...
New
blatyo
The best overview for how things are tied together is this presentation. Modules and functions are pretty well documented at this point, ...
New
wojtekmach
Hey everyone! Req is an HTTP client for Elixir that I’ve been working on for quite some time. There is already a lot of HTTP clients out...
New
josevalim
Hello everyone, We have just released NimbleCSV which is a small and fast CSV parsing library for Elixir. It allows developers to define...
New
zachdaniel
Ash Framework What is Ash? Ash Framework is a declarative, resource-oriented application development framework for Elixir. A resource can...
New
markmark206
simple_feature_flags is a tiny package that lets you turn features on or off based on which environment (e.g. localhost, staging, product...
New
New

Other popular topics Top

senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
TunkShif
This post is an instruction guide to help you setup your Neovim for Elixir development from scratch. It includes general information on h...
274 41539 114
New
Nvim
Anybody knows a comprehensive comparison of Django and Phoenix, thanks for the help. Where are they similar? Where do they differ the m...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
Lily
In templates/appointment/index.html.eex: <%= for appointment <- @appointments do %> <tr> <td><%= appoi...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New

We're in Beta

About us Mission Statement