Can the Elixir team fix the tree-sitter-elixir Rust crate's repository link?

Chat & Discussions Proposals: Ideas
,
1

To @jonatanklosko and @the-mikedavis:

I see that there is a Rust crate at crates.io: Rust Package Registry but it is pointing at https://github.com/tree-sitter/tree-sitter-elixir – and that no longer exists.

Can this be fixed to correctly point to GitHub - elixir-lang/tree-sitter-elixir: Elixir grammar for tree-sitter · GitHub, please? Assuming they are even the same which they might not be.

I am looking into making a PR to GitHub - ast-grep/ast-grep: ⚡A CLI tool for code structural search, lint and rewriting. Written in Rust · GitHub so Elixir can be included in the semantic / structural code search that this tool provides.

I am asking for this because the project’s Cargo.toml needs a valid crate to point at a language grammar before the small boilerplate required to add a language can work: ast-grep/crates/language/Cargo.toml at 782b175dacaf161cd567538f01929c5e78074e9f · ast-grep/ast-grep · GitHub

/CC @josevalim

2 Likes
2

To clarify further: I can and have started a PR to add Elixir to the ast-grep tool because Cargo allows GIT dependencies just fine. But I thought it would be more… hygienic might be the word… for us to have a proper Rust crate hosted on crates.io.

3

Notably you cannot publish a release to crates.io if it contains any git dependencies, so many packages that are actually CLIs or other end-user applications may forbid introducing such changes. On the projects’ side the stance is mainly motivated to protect the ability to cargo install the package by name.

I knew this from past friction; and Specifying Dependencies - The Cargo Book mentions that near the bottom of that section, but I couldn’t trivially find a page directly on crate.io to cite.

1 Like
4

Thanks for pointing this out, I overlooked it.

And yikes, I guess in that case repairing the crate is indeed a blocker if I am to contribute to ast-grep.

5

OK, I have forked the repo and made a bunch of changes that plug in the Elixir TreeSitter implementation – it seems that the moved repo is indeed doing the same things that the old (the one that no longer exists) was doing.

The only thing that is missing is replacing this line in Cargo.toml:

tree-sitter-elixir = { git = "https://github.com/elixir-lang/tree-sitter-elixir", optional = true }

with:

tree-sitter-elixir = { version = "0.1.0", optional = true }

(or whatever the version gets bumped to)

There’s a problem though, searching for $A != nil yields an error:

Multiple AST nodes are detected. Please check the pattern source $A != nil.

Looking for e.g. is_nil($A) works fine.

I have zero clue as to how TreeSitter works so this is still a mystery. Hopefully I can find some more time to experiment and see how this can be solved (if it can at all).

Here’s the forked repo. Running cargo test elixir is easy enough, and so is finding elixir.rs inside the project, if anybody wants to tinker.

F.ex. test_match("$A = 0", "a = 0"); also breaks with the “multiple nodes” error. Try and put it inside elixir.rs’s function test_elixir_pattern() and re-run cargo test elixir.

Finally, tinkering would be done like so:

cd /where/you/cloned/the/forked/repo
cargo b --release
export ASTGREP=$(pwd)/target/release/sg
cd /path/to/elixir/repo
$ASTGREP -p '$A != nil' --lang elixir
$ASTGREP -p 'is_nil($A)' --lang elixir
# etc.
6

I’ve also opened a pull request in the original repo.

7

…and posted an issue to the tree-sitter-elixir repo: Publish to crates.io with the current GitHub location · Issue #62 · elixir-lang/tree-sitter-elixir · GitHub

8

It seems that replacing one Rust macro incantation with another fixed the strange “multiple AST nodes” error. I’ll work on adding better and more convincing tests and will then ask the tool authors to merge the PR, after which we’ll be able to use ast-grep to search and replace Elixir code from the CLI. :partying_face:

…if the owners of the tree-sitter-elixir re-publish the Rust crate with the right GitHub repo URL that is.

9

tree-sitter-elixir has been fixed to have its correct GIT URL on crates.io.

I’ll proceed to enrich the tests in my PR and we’ll have Elixir support for ast-grep very soon.

1 Like
10

My PR has been merged.

We’ll have Elixir support in ast-grep come next release (so a matter of days).

3 Likes
11

ast-grep version 0.14.3 has been released and is now supporting Elixir.

Example usage:

# or "ex" for a shorthand name for Elixir.
$ sg -l elixir -p '$A != nil'
6 Likes
12

If anyone is interested, I have also made a PR to have their online playground support Elixir as well: Add Elixir support by dimitarvp · Pull Request #323 · ast-grep/ast-grep.github.io · GitHub.

And it’s merged.

Meaning that you can visit Playground | ast-grep and try out stuff without installing the tool.

3 Likes