Elixir → JavaScript Porting Initiative

I have an idea for a (probably non-trivial) improvement to the test suite. While porting :erlang.make_tuple/2, I made a mistake and wrote a test checking that making a tuple with size 0 raises an error. This is not true, an empty tuple is returned instead, but the test passed, since my implementation raised an error.

Would it be possible to compare the result of the JS implementation with the result of the actual Erlang function? I assume this would require implementing at least one of term_to_binary and binary_to_term. Does this seem too difficult to be worth the effort?

Thanks for the feedback! Have you seen the “Server-Side Consistency Tests” section in the Contributing Guide? The idea is that you implement matching tests in Elixir (in test/elixir/hologram/ex_js_consistency/erlang/) that mirror your JavaScript tests. These verify your JS implementation behaves identically to the actual OTP implementation. And if OTP behavior changes in the future, it’ll be caught.

Does this resolve the issue for your :erlang.make_tuple/2 case, or should we make this clearer in the docs?

Also, could you elaborate on your term_to_binary idea? I’m curious what you have in mind.

Ah, I misunderstood the point of the Elixir tests. I thought they were run using the JS implementation as well. The actual problem seems to be elsewhere, probably in `assert_error` reporting success even when there is no error. I’m investigating further.

As for the idea regarding comparing the two implementations, I think there would need to be a common serialization format, term_to_binary being the obvious choice. Then, a piece of code could be run both on the BEAM and in JS and the outputs could be compared. This should remove the need for duplicating tests.

EDIT: Yes, as far as I can tell, assert_error does not actually check that an error was raised. That explains my confusion when the empty tuple test passed even though there was no error.

Erlang consistency tests will eventually be automatically transpiled and run on the client. To enable that, we need to port some Erlang functions first (including some from phase 2). For now, matching tests are the simplest and most maintainable way to verify consistency IMO. The same applies to Elixir stdlib tests - they will be transpiled as well eventually to automatically verify consistency.

Could you share code snippets that reproduce the problem? I looked at your PR (https://github.com/bartblast/hologram/pull/363/files) and all CI checks pass, including tests using assert_error. If you have a specific case where it passes when it shouldn’t, please share the test code and what you expected vs. what happened.

I fixed the tests in the second commit, but the first commit contains the test that should not be passing.

test "raises ArgumentError when arity is not positive" do
  assert_error ArgumentError,
               build_argument_error_msg(1, "out of range"),
               {:erlang, :make_tuple, [0, :a]}
end

There is no error when applying the function, it simply returns the empty tuple. Therefore, I would expect the test to fail, since it should assert that an error was raised.

Looking at the definition of assert_error, I don’t immediately see anything asserting that there was actually an error.

Allright, I see the problem, fixed it here: https://github.com/bartblast/hologram/commit/482eac4f2fd6e74e2b15de40a221bfe3f3bf3afd

Thank you!

@bartblast I’m curious on your thoughts of what the return value of re.version should be, considering there is not a single version tied to JavaScript Regex capabilities?

❯ iex
Erlang/OTP 27 [erts-15.2.2] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:1] [jit]

Interactive Elixir (1.18.2) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> h :re.version/0

  @spec version() :: binary()

since: OTP 20.0

The return of this function is a string with the PCRE version of the system
that was used in the Erlang/OTP compilation.

                                                                                                        
❯ pcre-config --version
8.45

I simply return an empty bitstring for now, bit this test will always fail.

describe "version/0" do
  test "empty string" do
    assert :re.version() == ""
  end
end

Since JavaScript regexes aren’t compatible with the PCRE standard (which Elixir uses), Hologram will eventually transpile them to maintain compatibility.

For now, please hardcode the JavaScript implementation to return "8.44 2020-02-12". In both your JavaScript tests and Elixir consistency tests, use this regex pattern to verify the result format: ~r/^\d+\.\d+\s+\d{4}-\d{2}-\d{2}$/ (this should work in both Elixir and JS)

This way both test suites validate the version string format rather than an exact value, which keeps things consistent.

Eventually, :re.version/0 will fetch the PCRE version from the Hologram client-runtime, which will receive it from the server in the initial request. Initially we’ll be targeting a specific PCRE version to ensure consistency across the client-side regex behavior.

Thanks for flagging this!

Do we need an implementation of the Type.set() data type, in order to write functions to operate on them, e.g. :sets.to_list/1?