OptionParser result not consistent on iex shell, but doctests pass?

ubill · January 31, 2017, 1:39am

I have tested this one with Elixir release 1.4.1 and also the current master 17043411.

In the docs

Steps:
Run iex
compare with running

elixir -r test/elixir/test_helper.exs -pr test/elixir/option_parser_test.exs

Actual running result:

iex(22)> OptionParser.parse(["--source-path", "lib", "test/enum_test.exs", "--verbose"])
{[verbose: true], ["test/enum_test.exs"], [{"--source-path", "lib"}]}

When the docs and the source code indicates

iex> OptionParser.parse(["--source-path", "lib", "test/enum_test.exs", "--verbose"])
{[source_path: "lib", verbose: true], ["test/enum_test.exs"], []}

By right, since I compiled from sourse with make clean test, and also ran the unit test individually, the test case should have been covered already, but as you can see, what I get from actual run is not consistent with the docs and tests.

Don’t know if I should raise this as an issue on the elixir-lang github, or I am the one doing something wrong there?

ubill · January 31, 2017, 2:38am

This gets even stranger when I do it this way, all on the iex shell.

iex(1)> OptionParser.parse(["--source-path", "lib", "test/enum_test.exs", "--verbose"])
{[verbose: true], ["test/enum_test.exs"], [{"--source-path", "lib"}]}    # This is not following the doctests' behaviour to begin with.

iex(5)> OptionParser.parse(["--source-path", "lib", "test/enum_test.exs", "--verbose"]) === {[source_path: "lib", verbose: true], ["test/enum_test.exs"], []}
true    # ??? v(1) === v(5) ?

iex(7)> OptionParser.parse(["--source-path", "lib", "test/enum_test.exs", "--verbose"])
{[source_path: "lib", verbose: true], ["test/enum_test.exs"], []}    # ??? How did the result change to this now?

The iex line numbers in between are just repeated invokations of the preceding command (with same results.)

Now, in a new iex session,

OptionParser.parse(["--source-path", "lib", "test/enum_test.exs", "--verbose"]) === {[verbose: true], ["test/enum_test.exs"], [{"--source-path", "lib"}]}
true    # ??? The result just takes on whatever value I check it with?

OptionParser.parse(["--source-path", "lib", "test/enum_test.exs", "--verbose"])
{[verbose: true], ["test/enum_test.exs"], [{"--source-path", "lib"}]}    # Wasn't it supposed to be the result in the doctest?

# And, now this:
iex(8)> OptionParser.parse(["--source-path", "lib", "test/enum_test.exs", "--verbose"]) === {[source_path: "lib", verbose: true], ["test/enum_test.exs"], []}
true    # ???? but wasn't it the other result ?

iex(10)> OptionParser.parse(["--source-path", "lib", "test/enum_test.exs", "--verbose"])
{[source_path: "lib", verbose: true], ["test/enum_test.exs"], []}    # ???? And now result has switched to become this different one...

Ran the same by writing my own modules and doctests to and testing with those, doctests do not catch this kind of behaviour. (or in other words I don’t know what behaviour to expect.)

This makes it impossible to rely on to write a command line app.
As this is the case, it is impossible to use this to replace Python

NobbZ · January 31, 2017, 6:15am

OptionParser does not create atoms that are not known to the system. This is because atoms are not garbage collected and a strictly limited ressource (about a million is max).

Please use :switches option as describes in the docs, to make sure that axpected atoms do exist and also to make explicit what you expect.

ubill · January 31, 2017, 7:26am

Thanks NobbZ for the explanation about the atoms, that makes sense, and I will write my code in that way:

Testing again on the iex shell:

iex(1)> OptionParser.parse(["--source-path", "lib", "test/enum_test.exs", "--verbose"])
# parse:got here ["--source-path", "lib", "test/enum_test.exs", "--verbose"] opts: []
{[verbose: true], ["test/enum_test.exs"], [{"--source-path", "lib"}]}

iex(2)> OptionParser.parse(["--source-path", "lib", "test/enum_test.exs", "--no-verbose"])
# parse:got here ["--source-path", "lib", "test/enum_test.exs", "--no-verbose"] opts: []
{[], ["test/enum_test.exs"], [{"--source-path", "lib"}, {"--no-verbose", nil}]}

iex(3)> OptionParser.parse(["--source-path", "lib", "test/enum_test.exs", "--no-verbose"], switches: [{:verbose, :boolean}, {:"source_path", :string}])
# parse:got here ["--source-path", "lib", "test/enum_test.exs", "--no-verbose"] opts: [switches: [verbose: :boolean, source_path: :string]]
{[source_path: "lib", verbose: false], ["test/enum_test.exs"], []}

iex(4)> OptionParser.parse(["--source-path", "lib", "test/enum_test.exs", "--verbose"])                                   
# parse:got here ["--source-path", "lib", "test/enum_test.exs", "--verbose"] opts: []
{[source_path: "lib", verbose: true], ["test/enum_test.exs"], []}

but it still doesn’t explain the other thing - which is why the doctests still pass? No prior declaration of :"source_path" atom anywhere beforehand

1b. The example in the doctests, which can’t be reproduced at first use on the iex shell, is propagated into generated docs.

From my test in the iex shell, it seems that repeated function calls with the same arguments will yield different results (I suppose, because I ‘accidentally’ had ended up declaring the atom :"source_path" in the shell).
This seems surprising to the idea of functional programming as you’d expect function calls with same arguments to return the same result.

To reiterate, to fulfill my current usage, if I declare the atoms beforehand with switches param it looks like I’d be ok, or if I pick up the “string” values from the “invalid” part of parse function’s return value.

Apart from that, the behaviour, doctests and the docs is confusing and requires one to know something outside from the docs beforehand.

NobbZ · January 31, 2017, 8:39am

The doctests are read as a whole, so when the system executes the function, the result was already read and parsed and as such the atom is available in the atom pool.

But you are right, that the behaviour is not only inconsistent but indeterministic and that there should be better documentation about this. Since there was a very similar issue in the bugtracker (#5708) a few days ago, Because of that a new option has been added in #5709 to address this.

Since 5709 is merged I do assume, that it will be available with elixir 1.4.2. So the only elixir versions that have the problem you describe here will probably be 1.4.0 and 1.4.1. Allowing “unsafe” atom creation is an explicit opt in for newer elixirs while it was default before 1.4.0.

ubill · January 31, 2017, 9:05am

the result was already read and parsed

I see, and so after the above, then only the doctest function was executed, if this is the case then the retval that the doctest function call output, and asserted as OK makes sense.

Thanks NobbZ, I have to note though, that I am using the elixir source code from the latest(yesterday) master-17043411 (-v: 1.5.0-dev). I am not sure if the changes in 1.4.2 are inside that yet. Anyway this is just as an FYI.

Allowing “unsafe” atom creation is an explicit opt

How do I do that which you speak of? By listing inside the switches: opt in the parse function call,
or, this is a elixir/iex cmd line switch or mix.exs/config.exs option ?

NobbZ · January 31, 2017, 9:14am

As I said, in #5709 was an option introduced which allows to create even inexisting atoms on the fly. It seems to be allow_nonexistent_atoms. Just use it as an option.

I’d be carefull though and prefer the usage of :strict or :switches anyway, because of type annotations and making it unambigous.

When I start the program as do_something --verbose 3, is 3 meant to be the verbosity level or is it an argument to do_something? Both interpretations are valid in happen in the wild. Please do yourself a favor and name your expected arguments explicitely.

ubill · January 31, 2017, 9:18am

Thanks, that’s a good point there. I feel this ambiguity myself coding in other languages and making it explicit, as elixir gives the capability to, definitely helps.

josevalim · January 31, 2017, 9:46am

Based on this discussion I have pushed improvement to docs that favor the explicit use of :switches or :strict. The “dynamic” mode has been moved to a single section explaining how it works and when to use it.

NobbZ · January 31, 2017, 10:02am

Can you point out the commit? Right now I do not see any commit by you during the last year on https://github.com/elixir-lang/elixir/commits/master/lib/elixir/lib/option_parser.ex

josevalim · January 31, 2017, 10:35am

It is there. I posted the comment here and when I went back to the terminal noticed it didn’t push it through.