Hey there!
Stylistic note first: Elixir convention is 2 space indentation, examples with 4 spaces or tabs tend to look wrong. This applies both to the examples as well as the Exkorpion codebase itself.
Also in the code base you have camel case functions and function arguments with a trailing _ like https://github.com/wesovilabs/exkorpion/blob/master/lib/exkorpion/executor.ex#L11. snake case function names are idiomatic, and I’m not sure what the purpose of the trailing _ is. Elsewhere there are camelcase variable names, and the snake case convention applies to those too.
The idea is to make life easier to those developers who want to work under BDD.
What would you say makes life difficult for BDD developers?
Below a sample of how tests look when using Exkorpion
I think the simplicity of the examples doesn’t really help you here, because I have a hard time seeing how those examples are better than
describe "sum operation" do
test "sum positive numbers" do
assert sum(1, 2) == 3
end
test "sum negative numbers" do
assert sum(-1, -2) == -3
end
end
I appreciate how you’re trying to provide semantic meaning with the whole with given when thing but I’m concerned that the indirection just makes readability harder. For example
it "does multiple operations depending on vairable input" do
%{
with: fn ->
[
%{param1: 2, param2: 3, result: 5, op: fn a,b -> sum(a,b) end},
%{param1: 3, param2: -2, result: 5, op: fn a,b -> subs(a,b) end}
]
end,
given: fn ctx ->
%{a: ctx.param1, b: ctx.param2}
end,
when: fn ctx ->
%{c: ctx.op.(ctx.a, ctx.b)}
end,
then: fn ctx ->
assert ctx.c === ctx.result
end
}
end
Really just boils down to:
test "multiple operations" do
assert sum(2, 3) == 5
assert subs(3, -2) == 5
end
More problematically, when a test fails it serves to obscure the actual source of the error. For example with that last block if you change the results to 7 so as to make the tests fail the error message is
** (Exkorpion.Error.AssertionError) Assertion with === failed
stacktrace:
test/testing_test.exs:43: (test)
Unfortunately have we have lost the wonderful LHS / RHS comparison we normally get with ExUnit
code: sum(1, 1) == 3
lhs: 2
rhs: 3
stacktrace:
test/testing_test.exs:8: (test)
and Line 43 (indicated in the Exkorpion error output) is actually the line of the it "blah" line, not the line we actually make the assertion on.
When Exkorpion gives me an error I have to look through the test code to actually spot the line on which we are asserting but even when I find it it still isn’t clear what’s wrong because there’s several lines worth of indirection between the assertion and actual values. If there are several tests listed (like in your last example) I don’t even know which one it was that failed!
I hope this doesn’t come off as overly negative, I am always very appreciative when users new to a language contribute to that language, it always says something good about the user. I’m just concerned that in the case of this particular library the testing experience is a step backward from what ExUnit provides by default.
