Giving up on dynamic languages

[StefanHoutzager]

Had an interview about an elixir dev contractor position this week, with the tech lead and the commercial director. I have had less distracting interviews. I think I’ll have to give up maintaining my elixir knowledge and dynamic languages overall. I’ll say something about the interview and point to some discussions/presentations about typed languages, for those interested.

The interviewer had some questions about / suggestions for a function I wrote:

@spec format_HH_MM(integer) :: String.t
  defp format_HH_MM(nr) do
    nr = abs(nr)
    if nr < 10 do
      "0#{nr}"
    else
      "#{nr}"
    end
  end

Q: A doctest would be handy for this function. Did you write a unit test for it?
A: It is a private function, I could add a comment explaining it but I thought the function explains
itself, the calls to it make the use even more clear I think. I see no added value in a unit test here,
if the function had not been private.
Q: Did you write a test for a function calling this one.
A: This is an elixir learning project maintained by me only (this was an excuse, I did not feel like starting long discussions)
Q: You are using if else in this function. That is not usual in functional languages. How would you improve this function?
A: I answered with the usual alternatives. I said nothing about my opinion: in the case of this function I see no improvement with these alternatives (the interviewer clearly did).

The interviewer did not name the parameter of which the value is re"assigned". Which I find no problem either in this very short function.

Some links:

Gary Bernhardt, well known in the javascript world, seems to have found a way out of a lot of testing hassle. From test-driven to type-driven.

[…] Problems like these come up often in programming, especially in dynamic languages, but can be statically prevented without writing any automated tests and without doing any manual testing.

Execute Program

Guido van Rossum (python):

There was also an interesting discussion about maintainability of code. Rossum began by saying “I found that TypeScript is incredibly useful, and so we’re adding a very similar idea to Python… I’ve learned a painful lesson, that for small programs dynamic typing is great. For large programs, you have to have a more disciplined approach. And it helps if the language actually gives you that discipline, rather than telling you, ‘Well, you can do whatever you want.’”

A Conversation with the Creators Behind Python, Java, TypeScript, and Perl

“Type-Driven Program Synthesis” by Nadia Polikarpova

"Type-Driven Program Synthesis" by Nadia Polikarpova

She won an award from the independent national science foundation in 2020:
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1943623&HistoricalAwards=false

If the input is a formal specification, synthesis algorithms can not only create a program but also a proof that the program meets the given specification.

resyn.pdf

[amnu3387]

@StefanHoutzager

@spec format_HH_MM(integer) :: String.t
defp format_HH_MM(nr) do
nr = abs(nr)
if nr < 10 do
“0#{nr}”
else
“#{nr}”
end
end

A thing I notice with the name is that it’s really not intuitive. If I read format_HH_MM I will expect the function to return some form of strftime format, in the form of HH:MM or variants.

A better name might perhaps would be, integer_to_padded_string (?), maybe_pad_integer (?) I’m not saying that at the call site it’s not obvious what it will do, but just looking at the function in itself is confusing when reading its name (my personal opinion ofc).

There’s also the issue that either you won’t have negative numbers and the abs call is unneeded, or you have and you’re not padding with a leading zero when the values go from -9 to -1. In this case, given their are probably either hours or minutes that you’re padding, probably you don’t need abs and you should let the function call fail if a negative input is provided, or take care of the negative case as well in case it’s allowed?

As other options you might have used (not saying that you using if here is bad form, or anything, just pointing out other ways of writing it), I would also add a guard against negative values :

defp integer_to_padded_string(nr) when is_integer(nr) and nr >= 0 and nr < 10, do: "0#{nr}"
defp integer_to_padded_string(nr) when is_integer(nr) and nr > 0, do: "#{nr}"

This would make the function fail if it was not an integer, nor a positive number with a no matching clause error.

Then the spec could be written as:

@spec integer_to_padded_string(hours_or_minutes :: non_neg_integer()) :: String.t()

The variable naming is very context dependent in this case, it tells more about the context in which you will be calling this function and is not general, if you wanted the function to be used for padding any integer, then I would probably remove the argument naming in the spec.

Other option would be using case, which I find myself to be my default “go to” statement when I want to branch, and using your example.

defp format_HH_MM(nr) when is_integer(nr) do
	case abs(nr) do
	    nr when nr < 10 -> “0#{nr}”
	    nr -> “#{nr}”
	end
end

About the other points I have not much to say.

[StefanHoutzager]

@amnu3387

A thing I notice with the name is that it’s really not intuitive. If I read format_HH_MM I will expect the function to return some form of strftime format, in the form of HH:MM or variants.

Agreed, I noticed that too, forgot to mention.

@amnu3387

There’s also the issue that either you won’t have negative numbers and the abs call is unneeded, or you have and you’re not padding with a leading zero when the values go from -9 to -1 .

Am I missing something?
-9 is converted to 9 with the abs and “09” returned by the function?

The alternatives for if else you name I all made expiclit for the interviewer. Also cond do blabla.

@amnu3387

About the other points I have not much to say.

These I find the most important. There is an outside to every bubble.

[amnu3387]

@StefanHoutzager

Am I missing something?
-9 is converted to 9 with the abs and “09” returned by the function?

Yes, but it looses the sign.

[StefanHoutzager]

That is on purpose.

[lucaong]

Sorry for your bad interview experience. Unfortunately, that’s an experience that most of us can relate to.

Regarding the point of testing versus static typing, I personally don’t think that they are substitutes. There is some overlap, but only partial. Sure, there are some tests that merely assert something that could be done with types, but those are the least interesting tests. Valuable tests assert properties of our code that should likely stay invariant in face of change. They make it easier for us to evolve our code without introducing logical bugs.

As a trivial example, imagine we are writing a key/value store. With types, we can make sure that the basic interface is correct, something like (imaginary language here) get(store: Store<K, V>, key: K) => V and put(store: Store<K, V>, key: K, value: V) => Store<K, V>. This will help discovering early if we are misusing our interface. But this is not all there is to say about the correctness of our implementation. For example, we want to assert that setting a key to a value, and then getting the same key, yields that value and not something else. For that, we need a test:

test "setting a key to a value and then getting it returns the value" do
  store = put(store, "key", "value")
  assert get(store, "key") == "value"
end

Both typing and test are providing value here, and are not substitutes to each other.

[StefanHoutzager]

@lucaong

Sorry for your bad interview experience. Unfortunately, that’s an experience that most of us can relate to.

The bad experience has in great part to do with the unit-test obsession that I see here, in this group, and also with this interviewer. A special obsession of me you could name it.

@lucaong

Both typing and test are providing value here, and are not substitutes to each other.

See the work of Nadia Polikarpova (I sent links in my first mail) and you might discover new horizons.

[lucaong]

@StefanHoutzager

The bad experience has in great part to do with the unit-test obsession that I see here, in this group, and also with this interviewer. A special obsession of me you could name it.

Speaking for myself here, but I would not consider testing my obsession. It’s a practice that I find valuable in my profession, and that I think is worth mastering. If you do not, I am perfectly ok with that. I will contribute my view if I have something to add to the discussion, but I don’t need to change your mind on that. When hiring, I do consider testing skills important, and I am ok with potential candidates not wanting to work with me if they do not want to write tests. No bad blood.

@StefanHoutzager

See the work of Nadia Polikarpova (I sent links in my first mail) and you might discover new horizons.

I’d be very happy to “discover new horizons”. I am watching the video: it is definitely interesting, but I did not hear the speaker arguing that testing is useless. It is pushing forward what can be formally specified, which is quite impressive, similar to what languages like Idris do, but still not substituting the need of asserting that the runtime behavior of the program meets the business requirements.

Referring to my previous trivial example, the test asserts a property that involves both put() and get(), so it could not be in the type signature of one, or the other. How would you specify, in types, that get() should return what put() previously associated to that key?

[StefanHoutzager]

@lucaong

still not substituting the need of asserting that the runtime behavior of the program meets the business requirements

It is quite simple I think, I cited in my first message

@StefanHoutzager

If the input is a formal specification, synthesis algorithms can not only create a program but also a proof that the program meets the given specification.

business requirement == formal spec

@lucaong

How would you specify, in types, that get() should return what put() previously associated to that key?

I’ll leave that task to others here, I have some work to do for my customer now. Sorry.

[lucaong]

@StefanHoutzager

I’ll leave that task to others here, I have some work to do for my customer now. Sorry.

Well, allright then have a great day

[StefanHoutzager]

@lucaong

For that, we need a test:

test "setting a key to a value and then getting it returns the value" do
  store = put(store, "key", "value")
  assert get(store, "key") == "value"
end

Both typing and test are providing value here, and are not substitutes to each other.

Results from such tests will guarantee nothing. Imagine using a distributed kv store for real.

[StefanHoutzager]

@lucaong

Referring to my previous trivial example, the test asserts a property that involves both put() and get() , so it could not be in the type signature of one, or the other. How would you specify, in types, that get() should return what put() previously associated to that key?

As said your assertion guarantees nothing. Maybe you should say: when a function is impure you cannot trust it. Compile time with types as well as runtime with automated testing.
Likies not needed. I even don’t like it myself.

[lucaong]

@StefanHoutzager

As said your assertion guarantees nothing. Maybe you should say: when a function is impure you cannot trust it.

The code in my example is pure, no side effects there: it effectively could be code testing an Elixir map. The assertion guarantees exactly what it tests: that if you associate a key to a value and immediately after you get the value for that key, you get the value that you put in. My point is that you cannot enforce that with a type signature, hence types and tests are not substitutes.

@StefanHoutzager

Results from such tests will guarantee nothing. Imagine using a distributed kv store for real.

Distribution has no impact on the argument. How would you assert the correctness of a distributed system with static typing?

[StefanHoutzager]

@lucaong

The code in my example is pure, no side effects there

So you are not mutating state. A key value store stores state or am I missing something.

@lucaong

Distribution has no impact on the argument. How would you assert the correctness of a distributed system with static typing?

Huh? I just stated you cannot assert correctness with tests/types.

@StefanHoutzager

when a function is impure you cannot trust it. Compile time with types as well as runtime with automated testing.

[lucaong]

@StefanHoutzager

So you are not mutating state. A key value store stores state or am I missing something

The line store = put(store, key, value) and the type signature put(Store<K, V>, K, V) => Store<K, V> make it clear that the function returns a modified store, it’s not mutating state. If you want, there is state, but it’s not mutable.

But the thing is, it really doesn’t matter. You can still test code with side effects, and you cannot write a type signature that asserts that said side effect is correct. Which again is not a point against static typing, it’s saying that types and tests are not substitutes.

[StefanHoutzager]

@lucaong

types and tests are not substitutes

Agreed. Relevant discussion by Gary Bernhardt explaining this in detail: Ideology

My point is that a lot of automated tests are not relevant anymore with types. Research
by for example Nadia Polikarpova shows that even more automated testing can be made irrelevant:

@StefanHoutzager

If the input is a formal specification, synthesis algorithms can not only create a program but also a proof that the program meets the given specification.

Moreover a lot of unit testing is a waste BDD / TDD criticized

[lucaong]

If you are arguing that not all tests are useful, and that it is therefore possible to “overtest” by writing low-value tests that actually create more friction than value, I definitely agree. I also agree that static typing can help to keep the cost of change low (e.g. by making refactoring easier).

Not all tests are good. Tests are code, so bad tests are a painful reality.

I also agree that TDD is not the only true way, and definitely should not be practiced as a cult. I find it helps me guiding the solution when the problem space is clear, but I find it hinders me when I need to explore the problem first. I don’t think that TDD automatically leads to better software, I think it’s merely one technique at our disposal to use when it helps.

[StefanHoutzager]

About TDD a link that I did not post yet:
http://neverworkintheory.org/2016/10/05/test-driven-development.html
The post is by (that name should be enough to trigger interest in the whole post + the link to the academic paper, for some at least):

Dr. Greg Wilson has worked for 30 years in both industry and academia, and is the author or editor of several books on computing and two for children. He is best known as the co-founder of Software Carpentry, a non-profit organization that teaches basic computing skills to researchers.

He had a talk explicitly appreciated here Greg Wilson - What We Actually Know About Software Development, and Why We Believe It's True

I don’t program very much any more, but when I do, I feel that writing tests up front makes me more productive. I’d like to be able to say that these researchers and others must be measuring the wrong things, or measuring things the wrong way, but after so many years and so many different studies, those of us who believe might just have to accept that our self-assessment is wrong.

[StefanHoutzager]

Interesting paper from Coplien, a couple of tantalizing points

Unit testing is the least effective bug removal step that we know
(see the pdf, it contains tables that I cannot paste here)

“I need a safety net to catch inadvertent changes to program behavior; e.g., those I make
during refactoring.”
This pretends that we can change a function
in a way that changes only part of its behavior: that is, that we
can change a function without changing most aspects of its
externally observable behaviour. More to the point, it pretends
that we can have a high, but not perfect, degree of confidence
that we can write tests that examine only that portion of a
function’s behavior that that will remain constant across changes
to the code. That means that good unit test design must foresee
how the code may change. That in turn implies being able not
only to partition a method’s value mappings into disjoint cutsets
of evolution, but to be able to demonstrate how the code, or at
least the tests for the code, map onto those cutsets. I think that
programmers fool themselves into believing they can do this.
Unfortunately, there is no more preventing such assumptions
from being wrong than prevents the correctness of the change
itself being flawed. This is therefore a seriously flawed and
dangerous perspective.

https://rbcs-us.com/documents/Segue.pdf

[benwilson512]

This seems to be rapidly turning into a rehashing of Code inspections vs unit tests.

The following point applies to both this document and the last one: Anybody can put a bunch of text on the web. The existence of that PDF does not prove or debunk anything. It’s relatively long, and we’re all pretty busy. If you want us to read it, you should provide evidence that it is well regarded, or provide some kind of compelling summary. Otherwise it is unlikely to have the impact you wish.