I failed my tech challenge today for a job interview. I don't think I was given a fair chance

tech-challenge
interviews
#1

I’m finding it difficult to find employment, even more so in elixir. It would appear to me that employer’s standards are increasing at a rate that feels disproportionate to the available skill sets of the general population.

Today I was given a tech challenge that I was given 2 to 3 hours to complete. Normally I would not share the challenge but seeing as I was open about my skill level from the very beginning I feel I was done a disservice by my interviewer. Sadly this has not been the first time this has happened to me. It’s starting to cause me a great deal of distress in my personal life to the point to where I no longer want to be in this industry. I just don’t feel there is a good place for me anymore. That said I’ve been working in this industry for sometime so I can only imagine what its like for new comers. Frankly I’m sick of this meat grinder.

Any how I’m here to see if its just that I suck and I should just get out of the game and try to find some happiness else where in my life or if I’m seeing something that we are all seeing.

In short the position was posted for I believe 80k to 150k per year. That to me says there could be a wide range of skill sets but they stated the they wanted mid to senior level elixir developers. I mistakenly stated my skill sets fell into the 65k to 70k pay range before I knew what the position was listed for. From there on out I stated many times I’m not super noob but that I’m also very much learning and interested in learning more. I would rather be honest about my short coming then have it come out while on the job. That said they said they still wanted to have me do a tech challenge. Today I received the challenge and what I found was disheartening and demoralizing.

Here is the challenge as it was given to me. ( I get that posting this here is likely hurting my opportunity of finding employment and is a breach of trust but I’m at such a low point in my life I feel its better to get the truth from my peers then continually beat my self up over this )

The program accepts as arguments a list of one or more file paths (e.g. ./solution.rb file1.txt file2.txt ...).
The program also accepts input on stdin (e.g. cat file1.txt | ./solution.rb).
The program outputs a list of the 100 most common three word sequences in the text, along with a count of how many times each occurred in the text. For example: 231 - i will not, 116 - i do not, 105 - there is no, 54 - i know not, 37 - i am not …
The program ignores punctuation, line endings, and is case insensitive (e.g. “I love\nsandwiches.” should be treated the same as "(I LOVE SANDWICHES!!)")
The program is capable of processing large files and runs as fast as possible.
The program should be tested. Provide a test file for your solution.

So, set your own pace. No rush. And just so you have some bounds, the challenge was designed to take somewhere between 2 and 3 hours. You could try it against "Origin Of Species" as a test: http://www.gutenberg.org/cache/epub/2009/pg2009.txt .

I spent the first hour just trying to figure out the IO api. I also had little experience writing such algorithms as I’ve mostly have worked in website development and ended up spending a few hours looking up word count algorithms as I knew I would need to build a list of common words before I could find common sequences. I also knew I would need to dig into streams as I maybe loading large data sets and I needed to be mindful of that overhead. I ultimately hit a wall with just how I could get the cli’s api to work with both stdin or argument lists. At that point I just gave up frustrated by the whole experience feeling that opportunity passing by.

Am I wrong for feeling like someone who could competently do this challenge successfully in 2 to 3 hours time would be at a minimum asking for no less than 150k starting? Am I just crap for not being able to solve such a problem at a rate of 70k?

To me this test would have required I had a good understanding of the IO api and a strong comprehension of processing large data streams concurrently to have a well tested cli app. That does not feel like I was given a fair shot.

I’d like your input, please do not spare my feelings, I would rather have direct and honest feed back. Its very likely I’m just not cut out for this anymore and I need some guidance. As for that job I get it was just not gonna be a good fit and I need to move on. I just need to know I’m not going crazy for feeling like I do.

#2

I don’t think it makes sense to try map a salary range to a problem like this. As you said, these kind of exercises use a different set of skill and often libraries or stdlib functions that you don’t use in your everyday work. So I would not think of it as a junior/senior thing so much as a specialization you haven’t developed very far yet. Definitely you can get better at this kind of thing with practice, and sinking a lot of time into it may feel like a waste but I think learning this specialization still improves your over-all depth and capacity as a developer.

#3

What I hate about this kind of challenge is the artificial time constraints. And I flat-out don’t believe this is a two-hour challenge from scratch. In a normal sprint environment it would be at least 3 points, probably 5 if they want tests. Whoever gave you that time is either really bad at estimation or is actively lying. Either way, you dodged a bullet by not working there.

It’s clear they want someone with knowledge of Flow for this. That’s fine, just don’t expect people to learn it from scratch and start building stuff in three hours.

#4

don’t get me wrong I think there is lots of value in learning this type of skill set, I just don’t think the employer game me an honest and fair test

#5

You nailed it right on the head, I would have still tried to do this if it took me several days, but they poisoned the well with their over the top expectations of timeline.

#6

Somewhat apropos funny commentary on this culture of over-the-top expectations: https://www.reddit.com/r/programming/comments/80wz2k/why_your_programmers_just_want_to_code_hacker_noon/duze6jv/

#7

The challenge seems appropriate for a senior level candidate, but with several caveats:

  • 2-3 hours is far too short for somebody coming in cold. There are a lot of moving parts involved.
  • a partially complete challenge is perfectly acceptable. You can tell plenty about how somebody approaches a problem without fully functional software.

Don’t beat yourself up about it. Try working on some more algorithmic problems, like the upcoming Advent of Code.

#8

2-3 hours is actually a long time to expect someone to spend on an interview problem. We use a similar approach at my company but the problems only last an hour, and the problems we use aren’t quite as difficult (maybe “specialized” would be a better word). One approach to tackling these kind of interview problems is to reason through the problem out loud before you start writing any code. This can help the interviewers to identify if you have understood the requirements (very important) and if you have a basic understanding of the data structures and algorithms involved.

For instance, you might start by identifying that you will need to be able to read from one or more files or from stdin. And that you will need to concatenate any files together into a single stream. And that you will need to strip out any punctuation and newlines. You can probably gain points by asking specific questions where the requirements are vague, like “are all words separated by white space?”, “are hyphenated words considered one word or two?”, “what characters are word characters?”, etc.

Then identify any of the constraints that will impact your solution. Like “I need to be able to handle large files, so I don’t want to slurp the files into memory all at once - I’ll probably need to build up a data structure holding my triplets/counts incrementally”, or “I want it to be fast, so I probably don’t want to read through the stream more than once”. Also identify any assumptions you are making, like “I’m assuming that the number of unique words in the file(s) is small enough that a data structure containing my triplets/counts will fit into memory.”

Finally, break the problem down into manageable (and testable) parts. Articulate the parts to the reviewers in case you don’t finish (sometimes that’s ok). Make sure you talk through things as you go, and demonstrate the parts as you complete them (very important). Sometimes people move so fast that the reviewers have trouble keeping up, so you want to be sure they know that you finished each portion. Also, it can be very helpful to start with a small test set for development, i.e., not “Origin of Species.”

As you are working the problem, it’s perfectly fine to search the internet (Stack Overflow is great for this) for information. Particularly something like an IO API. It only gets a little sketchy if you are looking up really fundamental things like the syntax for reading a value from a Map. Or worse, if you don’t seem to even know about fundamental structures like Maps. DO NOT try to find a canned solution for the problem online - if the interviewers have done their homework, you will not. And even if you did, the interviewers will not be happy with you just copying someone else’s solution. This does not mean you can’t search for related algorithms.

A lot of times the interview problem is not about determining if you know some particular algorithm or esoteric data structure like a trie - it’s about seeing how you reason, how you write code, and what your code looks like. It can be tough when you are in an interview situation due to the extra pressure, but it really helps if you can not get flustered and just reason out loud.

Finally, many positions are not necessarily looking for someone that knows a particular language. If that is the case, be sure to solve the problem with the language you are most comfortable with. My colleague came up with the phrase “aspirational languages” to describe languages that people really want to learn (or think we want them to know), so they try to solve a problem using that language instead of one they are much more comfortable with.

Having said all that, to be candid, it probably shouldn’t take you an hour to understand how to stream in files (stdin is just another file - at least on Unix). On the other hand, everyone strikes out now and again, especially under interview pressures. Don’t let it get you too down.

#9

If the interviewer expect you to send back the code challenge solution in 3 hours, they might be doing it wrong. I guess you are expected to spend 3 hours to solve this and send the solution when you are ready.

TBH the code challenge looks fine, in my current company we give same challenge to all candidates, the result was somehow always unique and some were speak quality…

#10

I agree with this. After looking at this challenge, I think I’d be able to solve it in ~ 1.5 hours in Ruby. But don’t be disheartened! This is only because I already have extensive experience in:

  • Using Ruby outside of the Web environment
  • Using UNIX systems and ‘files-as-streams’.
  • Working with lazy enumerations.
  • Many datastructures and algorithms which I learned in the university, like ‘What is the fastest way to build a Histogram from a given piece of text’.

I think what is meant in the exercise is ‘on average, these kinds of problems might take between two and three hours to complete’ rather than ‘You have to finish it in these exact time constraints’. Everyone has their own skill-set, and looking at a single example does not at all help!

Besides that, though, I hope that the company in question actually works with aggregation and streaming of data, because otherwise I find such a challenge like this one to be a bad assessment tool.

:heart: Just keep on moving forward. There are enough cool companies out there that are looking for developers right now :slight_smile: .

#11

I do understand your frustration. I would probably be frustrated after such an experience as well.

From what you write, it is not even clear to me if there are restrictions about the language to use or if you can choose freely, but I assume, this has been talked about in the interview or is part of a not copied section of the assignment or even implied by the position you have applied to.

But I want to share my experience here as well, it can be different from what you have experienced:

The last interviews I did were after my regular job in the evening hours, even the interviewers did some extra hours to be able to talk to me.

When they asked me to do a live coding challenge of an hour or two, I declined and explained, that I’d need a day at least to prepare the computer (I do not bring my laptop by on interviews) and that it is already late and my family (and theirs) is waiting.

Therefore I offered to pick random contributions to open source, private pet projects and so on and talk about those instead.

Usually this offer was accepted.

Only one has declined the offer and I offered to do the challenge on another day, not in the evening which the interviewer declined, so the interview ended at that point.

Later I learned that I wouldn’t want to work with them anyway, as they expect flexibility from their employees, but are inable to give it as well.

Of course there might be situations were such assignments are inevitable, but if the company is known for a pair programming culture, ask for your peer :wink: In general, search for the weak spots in the assignment and ask questions. And if nothing helps, be honest with your potential employer, “Sorry, but as I have told you, I’m not quite matching the skill set you are expecting from your applicants, but I am eager to learn. Therefore I’d guess I need another 3 to 5 hours extra to learn about the modules I didn’t use in the past as I am originally a web developer.”

#12

Much agreed. I decided to refuse coding challenges and intelligence tests for about the same reasons. It mostly proved a loss of my time and a nuisance. I propose to show an application that I built and discuss the code. Often that offer is accepted.

#13

Are intelligence tests still a thing? I thought we were past this type of pseudo scientific malarkey already.

I’ve only encountered them once on a job hunt and it was for a company that dissolved shortly after opening due to poor management.

#14

Not often, but I even have been asked to show my degree (which is msc chemistry - not computer science - to make the request even more ridiculous) a couple of times. I refused the jobs immediately.

#15

I’ve not seen then in any of my interviews or application process when I applied for beeing a software developer, but during my studies, when I wanted to do some side jobs in supermarkets, they did them.

Also my wife, who is currently unemployed and cares for our children, had to do some courses “how to apply for jobs”, where the priority was put into those tests, and not how to write the application or do the interview…

So at least for german supermarkets and job agencies those tests seem still to be very important.

#16

I just talked with a former co worker about his application phase after he left us.

Those companies that insisted on convuluted problems without accepting counter offers were those that do not produce software as their main product, but use it internally or sell it as a by-product.

As he landed in such a company, he was able to figure out why they wanted to have him solve the challenge rather than talking about his github profile. It was mainly because the solution to the challenge was reviewed by someone from the development team, and not by those he talked with.

#17

Just people fond of bureaucracy, getting a questionable feeling of reassurance when an applicant behaves well and follows the rules, can be a cause.

#18

I definitely agree with this approach albeit it can be hard to do in the “heat of the moment” which, as others pointed out, makes live coding challenges tough. (unless they’re specifically testing how people work under pressure which is usually not needed.)

In this particular case I’d try to:

  • figure out the simplest possible thing that does something useful. E.g.: some 2 paragraphs of input text are hardcoded in code and we use naive algorithm to count the sequences.
  • choose another small piece to work on next, e.g. if optimising, I’d still read data from known place on disk instead of at the same time messing with ARGV and such.

The big caveat is it’s very easy for me to say this in the comfort of my own home, but I hope you’ll find this useful nonetheless. I definitely understand and have shared your frustration with such challenges.

#19

This is fairly simple task. I would even say that “beginner level” one.

To split this into separate parts let’s see at the requirements:

This mean that we need to read files somehow and if there is none then read stdin.

Ok, so we will need to chunk data by words. In general by word we understand whitespace separated string, and I will treat them as such.

Ok, so before we count the words we need to normalise strings to lowercase (assuming example from the above) and get rid of the “punctuation” (I assume that by punctuation they mean non alphanumeric characters, they do not say anything about digits, so I assume these also count as “words”).

So we need to use Stream and read lazily. This will greatly help us with point 1.

This is quite obvious.

So firstly, we need to know if there are any params, and if so, then provide content of the files that are pointed, and if not then use stdin.

  1. We need to know the params, so we open Elixir documentation and search for argv.
  2. We need to check what it returns (in C argv contains program name), so we create simple test program test.exs:
    IO.inspect System.argv
    And run it elixir test.exs and elixir test.exs foo and we see that it contains only arguments (no program name), perfect.
  3. We know that we need to read files and IO, and that we need to use Stream, so again in docs we search for File.stream (it seems that it will fail on non existent file, fortunately they didn’t said a word about error handling) and IO.stream. Nice, we have them both.
  4. Now we can create function to give us our stream of data:
    def stream, do: build_stream(System.argv())

defp build_stream([]), do: IO.stream(:stdio, :line) # situation when we do not have any args
defp build_stream(files) when is_list(files),
  do: files |> Enum.map(&File.stream!/1) |> Stream.concat()

Now we have reading. Step two isn’t grouping files, but normalising them:

  1. We need all data to be lowercase, however searching for that gives us nothing. But what about other name downcase? Bingo!

  2. Now we need to get rid of the pesky punctuation. As we agreed earlier, punctuation is everything that is not:

    • digit
    • letter
    • whitespace

    So now how we can “get rid” of that. The simplest solution? replace them with empty string.

    Our normalize/1 function:

    def normalize(str) when is_binary(str) do
  str
  |> String.downcase()
  |> String.replace(~r/[^0-9a-z ]/, "")
end

Great, now we have each line of the input normalised via:

stream
|> Stream.map(&normalize/1)

It is also worth dropping empty strings:

stream
|> Stream.reject(& &1 == "")

Next, we need to split them into the words. This is as simple as calling String.split on each line, however it would be easier for us to have it as an uniform stream of words instead of stream of line words. So instead of Stream.map we use Stream.flat_map. So we have:

stream
|> Stream.flat_map(&String.split/1)

Now we need to split them into chunks of three consecutive words via Stream.chunk_every/4 and then join it back to single, space separated string:

stream
|> Stream.chunk_every(3, 1, :discard) # we aren't interested in last, non full, entries
|> Stream.map(&Enum.join(&1, " "))

And now the main dish - counting, which in this situation is dumb easy:

stream
|> Enum.reduce(%{}, fn chunk, acc ->
  Map.update(acc, chunk, 1, & &1 + 1) # update element in map by 1; if there is none, add it with value 1
end)

Now we have map in form %{chunk => count} and we need to sort it by amount of occurrences:

map
|> Enum.sort_by(&elem(&1, 1), &>=/2)

And display top 100:

list
|> Enum.take(100)
|> Enum.each(fn {chunk, count} ->
  IO.puts([Integer.to_string(count), " - ", chunk])
end)

Whole program looks like this:

defmodule Input do
  def stream, do: build_stream(System.argv())

  defp build_stream([]), do: IO.stream(:stdio, :line) # situation when we do not have any args
  defp build_stream(files) when is_list(files),
    do: files |> Enum.map(&File.stream!/1) |> Stream.concat()

  def normalize(str) when is_binary(str) do
    str
    |> String.downcase()
    |> String.replace(~r/[^0-9a-z ]/, "")
  end
end

Input.stream()
|> Stream.map(&Input.normalize/1)
|> Stream.reject(& &1 == "")
|> Stream.flat_map(&String.split/1)
|> Stream.chunk_every(3, 1, :discard) # we aren't interested in last, non full, entries
|> Stream.map(&Enum.join(&1, " "))
|> Enum.reduce(%{}, fn chunk, acc ->
  Map.update(acc, chunk, 1, & &1 + 1) # update element in map by 1; if there is none, add it with value 1
end)
|> Enum.sort_by(&elem(&1, 1), &>=/2)
|> Enum.take(100)
|> Enum.each(fn {chunk, count} ->
  IO.puts([Integer.to_string(count), " - ", chunk])
end)

And you can check it on your own if it works.

#20

No, it is not. This is not an emergency services job. If I am evaluated on the basis of how quickly I can douse a fire or build a dog house then I’ll go apply for a firefighter or carpenter – not a programmer.

I fail to see how the real-time screen-sharing technical challenges tell you anything useful about the person outside of “how quickly can he/she switch context to what they were told just now” – and that’s not a valuable skill to have because we don’t work at assembly lines. We need our focus for hours-long stretches of time. And this is coming from a person who successfully passed 11 such challenges in the last 4 months. I still find them worthless even if most interviewers were praising me afterwards.

As @wojtekmach very accurately pointed out, you might not be in your best psychological state at the moment as well. And in other instances you might want the job so much that you cannot keep a cool head while talking. It happened to each and every one of us and it shouldn’t be a condemning factor.

Additionally, there are many programmers out there who are technical geniuses but aren’t very keen on the social aspect of the job. If the companies need the talent they have to make sure they can utilize their abilities with minimal bureaucracy (which only serves to constantly interrupt them anyway). I personally don’t have that problem but many others do.

With that caveat I am more inclined to agree with you. Even if I cannot finish the challenge in time, I give my thoughts and say what I would do next if I had one more hour. The way you approach the problem is what many people are impressed with and I am glad that’s the case because again, we don’t work assembly line jobs. Our productive tempo is never guaranteed at the micro-scale. Programmer’s seniority level should be gauged based on their monthly – or even tri-monthly – output, and not if they are excellent or bad in a particular day.