I failed my tech challenge today for a job interview. I don't think I was given a fair chance

tech-challenge
interviews

#21

You have spent hour and didn’t even asked interviewer for some clues? On my first interview as a developer I have been given clue: “Ask. We want to work with you, not rely on you as an encyclopaedic source of knowledge. A lot people do not know things, that is why we ask.”

This is just counting. I do not know what languages you have worked earlier with, but it is just: found phrase ↦ increment count in map. It is just that. Nothing more.

You have overthinked it. The whole point of the task description was that you should use Stream instead of Enum. You wasn’t supposed to think about any overhead (especially as it will be negligible in this case).

There was no “CLI API” - just arguments and stdin. You weren’t supposed to parse arguments or anything like that. You get list of files (as arguments) and if there is none, then you fall back to streaming stdin.


For me task is plain easy and great task for the interview (but a little poorly worded). Why?

  • It requires you to think “what is that large file”? 2 GiB, 5 GiB, 1 TiB?
  • It forces you to ask “what is a word?”
  • It forces you to ask “what is punctuation?”
  • It forces you to ask “what about unicode?”
  • Is easy to split into parts and talk about implementation step by step.
  • Most of the functions are already there in the standard library, so you do not need to implement much, just stick the pieces together.

#22

By the way: The regular expression should be changed to ~r/\W/ to not remove other in-word symbols (like -, for instance). (However, this assumes that standard unicode-compliant rules are used. If foo-bar should be the same as foobar then you cannot do this).

And to make that work with any unicode string (outside of the ASCII range), use ~r/\W/u.


#23

I try to avoid regex / replace those that I find in existing code by normal language btw. It’s not very readable.


#24

Slightly better version could use String.split(str, ~r/\W+/u) to probably slightly improve performance. Bigger improvement could be using trie instead of map to store entries. But all of that could be discussed later after the task is done as a possible future improvements in the solution. This leaves interviewer with:

  • fairly good solution that suits requirements that was provided fairly quickly
  • shows that you have vast knowledge
  • you show that you know that this isn’t optimal and that there can be possible future improvements, which you are aware of
  • you have used standard library instead of reinventing the wheel

And to be honest, this is slightly expanded version of task that @josevalim have presented in his “GenStage and Flow” presentation. So you could improve that even further, without much work, by using Flow. Another example for interviewer that you know what you do and that you can possibly do it better.


#25

I think you touched on something that I think you may have missed in my initial attempt to explain why I felt so bad about this whole test.

the:
shows that you have vast knowledge

Was my whole issue with this test. I never claimed to have a vast strong comprehension about elixir. To the contrary I made several attempts to explain my current skill level and short comings. While I agree with many of your points and that is why I hold my self accountable to some level I also feel it was equally their failure to accommodate. Too many assumptions had been made overall. I’m not so sure that was due to a lack of communication on my part. I thought that the only reason they had me take the test was because they understood my short comings and that was acceptable considering my will to grow. What I found is that all my attempts to explain myself fell on deaf ears and they wanted what they wanted in the end. And that is why I felt I was not given a fair chance overall. I’d like to figure out how I can learn from that more than the technical challenge. I’ve stated before that I would have been willing to give it as much time as it would have taken me but I feel like they put pressure on me for the sake of it rather than accommodate. And it is that I take issue with the most.


#26

(emphasis mine)

And my quote isn’t about Elixir either. In this post I tried to describe it as much as it is possible how total newbie could work through the process. Does it require some knowledge? Yes. Does this require Elixir specific knowledge? No.

Tries aren’t specific to Elixir. Flow and GenStage are more Elixir specific, but aren’t required by this task either. Mentioning them would be nice addition, not requirement.

From what you describe, it feels like both sides failed at the communication.

It seems like huge problem. They should explain you what you have done wrong. I already see some problems, for example that you have asked on forum instead of asking the interviewer directly. This is huge problem on both sides, as they should mark directly to ask them, and you shouldn’t ask about interview tasks in public, because it will make that solution not “yours”.

I believe that this was the case, however you have focused too much on how to solve first problem instead of even mocking it (as you didn’t know how to solve that) and solve rest of the problem. If they know that your knowledge of Elixir can be lacking, then this shouldn’t be a problem.

So next time, instead of tackling the problem as a whole, try to tackle them part by part, and “mock” the parts you do not know how to handle. To give you example from my experience:

During my studies I have subject where in the exam we were supposed to write sorting algorithm in Prolog (that will sort complex data in custom order). And it was 100% sure that there will be sorting. So what I have done? Learned by heart sorting predicate that use custom predicate to sort data. Now instead of figuring out whole sorting and comparing data, all I needed to figure out was comparison algorithm. My final program almost worked, it was looping through possible “solutions”, because there was missing ! somewhere (it is backtracing barrier). While my solution wasn’t perfect, I still have passed with A- because I solved most of the problem.

Whole idea of interview tasks is not to solve the problem, because no one give a damn about it TBH, it is about showing how you are thinking. Pseudo-code solution is better than no solution. Even no-code solution, where you describe ideas behind the steps you would take is better than nothing. Instead you have “spent the first hour just trying to figure out the IO api”. This is ⅓ of the whole time on something such trivial. Any “Hello World” example would give you basic example how it work.

Also top of the search results when we enter word counting elixir are:

End Exercism example even does it case-insensitively.

So as you know that you will have Elixir-based task on the interview, then it would be worth to at least see some tutorials as this is literally lesson 2 on Alchemist Camp and one of the easy tasks on Exercism.io.

And almost only thing you need to do to accommodate these solution to your situation is to add grouping words, to count them by 3, not individually.

So if you didn’t have heart to prepare yourself to the interview then maybe they didn’t care. You didn’t show respect for them, so they responded with the same.


#27

If company is expecting a person to be doing ETL it looks like a reasonable problem to give to be honest much better than algorithms on whiteboard type interviews that are pretty popular now.


#28

This looks like a reasonabble challenge. 2-3 hours is also a good time.

Hej, I think you should just move on and not complain too much about it.

It seems to me, you are kind of put down that you did not make it and that is ok.
Give yourself some time, get back up and try again. Learn, get better and improve yourself. And do not stop applying.

Of course I wish you find a job in this industry.

About your rage/being offended:
Ask yourself what is a fair chance?

Why should they alter the challenge just because you told them your skill level beforehand?

It is insanely hard to find good and matching people for a job, especially programmers.

I do not think it would be useful to adapt the challenge for every candidate just to get them into the company.

Either the company can afford to include people who need to learn a whole new skillset or they cannot.

Making an individual challenge seems very counter productive to me. In particular why should you get special treatment when every candidate has to be ranked somehow?

Isn‘t that what tests are about? To me this is very analogous to what happens in exams in school. All get the same challenge in order to build a point in time rank of a small focused set of skills.

So, also try to find a way to put down your rage/being offended mode.

Imagine, that at one point you might be at the other position and you are responsible for finding a good candidate.

If you make a mistake, it can make or break your project and team.
You do no want to know the turmoil caused when you have to lay that person off, because they are just learning but your project cannot afford the time and money to bring that person up to speed.

Well all good and fluffy clouds of course when you are hired by a VC backed next-big-thing™ company and you can just be the one who learns a whole set of skills on the job.


#29

Being at the other end of the table, I can tell you that the company I work at doesn’t care too much about ‘finishing’ the challenge. We mainly look at reasoning, style and documentation (in our business, in-depth defence is holy). That’s why we often accept code from (semi) personal projects.

@hauleth’s posts are incredible valuable for you, as he has written down many of the ‘checkboxes’ proje…scrum ma…agile captains value.

You will probably face more challenges. Don’t digg to deep, keep it simple, document (your workarounds) and if a step is too hard: skip it (mock!)

Ps. Never give up! If money is an issue, accept a job below your skill set and use your unused energy capacity in other ways to broaden your horizon. Mentoring, project management, analysis… outgrow your ‘developer’-label.


#30

This is just about the most sensible response in this thread, thank you.

I think most people responding as though the solution is simple not factoring in the headspace of performing in such a context as interviewing. Multiply that by the need to pay one’s bills, especially if one lives near the biggest cities for technology careers (with their staggering rents and costs-of-living).

Its simply not representative of how someone will perform on the job, or contribute to the company. Talking through apps they build is a much better indicator.


#31

I think it’s fine for companies to optimize interview process for their env. if they expect people to do ETL work with little ramp up time in high pressure env. it’s perfectly fine to optimize for that. It’s also perfectly fine for the candidate to make a decision to not work for such a company.


#32

My previous employer in Beijing used them.

I don’t think it’s uncommon, but it’s highly regional. In many places (i.e. Europe and N. America), your belief is the norm and politics would make such practices impossible or at least impractical. Even there, university entrance tests are often the same thing in disguise and are highly correlated with g.

It’s not hard to argue that filtering applicants by the brand of their universities, one is essentially doing a less efficient, but more palatable version of the same thing that similar counterparts in China do directly.


#34

I agree with others that this doesn’t map very well. It’s the sort of test I’d have done well on in 2011, when I was earning ~10k USD/year. When I was at the 120k range three years later, I was no better at that kind of test, but I was in a different city, knew a lot more about how the web works and had become far, far better at debugging larger projects. Meanwhile, my good friend was already earning about 150k and couldn’t solve the kind of test you shared in an efficient manner. He was very, very good at JavaScript, Angular, Webpack, Docker, etc, etc, etc, though.

Basically, it’s a matter of supply and demand. There are many, many different ways to win in the market, and there’s a lot of noise in the process. You’ll be perceived wildly differently by different potential employers and obviously tend to end up with those who had an unusually good impression of you. The same is true from their point of view—different candidates will perceive them, their compensation and their work wildly differently and it’s the ones who are most positive that will tend to pick them.


#35

Anyone else try this? (Except @hauleth since he provided a solution. :slight_smile: ) I gave it a go in ruby since it looks like that was the assignment language. The phrase “three word sentences” is a bit ambiguous, but I assumed trigrams. It’s been a few years since I programmed ruby. It took about 2 hours to get something working with no tests. I’d spend the rest of the time trying to make it more efficient since my solution is crap. Any tests would be tacked on at the end.

I have to agree, this isn’t really a good way to screen an applicant. I’d fail at this too. The pressure of doing the problem on an unfamiliar system without the tools and resources I like (especially Dash) would make it harder. If I was to give a problem like this I’d leave it open ended and ask the applicant to do a git commit every hour no matter where they were. That way I could look at how they worked through the problem over time to a proper finished solution.

Problems like this test how well you’d do at a hackathon. I don’t think companies should be looking for people who are good at hackathons.

I usually prefer to have an applicant talk through the system they are working on and drill down into different areas to see how much they really understand about their current work. And then drill down as deep as I possibly can on one small detail to get a sense of how deep their knowledge goes. By asking good questions you can get a good sense of how they think. This is hard work.

Then I ask them what books they are reading. :slight_smile:


#36

Sorry, but the OP knew what system he will use during the interview if it was job for “Elixir developer”. And As I said, there is a lot of examples of “word counter” in all basic tutorials for Elixir, because it is simple and easy to do. Only “trouble” there is that you need to chunk words in triplets, and even when you do not know about Enum.chunk_every, but you know basics of the Elixir, you can do it with simple pattern matching:

def triplets(list), do: triplets(list, [])

def triplets(list, acc) when length(list) < 3, do: acc
def triplets([a, b, c | rest], acc), do: triplets([b, c | rest], [[a, b, c] | acc)

This will not be the most performant solution, but hey, you have showed them that at least you know something.

It isn’t about completing the task, as @BartOtten said recruiter doesn’t give a damn whether your solution works or not, it is about process of getting there. And by reading what OP did, he just failed on entry, not because he didn’t know stuff, but because he spent 1/3rd of the time wandering on least important part of the task: how to read file. If it was me, who was recruiter I would expect questions, a lot of questions, especially when I have inexperienced developer in front of me.

Trigrams are something totally different.


About implementation in Ruby, it is almost as straightforward as the one in Elixir, here is golfed version I have done some time earlier (it is quite easy to “ungolf” it):

ARGF.each_line.lazy.flat_map{|l|l.downcase.split /\W+/}.reject(&:empty?).each_cons(3).reduce(Hash.new(0)){|a,e|a[e.join' ']+=1;a}.sort_by{|_,v|-v}[0..99].each{|c|puts [c[1],c[0]]*' - '}

#37

According to the wiki page you linked they are labelled as “word level trigrams”? See the example with the “Quick brown fox”.

I tried it in erlang. For me it was also quite a straight-forward exercise and didn’t take long. The big difference between my implementation and @hauleth’s was that erlang doesn’t have Stream and other nifty helper functions so I implemented them on my own with a slower result than if using Stream.

I think this was a good exercise. I can see this as part of an introductory CS course. I can’t comment if it is suitable as a recruitment tool.

As with a lot of things, once you know a concept reasoning about it becomes easy. Without that experience any task might seem harder than it is.

@hauleth’s initial post breaks down the task in small and understandable parts and state some assumptions going forward. In my view if the applicant is able to reason about the task like that I would consider it a success even if no lines of codes were actually written,


#38

It has been very insightful reading all the reviews. Thank you all.

A few things I wanted to update on now after reflecting on how I first wrote this.

First I didn’t want this post to come off as a complaint. Though I don’t know if I could have wrote this any other way. I really wanted to highlight what I felt I was observing as unacceptable level of expectations that I felt was more systemic beyond just this one given company. That said I still find it interesting that many of you feel that I should have been able to accomplish this task given the amount of time or at the very least I should have been close. This one is the hardest for me to agree with as there are many factors I disagree with. For example I personally have rarely encountered this type of problems solving while primarily being a web dev for the past decade. The employer should have known that my background was not directly alined to this type of work as I tried to be as transparent as I possibly could.

That said I think I also may have misunderstood the type of work this position was for as I assumed I’d be doing more web dev type work. In any case I still feel that the competence level needed to accomplish this type of work far exceeded the level of competence I felt I accurately portrayed myself as. I will agree that is naturally subjective but its also why I feel I’ve seen some of you aline with my opinion on this matter. With that I hope we all take something away from this, and that is this is clearly an area we all could benefit from being more open about and that its not a black and white issue.

The other big issue I take with this type of screening is that, if we are saying this should be done in 2 to 3 hours but realistically it can’t and we just want to see you work under pressure and see how far you can get. I think that is a horrible precedence for any company to set. But again I think that only highlights a higher level of expectation that I feel goes beyond acceptable.

At the very least I think it goes to show they were really just looking for someone with more qualifications than I felt I presented my self as. I tried to portray that as I reflected on my salary level. After reviewing some of the comments, I will agree that is not a very good measurement of skills, specifically specialized skills sets.

I do think that also highlights why this was a challenge in that it did feel like a very specialized challenge not alined with the type of position I thought I was getting in for. I will take accountability for being naive on that matter but I do think there is some equal blame on the part of the employer to identify that I was not a good fit too. In hindsight maybe this was not a good post as this was my chief issue and its not something that I now feel could be resolved here. I now see why this is viewed more as a complaint. And for that I’m sorry. You should all take comfort in that I’ve already moved on and this is not something I’m holding on to. Also thank you all for your kind words of inspiration. It really goes to show that even if the community here is trying to give criticism they mean it with the best intentions and you should all take pride in that.

I will also agree that my mental state does and will always have a factor in my performance as it does for most people. There is only so much of that I can control that as I’m only human trying to survive. I don’t think I’m unique in this matter and it should not hold as much weight in this conversation, I also don’t blame this on my in ability to solve this problem since I didn’t let that stop me from trying to solve the problem.

If there is any positive thing that we can take away from me sharing this experience I want it to be that I hope people find more ways to be inclusive and to stop trying to address employment as an analytical problem. Because I’m not a measurement, I’m a human being who deserves an equal opportunity at employment and so do you. And no matter how much you wish to defend this model its clearly failed me in this instance for better or worse. Its clear I have something to gain from this experience and that I have my many challenges ahead in my life and this wont likely be the last time I find my self here.

I’d like to take this opportunity to change this topic slightly from that of introspecting and fact checking each other on my poorly written post and try to move into a topic that is supportive to others who may find themselves in a similar issue. I think that will provide more value to the community as a whole vs a rant complaint style post that I did initially try to avoid but I now see how I failed to do so. That said I’m still glad I wrote this and I hope it helps someone else other than how its helped me. Thanks for being a supportive community.

Best.
– Josh Chernoff.


#39

I’ve posted a thread on that here:

(I’ve also moved this thread to chat, as it’s very much about your personal experience :slight_smile:)


#40

Very cool.


#41

I think it’s important to do research upfront on the company to save you time. Glassdoor and the like will generally have info even for smaller companies.