This seems like a pretty popular take on the interview process here on HN. However, there's one pretty glaring flaw that I've never seen discussed: if the interviewee can take the problem home, what prevents them setting up a hackathon with their friends at their place, and making the interview problem a team effort?
A couple of knowledgeable friends can do wonders, and if you also prep with them a little you can probably answer all relevant questions and beyond about the code on the Monday on-site interview.
Sure, that's cheating. And even if it wasn't, I would never do it for the wrong expectations it would set upon me if I actually got the job. But the ease of "embellishing" one's CV always comes up in these threads/posts, and while I wouldn't do that either, I kind of fail to see how this home project approach solves that problem.
> if the interviewee can take the problem home, what prevents them setting up a hackathon with their friends at their place, and making the interview problem a team effort?
Nothing, and that's fine. In fact we encourage candidates to use all resources on our take home interview test. I would probably be impressed if someone setup a hackathon to solve a rather simple take home problem.
A take home problem is simply another signal, and a jumping off point for discussions. If someone used some novel approach we'll discuss it and find out if they actually understand what they did.
The real surprising part is how many take home problems we have received that either do not compile, do not pass the test data we give, or look like the instructions were completely ignored. It ends up as a low effort FizzBuzz on our side.
> we encourage candidates to use all resources on our take home interview test
This sounds great. In my response to another comment I was worried if accepting-but-not-encouraging this kind of behavior would put the interviewees on uneven ground. But if you tell them in advance that basically anything goes, it's fair for all parties.
> The real surprising part is how many take home problems we have received that either do not compile, do not pass the test data we give, or look like the instructions were completely ignored.
Sounds horrible. Though I guess that's only a good thing. Seems like the assignment is doing its job in those cases pretty well :)
The problems are usually very simple. Something like implement a queue that can have multiple listeners, and write a test/driver to show the queue works. If someone really needed to pay someone to implement this problem, it will be very easy to figure out in the interview.
Like Fizzbuzz, the point is a quick way filter out candidates.
This is the primary reason that you shouldn't make hiring decisions based on the test/problem results alone.
The interview problem should only serve to be a conversation catalyst. If they got some help with the problem, outsourced the problem or simply googled prepackaged solutions to the problem then they are going to have a really hard time talking through the context of how they solved the problem.
What you're testing for here is communication skills and, if possible, a really basic datapoint that may indicate their technical ability. The latter is very subjective and incredibly difficult to test for but the former is arguably a lot more important if they are going to be working as a part of a team.
Totally agree, conversation catalyst is a great way to phrase it too. This is very much the approach I've taken having had very mixed outcomes from trying to do just one thing or the other. I find using a simple, 100% self contained task that should take a few hours and then using that to kick start discussion has provided the best way for us to hire and screen people. We've found using a task that is very simple to get a working solution but provides a lot of scope for edge cases or growth to be effective. It allows the candidate to not blow a whole weekend on something for the sake of it but gives us obvious directions to take the conversation.
Communication skills seem to be held in much lower regard despite them being the key thing for effective day to day work. I can manage / handle working with someone who needs more time than a "rockstar" or hasn't memorized TAOCP if they'are able to communicate effectively. Outside of very specific roles, non-trivial software development is a team effort. It's usually extremely obvious within 15 minutes if a candidate genuinely gets what they've coded or is winging it.
You and GP both bring up a really mind-opening view into this. "Conversation catalyst" indeed, sorry but I'm definitely stealing that one.
> Outside of very specific roles, non-trivial software development is a team effort.
Couldn't agree more, and I have to say I'm rather baffled that so few recruitment processes are fine-tuned for this. Or maybe I should say that the image one easily gets is so. I myself have had pretty well balanced interviews that also tried to assess things like this, but I've always kind of considered myself just lucky.
When one takes your points into consideration, I can definitely see how having the take-home assignments offer value above the usual whiteboarding+bullshit bingo.
We have an extension to the take home done paired on site that usually involves significant refactoring. If you don't have a clear understanding of the code you're sunk. Plus the presentation should weed out the most egregious cheats. I also strongly suspect that developers that can't solve the task well will mainly have friends that also can't solve the task well. If someone had skilled developers invested in them doing well I suspect they are most likely a skilled developer.
> We have an extension to the take home done paired on site that usually involves significant refactoring
That sounds actually brilliant. While some might argue that this comes back to the "whiteboarding under pressure", honestly if you have already written the code from scratch you definitely should be in a much better position than with the usual "implement a red-black tree insert, you have 15 minutes" whiteboarding tasks.
One view of this behaviour is cheating, another view is hustle. I mean, if you could arrange a hackathon to solve an interview question - you are hired my friend :)
Do you expect this guy to organise a hackathon to solve every problem you give him? And what if that havkathon now involves the people at your company working on his problems as well as their own?
Of course I don't expect him to organise a hackathon. If that's his way of solving the problem, I'll actually appreciate his effort. Shows great leadership skills. In fact, most of your success in a company comes from your ability to muster a team. Individual skills only go so far.
Sure I would, I have the $X/h to divide between them for helping me ;)
Seriously though, I guessed that many companies probably must have this kind of laid-back attitude on the matter, since it would be stupid to presume they are ignorant of my original point. Still, I think it might be a little unfair to the people going by the rules-as-written, which somewhat diminishes the consensus of awesomeness of this approach.
That is, if the interview process is not actively trying to choose for hustle... :)
No interview method is foolproof. If someone goes to those lengths, you'd find out soon enough that they're not actually competent enough to do their job after hiring them.
But I'd consider that fairly unlikely. Plus, starting from a position where you don't even trust someone enough to behave like a mature human being isn't a great way to begin a professional relationship.
I have solved and implemented workaround of this. Rather than only one home assignment of 2 hours, I give one assignment for home. Next assignment, which is enhancement of first assignment, is given when they appear of face 2 face interview.
This makes sure that if someone has not solved home assignment with other's help.
ELI5: you discuss the solution with the person, too. If they can't accurately describe it, they fail. They either have poor communication skills, don't understand what they wrote, or didn't write what they submitted. Those are all easy fails.
Compared to "traditional interviews". It's probably well worth taking the risk of occasionally hiring someone like this. I'd say it's less likely to occur than someone who has more or less memorized some interview question book algorithms and happens to get asked the right ones.
A couple of knowledgeable friends can do wonders, and if you also prep with them a little you can probably answer all relevant questions and beyond about the code on the Monday on-site interview.
Sure, that's cheating. And even if it wasn't, I would never do it for the wrong expectations it would set upon me if I actually got the job. But the ease of "embellishing" one's CV always comes up in these threads/posts, and while I wouldn't do that either, I kind of fail to see how this home project approach solves that problem.
Anyone able to ELI5?