Creating a take-home coding challenge that engages candidates

|

I’m Sorry, I Won’t Do Your Take Home Coding Exercise.” “Take Home Programming Interviews Suck.” “NO! Unequivocally, NO. Just No.” Do a search for take-home coding assignments, and you’ll quickly see just how polarizing they are. 

That being said, take-home coding challenges are an undeniably effective way of moving past soft skills as more companies opt out of whiteboard exercises as a technical skills assessment (in fact, you’ll find a list of more than 500 here who have erased whiteboards from their process). Inevitably, a few candidates will flatly refuse to do any coding challenge, either out of principle or in low-competition markets. But to keep top talent in your hiring pool, you still want to make sure that the candidates who do take this next step feel like it’s a valuable use of their time. 

Whether you’re a recruiter working with a hiring manager or a hiring manager yourself, follow the guidelines below to not advance candidates you match with on Seen, but to create a positive experience that doesn’t send the candidates venting online. 

Start with clear scoring criteria

Laying out the scoring criteria in the coding challenge before it’s assigned serves two purposes: reducing individual bias and structuring the post-challenge discussion. You won’t be able to fully eliminate subjectivity in the interview process, but having predetermined scoring factors laid out will certainly limit it. Slack, for instance, takes a transparent approach to grading coding challenges

The exercise is graded against a rigorous set of over 30 predetermined criteria. We’re looking for code that is clean, readable, performant, and maintainable … We do this to limit bias. If there are clear criteria, variations that might impact score but have nothing to do with the candidate (such as if the grader is having a good day) are less likely to influence the outcome.

That’s not to say you need to detail all of the criteria in the instructions, such as stating the candidate will be graded on how clean their code is—having purposefully vague instructions can be valuable to gain insight into the candidate’s thought process. In agency roles, for example, requirements can often be vague and assumptions can lead to costly consequences, so the questions candidates ask can be one of their performance factors. 

On the other hand, confusing or misleading instructions (e.g., saying something is optional when it’s a key scoring consideration) can add time to the exercise or send candidates down the wrong path for the project, hurting the hiring manager’s ability to assess the interviewee’s skills and adding time for both developer and code reviewer. Providing thoughtful guidelines will drive better results for both interviewer and interviewee. 

Align the challenge to the role 

One of the most frustrating things for a candidate to experience is receiving a take-home challenge that’s either peripheral or irrelevant to the role they’re interviewing for. For a QA developer candidate, it could make sense to provide a historical version of your company’s iOS game and have them set up some test scripts—that same exercise for an API integration engineer wouldn’t be as appropriate: they’ll question whether the focus of the role might be different than what’s been described or if the interviewer doesn’t understand the position.

A hiring manager doesn’t want their candidates to wonder what they’re actually interviewing for. You want your potential hires to have a sense of what day-to-day responsibilities will look like and hope that they feel both excited and capable to take on the role, whether that’s setting up test scripts or an infrastructure to parse API data, or designing out a basic web page. 

Don’t set unnecessary parameters

The reason why interviewers turn to exercises like pair programming interviews and take-home challenges is because these provide a more realistic environment than a whiteboard. Working with the hiring manager, you should strive to make the exercise as realistic as possible. 

If you’re setting explicit limitations, such as not allowing the candidate to use generators to set up an app or restricting them to a certain framework, make sure there’s a good reason for it since it will add time to an already time-intensive project. Any guardrails you set should help provide additional insight into the candidate. 

For example, does the team work within a limited tech stack and needs the candidate to be comfortable with that stack immediately? Do they want to see how the candidate picks up something unfamiliar? Or is it most valuable to understand how the candidate approaches the challenges using the frameworks, libraries and packages they’re most familiar with? 

As Atlassian describes in their approach to coding challenges, there is no right or wrong to any of these, but when setting up rules and shortcut limitations, know why you’re doing so. 

Allow them to add it to their portfolio

Some of the pushback from candidates on coding challenges is that it’s “free work”. There are several arguments against that logic, though: a challenge won’t be used as actual work-product, and candidates don’t get paid for any other part of the interview process, which can be as long as a take-home assignment.

But you don’t want your candidates to feel like they’ll put several hours of work down a black hole if they don’t get the job. Saying at the outset that they can add the project to their GitHub profile can add goodwill, though you should probably add the caveat to leave your company name off unless they don’t plan on offering the same challenge again. 

Have the hiring manager also consider candidates’ overall portfolios. While many won’t be able to share their past proprietary work, any individual work or contributions to open-source projects can help your hiring manager better evaluate and anticipate the quality of their work. 

Discuss the challenge

Finally, it’s good practice for the hiring manager to discuss the coding challenge after the interview: can the candidate justify all their code and explain their approach, or did they do a Google search and paste some code from a thread without a full understanding of it? 

From the candidate’s perspective, whether they move forward after the assignment or not, they’ll want some post-mortem of it. After all, they likely spent several hours on the project and will want to know the impact it had in the interview process. Encouraging the hiring manager to offer this feedback could prevent candidates from leaving negative reviews of the interview process because they don’t fully understand it. 

This is where the scoring criteria come in—if the candidate is offered a position with your company and the challenge was aligned to the role, it can offer insight into how performance is measured. If the candidate doesn’t move forward in the process, it’s just as useful for them to understand why not. At the very least, this feedback can help candidates better tackle their next coding challenge, and can even help them identify a skills gap that may have an impact on future positions they apply for. 

Take-home challenges will always be controversial. By guiding the hiring manager to build a coding challenge that emphasizes a positive candidate experience, you can make the process far less painful for the hiring manager and talent alike. 

Seen is here to help recruiters like you build up your talent sourcing strategy. Learn how Seen can help you find the perfect match

Recommended posts