For a development team, it’s essential that everyone on the team is helping you build a better product. But how can we consistently figure out which candidate will be the best fit for the role? Below, we explore that problem and establish a best-practice solution that works for teams needing to assess developers’ technical and intangible skills.

Hiring the best possible people is vital

Hiring is one of the most important functions of any company. In fact, in most companies, employees really are the most valuable asset. I’ll let some business greats hammer that point home:

“The secret of my success is that we have gone to exceptional lengths to hire the best people in the world.”
– Steve Jobs

“I hire people brighter than me and I get out of their way.”
– Lee Iacocca

How can we determine who will be the best fit?

Hiring a new member for your development team is tough and unfortunately speculative. And hiring mistakes are expensive – beyond compensation, there are substantial onboarding and training costs, in addition to having to make up for lost time.

Picture this common scenario:

• Candidate A looks great on paper and is very personable
• Candidate B is more soft-spoken or doesn’t appear to be have as much relevant experience

It’s easy to hire Candidate A without further thought, but what if Candidate B is a diamond in the rough? What if Candidate A’s previous work was an inaccurate representation of his actual skills, and he ends up holding the team back? How can we make sure we hire the best person for the job?

Past work is certainly valuable to assess, but it isn’t totally dependable. Maybe the work was done in a team, and our candidate wasn’t the majority contributor – we have little knowledge of the work’s context. We don’t want to think about this, but maybe the candidate didn’t actually write the code at all – authorship isn’t always totally verifiable. And even if we do know for fact that the candidate wrote something, that work is not directly comparable to any other candidates’ work.

Development trivia and traditional coding interview problems can give us some idea of the candidate’s knowledge level, but these are often irrelevant to the day-to-day work the candidate would end up actually doing, and they don’t tell us how the candidate works within an existing environment.

A short-term project would help us get to know a candidate’s ability, working style, and skills through working with them for a week, but most teams don’t have relevant, digestible work that can be easily and profitably contracted out, and candidates don’t always have the time or inclination to embark on such a thorough project with every company they look into. Furthermore, if there are a dozen candidates for one position, it is unfeasable to undertake such large projects with all of them.

A Better Solution: Microcosmic Testing

There is an approach that lets us more quickly, effectively, and accurately distinguish candidates’ tangible and intangible skills, which is called microcosmic testing (footnote: a microcosm is a system regarded as encapsulating in miniature the characteristic qualities or features of something much larger). It has five guiding tenets:

• Quick & straightforward setup
• Lean & digestible environment
• Clear job for the candidate to do
• Limited timebox for completion
• Deliverable result that is simple to assess and compare with others

Microcosmic testing lets us peer into how the candidate approaches a new system, a new problem, and the technologies he/she will be working with in day-to-day working life. We can get a strong grasp on each important aspect of a candidate, including thought process & flow, existing knowledge, initiative, thoughtfulness, accuracy, thoroughness, and tidiness. Perhaps most importantly, we are also able to directly compare Candidate A to Candidate B in a quantitative, unbiased way.

An Implementation

To demonstrate, let’s see how we use microcosmic testing where I work: Gather. In addition to intangible skills, we particularly needed to test for specific skills in AngularJS, API interfacing, and application testing. In the past, despite having technical and non-technical interviews as well as extensive reviews of previous work, we haven’t had a flawless history of high-productivity developers, and we needed to improve the accuracy of our assessments.

For context, Gather provides software-as-a-service to event venues to help them manage group events more effectively. Its platform is a relatively complex application with proprietary data involved, so we can’t realistically test candidates using the full application; to assess a new developer, we need something more lightweight. Microcosmic testing led us to create a miniature version of our full application, using similar technologies and only basic functionality. Our implementation, which we call DevQuest, follows the above five points of microcosmic testing:

• Setup: Less than 5 minutes with clear instructions in the README
• Environment: Small, well-organized, technologically relevant codebase which can be quickly perused and understood
• Job: Candidate chooses from a few quests (feature build-outs) which are detailed on the app’s homepage
• Timebox: The candidate gets an hour to work towards the goal of his/her chosen quest
• Deliverable: We can assess and compare the resulting pull request

We quickly built out a basic, feature-incomplete mini-app around our “venue booking management” functionality using Yeoman with the ng-poly generator (which is excellent, by the way) to quickly scaffold and keep structure consistent and predictable. We used json-server for a light database server, and since we can’t use customer data, we actually generate new sample data at each build using faker.js. npm and gulp then make it very easy to quickly get the environment set up.

Just like that, we have ourselves a fully functional AngularJS application, a database with a REST API, and actual data we can manipulate! From there on, all we had to do was build out a few routes, add some application-specific logic, and decide what (half-dozen or so) minor features or improvements we wanted candidates to build out (which we call quests). A good quest will touch on the most assessment-relevant parts of the application and be scope-limited to where the candidate should be able to make significant progress in an hour. After building out the basic functionality, we spent some time completing several of the quests ourselves to ensure that they were reasonable for the hour time-limit we enforce.

So when a candidate comes in to do DevQuest, we set them up with a laptop and the repository, explaining what DevQuest is and that they can choose from quests ranging from pure front-end to pure back-end. Our goal with each candidate is not so much for them to complete the feature but for them to think through the job thoroughly and get as far as they can. Candidates can ask questions, and, if they want to, they can keep working on their branch until they hit a good stopping point, though we do take that into consideration when scoring their effort afterwards.

The final deliverable is a pull request including the candidate’s changes and a clear, thorough explanation of their thought process, approach, difficulties encountered, things they would do differently, and what they would plan to do next. Having a pull request makes it simple to pull down and build a candidate’s branch, and it also becomes very easy to see, for instance, Candidate A’s solution vs. Candidate B’s vs our in-house version. Seeing how the candidate approached the problem and interacted with our team is very informative too – it allows us to see the context around their work and skills that we wouldn’t otherwise have insight into.

I encourage you to browse through our DevQuest mini-app and see if something like our approach to microcosmic testing is right for your team. If you’re keen to test your skills, set aside an hour, clone the repo, and try completing one of the quests yourself!

Our Results

After using DevQuest for microcosmic testing over a dozen candidates to supplement our soft- and hard-skills interviews, we have found that we have much more confidence in our assessments of candidates now. We’ve hired two developers through the process, and both of them are now exceptional members of the team. Throughout the screening process, we have been impressed by how much more confidence we now have in our assessment of developer candidates – a candidate’s performance on DevQuest can really distinguish the top performers from the rest.

Going Beyond

For us, microcosmic testing has become the foundation of our search for great developers, but the microcosmic testing approach can also be utilized for UX, QA, Copywriting, & Marketing positions as well: simply provide a controlled sandbox environment, a clear problem to solve, and a final deliverable that can be contrasted between candidates.

Microcosmic testing isn’t a one-size-fits-all solution for every type of company or every kind of job role, but when used effectively in combination with interviews for hard skills and soft skills, it can be an invaluable tool for consistently finding the best possible person for the job.

Do you utilize microcosmic testing? We’d love to hear about your experiences as well.