Online Coding Tests with Codility

I spent several hours today on an online coding test for an employer.  This isn’t my first test using Codility. Not only have I done several of the tests before, but I’ve actually recommended they be used in a previous employer, when I was involved in interviewing, So I thought I share some thoughts around approaching these type of tests.  Before I start, please note – I won’t be posting any solutions to the tests I’ve done.  But feel free to try this useful link - http://bit.ly/1wSuukg.

Codility is an online site that allows employers to set online coding tests for prospective developers.  It is effectively an automated screen to ensure that prospective developers have the necessary programming skills for the role. As it can take up a lot of time to arrange telephone calls for technical screening interviews, or onsite technical tests as part of the interview, Codility or the alternative CodeEval, can help save a lot of time.  Obviously, even when using something like Codility, you would still want to have a significant technical element to any subsequent interview.  Tools like Codility are an aid to the interview, but they certainly doesn’t replace it.

The actual test is sent to via an email with a personalized link to Codility.  Typically the link will only be valid for a set period (i.e. 24 hours).  On clicking the link, you are taken to landing page where you can choose to start the test, practice with a demo test or come back later.

Once the online IDE loads, you will be taken through a tour of the IDE features.

Some features in the IDE to be aware of:

• Problem description (left hand panel).
• Custom test cases (bottom of the left hand panel).  This allows you to define test cases for your solution, based on the problem description.
• Code Editor (top right hand panel).
• Countdown timer  (top right hand corner). Note, each task will typically take around 30-40 minutes.
• Select the language to code your solution (top right hand corner). Note, this can be limited by the company setting the test.
• Various buttons to run and submit your solution
• Console window (bottom right hand panel).  This will display the results of your code solution, including from your test cases.

Note, once the test starts, you can’t stop – the tests are time limited. 

The solutions require you to implement an algorithm described in the problem description. You may be asked to develop a solution with regard to performance, as expressed in the Big O Notation.  Check out this primer if it has been a while since you covered Big O notation!  Note, I have also seen a task that required you to correct an existing buggy implementation of an algorithm, so you may not always be starting a solution from scratch.

Now for my thoughts and tips for taking the test.

1. Practice using the demo test and examples on the Codility blog.
2. Refresh your knowledge on algorithms (i.e. sorting algorithm such as BubbleSort, or search algorithms such as BinarySearch), and specifically on Big O Notation.
3. Use the problem description to define and breakdown your solution.  I typically break the problem statement into single sentences and use this as comments to outline and stub out my code solution.  Obvious, but it is easy to get flustered when taking one of these test for the first time.
4. Use your usual IDE to develop your solution. The Codility site wants you to develop your code in their online IDE – I found this a real pain to use, and slow at times. My advice is to create a console application to host your solution code, and to develop in Visual Studio (or your IDE/code editor of choice) on your local machine. This offers a few advantages:

• It is a familiar coding environment that you already know.  The Codility IDE is OK, but it has a few quirks.  For example, I found the custom test cases feature a bit of a pain to use.  There is no learning curve required when you are using your usual IDE editor.
• You can benefit from automatic error highlighting and faster feedback on compilation errors.  The Codility IDE requires you to hit the Run button and then to read and decipher any errors in the console panel.  Your own IDE will typically give you more information about errors, and give it a lot faster.  This is important, as time is limited!
• You can make use of a proper unit testing framework, such as NUnit. As noted above, the test case feature in Codility is clunky, and limits you to only 5 test cases.  Forget that, use NUnit and define as many tests as you need.

I created my test console application when I was going through the demo test, to ensure it was working correctly prior to actually starting the test. For the demo (and also the test tasks), I simply copied the stub method to be developed from the Codility IDE into my test console application, and developed it there.  When I had successfully coded and tested it in Visual Studio, I then copied the completed method into the Codility IDE, and then re-tested before submitting.  Once the solution is submitted, you can modify it or resubmit.

When submitted, the solution is analysed by Codility, and a detailed breakdown of your solution is available to the employer who set the test.  You can check out an example here.  Your result obviously determines whether or not you get called for interview.

Previously, I’ve done quite well in the Codility tests, and actually enjoyed taking them.  They are a bit like code katas. Today’s test, however, was a disaster.   There were 3 tasks to be completed, instead of the usual 1, so that the total time for the test was 2 hours.  I took far too long on the first task, which meant I had to rush on the second task.  After spending some time on the second task,  I ended up having barely anytime at all for the last test, so I doubt I’ll get called back. Entirely my own fault.  Interestingly, the second task required the correction of a buggy implementation of an algorithm. I suspect this was deliberately added by the employer as the role I was applying for would involve dealing with legacy code.

Despite today’s cock-up, I still like the Codility tests, and would certainly like to see them more widely used – they’re a lot better than the usual FizzBuzz whiteboard exercise!

Update

Despite the problems I had with the Codility test, I was called for an interview. I was offered a job, but not the position I had actually applied for.  After some deliberation, I declined as I didn’t feel the offered job would push me enough.  But I was very happy to be given the opportunity!