The speed that we can run submissions in our sandbox has been a challenge for a while. For large complex submissions we'd often have to wait a few minutes for the backlog to clear. It worked really well to grind out the submission in actual machine cycles, but it created real usability and scalability challenges as the number of users on the system grew. We knew that we were going to need to come up with a faster and more user-friendly system.
The system we've moved to uses Docker, and it's really fast.
The new Codeeval Sandbox are hosted on the Latest AWS Linux instances with Kernel version 4.1.7. The good news is that this Kernel version supports the OverlayFS storage system which is the fastest solution for Docker. Using OverlayFS storage driver for docker gives us very big speed advantage. For example: running command docker run --rm --entrypoint=/bin/bash ubuntu:latest --version:
- On the AWS Linux with OverlayFS storage driver takes 0.393415212631 sec;
- On the Virtual Box Centos 6.7 with default storage driver takes 2.62845516205 sec;
To execute code for interpreted languages:
To execute the code for compiled languages: