-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for custom evaluators using compiled langauges #260
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't know too much ruby (or rails) and am not too familiar with the codebase but the code changes seem to make sense.
I tested it locally for a bit and it worked as expected but I'm not sure about raising an error when evaluator compilation fails.
This is probably outside the scope of this pr but in future it may be good to not display the message |
Yeah, the submission page doesn't really handle judging errors. I've committed some improvements in 9e7aa3e. |
CI is currently failing; need to merge #266 first. |
Ugh, now coveralls is failing... |
Seems to be fixed now |
Deployed and converted all existing C++ evaluators that were previously using cint.rb to use this instead. |
This adds an optional language field to the Evaluator class. All existing/new evaluators will default to having no language, which maintains the existing functionality (script is executed as-is).
Note that C++ evaluators were already "supported" through cint.rb, but that is much slower as it requires the evaluator to be recompiled on every test case. That functionality can be considered deprecated and removed later, after fixing existing evaluators.
Also note that for evaluators with no language, Pygments will attempt to automatically detect the language based on the code (see here).