Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for custom evaluators using compiled langauges #260

Merged
merged 5 commits into from
May 19, 2024

Conversation

Holmes98
Copy link
Member

This adds an optional language field to the Evaluator class. All existing/new evaluators will default to having no language, which maintains the existing functionality (script is executed as-is).

Note that C++ evaluators were already "supported" through cint.rb, but that is much slower as it requires the evaluator to be recompiled on every test case. That functionality can be considered deprecated and removed later, after fixing existing evaluators.

Also note that for evaluators with no language, Pygments will attempt to automatically detect the language based on the code (see here).

@coveralls
Copy link

coveralls commented Jan 28, 2024

Coverage Status

coverage: 37.481% (-0.01%) from 37.494%
when pulling 8d84fcd on compiled-evaluators
into 4fc9595 on master.

Copy link
Member

@BelgianSalamander BelgianSalamander left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know too much ruby (or rails) and am not too familiar with the codebase but the code changes seem to make sense.

I tested it locally for a bit and it worked as expected but I'm not sure about raising an error when evaluator compilation fails.

app/workers/judge_submission_worker.rb Outdated Show resolved Hide resolved
@BelgianSalamander
Copy link
Member

This is probably outside the scope of this pr but in future it may be good to not display the message This submission has not finished judging. Refresh this page in a minute or two to see the submission's score. when judging encounters an error (be it evaluator not compiling or some other error in the worker).

@Holmes98
Copy link
Member Author

Yeah, the submission page doesn't really handle judging errors. I've committed some improvements in 9e7aa3e.

@Holmes98
Copy link
Member Author

CI is currently failing; need to merge #266 first.

@Holmes98
Copy link
Member Author

Ugh, now coveralls is failing...

@BelgianSalamander
Copy link
Member

Seems to be fixed now

@Holmes98 Holmes98 merged commit ee7d417 into master May 19, 2024
6 checks passed
@Holmes98 Holmes98 deleted the compiled-evaluators branch May 19, 2024 06:44
@Holmes98
Copy link
Member Author

Deployed and converted all existing C++ evaluators that were previously using cint.rb to use this instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants