Welcome to Gradelang’s documentation!¶
Introduction¶
Gradelang is a new domain-specific language designed to alleviate many of the common pains in autograding.
Gradelang allows users to create arbitrarily complex testing conditions for executable files without being locked into
creating Python TestCase`s relying on :code:`Subprocess.run()
.
Driven by question
blocks, wrapped with optional setup
and teardown
blocks, each question
is executed separately, within it’s own process, with results recombined into a final report.
Example¶
setup {
# @/ represents a special thread-safe location,
# which is automatically cleaned up when a
# question exits, successfully or otherwise.
touch "@/temp.txt";
run "echo";
assert exit successful;
}
teardown {
# This is explicit cleanup.
remove "@/temp.txt";
}
output {
# Output formats with no filename are written to stdout.
json;
# Whereas, parameterized formats are written to disk.
markdown "results.md";
}
question 1 {
# Run the program, saving output.
run "echo", "hello world";
# Now let's run some checks.
assert exit successful;
# This checks both stdout and stderr
assert "hello" in stdout;
award 10;
}
question 2 {
run "echo", "hello world";
assert "goodbye" not in stdout;
award 10;
assert "hello" in stdout;
assert "hello" not in stderr;
award 10;
}
question 3 {
let x be Float(minvalue=1);
run "echo", x;
# If we want to just look at stdout.
assert x in stdout;
String y = "fish";
run "echo", y;
assert "fish" in stdout;
let z be String();
run "echo", z;
assert z in stdout;
let camel be Int(min_value=6);
run "echo", camel;
assert camel in stdout;
award 50;
}