Command testing by parsing #66
Conversation
This function takes a list of named codeblocks containing the testing function(s) per exercise section. To obtain these sections, the student code is being parsed (split on section titles following the pattern "### sectionname ###"). The section name has to match the named codeblock in the testcases list in order to run the testing functions.
There was a problem hiding this comment.
Please style your code consistently with the rest of the project (space after for and if, space between ) and {, space around ==/!=).
I feel like this doesn't handle mismatches between the exercise's requested sections and the student's provided sections. AFAICT, there is no message when the studnt has superfluous sections, and same for missing? In those cases, a warning to the student also feels appropriate.
Also, IIUC this throws away any code that is not in a section, right? (So all code before the first section, I guess). A student might set up some results/variables there, which could make the rest of the code invalid, even though their solution has correct results when run locally. You could add a warning for that case, but I'd prefer if we executed all submitted code, but just didn't do any tests for a preamble.
Co-authored-by: Charlotte Van Petegem <charlotte.vanpetegem@ugent.be>
By removing the section titles in the boilertemplate a student could avoid the test associated with this code section
|
|
That's correct luckily I made alternative testing methods which i will be using for the upcoming practica today. But thanks for noticing! |
| missing_sections <- setdiff(names(testcases), names(codeblocks)) | ||
| if (length(missing_sections) > 0) { | ||
| get_reporter()$add_message( | ||
| paste0("Parsing error: could not find rhe following section title(s): \r\n", |
There was a problem hiding this comment.
| paste0("Parsing error: could not find rhe following section title(s): \r\n", | |
| paste0("Parsing error: could not find the following section title(s): \n", |
| if (length(missing_sections) > 0) { | ||
| get_reporter()$add_message( | ||
| paste0("Parsing error: could not find rhe following section title(s): \r\n", | ||
| paste(sapply(missing_sections, function(x) {paste("###", x, "###")}), collapse = ',\r\n')) |
There was a problem hiding this comment.
| paste(sapply(missing_sections, function(x) {paste("###", x, "###")}), collapse = ',\r\n')) | |
| paste(sapply(missing_sections, function(x) {paste("###", x, "###")}), collapse = ',\n')) |
| paste0("Parsing error: could not find rhe following section title(s): \r\n", | ||
| paste(sapply(missing_sections, function(x) {paste("###", x, "###")}), collapse = ',\r\n')) | ||
| ) | ||
| get_reporter()$escalate("compilation error") |
There was a problem hiding this comment.
I'm not sure a compilation error makes a lot of sense? Isn't it more logical to add a runtime error (similar to if a function was missing), and still try to evaluate the other testcases?
This branch introduces the contextWithParsing function. This context can be used to test intermediate (printed/returned) results that are not saved in an object. This can be done by splitting the student code using comments following a certain pattern. The context function will run the code section by section alternated by the test defined for this section.