Feature request: let check50 eat URLs instead of github paths

[pazz]

Hey!
Thanks for check50, it looks awesome and I can't wait adopting it more widely.

One thing that's bugging me with check50/submit50 so far is the (perceived?) github lock-in.
I'd love to run both in combination with a locally hosted gitlab instance which can be used by our students without requiring to syncronize third party accounts.

Towards such a set-up, a first obvious missing feature is the ability to pull remote problem sets from any URL as a parameter.

Another though is that it'd be helpful if all hard-coded strings refering to Harwards support team could be configurable, perhaps as part of the problem set. I really don't want my students to spam you guys with bug reports in case my psets are buggy.

Cheers and hopefully talk to you soon in person, at this teachers workshop next week.
P

[cmlsharp]

So there's two ways GitHub is used by check50/submit50

1. where the actual checks are located (e.g. https://github.com/cs50/problems/tree/2019/ap). Does not require the student to have a GitHub account.
2. where check50 pushes the student's code to so that it can be checked by our backend. Does require the student to have a GitHub account.

It sounds like your use case really only requires a generalization of 2 to other platforms, not 1. As such it doesn't sound like the feature you're after really requires check50 to be able to pull from any URL (since doing so doesn't require that the students have a github account, just that whoever writes the checks does), but rather letting the URL check50/submit50 pushes to be configurable (since this is what requires your students to have a GitHub account in the first place).

Unfortunately generalizing 2 is way harder than generalizing 1. It wouldn't be hard to tell check50 or submit50 to push somewhere else (we could simply add a key to the .cs50.yaml spec, in fact we already allow you to push to a different GitHub org than ours), the hard part is what to do with our back end submit.cs50.io. For starters, the only way to log in to submit.cs50.io right now to see your submit50 results is with a GitHub account. Moreover, GitHub account IDs are how we identify users on the back end. While, in principle, we could support another remote service like BitBucket (at the cost of redeisigning a lot of submit.cs50.io's API), it's not clear how we could authenticate off your own internal GitLab server. I can't remember if the submit.cs50.io is still private, but if not the code is at https://github.com/cs50/submit.cs50.io

I guess allowing the URL that check50/submit50 pushes to would give someone the freedom to never put their student's code on GitHub so maybe it's worth implementing, but anyone doing that would be forfeiting their ability to use our backend at all.

You can still use check50 --local to use check50 with no GitHub account required (the checks will be run locally on your machine), but there's no equivalent of running submit50 locally obviously.

[pazz]

Thanks for your reply Chad. Quoting Chad Sharp (2020-07-06 19:17:49)

So there's two ways GitHub is used by check50. 1. is where the actual checks are located (e.g. https://github.com/cs50/ problems/tree/2020/ap) 2. is where check50 pushes the student's code to so that it can be run by our back end It sounds like your use case really only requires a generalization of 2 to other platforms, not 1. As such it doesn't sound like the feature you're after really requires check50 to be able to pull from any URL (since doing so doesn't require that the students have a github account, just that you do), but rather letting the URL check50/submit50 pushes to be configurable (since this is what requires your students to have a GitHub account in the first place).

I understand (also from reading your excellent docs!) that the submit.cs50.io backed is a completely different beast from check50. I just mentioned my "usecase" because that's what I had in mind as a longer-term ideal scenario for my department and I'm evaluating how feasible this would be. During last weeks teachers workshop I learned that the platform itself is not (yet?) open-sourced. Otherwise I'd discuss this over there. For this issue I really meant your point 1. From reading the check50 sources I have the impression that this really is a low-hanging fruit so to speak, and as a newcomer to check50 I was wondering why such an obvious feature was not considered. I say obvious feature because it clearly subsumes identifying psets by slugs, which ultimately are translated into URLs. My use case for this is that, at the moment, I only use check50 as an aid for marking and the feature would slightly simplify distributing the psets to my TAs.

Unfortunately generalizing 2 is way harder than generalizing 1. It wouldn't be hard to tell check50 or submit50 to push somewhere else (we could simply add a key to the .cs50.yaml spec, in fact we already allow you to push to a different GitHub org than ours), the hard part is what to do with our back end submit.cs50.io. For starters, the only way to log in to submit.cs50.io right now to see your submit50 results is with a GitHub account. Moreover, GitHub account IDs is how we identify users on the back end. While, in principle, we could support another remote service like BitBucket (at the cost of redeisigning a lot of submit.cs50.io's API), it's not clear how we could authenticate off your own gitlab server.

Local authentification is less of a problem, because it is done via LDAP. Don't get me wrong, I understand that this would likely require non-trivial refactoring. For now it is good to know that this is not currently possible. If you are considering open-sourcing the platform or implementing different back-ends I'd be very interested in that of course. Bottom line: This issue is really meant to ask about point 1. If you are not interested in the feature, fine with me. If you are, consider leaving this issue open and perhaps someone like me will eventually send a PR. Best wishes, P

[cmlsharp]

For this issue I really meant your point 1. From reading the check50 sources I have the impression that this really is a low-hanging fruit so to speak, and as a newcomer to check50 I was wondering why such an obvious feature was not considered. I say obvious feature because it clearly subsumes identifying psets by slugs,
which ultimately are translated into

It's low hanging fruit in a sense, except insofar as it intersects with design decisions on our back end (which is in charge of actually checking the student's code). Making the change to check50 or submit50 itself would be fairly trivial, the question is how it interacts with our back end. Currently slugs are used as branch names in the repos we have for students: these branch names would presumably have to become URLs however not all characters that are allowed in URLs are allowed in branch names, not to mention that it would be significantly less readable to students who visit their repo. There are already complaints about the long slugs used for psets, URLs would make them even longer. As you noted, submit.cs50.io is designed with slugs in mind. It'd have to get completely rewritten and there are a lot of UX/UI questions there. As such, there really isn't a PR to check50 that someone could make to add this feature since it is inexorably linked with the submit.cs50.io backend.

We've talked about letting checks be hosted elsewhere in the past, but what we always ended up running into was that it added a fair bit of backend complexity for very little benefit. Allowing checks to be hosted elsewhere doesn't prevent students from needing GitHub accounts, it really only prevents the teacher from needing one.

I'll leave this issue open though in case others want to comment. @Jelleas, thoughts?

[Jelleas]

There is definitely still talk on alternatives to GitHub, especially because GitHub is blocked in certain school districts. As @cmlsharp mentioned, a lot of the difficulty here has more to do with the remote backend, and honestly we're currently looking into alternative solutions there too. But that is not an easy change.

Putting the remote aside for a bit, it should be fairly doable to get check50 --local to consume any URL as an alternative to a slug. In fact we'd only need an alternative implementation of:

https://github.com/cs50/lib50/blob/ca41d702421250781663c3326f616be74e4e4981/lib50/_api.py#L108

So maybe that's worth pursuing. It would at least make check50 --local usable without git and GitHub.