Feedback links do not distinguish multiple submissions of the assignment.

[ykazakov]

If an assignment was submitted several times but feedback was provided only to one of the submissions, the student sees feedback as available / downloaded for all submissions of this assignment. The links in assignment list always lead to the last feedback, regardless when submission was made.

Here is the screenshot with the default nbgrader exchange:

Here is the screenshot with nbexchange:

This issue seems to be different from #8.

[perllaghu]

Thanks for this..... Yes, I can see that having a link for every submission is a problem.

What happens, in the default exchange, in the following scenario (all for 1 assignment):

• submit1
• submit2
• get feedback1
• submit3
• submit4
• get feedback2

.... does the interface link the different feedbacks to the submit2 & submit4 lines, or does the interface only link submit4 to feedback2?

[ykazakov]

What happens, in the default exchange, in the following scenario (all for 1 assignment)

All feedbacks are independent and are separately linked.

[perllaghu]

Yep.... we need to improve our code here.... thankee

[ykazakov]

I have some questions about the NBExchange API:

1. It seems that feedback timestamps are stored per notebook, not per submission. Is that right? When notebooks have different timestamps, in which feedback folder are they downloaded?
2. Does feedback_timestamp refer to the time when the feedback was issued or the timestamp of the submission to which the feedback is made?
3. When an assignment has several submissions (some with, some without feedbacks), will it be returned several times in the request .../assignments?course_id=$course_code (with different feedback times of notebook)? Or otherwise, how to get the list of all feedbacks for the particular assignment?

I just want to understand if this problem can be easily fixed (it is a bug) or requires changing the nbexchange data model.

[ykazakov]

OK, I figured out. Looks like feedback_timestamp is different from timestamp of the submission. The feedbacks are downloaded to the folder corresponding to feedback_timestamp. This is different from the default exchange, which downloads feedback to submissions to the folder corresponding to the timestamp of the submissions.

For the last question, I just tested the query and it returned me the output which basically corresponds to the screenshot, that is, only the timestamp of the last feedback_timestamp is provided for all submissions.

Details

{"success": true, "value": [{"assignment_id": "ps2", "student_id": 2, "course_id": "cs101", "status": "released", "path": "/var/run/nbexchange/storage/1/released/cs101/ps2/1727192330/8cc081b1-6dda-4305-90b1-98f207f24bb9.gz", "notebooks": [{"notebook_id": "part1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 15:38:50.199355 "}, {"assignment_id": "ps2", "student_id": 1, "course_id": "cs101", "status": "fetched", "path": "/var/run/nbexchange/storage/1/released/cs101/ps2/1727192330/8cc081b1-6dda-4305-90b1-98f207f24bb9.gz", "notebooks": [{"notebook_id": "part1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 15:39:05.471178 "}, {"assignment_id": "ps2", "student_id": 1, "course_id": "cs101", "status": "submitted", "path": "/var/run/nbexchange/storage/1/submitted/cs101/ps2/prof/1727192349/7b9e9f51-014d-4834-a439-44eb45cc904a.gz", "notebooks": [{"notebook_id": "part1", "has_exchange_feedback": true, "feedback_updated": false, "feedback_timestamp": "2024-09-24 15:39:09.944252 "}], "timestamp": "2024-09-24 15:39:09.959360 "}, {"assignment_id": "ps2", "student_id": 1, "course_id": "cs101", "status": "feedback_fetched", "path": "/var/run/nbexchange/storage/1/feedback/cs101/ps2/1727192381/9ea680c10225aefeb5a4379aa574f1da.html", "notebooks": [{"notebook_id": "part1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 15:39:48.680818 "}, {"assignment_id": "ps2", "student_id": 1, "course_id": "cs101", "status": "submitted", "path": "/var/run/nbexchange/storage/1/submitted/cs101/ps2/prof/1727192394/8eb23853-7f51-41a1-a6cb-d5ac84f06832.gz", "notebooks": [{"notebook_id": "part1", "has_exchange_feedback": true, "feedback_updated": false, "feedback_timestamp": "2024-09-24 15:39:09.944252 "}], "timestamp": "2024-09-24 15:39:54.865724 "}, {"assignment_id": "ps2", "student_id": 1, "course_id": "cs101", "status": "submitted", "path": "/var/run/nbexchange/storage/1/submitted/cs101/ps2/prof/1727192442/e65d5188-43c0-4161-a98a-17107a98ffba.gz", "notebooks": [{"notebook_id": "part1", "has_exchange_feedback": true, "feedback_updated": false, "feedback_timestamp": "2024-09-24 15:39:09.944252 "}], "timestamp": "2024-09-24 15:40:42.241833 "}, {"assignment_id": "ps2", "student_id": 1, "course_id": "cs101", "status": "feedback_fetched", "path": "/var/run/nbexchange/storage/1/feedback/cs101/ps2/1727192381/9ea680c10225aefeb5a4379aa574f1da.html", "notebooks": [{"notebook_id": "part1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 15:41:10.483238 "}, {"assignment_id": "ps2", "student_id": 1, "course_id": "cs101", "status": "feedback_fetched", "path": "/var/run/nbexchange/storage/1/feedback/cs101/ps2/1727192465/9ea680c10225aefeb5a4379aa574f1da.html", "notebooks": [{"notebook_id": "part1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 15:41:10.489525 "}, {"assignment_id": "ps2", "student_id": 1, "course_id": "cs101", "status": "feedback_fetched", "path": "/var/run/nbexchange/storage/1/feedback/cs101/ps2/1727192381/9ea680c10225aefeb5a4379aa574f1da.html", "notebooks": [{"notebook_id": "part1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 15:41:11.276428 "}, {"assignment_id": "ps2", "student_id": 1, "course_id": "cs101", "status": "feedback_fetched", "path": "/var/run/nbexchange/storage/1/feedback/cs101/ps2/1727192465/9ea680c10225aefeb5a4379aa574f1da.html", "notebooks": [{"notebook_id": "part1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 15:41:11.282210 "}, {"assignment_id": "ps2", "student_id": 1, "course_id": "cs101", "status": "feedback_fetched", "path": "/var/run/nbexchange/storage/1/feedback/cs101/ps2/1727192381/9ea680c10225aefeb5a4379aa574f1da.html", "notebooks": [{"notebook_id": "part1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 15:41:12.256730 "}, {"assignment_id": "ps2", "student_id": 1, "course_id": "cs101", "status": "feedback_fetched", "path": "/var/run/nbexchange/storage/1/feedback/cs101/ps2/1727192465/9ea680c10225aefeb5a4379aa574f1da.html", "notebooks": [{"notebook_id": "part1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 15:41:12.259033 "}, {"assignment_id": "ps2", "student_id": 1, "course_id": "cs101", "status": "feedback_fetched", "path": "/var/run/nbexchange/storage/1/feedback/cs101/ps2/1727192381/9ea680c10225aefeb5a4379aa574f1da.html", "notebooks": [{"notebook_id": "part1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 15:41:13.215329 "}, {"assignment_id": "ps2", "student_id": 1, "course_id": "cs101", "status": "feedback_fetched", "path": "/var/run/nbexchange/storage/1/feedback/cs101/ps2/1727192465/9ea680c10225aefeb5a4379aa574f1da.html", "notebooks": [{"notebook_id": "part1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 15:41:13.219322 "}, {"assignment_id": "ps2", "student_id": 1, "course_id": "cs101", "status": "feedback_fetched", "path": "/var/run/nbexchange/storage/1/feedback/cs101/ps2/1727192381/9ea680c10225aefeb5a4379aa574f1da.html", "notebooks": [{"notebook_id": "part1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 15:41:17.607199 "}, {"assignment_id": "ps2", "student_id": 1, "course_id": "cs101", "status": "feedback_fetched", "path": "/var/run/nbexchange/storage/1/feedback/cs101/ps2/1727192465/9ea680c10225aefeb5a4379aa574f1da.html", "notebooks": [{"notebook_id": "part1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 15:41:17.610327 "}, {"assignment_id": "ps1", "student_id": 2, "course_id": "cs101", "status": "released", "path": "/var/run/nbexchange/storage/1/released/cs101/ps1/1727167602/5171a950-44b4-48e5-b7aa-ca8b6df846e1.gz", "notebooks": [{"notebook_id": "problem1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}, {"notebook_id": "problem2", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 08:46:42.239274 "}, {"assignment_id": "ps1", "student_id": 1, "course_id": "cs101", "status": "fetched", "path": "/var/run/nbexchange/storage/1/released/cs101/ps1/1727167602/5171a950-44b4-48e5-b7aa-ca8b6df846e1.gz", "notebooks": [{"notebook_id": "problem1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}, {"notebook_id": "problem2", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 08:47:06.397647 "}, {"assignment_id": "ps1", "student_id": 1, "course_id": "cs101", "status": "submitted", "path": "/var/run/nbexchange/storage/1/submitted/cs101/ps1/prof/1727167628/fcc7b6dd-4d04-4ca4-b2a8-c697d811713c.gz", "notebooks": [{"notebook_id": "problem1", "has_exchange_feedback": true, "feedback_updated": false, "feedback_timestamp": "2024-09-24 16:13:22.268188 "}, {"notebook_id": "problem2", "has_exchange_feedback": true, "feedback_updated": false, "feedback_timestamp": "2024-09-24 16:13:22.268188 "}], "timestamp": "2024-09-24 08:47:08.571904 "}, {"assignment_id": "ps1", "student_id": 1, "course_id": "cs101", "status": "feedback_fetched", "path": "/var/run/nbexchange/storage/1/feedback/cs101/ps1/1727167682/7f00275cbec90f82b44c7411413fe920.html", "notebooks": [{"notebook_id": "problem1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}, {"notebook_id": "problem2", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 08:48:07.267073 "}, {"assignment_id": "ps1", "student_id": 1, "course_id": "cs101", "status": "feedback_fetched", "path": "/var/run/nbexchange/storage/1/feedback/cs101/ps1/1727167682/f2e38b3da22c73945aa75201564a1772.html", "notebooks": [{"notebook_id": "problem1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}, {"notebook_id": "problem2", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 08:48:07.269869 "}, {"assignment_id": "ps1", "student_id": 1, "course_id": "cs101", "status": "submitted", "path": "/var/run/nbexchange/storage/1/submitted/cs101/ps1/prof/1727169244/6d5f8216-a10e-4afe-aa21-bca8046fdcc8.gz", "notebooks": [{"notebook_id": "problem1", "has_exchange_feedback": true, "feedback_updated": false, "feedback_timestamp": "2024-09-24 16:13:22.268188 "}, {"notebook_id": "problem2", "has_exchange_feedback": true, "feedback_updated": false, "feedback_timestamp": "2024-09-24 16:13:22.268188 "}], "timestamp": "2024-09-24 09:14:04.975736 "}, {"assignment_id": "ps1", "student_id": 1, "course_id": "cs101", "status": "submitted", "path": "/var/run/nbexchange/storage/1/submitted/cs101/ps1/prof/1727192141/dcbc28c8-f014-489a-b2c4-0e78f2d35c12.gz", "notebooks": [{"notebook_id": "problem1", "has_exchange_feedback": true, "feedback_updated": false, "feedback_timestamp": "2024-09-24 16:13:22.268188 "}, {"notebook_id": "problem2", "has_exchange_feedback": true, "feedback_updated": false, "feedback_timestamp": "2024-09-24 16:13:22.268188 "}], "timestamp": "2024-09-24 15:35:41.182870 "}, {"assignment_id": "ps1", "student_id": 1, "course_id": "cs101", "status": "submitted", "path": "/var/run/nbexchange/storage/1/submitted/cs101/ps1/prof/1727192145/b9466011-fa72-4f3c-a90e-1087e584b6f7.gz", "notebooks": [{"notebook_id": "problem1", "has_exchange_feedback": true, "feedback_updated": false, "feedback_timestamp": "2024-09-24 16:13:22.268188 "}, {"notebook_id": "problem2", "has_exchange_feedback": true, "feedback_updated": false, "feedback_timestamp": "2024-09-24 16:13:22.268188 "}], "timestamp": "2024-09-24 15:35:45.082283 "}, {"assignment_id": "ps1", "student_id": 1, "course_id": "cs101", "status": "submitted", "path": "/var/run/nbexchange/storage/1/submitted/cs101/ps1/prof/1727192234/ddf4df78-ba91-496f-a4e9-b589cf141659.gz", "notebooks": [{"notebook_id": "problem1", "has_exchange_feedback": true, "feedback_updated": false, "feedback_timestamp": "2024-09-24 16:13:22.268188 "}, {"notebook_id": "problem2", "has_exchange_feedback": true, "feedback_updated": false, "feedback_timestamp": "2024-09-24 16:13:22.268188 "}], "timestamp": "2024-09-24 15:37:14.632649 "}, {"assignment_id": "ps1", "student_id": 1, "course_id": "cs101", "status": "feedback_fetched", "path": "/var/run/nbexchange/storage/1/feedback/cs101/ps1/1727167682/7f00275cbec90f82b44c7411413fe920.html", "notebooks": [{"notebook_id": "problem1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}, {"notebook_id": "problem2", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 16:28:44.451722 "}, {"assignment_id": "ps1", "student_id": 1, "course_id": "cs101", "status": "feedback_fetched", "path": "/var/run/nbexchange/storage/1/feedback/cs101/ps1/1727167682/f2e38b3da22c73945aa75201564a1772.html", "notebooks": [{"notebook_id": "problem1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}, {"notebook_id": "problem2", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 16:28:44.458735 "}, {"assignment_id": "ps1", "student_id": 1, "course_id": "cs101", "status": "feedback_fetched", "path": "/var/run/nbexchange/storage/1/feedback/cs101/ps1/1727167682/7f00275cbec90f82b44c7411413fe920.html", "notebooks": [{"notebook_id": "problem1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}, {"notebook_id": "problem2", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 16:31:04.557948 "}, {"assignment_id": "ps1", "student_id": 1, "course_id": "cs101", "status": "feedback_fetched", "path": "/var/run/nbexchange/storage/1/feedback/cs101/ps1/1727167682/f2e38b3da22c73945aa75201564a1772.html", "notebooks": [{"notebook_id": "problem1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}, {"notebook_id": "problem2", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 16:31:04.562291 "}, {"assignment_id": "ps1", "student_id": 1, "course_id": "cs101", "status": "fetched", "path": "/var/run/nbexchange/storage/1/released/cs101/ps1/1727167602/5171a950-44b4-48e5-b7aa-ca8b6df846e1.gz", "notebooks": [{"notebook_id": "problem1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}, {"notebook_id": "problem2", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 16:35:00.386687 "}, {"assignment_id": "ps1", "student_id": 1, "course_id": "cs101", "status": "feedback_fetched", "path": "/var/run/nbexchange/storage/1/feedback/cs101/ps1/1727167682/7f00275cbec90f82b44c7411413fe920.html", "notebooks": [{"notebook_id": "problem1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}, {"notebook_id": "problem2", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 16:35:05.629488 "}, {"assignment_id": "ps1", "student_id": 1, "course_id": "cs101", "status": "feedback_fetched", "path": "/var/run/nbexchange/storage/1/feedback/cs101/ps1/1727167682/f2e38b3da22c73945aa75201564a1772.html", "notebooks": [{"notebook_id": "problem1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}, {"notebook_id": "problem2", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 16:35:05.633403 "}, {"assignment_id": "ps1", "student_id": 1, "course_id": "cs101", "status": "feedback_fetched", "path": "/var/run/nbexchange/storage/1/feedback/cs101/ps1/1727167682/7f00275cbec90f82b44c7411413fe920.html", "notebooks": [{"notebook_id": "problem1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}, {"notebook_id": "problem2", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 16:44:16.609443 "}, {"assignment_id": "ps1", "student_id": 1, "course_id": "cs101", "status": "feedback_fetched", "path": "/var/run/nbexchange/storage/1/feedback/cs101/ps1/1727167682/f2e38b3da22c73945aa75201564a1772.html", "notebooks": [{"notebook_id": "problem1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}, {"notebook_id": "problem2", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 16:44:16.610543 "}, {"assignment_id": "ps1", "student_id": 1, "course_id": "cs101", "status": "feedback_fetched", "path": "/var/run/nbexchange/storage/1/feedback/cs101/ps1/1727196238/5aa38f4049b9eb1ae8fb5407f5c0427c.html", "notebooks": [{"notebook_id": "problem1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}, {"notebook_id": "problem2", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 16:44:16.611365 "}, {"assignment_id": "ps1", "student_id": 1, "course_id": "cs101", "status": "feedback_fetched", "path": "/var/run/nbexchange/storage/1/feedback/cs101/ps1/1727196238/637d63b54d9808c639636ddb0053b5f6.html", "notebooks": [{"notebook_id": "problem1", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}, {"notebook_id": "problem2", "has_exchange_feedback": false, "feedback_updated": false, "feedback_timestamp": null}], "timestamp": "2024-09-24 16:44:16.616706 "}]}

[perllaghu]

I'm happy to change things to be more in-line with standard nbgrader plugins.

I think our reasoning [some 4 years ago] was that we wanted to have a timestamp for the student of when the instructor had actually marked their work [rather than when they'd submitted it - they already knew that!]

[ykazakov]

I think our reasoning [some 4 years ago] was that we wanted to have a timestamp for the student of when the instructor had actually marked their work [rather than when they'd submitted it - they already knew that!]

What for??

The main problem with different timestamps: how do students know which exactly submission has been marked? The instructor can mark earlier submission at a later timepoint when another submission was already made.
Is it possible to obtain this information from nbexchange database?

[perllaghu]

What for??

I don't fully remember..... I suspect it was probably something nebulous like "It'll be useful somehow"

Is it possible to obtain this information from nbexchange database?

Yes... the exchange should timestamp every action, so that can be released in metadata - actually, isn't that in the metadata lump you shared?

[ykazakov]

the exchange should timestamp every action

That is good, but the time stamp of feedback is not enough to identify the submission (not the assignment!) for which the feedback was provided.

actually, isn't that in the metadata lump you shared?

As far as I can tell, the response for assignments makes it look like the last feedback is provided to every submission of the assignment, i.e., just like it is shown in the form. Also, there is no information (timestamps) of the previous feedbacks.