Run tests for spec compliance

[acka47]

LDN receiver: https://linkedresearch.org/ldn/tests/receiver
For WebSub tests see https://websub.rocks/

[acka47]

As the test server is deployed (#12), I tried out https://linkedresearch.org/ldn/tests/receiver on https://test.skohub.io/inbox?target=http://test.lobid.org/gnd/118696432 with data {"foo": "bar"}. Strangely, it says that the server would repsond with HTTP 400, Invalid Content-Type to the POST request. It says on the page that the following is tested:

• Accepts POST requests.
• Responds to POST requests with Content-Type: application/ld+json with status code 201 Created or 202 Accepted.

When I do exactly this with curl, it works:

$ curl -i -X POST -H "Content-Type: application/ld+json" "https://test.skohub.io/inbox?target=http://test.lobid.org/gnd/118696432" -d '{"foo": "bar"}'
HTTP/1.1 202 Accepted
Date: Tue, 30 Apr 2019 08:19:34 GMT
Server: Apache/2.4.18 (Ubuntu)
X-Powered-By: Express
Content-Length: 0

This is is quite confusing and gives the impression that the test does not test what it says it does. @csarven, can you bring some clarity into this?

[csarven]

The payload must be in JSON-LD ({"foo": "bar"} alone is not). So, try as compacted or expanded JSON-LD?

Edit: I think 400 response is correct in this case (while the Content Type says it is JSON-LD, the payload is not). Although I'm not sure right now why you get a 202. Is https://test.skohub.io/inbox?target=http://test.lobid.org/gnd/118696432 the URL of an Inbox or is that a target resource that has an inbox? Perhaps check config's maxPayloadSize size? If the payload goes over, it'll probably result in 202.

[acka47]

Is https://test.skohub.io/inbox?target=http://test.lobid.org/gnd/118696432 the URL of an Inbox or is that a target resource that has an inbox?

https://test.skohub.io/inbox?target=http://test.lobid.org/gnd/118696432 is the URL for the inbox of the target resource http://test.lobid.org/gnd/118696432.

$ curl -I http://test.lobid.org/gnd/118696432
HTTP/1.1 200 OK
Date: Tue, 30 Apr 2019 10:38:02 GMT
Server: Apache/2.4.10 (Linux/SUSE)
Link: <https://test.skohub.io/inbox?target=http://test.lobid.org/gnd/118696432>; rel=http://www.w3.org/ns/ldp#inbox, <https://test.skohub.io/hub>; rel=hub, <http://test.lobid.org/gnd/118696432>; rel=self
Access-Control-Allow-Origin: *
Content-Type: application/json
Content-Length: 5215
Via: 1.1 test.lobid.org

However, I do not get the test at https://linkedresearch.org/ldn/tests/receiver to be successful. Whether I use the default data that is JSON-LD or post {"@context": {"foo": "http://example.org/foo"},"foo": "bar"} instead, I will get HTTP 400, Invalid Content-Type.

Perhaps check config's maxPayloadSize size? If the payload goes over, it'll probably result in 202.

@literarymachine has built it to always respond 202 because the notification won't be stored and does not get a URI but will be send via Websub to subscribers.

[csarven]

$ curl -iX POST -H'Content-Type: application/ld+json; charset=utf-8' https://test.skohub.io/inbox?target=http://test.lobid.org/gnd/118696432 -d '{ "@id": "http://example.net/note#foo", "http://schema.org/citation": { "@id": "http://example.org/article#results" } }'

Make sure params beside the mimetype eg. charset, profile etc are taken into account. Right now, 400 seems correct the way your server is set up.

Edit: put another way (perhaps more importantly), the server shouldn't end up in 400 if all it wants to do is ignore such params in Content-Type. The Content-Type value is valid after all, and it is perfectly normal for a sender to send that (even if they're not required as per LDN spec).

For reference, the LDN Test Suite sends out the equivalent of:
$ curl -iX POST -H'Slug: .jsonld' -H'Content-Type: application/ld+json; profile="http://example.org/profile"; charset=utf-8' https://test.skohub.io/inbox?target=http://test.lobid.org/gnd/118696432 -d '{ "@id": "http://example.net/note#foo", "http://schema.org/citation": { "@id": "http://example.org/article#results" } }'

Note the Slug. It is okay to ignore that as well.. and I think your server is not having an issue with that but profile or charset are leading to the 400. So, ignore them (and return 201/202) if you think that's what should happen.

[acka47]

@csarven, thanks for the clarification. This is very helpful.

I opened #13 to support the profile and charset parameter.

[acka47]

Running the WebSub tests we noticed two things:

1. A conceptional flaw in our current architecture: Services are not actually subscribing to a topic/concept but to the notifications sent to it, so it is the inbox that should be subscribed to and not the resource itself.
2. Websub expects the payload of a push notification to be identical with the content of the topic, i.e. the current content of the web resource subscribed to. See 7. Content Distribution in the the spec, where it reads:

The content distribution request MUST have a Content-Type Header corresponding to the Content-Type of the topic, and MUST contain the full contents of the topic URL, with an exception allowed as described below.

For Atom ([RFC4287]) and RSS ([RSS-2.0]) feeds, the hub MAY remove already-delivered atom:entry or rss:item elements from the feed.

I reopened #2 to cover 1.). We will have to think about how to deal with 2.)

[acka47]

Regarding 2.), I see two options:

1. Implement Websub without 100% spec-compliance
2. Switch to using ActivityPub's subscription mechanism with ActivityStream's Follow activity (as:Follow). See https://www.w3.org/TR/social-web-protocols/#subscribing-with-follow for a short description. Notifications regarding new resources about a subject could then be sent using the as:Add activity (see also https://www.w3.org/TR/activitypub/#add-activity-inbox).

I tend towards the second solution. I did not follow up with ActivityPub in the past but already thought that it might be a good fit because of being JSON-LD-based as well. With this problem with Websub occuring, I think we should seriously consider using ActivityPub instead. We will definitely have no problem regarding the payload of an Add action and transient objects without URI are also supported (from "Object Identifiers" paragraph):

[A]ll objects distributed by the ActivityPub protocol MUST have unique global identifiers, unless they are intentionally transient (short lived activities that are not intended to be able to be looked up, such as some kinds of chat messages or game notifications)

Furthermore, get possibilities of expanding Skohub with other ActivityPub/Stream features like the client to server protocol, likes etc.

[acka47]

I just tested the current setup again with https://linkedresearch.org/ldn/tests/receiver#test-response on inbox https://test.skohub.io/inbox?target=https%3A%2F%2Fw3id.org%2Fclass%2Fesc%2Fn0111:

We should take a look at this in December.

Regarding testing ActivityPub, there currently exists no test suite, see https://socialhub.activitypub.rocks/t/the-activitypub-test-suite/290/3.

[literarymachine]

I just tested the current setup again inbox https://test.skohub.io/inbox?target=https%3A%2F%2Fw3id.org%2Fclass%2Fesc%2Fn0111

As defined in the wiki, the parameter now is ?actor=.

Also, our actors are currently not the URIs of the concepts, but relative paths:

https://skohub.io/inbox?actor=hbz/vocabs-edu/w3id.org/class/esc/n001

This is due to the fact that we currently have to map them to webfinger user@domain because of our switch to Mastodon-compliant ActivityPub. We should definitely have another look at all of this after SWIB.