Remove or re-work section 6.1.2 Links

[benfrancis]

I'm sorry I'm only just getting to review this after PR #240 has already landed.

@mlagally Did you take into account the previous discussion on this topic in #114?

Review comments below.

6.1.2 Links

The following keywords are defined for the HTTP profiles and MUST be supported by all consumers.

In principle I like the idea of defining a finite set of link relation types which Consumers MUST support and Producers MAY use, but I'm not sure how it can work in practice (see below).

What does "keywords" mean here? Does it mean Consumers must support all of the included members of the link object, and all of the listed link relation types? I'm going to assume that's the case.

type

WebThings currently uses links to link to WebSocket endpoints, which don't have a type member, just a href and a rel member, because the type member wouldn't make sense.

We are discussing changing these links to forms, but there are still unresolved issues in the Thing Description specification which may prevent that (see w3c/wot-thing-description#1324).

I suggest it would be fine for type to remain optional as in the Thing Description specification. It being mandatory doesn't really gain anything anyway as far as I can tell (see below).

hreflang

I see there's already an issue filed to make this optional, which I agree with.

6.1.2.1 Link Relation Types

How would all of the assertions in this section be tested?

icon

What does it mean have mandatory support for icons? What if a Consumer doesn't have a screen on which to render an icon? The specification limits icons to PNG and JPEG. What about WEBP, GIF and SVG, which are also widely supported on the web? For WebThings we prefer SVG icons for Things so that they are scaleable, so would want SVG support and the "any" value for sizes.

type

What does a Consumer actually need to do with this kind of link in order to be conformant? My understanding is that Thing Models are only used internally by applications for generating Thing Descriptions and are not intended to be used by Consumers in any way.

service-doc

Again, what if a Consumer can't render, Markdown, HTML or PDF files?

collection

What would the mandatory type value be for this kind of link relation? Does it have to be a Thing Description, a Directory Description, or can it be any kind of resource? (See: w3c/wot-thing-description#1567).

What must a Consumer do with this kind of link relation in order to be compliant? Present a clickable link to the collection to the user? What happens when the user clicks that link? What if there is no user interface?

item

Does the target type have to be a Thing Description? What must a Consumer do with this kind of link relation in order to be compliant?

6.1.2.2 Media Types for Link Targets

The following media types from IANA are defined for the link targets of compliant TDs. They MUST be supported by all consumers.

What must a Consumer do with these links in order to comply with this assertion? This seems like quite an arbitrary list and I have many questions.

Does a Consumer have to be able to render text, HTML and PDF documents? Do all Consumers have to have a web rendering engine in order to render HTML? What if they don't have a screen?

Do all Consumers need to implement a markdown renderer? Which version of markdown? There are many.

What must a Consumer do with machine-readable JSON files, Thing Descriptions and octet streams?

Must a Consumer be able to render JPEGs and PNGs? What if it doesn't have a screen?

---

In WebThings, aside from the WebSocket links which we would like to replace with Forms, our main use case for Links is providing a link to an HTML user interface for a Thing, which is the example I provided in #114. E.g.:

{
  "href": "https://user.webthings.io/things/mything1",
  "rel": "alternate",
  "type": "text/html"
}

I would like it if we could make support for this part of the HTTP Profiles, but again what would it mean that it is mandatory for all Consumers to support this? What happens if a Consumer can't render an HTML page?

In conclusion, whilst in principle I like the idea of a finite set of link relation types that Consumers MUST support and Producers MAY use, in practice I'm not sure it would actually work. I'm struggling to imagine how these assertions can be implemented, or tested.

[mlagally]

6.1.2 Links

The following keywords are defined for the HTTP profiles and MUST be supported by all consumers.

In principle I like the idea of defining a finite set of link relation types which Consumers MUST support and Producers MAY use, but I'm not sure how it can work in practice (see below).

What does "keywords" mean here? Does it mean Consumers must support all of the included members of the link object, and all of the listed link relation types? I'm going to assume that's the case.

The intention is to have a restricted set of feature that consumers must be prepared to handle.
It depends on the actual use cases of the consumer, HOW these are handled.

A consumer without a screen certainly is not able to render any icons, but it might be able to print PDF documents. Others may just ignore these.

type

WebThings currently uses links to link to WebSocket endpoints, which don't have a type member, just a href and a rel member, because the type member wouldn't make sense.

We are discussing changing these links to forms, but there are still unresolved issues in the Thing Description specification which may prevent that (see w3c/wot-thing-description#1324).

I suggest it would be fine for type to remain optional as in the Thing Description specification. It being mandatory doesn't really gain anything anyway as far as I can tell (see below).

6.1.2.1 Link Relation Types

How would all of the assertions in this section be tested?

icon

What does it mean have mandatory support for icons? What if a Consumer doesn't have a screen on which to render an icon? The specification limits icons to PNG and JPEG. What about WEBP, GIF and SVG, which are also widely supported on the web? For WebThings we prefer SVG icons for Things so that they are scaleable, so would want SVG support and the "any" value for sizes.

We could also include SVG, however, since SVG rendering is more costly to implement, we should require PNG or JPEG to have a fallback icon.

type

What does a Consumer actually need to do with this kind of link in order to be conformant? My understanding is that Thing Models are only used internally by applications for generating Thing Descriptions and are not intended to be used by Consumers in any way.

Wheter TMs should be included in the profile is yet an open question.

service-doc

Again, what if a Consumer can't render, Markdown, HTML or PDF files?

see above.

collection

What would the mandatory type value be for this kind of link relation? Does it have to be a Thing Description, a Directory Description, or can it be any kind of resource? (See: w3c/wot-thing-description#1567).

What must a Consumer do with this kind of link relation in order to be compliant? Present a clickable link to the collection to the user? What happens when the user clicks that link? What if there is no user interface?

It depends on the use cases and capabilites of the consumer. In some scenarios a drill down into collection members may be desirable, in others this might be not the case.

item

Does the target type have to be a Thing Description? What must a Consumer do with this kind of link relation in order to be compliant?

Great question. My assumption was that we have TDs as items. Do you see others?
Consumer behavior depends on the use cases, see previous comments.

6.1.2.2 Media Types for Link Targets

The following media types from IANA are defined for the link targets of compliant TDs. They MUST be supported by all consumers.

What must a Consumer do with these links in order to comply with this assertion? This seems like quite an arbitrary list and I have many questions.

Should be discussed in a call.

Does a Consumer have to be able to render text, HTML and PDF documents? Do all Consumers have to have a web rendering engine in order to render HTML? What if they don't have a screen?

Do all Consumers need to implement a markdown renderer? Which version of markdown? There are many.

What must a Consumer do with machine-readable JSON files, Thing Descriptions and octet streams?

Must a Consumer be able to render JPEGs and PNGs? What if it doesn't have a screen?

In WebThings, aside from the WebSocket links which we would like to replace with Forms, our main use case for Links is providing a link to an HTML user interface for a Thing, which is the example I provided in #114. E.g.:

{
  "href": "https://user.webthings.io/things/mything1",
  "rel": "alternate",
  "type": "text/html"
}

I would like it if we could make support for this part of the HTTP Profiles, but again what would it mean that it is mandatory for all Consumers to support this? What happens if a Consumer can't render an HTML page?

See comments above. Let's discuss in the next call.

In conclusion, whilst in principle I like the idea of a finite set of link relation types that Consumers MUST support and Producers MAY use, in practice I'm not sure it would actually work. I'm struggling to imagine how these assertions can be implemented, or tested.

The intention of this section is to have a finite set with clear semantics. Consumer behavior depends on uses cases and capabilities (e.g. whether they have a screen, rendering engine for specific types, wheter the use cases require to prender documents, e.g. user manuals ...)

[benfrancis]

@mlagally wrote:

The intention is to have a restricted set of feature that consumers must be prepared to handle.

I understand and agree with the objective, but I don't think the specification does or can achieve this, because of the ambiguity in what "handle" actually means and the difference in capabilities of different consumers.

A consumer without a screen certainly is not able to render any icons, but it might be able to print PDF documents. Others may just ignore these.

If they can be ignored then supporting them is not mandatory. If it's not mandatory then it's only a recommendation, which is already provided by the Thing Description specification.

We could also include SVG, however, since SVG rendering is more costly to implement, we should require PNG or JPEG to have a fallback icon.

I don't really agree with this, but the important point is that some consumers may not be able to render any icons because they don't have a graphical display to render them to. E.g. think of a command-line tool or software library which acts as a WoT consumer.

Wheter TMs should be included in the profile is yet an open question.

The consensus so far seems to be to exclude them altogether (see #242), but this section appears to say that supporting at least the link relation part of Thing Models is mandatory.

It depends on the use cases and capabilites of the consumer. In some scenarios a drill down into collection members may be desirable, in others this might be not the case.

Can you be more explicit? What would a consumer need to implement in order to comply with this assertion, and how would we write a test to verify that it has?

My assumption was that we have TDs as items. Do you see others?

No, but I'm also not sure about what Consumers need to implement in order to support Thing Descriptions as items.

Consumer behavior depends on the use cases, see previous comments.

Again, if the behaviour depends on use cases, how can it be tested?

The intention of this section is to have a finite set with clear semantics. Consumer behavior depends on uses cases and capabilities (e.g. whether they have a screen, rendering engine for specific types, wheter the use cases require to prender documents, e.g. user manuals ...)

In conclusion, I understand and agree with the objective but I don't think the current text can or does achieve this.

Unless we can be much more explicit about what implementing these different types of link relations involves, to the extent that they can be tested in an implementation report, I think we may have to remove this section. Perhaps the current recommendations in the Thing Description specification are enough.

[mlagally]

@mlagally wrote:

The intention is to have a restricted set of feature that consumers must be prepared to handle.

I understand and agree with the objective, but I don't think the specification does or can achieve this, because of the ambiguity in what "handle" actually means and the difference in capabilities of different consumers.

A consumer without a screen certainly is not able to render any icons, but it might be able to print PDF documents. Others may just ignore these.

If they can be ignored then supporting them is not mandatory. If it's not mandatory then it's only a recommendation, which is already provided by the Thing Description specification.

We could also include SVG, however, since SVG rendering is more costly to implement, we should require PNG or JPEG to have a fallback icon.

I don't really agree with this, but the important point is that some consumers may not be able to render any icons because they don't have a graphical display to render them to. E.g. think of a command-line tool or software library which acts as a WoT consumer.

Wheter TMs should be included in the profile is yet an open question.

The consensus so far seems to be to exclude them altogether (see #242), but this section appears to say that supporting at least the link relation part of Thing Models is mandatory.

It depends on the use cases and capabilites of the consumer. In some scenarios a drill down into collection members may be desirable, in others this might be not the case.

Can you be more explicit? What would a consumer need to implement in order to comply with this assertion, and how would we write a test to verify that it has?

My assumption was that we have TDs as items. Do you see others?

No, but I'm also not sure about what Consumers need to implement in order to support Thing Descriptions as items.

Consumer behavior depends on the use cases, see previous comments.

Again, if the behaviour depends on use cases, how can it be tested?

The intention of this section is to have a finite set with clear semantics. Consumer behavior depends on uses cases and capabilities (e.g. whether they have a screen, rendering engine for specific types, wheter the use cases require to prender documents, e.g. user manuals ...)

In conclusion, I understand and agree with the objective but I don't think the current text can or does achieve this.

Unless we can be much more explicit about what implementing these different types of link relations involves, to the extent that they can be tested in an implementation report, I think we may have to remove this section. Perhaps the current recommendations in the Thing Description specification are enough.

There seems to be a misconception about interoperability testing / conformance of the consumer.
The fundamental assumption is that the manufacturer of a device and the manufacturer of a consumer are not the same company, and that the TD is the only machine-readable exchange of information.

Consumers have different capabilities, some have a screen, others don't. The profile compliant TD must support both, i.e. include icons that can be rendered on devices with a screen.
Similarly, if there's a PDF manual available on the Web, a device should be able to provide a link in an unambiguous way, i.e. tell the consumer where the manual can be found. However the presence of these links in a TD does not put the obligation on the consumer to be able to process all of them, but can use those, that match best to the capabilities of the consumer.

[benfrancis]

@mlagally wrote:

There seems to be a misconception about interoperability testing / conformance of the consumer.
The fundamental assumption is that the manufacturer of a device and the manufacturer of a consumer are not the same company, and that the TD is the only machine-readable exchange of information.

I don't there's a misconception, this is my understanding too.

What I'm saying is that the assertions in the specification have to be implementable, and testable.

Consumers have different capabilities, some have a screen, others don't. The profile compliant TD must support both, i.e. include icons that can be rendered on devices with a screen.

But that isn't what the current text says. It says:

"The following keywords are defined for the HTTP profiles and MUST be supported by all consumers."

What would "supporting" links to icons involve, if not rendering them on a screen?

Similarly, if there's a PDF manual available on the Web, a device should be able to provide a link in an unambiguous way, i.e. tell the consumer where the manual can be found. However the presence of these links in a TD does not put the obligation on the consumer to be able to process all of them, but can use those, that match best to the capabilities of the consumer.

Again, using only those links which match the capabilities of the consumer is very different to what the text currently says. It says that Consumers MUST support all of the keywords, of which text/pdf media type is one. What does "supporting" a link to a PDF mean if not being able to "process" it?

[mlagally]

@benfrancis
I consider it very bad practice to just create a PR that removes an entire section.
Please consider creating a constructive proposal to improve the current text.

[benfrancis]

@mlagally wrote:

I consider it very bad practice to just create a PR that removes an entire section.
Please consider creating a constructive proposal to improve the current text.

@mlagally In #273 you asked me to create a PR to address the concerns I raised in #255.

As I explained in the PR description of #289, removing the section is the only way I can think of to resolve those concerns, but I am open to alternative proposals because I like the underlying idea - I just don't think it can be specified in a way which is implementable in all consumers.

[benfrancis]

Perhaps another direction for this section would be to identify particular combinations of link relation types and media types which should be interpreted in a particular way.

For example:

• If a Consumer encounters a link with "rel": "icon" and "type": "image/*" and it is capable of rendering images in the provided format, then it SHOULD interpret the link as an icon for the Thing and display it to the user.
• If a Consumer encounters a link with "rel": "alternate" and "type": "text/html" and it is capable of rendering an HTML page and accepting user input, then it SHOULD interpret the link as a user interface for the Thing and provide a means for the user to follow that link and view and interact with the HTML page.
• If a Consumer encounters a link with "rel": "service-doc" and "type": "text/plain", "type": "text/html" or "type": "text/pdf", and is capable of rendering documents in the provided format, then it SHOULD interpret the link as a user manual for the Thing and provide a means for the user to follow that link and read the user manual.
• If a Consumer encounters a link with "rel": "item" and "type": "application/td+json" and is capable of rendering a hierarchical tree of Things, then it should interpret the link as an indication that the target is a sub-Thing of the current Thing and render this in a meaningful way to the user.
• If a Consumer encounters a link with "rel": "collection" and "type": "application/td+json" and is capable of rendering a hierarchical tree of Things, then it should interpret the link as an indication that the target describes a Thing (e.g. a group, system of Things or Thing Directory) which contains the current Thing and render this in a meaningful way to the user.

[benfrancis]

Thank you for including some more testable assertions in the style I suggested above.

However, I am currently writing implementation reports for two WoT Profile implementations and I'm still having difficulty in testing the other assertions in this section.

5.6 Links

The following keywords are defined for links in the HTTP profiles and MAY be present in profile-compliant TDs with the constraints defined by this section.

What is the intended meaning of this assertion? Because of the way the assertion is worded it doesn't really have any effect, since the Thing Description specification already implies that Thing Descriptions "may" use these terms. The inclusion of the hreflang member in the table also has no effect since it is listed as optional but is already optional.

Is the intention that members other than those listed in the table are allowed in Link objects (already true), that Things must only use these members (if so why?), that they must only use these members if the provided constraints are observed, or that Things must always use these members (not actually what the assertion says)?

Does "following keywords" refer only to the first table in section 5.6 (which has a column header "Keyword") or also to the table in section 5.6.1 (which does not have the "Keyword" header)?

Other keywords for links MAY be present in a TD, however their interpretation is undefined in the context of the HTTP profiles

Does "keywords" mean members of the Link object, or values of the rel member? Again, the way this assertion is worded means it doesn't really have any effect since all it does is allow something which is already allowed and state what is not defined rather than what is.

These other link types MAY be ignored by all profile-compliant consumers.

What is meant by "link types"? Does it mean "link relation types" from section 5.6.1? If so should the assertion be in that section?

Again, links may already be ignored by all Consumers not only profile-compliant Consumers. What additional constraint is intended to be communicated here?

5.6.1 Link Relation Types

There is no assertion in this section, so it's not clear how to test it.

The table appears to define constraints on how certain relation types are used, but it's very ambiguous about how a Thing or a Consumer would implement these constraints. E.g. Is it mandatory for a Consumer to support icons with the format image/png and image/jpeg? Is it allowed for a Thing to use another image type for icons? What should the type of the target of a collection link be?

5.6.2 Media Types for Link Targets

The following media types from IANA MAY be used as the link targets of profile compliant TDs with the constraints in this section.

As above, this assertion does not really have any effect on its own, since these types "may" be used by any Thing Description.

Is the intended meaning that Things should only use these media types (if so, why?), or that Consumers should try to support them all (probably not practical)?

Other media types MAY be present in a TD, however their heir interpretation is undefined in the context of the HTTP profiles and they MAY be ignored by all profile-compliant consumers.

Again, this assertion only allows something which is already allowed and states what is not defined, not what is. What is the intended meaning?

---

In conclusion, a lot of the content in this section is still quite ambiguous and difficult to test.

I can think of a couple of ways to improve it:

1. Remove all of the content in this section up until the collection of more testable assertions at the end starting with common-constraints-links-media-types-4
2. Re-write the section as a single table with "rel", "type" and "Meaning" headings (which defines how certain links should be interpreted). Then add a couple of assertions at the start which make it clear what the expectations are for both Things and Consumers (probably more as best practice guidelines on how to use and interpret links for common use cases, rather then any kind of list of mandatory features).

The former would be the quickest fix. If the latter is preferred, I can try to find time to draft some replacement text.

See also:

• Unclear character set constraints and non-UTF-8 html #386
• Column names  #333
• Create sub-assertions for Keywords in the Links section  #348