No way to wildcard subscriptions or subscribe to subtrees

[bdamm]

Suppose there is a topic structure such as /${tenant-id}/${device-mac}/${event-id}

Suppose a subscriber wants to receive all events with a specific ID. There is no way to do that in this protocol.

Suppose a subscriber wants to receive all events from a specific device-mac.
Suppose a subscriber wants to receive all events from a specific tenant-id.

How could this be efficiently realized at all, even with complete freedom to structure the topic tree in any way? With the current protocol I don't see how to do it. Subscribers would need to subscribe to millions of potential topics, even though there may not be millions of actual topics. Even worse, subscribers would need intimate knowledge of all possible topics (all possible IDs and MACs) which may not even be possible.

[mattaylor]

This proposal could be extended to support topic wildcard subscriptions following MQTT conventions (+ for single path segments and # for multiple path segments).
When a wildcard is included in the subscription, the broker is responsible for notifying all subscribers when any resources change on any matching topics.
When a notify response is then delivered to the subscriber on a wildcard topic, the broker MUST include the URI-PATH option in the message which indicates the full topic path of the resource being sent.

E.G:

    Client1   Client2                                          Broker
       |          |                   Subscribe                    |
       |          | --- GET /ps/+ Obs:0 Token:XX ----------------> | 
       |          | <--- ACK ------------------------------------- | 
       |          |                                                |
       |          |                    Publish                     |
       | ---------|---- PUT /ps/topic1 "1033.3"  ----------------> |
       |<---------|---- 2.01 Location:/ps/topic1 ----------------- |
       |          |                    Notify                      |
       |          | <-- 2.05 "1033.3" Obs:11 Uri-Path:/ps/topic1 - |

[jaimejim]

Dear @bdamm and @mattaylor, sorry for the late response to this.

As I mentioned in other issues, the draft has now undergone many changes.
You can find the editor's version of the draft and an early implementation.

In relation to # and + wildcards, the currnt draft does not define them. The current spec does not mandate for the topics to be hierarchically organized within the same broker, the path is a URL to a resource, unlike the MQTT topic structure.

The issue is at least twofold:

1. Finding all topic_data URLs

I htink this part is solved, there are many ways to find the topic resources and if your broker had a specific hierarchy you could design wildcards. For example a collection with some topics under it on a predefined path like /ps and under it the topic data under another predefined path like /ps/data could be used to set wildcard subscription to all topic_data resources. Another mechanism would be to use FETCH with "resource_type" : "core.ps.data" , you would get a series of topic_data URIs.

As @mattaylor points out, you can also subscribe to the collection resource itself /ps with rt=core.ps.data and obs to get notified of new topic_data resources.

The usual discovery mechanisms apply https://core-wg.github.io/coap-pubsub/ps-rev/draft-ietf-core-coap-pubsub.html#name-discovery

2. Subscribing to them in one single message

Assuming that under the topic collectiong there are 100 resources like ps/data/ (they can also be labeled with rt=core.ps.data). How does a client observe all of them with one GET message?

This part that is not there yet. You could craft specific FETCH + obs messages that contain topic data URIs and subscribe to them in one go.

@cabo for example mentions these two potential payloads:
{ match: “/my/path/*” } or { prefix: “/my/path/“ }