Centrifugo v4 roadmap

[FZambia]

Soon after Centrifugo v3 we are starting to think about v4 release. Let's look at v4 plans.

New client protocol adoption

More human-readable JSON, one-shot encode/decode (more performance), simpler introspection if required. The work already started in #499. Note that possibility to try (switch to) client protocol v2 will appear in Centrifugo v3 (possibly in v3.2.0), in v4 the plan is to make new protocol version default.

Standartization of client SDK API

All client SDKs will work according to a common spec. There is a prototype of if in Centrifugo v4 WIP docs.

Application-level pings

Protocol V2 may have a new approach to client-server pings. Pings will be sent on application protocol level, clients won't send pings to server at all – the will be able to rely on server heartbeats to understand that connection is still active. This may help to reduce CPU usage since less ping/pong frames will travel around. Also, this will unify format of pings throughout our unidirectional transports (uni Websocket, Eventsource (SSE), uni http streaming).

Disconnects: expose code to client disconnect handler and remove JSON from reason

This will allow handling disconnect codes in a custom way. Will add some performance, readability, WebTransport interop. This is also part of client protocol v2 and details can be found in #499.

Better channel security model

• Namespaces should be protected by default (introduce public boolean channel option and remove protected option - namespaces will inherit protected option behavior by default). This should resolve some confusion about current channel behavior, for example when channels sent in JWT users often think that nobody else will be able to subscribe on them. While this is not true till protected option is not enabled in namespace.
• Strict channel symbol validation (allow only ASCII, it was always documented but not really enforced at the moment)

Removing usage of $ prefix to mark private channel

See #507 for motivation and details.

Faster Redis engine

Redis engine based on https://github.com/go-redis/redis should provide more perf and reduce dependencies. Additional details: centrifugal/centrifuge#210. This is not really required to be part of Centrifugo v4 - this may come in v3 or soon after v4 release. But I think it's important to mention. The work is pretty big and not really trivial but may be worth it.

When Centrifugo v4 will be released?

For Centrifugo v3 we had strict deadline right from the announce post. For v4 there is no deadline defined yet. This post will be updated as soon as things will evolve.

If anyone is interested to discuss these changes and maybe adjust them somehow at an early stage – feel free to reach out.

[FZambia]

One more possible addition in terms of Protocol V2 is Centrifugo own bidirectional emulation layer. It may allow removing SockJS in most cases I believe. And get rid of sticky sessions for fallback transports. Client API may look like this:

const transports = [
    {
        transport: 'websocket',
        endpoint: 'ws://example.com/connection/websocket'
    },
    {
        transport: 'http_stream',
        endpoint: 'http://example.com/connection/http_stream'
    },
    {
        transport: 'sse',
        endpoint: 'http://example.com/connection/sse'
    },
];
const centrifuge = new Centrifuge(transports, {protocolVersion: 'v2'});

This is mostly useful for browser only. For mobile clients we will only support our main WebSocket transport.

Centrifuge-js may try transports in order on the first handshake. So most users will be connected to WebSocket and some to HTTP-based fallback transports which work over built-in emulation layer (similar way to SockJS but without SockJS involved at all, and no real need to use sticky sessions in cluster mode).

[matpuk]

Hello!

It's me again. Here are my ideas:

1. Multiple JWKS endpoints support. Currently, we are forced to use single RSA cert to issue JWT tokens from different keycloak realms because we can't configure centrifugo to select proper JWKS endpoint based on JWT "iss" claim. It would be nice to have a possbility to use JWT 'iss' field as an actual JWKS endpoint with possibility to define a whitelist of allowed 'iss' URLs prefixes.
2. Provide complete JWT token content in the 'meta' field when calling any proxy backend. Including connection proxy. Right now it's impossible to identify client in the connection proxy by provided JWT token. And centrifugo forces us to duplicate data in JWT token just to make 'meta' property passed to other proxies.

[FZambia]

@matpuk thanks for suggestions.

Could you elaborate more on both ideas – extend them with examples and desired API/configuration changes? I think both can be implemented even as part of Centrifugo v3.

I think I mostly understood 1 – but more real-life examples with real endpoints/iss will help.
For 2 – if you are using JWT then connection proxy won't be called now. So I am a bit confused in understanding what you suggested. So again - examples will help to dive into this.

[matpuk]

Hi, @FZambia.

Let's start with point 2. Centrifugo v3 has cool feature - possibility to attach private metadata to a connection from connection proxy. But in case of JWT, the proxy is not called. The only way to use 'meta' property is to put it inside JWT token, so Centrifugo will pass it to other proxies. And here go problems. For example, I have 'tenantID' property in my JWT token and want to know it in subscription proxy. For this I'm forced to duplicate this property in my JWT token:

{
  "sub": "1234567890",
  "name": "John Doe",
  "iat": 1516239022,
  "tenantID": "432",
  "meta": { "tenantID": "432" }
}

And for every additional JWT token data I want to have access in my proxies, I need to duplicate it inside 'meta'.

There are two possible solutions for this problem:

1. Enable connection proxy calling when using JWT tokens and provide complete token body to it. This way backend can populate connection meta with any data it wants.
2. Provide complete JWT token body for every proxy call.

[FZambia]

@matpuk I see. What I think:

Enable connection proxy calling when using JWT tokens and provide complete token body to it. This way backend can populate connection meta with any data it wants.

If you can afford working this way then theoretically you can set token to connection custom data (sent from client in first request and proxied to the backend). Then you need to validate JWT yourself on the backend side in proxy request handler. And after authenticating attach required metadata to the connection.

Though I understand this can be not very handy especially since you are already relying on Centrifugo to validate tokens, using JWKS, etc.

Provide complete JWT token body for every proxy call.

In general, seems reasonable to have (especially since we already support proxying custom headers) – just need to think on accurate implementation. Can't say any ETAs for this at the moment - could you plz open a separate issue?

[matpuk]

@FZambia:

Enable connection proxy calling when using JWT tokens and provide complete token body to it. This way backend can populate connection meta with any data it wants.

If you can afford working this way then theoretically you can set token to connection custom data (sent from client in first request and proxied to the backend). Then you need to validate JWT yourself on the backend side in proxy request handler. And after authenticating attach required metadata to the connection.

I'm thinking to go this way, but there is a drawback - I will be forced to implement connection attempts throttling and invalid connections discarding in my backend's python code instead of leaving this to fast Go-implemented Centrifugo 😞

Though I understand this can be not very handy especially since you are already relying on Centrifugo to validate tokens, using JWKS, etc.

Provide complete JWT token body for every proxy call.

In general, seems reasonable to have (especially since we already support proxying custom headers) – just need to think on accurate implementation. Can't say any ETAs for this at the moment - could you plz open a separate issue?

Done.

[FZambia]

Some updates about v4 progress. Given the huge amount of work we decided to choose a bit simpler plan:

• Centrifugo v4 will by default work over client protocol v2
• It will be possible to turn on client protocol v1 as a default on a server side - this way all current client SDKs will be able to connect to Centrifugo v4
• We will release a new set of client SDKs which only work using client protocol v2 semantics.
• It's possible to connect to Centrifugo and explicitly select protocol v2 using url param cf_protocol_version=v2
• So it will be still possible to migrate smoothly but we won't support both protocols in new client SDK versions

I started a doc about upcoming client API standartization (which should be part of Centrifugo v4). It's here:

Client API v2 description - this is a page on v4 WIP docs.

Expect changes in it since I'd like to validate this with interested parties, and feel free to ask if sth needs clarification. At this point we have branches in all client SDKs which are close to this spec.

[FZambia]

Option use_unlimited_history_by_default which was used to help migrating from Centrifugo v2 to Centrifugo v3 (see description in v3 migration guide) will be removed in v4.

[FZambia]

Updated issue description to mention work on #507 in Centrifugo v4

[FZambia]

Opened a PR #519 with changes for Centrifugo v4

[FZambia]

For those interested in trying out new SDK API - it's already possible to try using SDK from a branch where we support client protocol v2 and adding URL parameter ?cf_protocol_version=v2 when connecting to Centrifugo v3.2.2. There still may be some tweaks before v4 release. Hope Centrifugo v4 beta will be available in the beginning of July.

PRs in SDK from branches with new protocol and API support:

• centrifuge-js - Centrifuge-js v3: Typescript, client protocol v2, new API iteration, WebSocket emulation centrifuge-js#186
• centrifuge-dart - Support client protocol v2 and new API iteration  centrifuge-dart#62
• centrifuge-java - Support client protocol v2 centrifuge-java#32
• centrifuge-swift - Client protocol v2 centrifuge-swift#58
• centrifuge-go - Support client protocol v2 and new API iteration centrifuge-go#59

Don't hesitate to ask any question in our community rooms.

[FZambia]

Centrifugo v4.0.0 beta.1 just released.

Also, centrifuge-js v3.0.0-beta.0 was released yesterday.

Obviously, these releases are not for production usage, and some changes can still happen. Please reach out in community chat if you want to try beta release but came across questions/issues - highly appreciate the feedback.

[FZambia]

Centrifugo v4 released.