View models: sequential processing, discovering (and fixing) "holes" in events

[konrad-garus]

I have some questions regarding the view model "side" of CQRS in your libraries. Let's use the sample app as an example.

I'm using the term "view model" here in the CQRS sense, but it may involve "denormalizer" and "view model" (and possibly even some other libs?) since I am not yet sure what their respective responsibilities and boundaries are.

Suppose the following commands are executed in the following order:

• Item A created (1)
• Item A updated (2)
• Item B created (3)

On the view model, do the libraries support globally sequential processing of these events, so that for example (3) is guaranteed to execute after (2)?

---

Let's say there has been a network partition or the view model has been down for a while. In the meanwhile, the following events have been written:

• Item A updated (4)
• Item C created (5)

How can the view model detect this situation and catch up?

[adrai]

cqrs-eventdenormalizer and cqrs-saga both have a revisionGuard.
If your events contain a revision i.e. a feature of cqrs-domain it's possible to sequentialize all events of a given event stream (same aggregate)
So your example will not be catched by the revisionGuard feature because the events belong to different aggregates...
May I ask why you need to detect this situation? What's your use case?

[konrad-garus]

The serialization (my first question) is a nice-to-have optional feature. Sometimes it can simplify the view model code quite a bit.

What about the second question though, detecting holes within an aggregate, or unhandled events for not-yet-seen aggregates?

[adrai]

There are multiple ways to detect holes within an event stream:

1. On startup: use the function getLastEvent to ask your replay service if you missed something... (i.e. via rest)
2. at runtime: thanks to the revisionGuard listen to the onEventMissing function and ask your replay service to send you the missing events... (i.e. via rest)
3. cron job: rebuild you ReadModel (my definition for the whole collection of all your ViewModels) in a scheduled way replay. And again ask your replay service to send you all desired events... (i.e. via rest)

[konrad-garus]

When I have a hole because I missed the "create" item, and "update" is received, the onEventMissing callback is not fired. How can I catch this?

  denormalizer:revisionGuard no revision in store => retry [concatenatedId]=92cce6d1-f69b-4d40-9e1f-1bb223f8219d, [revInStore]=null, [revInEvt]=4 +12s
  denormalizer:revisionGuard no revision in store => retry [concatenatedId]=92cce6d1-f69b-4d40-9e1f-1bb223f8219d, [revInStore]=null, [revInEvt]=4 +711ms
  denormalizer:revisionGuard no revision in store => retry [concatenatedId]=92cce6d1-f69b-4d40-9e1f-1bb223f8219d, [revInStore]=null, [revInEvt]=4 +991ms
  denormalizer:revisionGuard finished loops for retry => proceed [concatenatedId]=92cce6d1-f69b-4d40-9e1f-1bb223f8219d, [revInStore]=null, [revInEvt]=4 +556ms
  denormalizer:revisionGuard first revision to store [concatenatedId]=92cce6d1-f69b-4d40-9e1f-1bb223f8219d, [revInStore]=null, [revInEvt]=4 +2ms
  denormalizer:eventDispatcher no version found, handling as version: 0 +1ms
  denormalizer:eventDispatcher no aggregate found +0ms
  denormalizer:eventDispatcher no context found +3ms
  denormalizer:eventDispatcher no version found, handling as version: 0 +3ms
  denormalizer:eventDispatcher no aggregate found +2ms
  denormalizer:eventDispatcher no context found +0ms
  denormalizer:viewBuilder [itemChanged] found viewmodel id in event +3ms
  denormalizer:viewBuilder [itemChanged], [eventId]=1ab132c2-aa20-450f-8fd7-a8cff342d669 call denormalizer function +9ms
  denormalizer:viewBuilder [itemChanged], [eventId]=1ab132c2-aa20-450f-8fd7-a8cff342d669 generate new id for notification +4ms
  denormalizer:eventDispatcher no version found, handling as version: 0 +5ms
  denormalizer:eventDispatcher no aggregate found +1ms
  denormalizer:eventDispatcher no context found +0ms
  denormalizer publish a notification +2ms
  denormalizer emit notification: { correlationId: 'msg24',
  payload: { id: '92cce6d1-f69b-4d40-9e1f-1bb223f8219d', text: 'new text' },
  collection: 'item',
  name: 'create',
  id: '4b82fb72-11c4-420e-a942-db746c8dd18c' } +1ms
  denormalizer publish an event +0ms

Secondly, since network and processes can go down any time, I am really concerned that the read model can be missing something any time for random reasons, without me knowing about it.

By the way, this is a nice property of a globally ordered event store. When the last (global) event in store is #12050 and a view model is at #12036, you know that you are missing something. And it's easy to query things without getting duplicates or missing anything, since relying on server-assigned commitStamp is usually a bad idea.

[adrai]

Oh... there is an undocumented feature...
set startRevisionNumber to 1 in the revisionGuard options this will fix your issue...