WIP: multi-slot WOV arbiter with VAD gate and D0I3/S0iX support

[lgirdwood]

Multi-Slot Wake-On-Voice (WOV) Architecture & Arbitration

Overview

The Multi-Slot WOV subsystem lets a single DMIC feed up to 3 concurrent keyword detectors
running on the DSP. A single shared KPB sits between the VAD gate and the fan-out mixin,
recording a pre-roll window (6 seconds on TigerLake, 2.1 seconds on other platforms) for all
three slots in one 192 KB ring buffer — versus 576 KB in a per-slot design. When any detector
fires the wov_arbiter drains the KPB ring buffer to the host via an ALSA compress device and
pauses the other detectors. When the host closes the stream the arbiter resumes all detectors.

The ALSA compress framework (snd_compr) decouples HDA DMA from the audio sample-rate clock.
During the pre-roll burst the KPB draining EDF task fills HDA DMA fragments as fast as the bus allows
rather than at the rate-locked 16 kHz × sample-size pace of a regular PCM device.
After the pre-roll drains the live DMIC stream continues over the same compress device at realtime rate.

This design keeps host DMA live and eliminates the wakeup latency normally incurred by starting
DMA after detection.

---

System Architecture

Component Graph

graph TD
    subgraph P100["Pipeline 100 — Capture, Gating & Pre-Roll  (Core 0)"]
        DAI["DAI Copier\nHDA Analog\ndai_index=1"]
        VAD["vad_gate\n(energy estimator)"]
        KPB["kpb\n**shared** 192 KB ring buffer\n(6 s pre-roll for all slots)"]
        MIX["mixin\n(1→3 fan-out)"]
        DAI --> VAD --> KPB --> MIX
    end

    subgraph P101["Pipeline 101 — Slot 0  (Core 0)"]
        MO0["mixout 0"]
        D0["detect_test\nSlot 0\n(Male 80–170 Hz)"]
        MO0 --> D0
    end

    subgraph P102["Pipeline 102 — Slot 1  (Core 0)"]
        MO1["mixout 1"]
        D1["detect_test\nSlot 1\n(Female 175–270 Hz)"]
        MO1 --> D1
    end

    subgraph P103["Pipeline 103 — Slot 2  (Core 1)"]
        MO2["mixout 2"]
        D2["detect_test\nSlot 2\n(Child 275–500 Hz)"]
        MO2 --> D2
    end

    subgraph P104["Pipeline 104 — Arbitration & Host Capture  (Core 0)"]
        ARB["wov_arbiter\n(first-wins)"]
        HC["host-copier\ncomprC0D11\n'DMIC Multi-WOV'"]
        ARB --> HC
    end

    MIX --> MO0
    MIX --> MO1
    MIX --> MO2

    D0 --> ARB
    D1 --> ARB
    D2 --> ARB

    D0 -- "Notifier WOV_DETECT\n(slot_id=0)" --> ARB
    D1 -- "Notifier WOV_DETECT\n(slot_id=1)" --> ARB
    D2 -- "Notifier WOV_DETECT\n(slot_id=2)" --> ARB
    ARB -- "Notifier WOV_CTRL\n(PAUSE/RESUME)" --> D0
    ARB -- "Notifier WOV_CTRL\n(PAUSE/RESUME)" --> D1
    ARB -- "Notifier WOV_CTRL\n(PAUSE/RESUME)" --> D2

    %% Row layout hints (invisible ~~~ edges — dagre rank control)
    %% Row 0: P100  |  Row 1: P101 / P102 / P103  |  Row 2: P104
    KPB ~~~ MO0
    MO0 ~~~ ARB

    style VAD fill:#2d5a27,stroke:#555
    style KPB fill:#4a3a00,stroke:#555
    style ARB fill:#1c4966,stroke:#555
    style D0  fill:#663300,stroke:#555
    style D1  fill:#660033,stroke:#555
    style D2  fill:#003366,stroke:#555

Loading

Pipeline State Transitions & Re-Arm Cycle

Usage Flow (Compress Device Open Once)

The compress device is opened once at startup and kept open across all WOV cycles.
No pipeline close/reopen is needed between triggers.

Host                        Kernel / ASoC               Firmware
────                        ─────────────               ────────
open comprC0D11         ──► SET_PIPELINE_STATE(RUNNING) ──► VAD: vad_active=false (threshold default=0 → open)
                                                             KPB: KPB_STATE_BUFFERING (shared ring buffer filling, all 3 slots)
                                                             detect_tests: dp_thread running, listening

[optional] write
vad_gate_cfg_100 TLV     ──► MODULE_LARGE_CONFIG_SET  ──► threshold, onset, hangover updated

write wov_init_1NN TLV   ──► MODULE_LARGE_CONFIG_SET  ──► wov_slot_id programmed (slot 0/1/2)
amixer cset wov_mute on  ──► MODULE_LARGE_CONFIG_SET  ──► detection armed for that slot

─────────────────────── Listening (blocking) ───────────────────────────
read() blocks on comprC0D11
arbiter writes silence (zero-fill) into HDA DMA fragments at realtime rate

WOV Trigger Sequence

[voice detected]                                     ──► detect_test auto/real trigger
                        ◄── SOF_IPC4_NOTIFY_PHRASE_DETECTED  (word_id = slot_id)
                        ◄── snd_sof_compr_fragment_elapsed() wakes blocked read()
read() returns PCM ◄────────────────────────────────────────
  (burst: 6 s pre-roll arrives fast, then realtime)

DSP-Initiated Re-Arm (VAD Silence Path)

This is the no-reset re-arm: compress device stays open, pipeline stays RUNNING.

[speech ends, ambient noise below threshold for hangover_frames]
                                                     ──► vad_update_energy(): energy < threshold
                                                         for hangover_frames (default 200 × 10ms = 2s)
                                                         → notifier_event(NOTIFIER_ID_VAD_SILENCE)
                                                         → arb_on_vad_silence():
                                                             active_slot = WOV_ARB_NO_ACTIVE
                                                             broadcast WOV_ARB_CMD_RESUME
                                                         → detect_tests: cd->paused=false, cd->detected=0
                                                         → KPB resumes BUFFERING (shared ring buffer, drain complete)
                                                         → DSP clock → WOVCRO (38.4 MHz)

poll vad_gate_status_100 ────────────────────────────►  MODULE_LARGE_CONFIG_GET(param_id=2)
(TLV ioctl read)                                        returns energy + vad_active=false

Host detects silence, drains remaining compress data
read() blocks again on comprC0D11 ◄───────────────── arbiter back to silence-fill (zero frames)
─────────────────────── Re-armed, listening ─────────────────────────────────

Between-Cycle Re-Arm (Drain + VAD Gate Kcontrol)

The compress device stays RUNNING across all WOV cycles — no compress_stop() /
compress_start() is issued between triggers. The host re-arms by draining and
polling the vad_gate_status_100 kcontrol:

[keyword audio delivered; speech is fading]
                                                     ──► vad_gate energy drops below threshold
                                                         → NOTIFIER_ID_VAD_SILENCE
                                                         → arb_on_vad_silence():
                                                             active_slot = WOV_ARB_NO_ACTIVE
                                                             broadcast WOV_ARB_CMD_RESUME
                                                         → detect_tests resume DP threads

poll vad_gate_status_100 ──► MODULE_LARGE_CONFIG_GET ──► vad_active=false confirmed

[host: drain remaining compress data — read until 600 ms idle]
read() blocks on comprC0D11 ◄───────────────────────── arbiter back to silence-fill
─────────────────────── Re-armed, same compress handle ──────────────────────────

No compress_stop/start between cycles. compress_stop() is issued only when the
application exits (or the pipeline is torn down). Calling compress_stop() between
cycles is unnecessary and resets DMA state, adding latency to the next trigger.

State Transition Summary

State	Transition	Trigger	Compress device
Listening → Active	WOV_DETECT	detect_test fires	Stays OPEN
Active → Listening	VAD_SILENCE + drain + kcontrol poll	hangover expires, host drains	Stays OPEN
Active → Listening	STOP/PAUSE	pipeline teardown / app exit	Closed on compress_close()
Any → closed	compress_close()	host exits	Closed

---

SOF Notifier Inter-Module Signaling

The SOF Notifier system (src/include/sof/lib/notifier.h) is SOF's intra-DSP
publish/subscribe bus. It works on all platforms (no CONFIG_AMS required) and
is already used for KPB client events. Signals are delivered synchronously to
all registered listeners on the calling core.

Signal Catalog

Notifier ID	Direction	Payload struct	Purpose
NOTIFIER_ID_WOV_DETECT	detector → arbiter	struct wov_detect_notif { uint8_t slot_id; }	Announce keyword detection
NOTIFIER_ID_WOV_CTRL	arbiter → all detectors	struct wov_ctrl_notif { uint8_t cmd; }	Pause/resume detectors
NOTIFIER_ID_VAD_SILENCE	vad_gate → arbiter	NULL (no payload)	Silence hangover expired — re-arm for next trigger

cmd values: WOV_ARB_CMD_PAUSE, WOV_ARB_CMD_RESUME (defined in wov_arbiter.h).

NOTIFIER_ID_VAD_SILENCE is fired by vad_update_energy() when the IIR energy drops
below threshold for hangover_frames consecutive frames. The arbiter's
arb_on_vad_silence() callback handles it: resets active_slot = WOV_ARB_NO_ACTIVE
and broadcasts WOV_ARB_CMD_RESUME to all detectors so they re-arm without any
pipeline RESET or compress device close/reopen.

Full Detect-to-Drain Sequence

sequenceDiagram
    autonumber
    participant DMIC  as DMIC (HW)
    participant KPB   as KPB (shared P100)
    participant DET   as detect_test (slot N)
    participant ARB   as wov_arbiter
    participant HOST  as Host Compress (comprC0D11)
    participant OTHER as detect_test (slots ≠ N)

    Note over DMIC,OTHER: Listening state — shared KPB accumulating pre-roll for all slots

    loop Every 1 ms (LL period)
        DMIC->>KPB: DAI DMA frames
        KPB->>DET: sel_sink copy
    end

    loop Every 20 ms (DP batch)
        DET->>DET: run algorithm on 320-frame batch
    end

    Note over DET,ARB: Keyword detected on slot N

    DET->>HOST: ① IPC4 SOF_IPC4_NOTIFY_PHRASE_DETECTED\n   (word_id = slot_id)
    DET->>KPB: ② notifier_event(WOV_DETECT) [KPB already wired via kpb_client]
    DET->>ARB: ③ notifier_event(NOTIFIER_ID_WOV_DETECT, slot_id=N)

    ARB->>ARB: active_slot = N
    ARB->>OTHER: notifier_event(NOTIFIER_ID_WOV_CTRL, cmd=PAUSE)
    OTHER->>OTHER: cd->paused = true\n(stops DP batching)

    KPB->>ARB: stream pre-roll (up to 6 s) via host_sink
    ARB->>HOST: route slot-N audio to host PCM

    Note over HOST,ARB: Host finishes reading / closes PCM

    HOST->>ARB: trigger STOP (ALSA hw_free / snd_pcm_close)
    ARB->>ARB: active_slot = NO_ACTIVE
    ARB->>OTHER: notifier_event(NOTIFIER_ID_WOV_CTRL, cmd=RESUME)
    ARB->>DET:  notifier_event(NOTIFIER_ID_WOV_CTRL, cmd=RESUME)
    OTHER->>OTHER: cd->paused = false\ncd->detected = 0\nresumed listening

Loading

[lgirdwood]

@gkdeepa @naveen-manohar fyi