Seed Peer cached pieces are not re-announced after Scheduler restart

[lsantos-voleon]

Hi Dragonfly team,

We are running Dragonfly v2.4.4-rc.3.

We observed a behavior where Seed Peers can have image layer pieces in local cache, but after Scheduler restart the Scheduler no longer knows that those Seed Peers have those pieces. When a client requests the same image layer, the Scheduler triggers a Seed Peer download task, but if the Seed Peer already has all pieces locally, it completes from local cache and does not re-announce piece ownership back to the Scheduler.

As a result, the Scheduler still has no candidate parents for the requesting client and eventually sends it to BACK_TO_SOURCE.

Source references

In the v2 Seed Peer trigger path, TriggerDownloadTask calls the Seed Peer's DownloadTask RPC and drains the stream, but does not parse returned piece metadata into Scheduler task state:

https://github.com/dragonflyoss/dragonfly/blob/v2.4.4-rc.3/scheduler/resource/standard/seed_peer.go#L120-L154

On the client side, piece ownership is announced through DownloadPieceFinishedRequest / DownloadPieceBackToSourceFinishedRequest:

https://github.com/dragonflyoss/client/blob/main/dragonfly-client/src/resource/task.rs#L1192-L1203
https://github.com/dragonflyoss/client/blob/main/dragonfly-client/src/resource/task.rs#L1489-L1502

But if all interested pieces are already available locally, the client returns before downloading through the Scheduler path:

https://github.com/dragonflyoss/client/blob/main/dragonfly-client/src/resource/task.rs#L453-L457

Reproduction / observed behavior

1. Preheat an image to all Seed Peers.
2. Confirm preheat completes.
3. Restart all Schedulers.
4. Pull the same image from a normal client.
5. The Seed Peer still has the content in local cache, but the fresh Scheduler does not know that.
6. The client falls back to source because no candidate parents are available.

We observed this in metrics as client-side BACK_TO_SOURCE traffic after Scheduler restart, even though the image had been preheated to Seed Peers before the restart.

When we cleared Seed Peer cache, restarted Schedulers and Seed Peers, then preheated again after Scheduler restart, the same image pull came fully from P2P. This suggests the Scheduler only knows about pieces downloaded/announced during its current lifetime.

Questions

1. Is this expected behavior for the v2 Scheduler protocol?
2. Is there a supported way to persist or reconstruct the standard task peer-piece mapping after Scheduler restart?
3. Is there a recommended way for Seed Peers to re-announce cached pieces to a fresh Scheduler?
4. What would be the downside of having a Seed Peer announce cached pieces when the Scheduler triggers DownloadTask and the Seed Peer finds the requested task already complete locally?
5. If this is currently expected behavior, should clients always keep back-to-source enabled to tolerate Scheduler restarts?

[BraveY]

This phenomenon does exist and has been successfully reproduced. How was it considered during the initial design? Why wasn’t peer-piece mapping persistence taken into account, such as storing it in Redis? @gaius-qi