CPU side double pixel buffer

[Dy11on]

Hi, I saw that in issue #170 there was some talks about a GPU side pixel buffer and I was wondering if there was also a need for a CPU side pixel buffer? I've been working on an implementation for a CPU side buffer in my fork and could make a PR when its finished which would be soon.

[JMS55]

Hi, unless I'm misunderstanding what you mean by it, Pixels already uses a CPU side pixel buffer.
Pixels::get_frame() returns a slice of bytes that you modify, and then that gets used to render a texture to a window.

I'm assuming you mean something different, so please comment and leave some more details on the problem your code solves, thanks!

[Dy11on]

Hello! I didn't explain the initial issue well at all lol. I may also not be 100% understanding how the pixel buffer is currently working but it seems to me that theres only one buffer right now so it might be prone to a double buffer issue where the buffer is in the middle of an update but the screen renders so half the screen is displaying an old frame and half of it is displaying a new frame.

I was wondering if there needed to be a double buffer which is when you write the data to another buffer and when the update is ran a pointer swaps the buffers and the screen is only updated with a completed buffer and a case where the buffer is overwritten when it is displayed never happens. This might not be the best description, but I recently read this article on it and while I was using pixels I saw there was only one buffer and thought that this might be an issue?

https://gameprogrammingpatterns.com/double-buffer.html

[parasyte]

it seems to me that theres only one buffer right now so it might be prone to a double buffer issue where the buffer is in the middle of an update but the screen renders so half the screen is displaying an old frame and half of it is displaying a new frame.

This should not be possible, assuming you have appropriately set the PresentMode for your GPU (and graphics stack). The CPU side buffer (currently named Pixels::pixels) is the third buffer! This buffer can never be displayed on screen. It can only explicitly be uploaded to the GPU texture (the second buffer, aka back buffer) on demand. That happens in Pixels::render_with(). This function ultimately calls wgpu::Queue::submit and that's where Vsync happens.

All of this magic is built on top of what is called a swap chain in wgpu (and Vulkan): https://en.wikipedia.org/wiki/Swap_chain

You can see this happening with trace logging level enabled. E.g.:

[2021-06-05T01:13:23Z INFO  wgpu_core::device] creating swap chain SwapChainDescriptor { usage: RENDER_ATTACHMENT, format: Bgra8UnormSrgb, width: 320, height: 240, present_mode: Fifo }

This is the render target. which Pixels calls the surface texture. This is not the texture that receives the pixel buffer data from Pixels::render_with()! This render target is the primary display buffer. So you have no less than 3 buffers before your image is displayed on screen.

[2021-06-05T01:13:23Z INFO  wgpu_core::device] Created texture Valid((0, 1, Vulkan)) with TextureDescriptor { label: Some("pixels_source_texture"), size: Extent3d { width: 320, height: 240, depth_or_array_layers: 1 }, mip_level_count: 1, sample_count: 1, dimension: D2, format: Rgba8UnormSrgb, usage: COPY_DST | SAMPLED }

This is the second buffer (aka back buffer). This is the texture that receives pixel data.

The last log snippet was created by adding a bunch of trace!() calls around the main event loop handler in the minimal_winit example. It's a big ol' wall of text, so it's collapsed below and you can expand it at your leisure.

Trace log of a single frame, annotated

// First, we enter the event loop and notice that the event being handled is `Event::RedrawRequested`.
[2021-06-05T01:13:24Z TRACE minimal_winit] event_loop BEGIN

// And we get a mutable pointer to the pixel buffer (aka third buffer).
[2021-06-05T01:13:24Z TRACE minimal_winit] pixels::get_frame()

// Then we ask Pixels to render it through the GPU pipeline and put it on the window so the human can see it.
[2021-06-05T01:13:24Z TRACE minimal_winit] pixels::render()

// Now we are in wgpu land. I don't know the full inner workings of this crate, but I can take an educated guess...
// Also I did not add any trace!() logs to the Pixels crate while capturing this, so you can't always tell when the code is entering and leaving wgpu.
// PendingTransition has to do with GPU memory barriers (synchronization primitives). I can't personally go into any more detail on this.
[2021-06-05T01:13:24Z TRACE wgpu_core::track] 	texture -> PendingTransition { id: Valid((0, 1, Vulkan)), selector: TextureSelector { levels: 0..1, layers: 0..1 }, usage: COPY_SRC | COPY_DST | SAMPLED | ATTACHMENT_READ | ATTACHMENT_WRITE | STORAGE_LOAD | STORAGE_STORE | READ_ALL | WRITE_ALL | ORDERED | UNINITIALIZED..COPY_DST }

// Pixels has called Device::create_command_encoder()
[2021-06-05T01:13:24Z TRACE wgpu_core::command::render] Encoding render pass begin in command buffer (0, 1, Vulkan)
[2021-06-05T01:13:24Z TRACE wgpu_core::command::bind] 	Binding [0] = group Valid((0, 1, Vulkan))
[2021-06-05T01:13:24Z TRACE wgpu_core::command::render] Merging (0, 1, Vulkan) with the render pass

// The pipeline has a single texture (second buffer) and texture sampler. Here it's doing all the drawing...
[2021-06-05T01:13:24Z TRACE wgpu_core::command] Command buffer (0, 1, Vulkan) TrackerSet {
        buffers: {
            (
                0,
                1,
            ): Unit {
                first: None,
                last: UNIFORM,
            },
        },
        textures: {
            (
                0,
                1,
            ): TextureState {
                mips: [
                    RangedStates {
                        ranges: [
                            (
                                0..1,
                                Unit {
                                    first: None,
                                    last: SAMPLED,
                                },
                            ),
                        ],
                    },
                ],
                full: false,
            },
        },
        views: {
            (
                0,
                1,
            ): PhantomData,
        },
        bind_groups: {
            (
                0,
                1,
            ): PhantomData,
        },
        samplers: {
            (
                0,
                1,
            ): PhantomData,
        },
        compute_pipes: {},
        render_pipes: {
            (
                0,
                1,
            ): PhantomData,
        },
        bundles: {},
        query_sets: {},
    }
[2021-06-05T01:13:24Z TRACE wgpu_core::device::queue] Stitching command buffer (0, 1, Vulkan) before submission
[2021-06-05T01:13:24Z TRACE wgpu_core::track] 	buffer -> PendingTransition { id: Valid((0, 1, Vulkan)), selector: (), usage: COPY_DST..UNIFORM }
[2021-06-05T01:13:24Z TRACE wgpu_core::track] 	texture -> PendingTransition { id: Valid((0, 1, Vulkan)), selector: TextureSelector { levels: 0..1, layers: 0..1 }, usage: COPY_DST..SAMPLED }
[2021-06-05T01:13:24Z TRACE wgpu_core::device::queue] Device after submission 1: TrackerSet {
        buffers: {
            (
                0,
                1,
            ): Unit {
                first: Some(
                    COPY_DST,
                ),
                last: UNIFORM,
            },
        },
        textures: {
            (
                0,
                1,
            ): TextureState {
                mips: [
                    RangedStates {
                        ranges: [
                            (
                                0..1,
                                Unit {
                                    first: Some(
                                        COPY_SRC | COPY_DST | SAMPLED | ATTACHMENT_READ | ATTACHMENT_WRITE | STORAGE_LOAD | STORAGE_STORE | READ_ALL | WRITE_ALL | ORDERED | UNINITIALIZED,
                                    ),
                                    last: SAMPLED,
                                },
                            ),
                        ],
                    },
                ],
                full: true,
            },
        },
        views: {
            (
                0,
                1,
            ): PhantomData,
        },
        bind_groups: {
            (
                0,
                1,
            ): PhantomData,
        },
        samplers: {
            (
                0,
                1,
            ): PhantomData,
        },
        compute_pipes: {},
        render_pipes: {
            (
                0,
                1,
            ): PhantomData,
        },
        bundles: {},
        query_sets: {},
    }

// This is where the frame is displayed on the window. This happens implicitly when the swap chain texture (aka render target) is dropped. And that happens _synchronously_ at the end of `Pixels::render_with()`.
[2021-06-05T01:13:24Z DEBUG wgpu_core::swap_chain] Presented. End of Frame

// Which means once the code gets back to the application, there is nothing left for the GPU to do, and the user's code cannot do anything to stomp on an in-flight render pass.
[2021-06-05T01:13:24Z TRACE minimal_winit] pixels::render() DONE
[2021-06-05T01:13:24Z TRACE minimal_winit] event_loop DONE

And that is why, this scenario should not be possible to experience in practice with Pixels. However, if you have a method to reproduce this kind of screen tearing that the linked article describes, I would love to hear about it so I can investigate further. As it stands, I'm convinced this is not an issue.

[Dy11on]

Ohh okay thanks for taking the time to explain all of this to me it was really interesting. I should have read more of the wgpu documentation before raising the issue sorry about that 😄. I'll close out this issue now

[parasyte]

No worries. This can serve as more documentation of the Pixels internals. There is another wall-of-text with more information here: #136 (comment)