Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mpp encoding YUYV to H264 / H265 (RK3568 / RK3588) #253

Open
avafinger opened this issue Oct 9, 2021 · 49 comments
Open

mpp encoding YUYV to H264 / H265 (RK3568 / RK3588) #253

avafinger opened this issue Oct 9, 2021 · 49 comments

Comments

@avafinger
Copy link

avafinger commented Oct 9, 2021 •
edited
Loading

Encoding a camera output /dev/video0 (YUYV) to H264 or H265 get lots of horizontal trashed lines as in the frame sample.
2021-10-09-175443_1282x750_scrot

How can i get rid of the noise?

testing with:
mpi_enc_test -i /dev/video0 -f 8 -w 1280 -h 720 -o out_1280x720.h264 -t 7 -n 150 -d 1

Info:

mpp[798]: mpp_info_test: normal version log:
mpp[798]: mpp_info: mpp version: 6dadc7e1 author: Ding Wei      2021-09-02 [hal_h265d]: Use MPP_OK default, avoid the api->control is NULL
mpp[798]: mpp_info_test: history version log:
mpp[798]: mpp_info: mpp version history 10:
mpp[798]: mpp_info: 6dadc7e1 author: Ding Wei      2021-09-02 [hal_h265d]: Use MPP_OK default, avoid the api->control is NULL  (HEAD -> develop, origin/develop, origin/HEAD)
mpp[798]: mpp_info: 9036159c author: Herman Chen   2021-08-30 [jpegd_vdpu2]: Fix error on first jpegd vdpu2 task
mpp[798]: mpp_info: 264aca48 author: Herman Chen   2021-08-30 [utils]: Remove extra NULL option print
mpp[798]: mpp_info: 1263919d author: Herman Chen   2021-08-30 [vepu]: Fix error on encoding yuyv format
mpp[798]: mpp_info: b394dcea author: Herman Chen   2021-08-26 [jpege]: Set default rc_mode to VBR
mpp[798]: mpp_info: 0d11f480 author: Yandong Lin   2021-08-26 [h264d_sei]: fix parse sei crash with invalid sps id
mpp[798]: mpp_info: 4f06d9dc author: Yandong Lin   2021-08-26 [mpp]: support dump pkt data for dec
mpp[798]: mpp_info: 196b918d author: sayon.chen    2021-08-25 [h265e_dpb]: Mark unused ref which no show in dpb
mpp[798]: mpp_info: fb3e07d9 author: sayon.chen    2021-08-25 [rc_v2]: Reorganize ins_ratio calc code
mpp[798]: mpp_info: 73400764 author: Yandong Lin   2021-08-25 [mpp_time]: fix the interval time calculate wrong
@avafinger
Copy link
Author

I found out if i encode only one frame I don't get broken horizontal lines.

@avafinger
Copy link
Author

Some additional info:

  • the first 4 frames are clean
  • from frame 5 onwards, frames are corrupt (H264,H265,VP8)

frame 1
frame-1

frame 4
frame-4

frame 5
frame-5

@HermanChen
Copy link
Collaborator

Which chip are you using?

@avafinger
Copy link
Author

avafinger commented Oct 12, 2021 •
edited
Loading

Chip: RK3568
Kernel: 4.19
Camera: h264 usb sensor AR0330

PS: Decoding H264 to I420 works, encoding has the limitation, works always for the first 4 frames.

@avafinger
Copy link
Author

@HermanChen
I tried to reproduce the error with the same setup on RK3399 but the kernel crashed badly while encoding and the board rebooted. Tested with various kernels on rk3399, mpp crashed the kernel.

On rk3568 encoding and decoding is really stable and a joy. I don't have a camera with NV12 to test and see what would happen during encoding.

Any idea?

@avafinger
Copy link
Author

@HermanChen
Just in time, I rebuilt the same kernel on rk3399 and used the same setup, it works for RK3399!

@avafinger
Copy link
Author

I tested the kernel 4.19.161 (rockchip developers) on rk3399 which detects my ov4689 and throws a lot of kernel messages and put the kernel in a halt state

sensor ov4689 with NV12

sudo mpi_enc_test -i /dev/video0 -f 0 -w 1920 -h 1080 -o out_200frames_1920x1080.h264 -t 7 -n 200
[sudo] password for ubuntu: 
mpp[747]: mpi_enc_utils: cmd parse result:
mpp[747]: mpi_enc_utils: input  file name: /dev/video0
mpp[747]: mpi_enc_utils: output file name: out_200frames_1920x1080.h264
mpp[747]: mpi_enc_utils: width      : 1920
mpp[747]: mpi_enc_utils: height     : 1080
mpp[747]: mpi_enc_utils: format     : 0
mpp[747]: mpi_enc_utils: type       : 7
mpp[747]: mpi_enc_test: mpi_enc_test start
mpp[747]: mpi_enc_test: open camera device
mpp[747]: camera_source: get width 1920 height 1080
mpp[747]: camera_source: get dma buf(0)-fd: 5
mpp[747]: mpp_rt: NOT found ion allocator
mpp[747]: mpp_rt: found drm allocator
mpp[747]: camera_source: get dma buf(1)-fd: 8
mpp[747]: camera_source: get dma buf(2)-fd: 9
mpp[747]: camera_source: get dma buf(3)-fd: 10
mpp[747]: mpi_enc_test: new framecap ok
mpp[747]: mpp_info: mpp version: 6cc2ef5f author: Herman Chen   2021-09-17 [mpp_list]: Add list_mode and list_move_tail
mpp[747]: mpi_enc_test: 0x55aff4f060 mpi_enc_test encoder test start w 1920 h 1080 type 7
mpp[747]: mpp_enc: MPP_ENC_SET_RC_CFG bps 7776000 [486000 : 8262000] fps [30:30] gop 60
mpp[747]: h264e_api_v2: MPP_ENC_SET_PREP_CFG w:h [1920:1080] stride [1920:1080]
mpp[747]: mpp_enc: send header for set cfg change input/format 
mpp[747]: mpp_enc: mode vbr bps [486000:7776000:8262000] fps fix [30/1] -> fix [30/1] gop i [60] v [0]
mpp[747]: mpp_serivce: mpp_service_cmd_poll ioctl MPP_IOC_CFG_V1 failed ret -1 errno 110 Connection timed out
mpp[747]: hal_h264e_vepu2_v2: hal_h264e_vepu2_wait_v2 poll cmd failed 110
mpp[747]: mpi_enc_test: 0x55aff4f060 encoded frame 0    size 200    
mpp[747]: mpp_serivce: mpp_service_cmd_poll ioctl MPP_IOC_CFG_V1 failed ret -1 errno 110 Connection timed out
mpp[747]: hal_h264e_vepu2_v2: hal_h264e_vepu2_wait_v2 poll cmd failed 110
mpp[747]: mpi_enc_test: 0x55aff4f060 encoded frame 1    size 0      
mpp[747]: mpp_serivce: mpp_service_cmd_poll ioctl MPP_IOC_CFG_V1 failed ret -1 errno 110 Connection timed out
mpp[747]: hal_h264e_vepu2_v2: hal_h264e_vepu2_wait_v2 poll cmd failed 110
mpp[747]: mpi_enc_test: 0x55aff4f060 encoded frame 2    size 0      
mpp[747]: mpp_serivce: mpp_service_cmd_poll ioctl MPP_IOC_CFG_V1 failed ret -1 errno 110 Connection timed out
mpp[747]: hal_h264e_vepu2_v2: hal_h264e_vepu2_wait_v2 poll cmd failed 110
mpp[747]: mpi_enc_test: 0x55aff4f060 encoded frame 3    size 0      
mpp[747]: mpp_serivce: mpp_service_cmd_poll ioctl MPP_IOC_CFG_V1 failed ret -1 errno 110 Connection timed out
mpp[747]: hal_h264e_vepu2_v2: hal_h264e_vepu2_wait_v2 poll cmd failed 110
mpp[747]: mpi_enc_test: 0x55aff4f060 encoded frame 4    size 0      
mpp[747]: mpp_serivce: mpp_service_cmd_poll ioctl MPP_IOC_CFG_V1 failed ret -1 errno 110 Connection timed out
mpp[747]: hal_h264e_vepu2_v2: hal_h264e_vepu2_wait_v2 poll cmd failed 110
mpp[747]: mpi_enc_test: 0x55aff4f060 encoded frame 5    size 0

@avafinger
Copy link
Author

avafinger commented Oct 13, 2021 •
edited
Loading

@HermanChen

I would like to add this info to not confuse you (there are two different issues):

  1. Grabbing YUV (200 frames) and encode it to H264 / H265 works only for the first 4 frames, the remaining frames are corrupt with that horizontal lines, this is on RK3568, camera is USB, YUV,H264 and MJPEG
    sudo mpi_enc_test -i /dev/video0 -f 8 -w 1280 -h 720 -o out_200F_1280x720.h264 -t 7 -n 200
  2. Grabbing YUV (200 frames) and encode it to H264 / H265 works fine for the 200 frames on RK3399 (same kernel as 3568), camera is USB, YUV,H264 and MJPEG. I have here two ov4689 and one USB attached. /dev/video0 is ov4689 and /dev/video10 is USB.
    sudo mpi_enc_test -i /dev/video10 -f 8 -w 1280 -h 720 -o out_200F_1280x720.h264 -t 7 -n 200
  3. Grabbing NV12 (200 frames) and encode it to H264 / H265 breaks the kernel on RK3399, this is on RK3399 with the same kernel used in RK3568, camera is MIPI CSI ov4689
sudo mpi_enc_test -i /dev/video0 -f 0 -w 1920 -h 1080 -o out_1frames_1920x1080.h264 -t 7 -n 2
[sudo] password for ubuntu: 
mpp[642]: mpi_enc_utils: cmd parse result:
mpp[642]: mpi_enc_utils: input  file name: /dev/video0
mpp[642]: mpi_enc_utils: output file name: out_1frames_1920x1080.h264
mpp[642]: mpi_enc_utils: width      : 1920
mpp[642]: mpi_enc_utils: height     : 1080
mpp[642]: mpi_enc_utils: format     : 0
mpp[642]: mpi_enc_utils: type       : 7
mpp[642]: mpi_enc_test: mpi_enc_test start
mpp[642]: mpi_enc_test: open camera device
mpp[642]: camera_source: get width 1920 height 1080
mpp[642]: camera_source: get dma buf(0)-fd: 5
mpp[642]: mpp_rt: NOT found ion allocator
mpp[642]: mpp_rt: found drm allocator
mpp[642]: camera_source: get dma buf(1)-fd: 8
mpp[642]: camera_source: get dma buf(2)-fd: 9
mpp[642]: camera_source: get dma buf(3)-fd: 10
mpp[642]: mpi_enc_test: new framecap ok
mpp[642]: mpp_info: mpp version: 6cc2ef5f author: Herman Chen   2021-09-17 [mpp_list]: Add list_mode and list_move_tail
mpp[642]: mpi_enc_test: 0x558eaef060 mpi_enc_test encoder test start w 1920 h 1080 type 7
mpp[642]: mpp_enc: MPP_ENC_SET_RC_CFG bps 7776000 [486000 : 8262000] fps [30:30] gop 60
mpp[642]: h264e_api_v2: MPP_ENC_SET_PREP_CFG w:h [1920:1080] stride [1920:1080]
mpp[642]: mpp_enc: send header for set cfg change input/format 
mpp[642]: mpp_enc: mode vbr bps [486000:7776000:8262000] fps fix [30/1] -> fix [30/1] gop i [60] v [0]
mpp[642]: mpp_serivce: mpp_service_cmd_poll ioctl MPP_IOC_CFG_V1 failed ret -1 errno 110 Connection timed out
mpp[642]: hal_h264e_vepu2_v2: hal_h264e_vepu2_wait_v2 poll cmd failed 110
mpp[642]: mpi_enc_test: 0x558eaef060 encoded frame 0    size 200    
mpp[642]: mpp_serivce: mpp_service_cmd_poll ioctl MPP_IOC_CFG_V1 failed ret -1 errno 110 Connection timed out
mpp[642]: hal_h264e_vepu2_v2: hal_h264e_vepu2_wait_v2 poll cmd failed 110
mpp[642]: mpi_enc_test: 0x558eaef060 encoded frame 1    size 0      
mpp[642]: mpi_enc_test: 0x558eaef060 encode max 2 frames
mpp[642]: mpi_enc_test: 0x558eaef060 mpi_enc_test success total frame 2 bps 24000
mpp[642]: mpp_buffer: ~MppBufferService cleaning misc group

Kernel halt with:
kernel

Obs: kernel used is rock solid, i can compile kernels on the board and heavy stuff, not a single problem so far.

@avafinger
Copy link
Author

avafinger commented Oct 13, 2021 •
edited
Loading

@HermanChen

Please, note that the size 1920x1080 is the reason for the kernel crash!

Works:
sudo mpi_enc_test -i /dev/video0 -f 0 -w 1280 -h 720 -o out_250.h264 -t 7 -n 250 -d 1

Kernel crash:
sudo mpi_enc_test -i /dev/video0 -f 0 -w 1920 -h 1080 -o out_250.h264 -t 7 -n 250 -d 1

@HermanChen
Copy link
Collaborator

We can not help much on the camera or sensor part...
The camera_source in mpp is just a demo and may not work for all the platform.
I think it is generally a sdk / environment issue.

@avafinger
Copy link
Author

avafinger commented Oct 14, 2021 •
edited
Loading

@HermanChen
Apart from the camera/sensor code that is possibly causing the issues, can you give some info about the:

mpp_serivce: mpp_service_cmd_poll ioctl MPP_IOC_CFG_V1 failed ret -1 errno 110 Connection timed out

is there any restriction to use the encoder that would limit its use?

@avafinger
Copy link
Author

Closing it for now, until the camera code is fixed somehow.

Please, if you can comment about the mpp_service_cmd_poll ioctl MPP_IOC_CFG_V1 error I would appreciate it.

@avafinger
Copy link
Author

@HermanChen
I revised the camera code and there seems nothing wrong. Despite the forgotten mapping:

mpp/utils/camera_source.c

Line 217 in 6cc2ef5

ctx->fbuf[i].start = mmap(NULL, buf.length,

The only sample available with DRM allocator:

mpp/utils/camera_source.c

Line 254 in 6cc2ef5

info.type = MPP_BUFFER_TYPE_EXT_DMA;

is with this cam source, and this takes down the kernel when buffer size is 1920x1080. I synced my kernel with @JeffyCN s kernel which seems to have mpp driver updated. I increased CMA and dma buf to 4M, just in case.

The memory size allocated by mpp is 1920x1080*2 , what about the alignment?
Could you review the mpp_buffer_import for this case?

@HermanChen
Copy link
Collaborator

the buffer size is for decoder or encoder?

@avafinger
Copy link
Author

For the encoder. Decoder is working 100% for any size.
If I remember, the mpp driver displayed some info about the buffer size is smaller than the frma size, but it was not a warning, just info.

@avafinger
Copy link
Author

avafinger commented Oct 26, 2021 •
edited
Loading

@HermanChen
I was able to reproduce the message with Debian.
mpp[7534]: hal_h264e_vepu_v2: warnning: input buffer size 0xa2800 is smaller than required size 0x1fe000
mpp[7534]: hal_h264e_vepu_v2: warnning: input buffer size 0xa2800 is smaller than cb requirement 0x1fa400 + 0x7f800

Why is this buffer (size= 0xa2800) small? Who is setting this value? It comes from:
size_t size = mpp_buffer_get_size(buffer);

0x1fe00 = 1920 x 1088

mpi_enc_test -i /dev/video10 -f 0 -w 1920 -h 1080 -o out_200frames_1920x1080.h264 -t 7 -n 10 -d 1
mpp[7534]: mpi_enc_utils: skip invalid opt d
mpp[7534]: mpi_enc_utils: cmd parse result:
mpp[7534]: mpi_enc_utils: input  file name: /dev/video10
mpp[7534]: mpi_enc_utils: output file name: out_200frames_1920x1080.h264
mpp[7534]: mpi_enc_utils: width      : 1920
mpp[7534]: mpi_enc_utils: height     : 1080
mpp[7534]: mpi_enc_utils: format     : 0
mpp[7534]: mpi_enc_utils: type       : 7
mpp[7534]: mpi_enc_test: mpi_enc_test start
mpp[7534]: mpi_enc_test: open camera device
mpp[7534]: camera_source: get width 1920 height 1080
mpp[7534]: camera_source: get dma buf(0)-fd: 5
mpp[7534]: mpp_rt: NOT found ion allocator
mpp[7534]: mpp_rt: found drm allocator
mpp[7534]: camera_source: get dma buf(1)-fd: 8
mpp[7534]: camera_source: get dma buf(2)-fd: 9
mpp[7534]: camera_source: get dma buf(3)-fd: 10
mpp[7534]: mpi_enc_test: new framecap ok
mpp[7534]: mpp_info: mpp version: 6cc2ef5f author: Herman Chen   2021-09-17 [mpp_list]: Add list_mode and list_move_tail
mpp[7534]: mpi_enc_test: 0x559af530a0 mpi_enc_test encoder test start w 1920 h 1080 type 7
mpp[7534]: mpp_enc: MPP_ENC_SET_RC_CFG bps 7776000 [486000 : 8262000] fps [30:30] gop 60
mpp[7534]: h264e_api_v2: MPP_ENC_SET_PREP_CFG w:h [1920:1080] stride [1920:1080]
mpp[7534]: mpp_enc: send header for set cfg change input/format 
mpp[7534]: mpp_enc: mode vbr bps [486000:7776000:8262000] fps fix [30/1] -> fix [30/1] gop i [60] v [0]
mpp[7534]: hal_h264e_vepu_v2: warnning: input buffer size 0xa2800 is smaller than required size 0x1fe000
mpp[7534]: hal_h264e_vepu_v2: warnning: input buffer size 0xa2800 is smaller than cb requirement 0x1fa400 + 0x7f800
mpp[7534]: mpi_enc_test: 0x559af530a0 encoded frame 0    size 49156  
mpp[7534]: hal_h264e_vepu_v2: warnning: input buffer size 0xa2800 is smaller than required size 0x1fe000
mpp[7534]: hal_h264e_vepu_v2: warnning: input buffer size 0xa2800 is smaller than cb requirement 0x1fa400 + 0x7f800
mpp[7534]: mpp_serivce: mpp_service_cmd_poll ioctl MPP_IOC_CFG_V1 failed ret -1 errno 110 Connection timed out
mpp[7534]: hal_h264e_vepu2_v2: hal_h264e_vepu2_wait_v2 poll cmd failed 110
mpp[7534]: mpi_enc_test: 0x559af530a0 encoded frame 1    size 48956  
mpp[7534]: hal_h264e_vepu_v2: warnning: input buffer size 0xa2800 is smaller than required size 0x1fe000
mpp[7534]: hal_h264e_vepu_v2: warnning: input buffer size 0xa2800 is smaller than cb requirement 0x1fa400 + 0x7f800
mpp[7534]: mpp_serivce: mpp_service_cmd_poll ioctl MPP_IOC_CFG_V1 failed ret -1 errno 110 Connection timed out
mpp[7534]: hal_h264e_vepu2_v2: hal_h264e_vepu2_wait_v2 poll cmd failed 110
mpp[7534]: mpi_enc_test: 0x559af530a0 encoded frame 2    size 48956  
mpp[7534]: hal_h264e_vepu_v2: warnning: input buffer size 0xa2800 is smaller than required size 0x1fe000
mpp[7534]: hal_h264e_vepu_v2: warnning: input buffer size 0xa2800 is smaller than cb requirement 0x1fa400 + 0x7f800
mpp[7534]: mpp_serivce: mpp_service_cmd_poll ioctl MPP_IOC_CFG_V1 failed ret -1 errno 110 Connection timed out
mpp[7534]: hal_h264e_vepu2_v2: hal_h264e_vepu2_wait_v2 poll cmd failed 110
mpp[7534]: mpi_enc_test: 0x559af530a0 encoded frame 3    size 48956  
mpp[7534]: hal_h264e_vepu_v2: warnning: input buffer size 0xa2800 is smaller than required size 0x1fe000
mpp[7534]: hal_h264e_vepu_v2: warnning: input buffer size 0xa2800 is smaller than cb requirement 0x1fa400 + 0x7f800
mpp[7534]: mpp_serivce: mpp_service_cmd_poll ioctl MPP_IOC_CFG_V1 failed ret -1 errno 110 Connection timed out
mpp[7534]: hal_h264e_vepu2_v2: hal_h264e_vepu2_wait_v2 poll cmd failed 110

@avafinger avafinger reopened this Nov 2, 2021
@philfleck
Copy link

Hi @avafinger, Hi @HermanChen
could you resolve the issue (artifacts when encoding video from eg uvccamera with YUYV)? I am facing a similar problem but on the RK3588 (nanopc t6) and the RK3568B2.

Any help appreciated
best p

@avafinger
Copy link
Author

Can you share your command? I will check if i can reproduce here.

@avafinger
Copy link
Author

avafinger commented Oct 2, 2023 •
edited
Loading

I can only speculate, i think the problem is with RGA conversion with dmabuf.

If not using dmabuf you face this error:

 RgaBlit(1465) RGA_BLIT fail: Invalid argument
 RgaBlit(1466) RGA_BLIT fail: Invalid argument
fd-vir-phy-hnd-format[0, 0x7f8b3ce000, (nil), 0, 0]
rect[0, 0, 640, 480, 1280, 480, 65536, 0]
f-blend-size-rotation-col-log-mmu[0, 0, 0, 0, 0, 0, 1]
fd-vir-phy-hnd-format[29, (nil), (nil), 0, 0]
rect[0, 0, 640, 480, 1280, 480, 65536, 0]
f-blend-size-rotation-col-log-mmu[0, 0, 0, 0, 0, 0, 1]
This output the user patamaters when rga call blit fail

with dmabuf (render):
gstreamer_640x480

without dmabuf (render):
ffplay_640x480

Unfortunately, i can't do anything about it.

Maybe Rockchip ( @HermanChen ) can..

@avafinger
Copy link
Author

You should try with different window sizes, i know there is/are problems with alignment / stride with RGA.
The camera i tested here on rk3588 is limited to 640x480.

@philfleck
Copy link

philfleck commented Oct 3, 2023 •
edited
Loading

hi thanks for the input,

i used something among this line, without setting strides. i tested with a Microsoft lifecam 720p@30fps, same behavior with an logitech brio and a logitech c920. btw i did not come along the rga error ...

mpi_enc_test -i /dev/video0 -f 8 -w 1280 -h 720 -o out_1280x720.h264 -t 7

best p

@philfleck
Copy link

btw, the artefacts appear across all resolutions ...

@avafinger
Copy link
Author

avafinger commented Oct 3, 2023 •
edited
Loading

I'll leave it open.
Maybe someone has found a workaround and can post their findings here...

@JeffyCN

@avafinger avafinger reopened this Oct 3, 2023
@avafinger avafinger changed the title mpp encoding YUYV to H264 / H265 (RK3568) mpp encoding YUYV to H264 / H265 (RK3568 / RK3588) Oct 3, 2023
@belveder79 belveder79 mentioned this issue Oct 3, 2023
Closed
@nyanmisaka
Copy link
Contributor

You should use ioctl(DMA_BUF_IOCTL_SYNC) to synchronize the cacheable DMA-BUF before accessing it, and call it again after accessing it to invalidate the cache.

https://github.com/airockchip/librga/blob/fb3357d09008222bc5e27bdaadf74a0c5ea4c86e/samples/allocator_demo/src/rga_allocator_dma_cache_demo.cpp

https://github.com/airockchip/librga/blob/fb3357d09008222bc5e27bdaadf74a0c5ea4c86e/samples/utils/allocator/dma_alloc.cpp#L81

https://www.kernel.org/doc/html/v5.9/driver-api/dma-buf.html#cpu-access-to-dma-buffer-objects

@avafinger
Copy link
Author

The odd thing it only happens with YUYV, NV12 is fine...

@philfleck
Copy link

You should use ioctl(DMA_BUF_IOCTL_SYNC) to synchronize the cacheable DMA-BUF before accessing it, and call it again after accessing it to invalidate the cache.

https://github.com/airockchip/librga/blob/fb3357d09008222bc5e27bdaadf74a0c5ea4c86e/samples/allocator_demo/src/rga_allocator_dma_cache_demo.cpp

https://github.com/airockchip/librga/blob/fb3357d09008222bc5e27bdaadf74a0c5ea4c86e/samples/utils/allocator/dma_alloc.cpp#L81

https://www.kernel.org/doc/html/v5.9/driver-api/dma-buf.html#cpu-access-to-dma-buffer-objects

is this shown within the mpi_enc_test examples?

@nyanmisaka
Copy link
Contributor

You should use ioctl(DMA_BUF_IOCTL_SYNC) to synchronize the cacheable DMA-BUF before accessing it, and call it again after accessing it to invalidate the cache.
https://github.com/airockchip/librga/blob/fb3357d09008222bc5e27bdaadf74a0c5ea4c86e/samples/allocator_demo/src/rga_allocator_dma_cache_demo.cpp
https://github.com/airockchip/librga/blob/fb3357d09008222bc5e27bdaadf74a0c5ea4c86e/samples/utils/allocator/dma_alloc.cpp#L81
https://www.kernel.org/doc/html/v5.9/driver-api/dma-buf.html#cpu-access-to-dma-buffer-objects

is this shown within the mpi_enc_test examples?

Nope, but it's a part of the librga sample. IMHO mpi_enc_test should not be responsible for this. It assumes that the user's input is a normal file rather than a device file.

@HermanChen
Copy link
Collaborator

This error seems a cache issue. Any cpu copy on the image?

@avafinger
Copy link
Author

avafinger commented Oct 7, 2023 via email

@HermanChen
Copy link
Collaborator

Check the stream from mpi_enc_test first, if there is the lines on the stream then it is an error on copying frame before mpi_enc_test. If there is no lines in the steam then it is the decoding and render issue.

@HermanChen
Copy link
Collaborator

How to reach you by email? @avafinger

@avafinger
Copy link
Author

avafinger commented Oct 7, 2023 via email

@avafinger
Copy link
Author

avafinger commented Oct 7, 2023 via email

@avafinger
Copy link
Author

avafinger commented Oct 7, 2023 via email

@hbiyik
Copy link

hbiyik commented Nov 15, 2023

now i have the same issue, i have not investigated fully but here are my observations with rk3588

  1. it happens when using decoder with TIMEOUT_NON_BLOCKING with external dma32 buffers, i have a different architecture where there is TIMOUT_BLOCKING with internal buffers is used when decoding, it does not happen there. (i do not know which is relevant)
  2. in my case it also happen when i do not use rga as well, so pure decoder output
  3. it happens with mpeg1/2 or AV1 decoders does not happen with h264 or hevc decoder. (at least in my code)

I also have a feeling it is some form of synronization issue somewhere i will dig into it.

@avafinger
Copy link
Author

avafinger commented Nov 15, 2023 via email

@hbiyik
Copy link

hbiyik commented Nov 16, 2023

always happens when i play this file: https://github.com/hbiyik/FFmpeg/files/13342554/video_green.mkv.zip

typically with ffplay. i really think it is a dmabuf sync issue.

@avafinger
Copy link
Author

avafinger commented Nov 16, 2023 via email

@avafinger
Copy link
Author

avafinger commented Nov 16, 2023 via email

@hbiyik
Copy link

hbiyik commented Nov 16, 2023 •
edited
Loading

Yeah, this was a bad example due to another bug, ffplay gets stuck after 4th frame in this file.

I remembered now running mpv --hwdec=rkmpp video_green.mkv should reproduce it, like expkained here

But you are right, even mpv uses drmprime to render in this case ffmpeg decoder is using rga to convert 16byte aligned strides to 64, to satisfy panforks egl
So you are right in my case it is also rga.

@avafinger
Copy link
Author

avafinger commented Nov 17, 2023 via email

@avafinger
Copy link
Author

avafinger commented Nov 17, 2023 via email

@hbiyik
Copy link

hbiyik commented Nov 17, 2023 •
edited
Loading

in drm prime (--hwdec=rkmpp) it is not possible to output in planar form,

how ever you can do

FFMPEG_RKMPP_PIXFMT=yuv422p mpv --msg-level=ffmpeg=debug somefile.mp4,

this will force the output to yuv422p, still i am not sure how the mpp will output yuyv422, i assume it should output NV16 then yuv422p if size is below <4k, else you will have libyuv conversion.

You can see the decoder flow in log if you do --msg-level=ffmpeg=debug

@hbiyik
Copy link

hbiyik commented Nov 20, 2023

@avafinger
i fixed my problem in hbiyik/FFmpeg@65d4cf4, fyi, this happens when the input size of picture is small, and rga is faster than dma, you have to sync the dma with DMA_BUF_IOCTL_SYNC

@hbiyik
Copy link

hbiyik commented Dec 1, 2023

now the similar inferface is implemented in mpp e15972e

@avafinger
Copy link
Author

avafinger commented Dec 2, 2023 via email

@avafinger
Copy link
Author

avafinger commented Dec 25, 2023 via email

@p3ngu19z
Copy link

p3ngu19z commented Apr 19, 2024 •
edited
Loading

Any progress to resolve?
I have similar problem with on radxa zero 3w and radxa-zero3_debian_bullseye_xfce_b6.img build
Camera YUYV only

UPD:
Parameter io-mode=dmabuf-import fix this artifacts:
gst-launch-1.0 v4l2src device=/dev/video0 io-mode=dmabuf-import

@qqzlqqzlqqzl
Copy link

好东西，问题解决了

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@HermanChen @philfleck @nyanmisaka @avafinger @hbiyik @p3ngu19z @qqzlqqzlqqzl