Add support for OV13850 DUAL camera #2
Comments
|
Hey @avafinger , I don't have another OV13850 but I have a MCAM400, that I wanted to test anyway. The weekend is close and I will try to figure out how to run the dual camera set up, I comment as soon as I found out how. My current guess is that you probably have not activated the corresponding MIPI port because the second camera doesn't run on the Good luck and greetings, |
|
Update info: I tested with cam and camera 1 (1-0010) is somehow working, but (2-0010) failed for the reason: gpio2-11 is pinctrl-0 = <&cam1_default_pins &cif_clkout_a>; cif_clkout_a is shared by the two cameras, interesting is the same DT as in FE image and that is working with dual-camera. I decided to try it with Kernel 4.19.111 and have the same error. Built FE Kernel 4.4.179 and the same thing. Can you spot what is missing? cam -l cam --camera=1 --capture=20 --stream height=1080,width=1920,pixelformat=YUYV,role=video cam --camera=2 --capture=20 --stream height=1080,width=1920,pixelformat=YUYV,role=video Attached is a working DT from FE Image with a working dual-camera, just in case you would like to have a look. There, the cif_clkout_a is also shared but the FE kernel does not complain. |
|
Hey @avafinger , I have looked into this issue this weekend and have had the same problem, here is my current attempt: The issue really seems to be the sharing of the cif_clkout_a GPIO, something must have changed within other parts of the kernel. That is why I started to communicate with the Video 4 Linux team, I will give you feedback as soon as I know more. Greetings, EDIT: If you want you can check it out: initBasti/Linux_kernel_media_tree_fork@bebcd38 EDIT-2: |
That's what I suspected, I am not able to run a single camera on CSI-2. Porting this is beyond my skills/time. Is Helen kindly enough to start with that or you plan to start the porting? We still have the same problem of sharing the pins for the two cameras (multi-camera or dual-camera). It is not possible to share and set the same pins, the kernel will complain. Can you ask Helen how would that be possible? Or how she would approach the problem? |
|
@avafinger , Greetings, |
|
I had some free time and tested cif_clkout_b and it worked on the legacy kernel. dual-camera running! (I think multi-camera will work also) Thank you. and on isp1:
I am puzzled how FE solved this, they implemented this in the driver. Anyway, do you have the ov4689 with or without IR Cut filter? |
|
@avafinger
With the IR cut filter. Greetings, |
|
Here is my dev kernel for the NanoPi M4 you can have as a reference for the NanoPC-T4. For reference, Kernel 4.19.111 with dual-camera is here: DTS for OV13850 and OV4689: Camera 2 Pin config as you proposed: Hope you are making progress with the porting with mipi_dphy_tx1rx1. BR, |
|
Hi @initBasti Did you have any progress with mipi_dphy_tx1rx1 ? BR, |
|
Hey @avafinger , Not yet, I am currently finishing the first version of the mainline OV13850 sensor driver. What I currently know is that one of the Kernel developers specializing in Rockchip hardware (mmind), has made an attempt on that project in the past (here), but he was not able to get it to work at that time. Once I have finished the mentioned driver, I will have a look into it and see if I can make it work. Greetings, |
|
Hey @avafinger , I have good news, Heiko Stübener has found some time to work on the missing MIPI CSI for the second camera. You can find his work here. I have played around with these changes on my tree at the weekend, but I was not able to make it work for now as there are still some problems with the second instance of the ISP and with my OV4689 driver. Greetings, |
|
Hi @initBasti , Thank you for the update. This weekend i spent a lot of time with Kernel 4.19.161 (Rockchip) with the suggested cif_clkout_b for the second camera despite FE using cif_clkout_a for both cameras. I have not found where in the code the multiplexed pin is allowed. If I am not mistaken, FE 4.19 worked with cif_clkout_a for both cameras, after so many changes and tries I am not sure anymore. Anyway, FYI, it is all good for 4.19.y , except for the noise. Note that both drivers (ov4689 and ov13850) have been updated. There is better quality but with noise, I will try to get in touch with Rockchip and see if I can get it fixed. The attributes like module name, module facing,etc.. are used by the rkisp engine (bsp), and that works well with cif_clkout_b. I am able to grab images from both cameras at the same time. For the mainline, i can't say there is noise or not since the image is a little dark, if you work out the 3A to improve it, let me know. I will try your/his changes soon and report back. PS: I asked about your OV4689 sensor just because I have both, with and without IR. Interestingly the camera w/o IR gives me a pinkish image (or red tone if you prefer), many sources would say the image is pinkish due to IR being active during daylight, but this camera is without IR. Some sources say it is pinkish precisely because there is no IR on the sensor. BR |
|
Do you know what is the purpose of this hack? |
|
I think you missed: My first try (using Heiko patch), Kernel 5.11.0-rc5:
What is your experience? |
|
I have noticed Heiko has patches for the HDMI to DSI: mmind/linux-rockchip@5044a21 Do you think it is relevant? |
|
Second attempt, kernel 5.11-rc6 (cif_clkout_b): Note this : And Heiko Hack for the ov13850: mmind/linux-rockchip@9641b7a Unfortunately, HDMI is gone.... Can you check with Heiko if he is using his board headless? Complete boot log: Without the Heiko's patch (single camera): |
|
Fixed the gpio mistake, dual-camera shot, unfortunately, it breaks hdmi (drm) on the mainline kernel. Maybe you have better luck. |
|
Hey @avafinger , first of all sorry for the delayed response, I have been working on this issue and I am finally at a point where it works for me.
I have noticed a lot of noise as well, but I believe those are also fixed with the 3A algorithms. The reason for this belief is that I have tried both cameras on the official friendly elec image, where they have pre-installed a video capture software, which produces a quite clean output.
Ok, well then that means that my driver is not compatible with that kernel. As the rkisp1 engine from the mainline kernel doesn't expect such arguments.
This is on my todo list for a few months now ;) I will message you as soon as I am at a point where you can test. Could you provide me an Email address?
Nope, and I didn't require it as well.
True this was one of the issues that I encountered, thank you for your good eyes :).
I have not played around with HDMI until now and I just noticed, that there might be problems. What I have experienced with the setup that I explain further below is that everything works if I have no HDMI devices connected during boot. The media device graph can be created and the cam command finishes everything until the capture but then it halts. This means that the TX/RX seems to be reserved for the TX and therefore not available for the RX. But the rkisp1 has 3 different PHY a TX, an RX, and a TX/RX, so maybe there is a way to take the pure TX for the HDMI out. I am not completely sure about this theory though, I will mention this is Heiko's upstream patch series mentioned below. I have been able to capture from both cameras at the same time using the following setup: Heiko Stübner's upstream patch series: The image of the OV4689 looks very bad and has big green patches I currently investigate if it is a problem with the PHY, ISP instance, or the camera driver. Greetings, |
Do you mean for the second camera? |
I have tested if it is the camera or the Image Signal Processor instance, by switching their ports so that the OV13850 is connected to the 2nd ISP instance and the OV4689 to the 1st instance. And the image was still bad so it's not the second camera but the failure is specifically at the OV4689 (I am really not surprised as I haven't worked a lot on that driver so far). |
|
I get a good image but with the wrong window size or cropped, certainly something wrong. Maybe you can spot what is wrong: Here is a video (120 frames) that should be 1920x1080 but got a 1920x1520 frame size with the real image smaller, resized, or cropped. Where you see Blue is Red. The camera is w/o IR cut. https://drive.google.com/file/d/14UkhxQx7mA1w9xo748WxmakIGM-70qHd/view?usp=sharing No progress with dual-camera, my kernel is based on mainline kernel. |
|
Hey @avafinger ,
That is interesting when you use the setup that I described here:
Then you cannot stream with two cameras? Hmmm, I updated this repository maybe you can try to build a new kernel with the patches that I use. The setup that you use looks correct but I am not very familiar with mjpeg streamer. It looks like the top left crop that you created during media device configuration is cropped down even further at some point. I believe that the color issues are probably related to the 3A algorithms again. Greetings, |
|
Hi @initBasti ,
mjpeg streamer just grabs the yuv frame and sends it to the web client encoded in jpeg format. No changes during this process. Anyway, my setup is a bit different than yours, I will try a few things I have in mind. I have everything working with this kernel, hdmi,panfrost, wlan,eth,bt,rt5651, and hdmi-sound and would like not to lose these features.
This only happens with the sensor w/o IR cut. Works fine the one with IR. I think I have a clue, I will test a few things. Thanks, |
|
Hi @initBasti In my last attempt, i fixed my issues and now I have the image with the correct size but I found an interesting thing, i tested my changes with those two cameras (below). The left camera does not work, the right camera works (with HDMI working).
The left camera gives me a timeout during the select, although i can get info from the camera and set parameters. Here is the final output image of the right camera:: |
|
@avafinger |
|
@initBasti |
|
@initBasti Meanwhile, I got some ov5647 sensors with IR (auto cut) for 4B+, and as soon as I get the 4B+ board will try some experiments.
|
|
Hi @initBasti It's been a while since your last post, I wonder if you worked on libcamera (or ov4689) and can get a decent image from ov4689 |
|
Hey @avafinger , Oh, I am terribly sorry for replying so late. I have not yet worked with OV5647, but as I build camera test boxes for the libcamera project with different platforms + camera combinations, I will probably work with it in the near future. Greetings, |
|
No problems. Thank you for your reply. I just tested ov4689, kernel 5.13, with the latest libcamera but got a few more errors than the previous version. I don't know how to fix it, maybe you could give some hints or directions, or it is just my pipeline setup (cam don't rely on the manual pipeline, right?). About the ov5647, i have read it is on pair with RPI but not able to test it. I have not received my 4B+, that will take some time... Regards, |
|
Hey @avafinger , so the problem boils down to the missing IOCTL for getting the current selection, I had the same problem with the OV13850 and I was able to solve it by adding the following function. I have declared those macros here and I got the values from the documentation (this part could be tricky for the OV4689 and don't expect support from Omnivision, maybe you can find out what works by trying). I hope this helps. Greetings, |
|
Thank you, I missed that work but yesterday i had the following on my tests: But latest libcamera seems to require V4L2_CID_CAMERA_ORIENTATION. |
|
sorry to hijack this conversation... |
|
My board is on the way. I expect to get it at the end of this month, hopefully. |
|
I digged out the device tree from the rockpro64 I fixed with the help of @initBasti some time ago for the OV13850... trying some new tweaks tomorrow morning to see if that works for either the OV13850 or the MCAM400... what we have so far is mpp working and @philfleck is looking into this based on the 5.10.y kernel blob which seems to work already quite ok... funny that there is around 10 people only which you run across virtually all the time :-) |
|
Yeah. I think you need to tweak a bit for the dual sensor. Post your findings / changes. Encoder and Decoder are working great for NV12, but YUYV is a trouble. |
|
you are my hero! I did not find that post when I was googling for hours... will check out tomorrow immediately... thx a lot |
|
so quick reply... it does not work yet... either there is an essential bit missing in the description/files, or I'm still doing something wrong... will keep you posted... |
|
Did you get at least the video devices? ls -la /dev/video* |
|
hi, yes, a ton actually, and 4 media devices that link all of those: https://www.dropbox.com/scl/fi/5e2evasrczfkznhyp55xx/graph0.pdf?rlkey=cv9ycjosjq1glduoemta2f2vv&dl=0 but the issue is that the kernel log is full of issues and I'm not sure if it happens because I compiled the sensor driver as a module or not... |
|
Which kernel version? |
|
Rockchip-Kernel from NanoPC-T6 WIKI - https://github.com/friendlyarm/kernel-rockchip PS, did not make a difference to use the OV as a module or not... |
|
Did you attach the 2 sensors? |
|
Yes… both seem to work, but i just grabbed yuv images yet… still need to
check the data…
avafinger ***@***.***> schrieb am Mi. 4. Okt. 2023 um 15:24:
… Did you attach the 2 sensors?
—
Reply to this email directly, view it on GitHub
<#2 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAA3R4FEU26KWXH6TPSNJYLX5VPSVAVCNFSM4WCVSWDKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCNZUGY4DOMZUHE4A>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Hi @initBasti
I have added support for a dual camera that seems to be OK but I can not figure out how to configure the cameras.
Here is the device tree for the second camera:
rk3399.dtsi:
rk3399-nanopi-*:
In theory, both OV13850 are loaded and available:
and
Do you have another camera? Anyway below is all information that I think is relevant, can you spot how it should be configured?
All my attempts to configure the cameras failed, although both cameras have the same bus...
The text was updated successfully, but these errors were encountered: