Skip to content

feat(linux): Add IMX219 libcamera soft ISP documentation for AM62PX and AM62AX #827

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
thorstenl312 wants to merge 1 commit into
base: master
Choose a base branch
from
Open

feat(linux): Add IMX219 libcamera soft ISP documentation for AM62PX and AM62AX #827

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 75 additions & 2 deletions ...ce/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/CSI2RX_AM62AX.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -194,8 +194,8 @@
<https://carinastudio.azurewebsites.net/PixelViewer/>`__, or converted
using OpenCV.

Capture to Display using ISP
============================
Capture to Display using tiovxisp

Check warning on line 197 in source/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/CSI2RX_AM62AX.rst

View workflow job for this annotation

GitHub Actions / vale 

[vale] reported by reviewdog 🐶
[RedHat.Spelling] Verify the word 'tiovxisp'. It is not in the American English or Red Hat terminology spelling dictionaries used by Vale.

Raw Output:
{"message":"[RedHat.Spelling] Verify the word 'tiovxisp'. It is not in the American English or Red Hat terminology spelling dictionaries used by Vale.","location":{"path":"source/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/CSI2RX_AM62AX.rst","range":{"start":{"line":197,"column":26},"end":{"line":197,"column":34}}},"severity":"WARNING","code":{"value":"RedHat.Spelling","url":"https://redhat-documentation.github.io/vale-at-red-hat/reference-guide.html#spelling"}}
========================================

To use the full capture to display pipeline, you can use gstreamer to call
the required ISP components to convert the raw frames, apply
Expand Down Expand Up @@ -247,6 +247,79 @@
sink_1::startx="<640>" sink_1::starty="<480>" sink_1::widths="<640>" sink_1::heights="<480>" ! \
queue ! kmssink driver-name=tidss plane-properties=s,zpos=1

IMX219 Camera with libcamera Soft ISP

Check warning on line 250 in source/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/CSI2RX_AM62AX.rst

View workflow job for this annotation

GitHub Actions / vale 

[vale] reported by reviewdog 🐶
[RedHat.Definitions] Define acronyms and abbreviations (such as 'ISP') on first occurrence if they're likely to be unfamiliar.

Raw Output:
{"message":"[RedHat.Definitions] Define acronyms and abbreviations (such as 'ISP') on first occurrence if they're likely to be unfamiliar.","location":{"path":"source/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/CSI2RX_AM62AX.rst","range":{"start":{"line":250,"column":35},"end":{"line":250,"column":38}}},"severity":"INFO","code":{"value":"RedHat.Definitions","url":"https://redhat-documentation.github.io/vale-at-red-hat/reference-guide.html#definitions"}}

Check warning on line 250 in source/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/CSI2RX_AM62AX.rst

View workflow job for this annotation

GitHub Actions / vale 

[vale] reported by reviewdog 🐶
[RedHat.Spelling] Verify the word 'libcamera'. It is not in the American English or Red Hat terminology spelling dictionaries used by Vale.

Raw Output:
{"message":"[RedHat.Spelling] Verify the word 'libcamera'. It is not in the American English or Red Hat terminology spelling dictionaries used by Vale.","location":{"path":"source/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/CSI2RX_AM62AX.rst","range":{"start":{"line":250,"column":20},"end":{"line":250,"column":29}}},"severity":"WARNING","code":{"value":"RedHat.Spelling","url":"https://redhat-documentation.github.io/vale-at-red-hat/reference-guide.html#spelling"}}

Check warning on line 250 in source/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/CSI2RX_AM62AX.rst

View workflow job for this annotation

GitHub Actions / vale 

[vale] reported by reviewdog 🐶
[RedHat.Headings] Use sentence-style capitalization in 'IMX219 Camera with libcamera Soft ISP'.

Raw Output:
{"message":"[RedHat.Headings] Use sentence-style capitalization in 'IMX219 Camera with libcamera Soft ISP'.","location":{"path":"source/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/CSI2RX_AM62AX.rst","range":{"start":{"line":250,"column":1},"end":{"line":250,"column":38}}},"severity":"INFO","code":{"value":"RedHat.Headings","url":"https://redhat-documentation.github.io/vale-at-red-hat/reference-guide.html#headings"}}
=====================================

IMX219-based modules on the 15-pin FFC connector can also be used with

Check warning on line 253 in source/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/CSI2RX_AM62AX.rst

View workflow job for this annotation

GitHub Actions / vale 

[vale] reported by reviewdog 🐶
[RedHat.PassiveVoice] 'be used' is passive voice. In general, use active voice. Consult the style guide for acceptable use of passive voice.

Raw Output:
{"message":"[RedHat.PassiveVoice] 'be used' is passive voice. In general, use active voice. Consult the style guide for acceptable use of passive voice.","location":{"path":"source/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/CSI2RX_AM62AX.rst","range":{"start":{"line":253,"column":59},"end":{"line":253,"column":66}}},"severity":"INFO","code":{"value":"RedHat.PassiveVoice","url":"https://redhat-documentation.github.io/vale-at-red-hat/reference-guide.html#passive-voice"}}

Check warning on line 253 in source/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/CSI2RX_AM62AX.rst

View workflow job for this annotation

GitHub Actions / vale 

[vale] reported by reviewdog 🐶
[RedHat.Definitions] Define acronyms and abbreviations (such as 'FFC') on first occurrence if they're likely to be unfamiliar.

Raw Output:
{"message":"[RedHat.Definitions] Define acronyms and abbreviations (such as 'FFC') on first occurrence if they're likely to be unfamiliar.","location":{"path":"source/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/CSI2RX_AM62AX.rst","range":{"start":{"line":253,"column":36},"end":{"line":253,"column":39}}},"severity":"INFO","code":{"value":"RedHat.Definitions","url":"https://redhat-documentation.github.io/vale-at-red-hat/reference-guide.html#definitions"}}
libcamera's CPU soft ISP as an alternative to the tiovxisp hardware ISP

Check warning on line 254 in source/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/CSI2RX_AM62AX.rst

View workflow job for this annotation

GitHub Actions / vale 

[vale] reported by reviewdog 🐶
[RedHat.Definitions] Define acronyms and abbreviations (such as 'ISP') on first occurrence if they're likely to be unfamiliar.

Raw Output:
{"message":"[RedHat.Definitions] Define acronyms and abbreviations (such as 'ISP') on first occurrence if they're likely to be unfamiliar.","location":{"path":"source/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/CSI2RX_AM62AX.rst","range":{"start":{"line":254,"column":69},"end":{"line":254,"column":72}}},"severity":"INFO","code":{"value":"RedHat.Definitions","url":"https://redhat-documentation.github.io/vale-at-red-hat/reference-guide.html#definitions"}}

Check warning on line 254 in source/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/CSI2RX_AM62AX.rst

View workflow job for this annotation

GitHub Actions / vale 

[vale] reported by reviewdog 🐶
[RedHat.Spelling] Verify the word 'tiovxisp'. It is not in the American English or Red Hat terminology spelling dictionaries used by Vale.

Raw Output:
{"message":"[RedHat.Spelling] Verify the word 'tiovxisp'. It is not in the American English or Red Hat terminology spelling dictionaries used by Vale.","location":{"path":"source/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/CSI2RX_AM62AX.rst","range":{"start":{"line":254,"column":51},"end":{"line":254,"column":59}}},"severity":"WARNING","code":{"value":"RedHat.Spelling","url":"https://redhat-documentation.github.io/vale-at-red-hat/reference-guide.html#spelling"}}

Check warning on line 254 in source/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/CSI2RX_AM62AX.rst

View workflow job for this annotation

GitHub Actions / vale 

[vale] reported by reviewdog 🐶
[RedHat.Definitions] Define acronyms and abbreviations (such as 'ISP') on first occurrence if they're likely to be unfamiliar.

Raw Output:
{"message":"[RedHat.Definitions] Define acronyms and abbreviations (such as 'ISP') on first occurrence if they're likely to be unfamiliar.","location":{"path":"source/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/CSI2RX_AM62AX.rst","range":{"start":{"line":254,"column":22},"end":{"line":254,"column":25}}},"severity":"INFO","code":{"value":"RedHat.Definitions","url":"https://redhat-documentation.github.io/vale-at-red-hat/reference-guide.html#definitions"}}

Check warning on line 254 in source/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/CSI2RX_AM62AX.rst

View workflow job for this annotation

GitHub Actions / vale 

[vale] reported by reviewdog 🐶
[RedHat.Spelling] Verify the word 'libcamera's'. It is not in the American English or Red Hat terminology spelling dictionaries used by Vale.

Raw Output:
{"message":"[RedHat.Spelling] Verify the word 'libcamera's'. It is not in the American English or Red Hat terminology spelling dictionaries used by Vale.","location":{"path":"source/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/CSI2RX_AM62AX.rst","range":{"start":{"line":254,"column":1},"end":{"line":254,"column":12}}},"severity":"WARNING","code":{"value":"RedHat.Spelling","url":"https://redhat-documentation.github.io/vale-at-red-hat/reference-guide.html#spelling"}}
pipeline. The IMX219 outputs raw Bayer frames - libcamera handles the
debayering in software.

Applying the sensor overlay
---------------------------

At the u-boot prompt:

.. code-block:: text

=> setenv name_overlays ti/k3-am62x-sk-csi2-imx219.dtbo
=> boot

To apply persistently, edit :file:`uEnv.txt` on the boot partition:

.. code-block:: console

$ vi /run/media/boot-mmcblk1p1/uEnv.txt

.. code-block:: text

name_overlays=ti/k3-am62x-sk-csi2-imx219.dtbo

Verifying the sensor is detected
---------------------------------

.. code-block:: console

$ cam -l
[...] INFO Camera camera_manager.cpp:223 Adding camera \
'/base/bus@f0000/i2c@20020000/i2c-mux@71/i2c@1/camera@10' for pipeline handler simple
Available cameras:
1: External camera 'imx219' (...)

The sensor is listed as ``External camera`` because the device tree sets
``orientation = <MEDIA_ORIENTATION_EXTERNAL>``. libcamera requires this
property to expose ``V4L2_CID_CAMERA_ORIENTATION`` and will refuse to open
the sensor without it.

.. note::

The ``Rotation control not available`` warning is expected and harmless.

Capturing frames
----------------

.. code-block:: console

$ export LIBCAMERA_SOFTISP_MODE=cpu
$ cam -c 1 --capture=1 --file

The frame is saved as ``frame-cam0-stream0-000000.bin`` in the current
directory in ``ABGR8888`` format.

libcamera capture to display
-----------------------------

.. code-block:: console

$ systemctl stop emptty
$ export LIBCAMERA_SOFTISP_MODE=cpu

# 1920x1080 — expected ~15 fps
$ gst-launch-1.0 libcamerasrc ! video/x-raw,format=RGBA,width=1920,height=1080 ! kmssink driver-name=tidss

# 640x480 — expected ~30 fps
$ gst-launch-1.0 libcamerasrc ! video/x-raw,format=RGBA,width=640,height=480 ! kmssink driver-name=tidss

CSI2RX testing details
======================

Expand Down
76 changes: 74 additions & 2 deletions ...ce/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/CSI2RX_AM62PX.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,8 +129,8 @@ than the default (640x480 UYVY):

$ echo "on" > /sys/devices/platform/bus@f0000/20020000.i2c/i2c-2/i2c-4/4-003c/power/control

Capture to display
==================
OV5640 Capture to Display
=========================

If a display (HDMI or LVDS) is connected then use the following steps to view the camera frames:

Expand Down Expand Up @@ -193,6 +193,78 @@ should resume from where it left (as long as the sensor supports it).

Only TEVI OV5640 and IMX219 are known to work reliably when system is suspended with capture running.

IMX219 Camera with libcamera Soft ISP
======================================

|__PART_FAMILY_NAME__| SK supports IMX219-based camera modules on the same
15-pin FFC connector. The IMX219 outputs raw Bayer frames - libcamera
debayering is done in software on the CPU.

Applying the sensor overlay
---------------------------

At the u-boot prompt:

.. code-block:: text

=> setenv name_overlays ti/k3-am62x-sk-csi2-imx219.dtbo
=> boot

To apply persistently, edit :file:`uEnv.txt` on the boot partition:

.. code-block:: console

$ vi /run/media/boot-mmcblk1p1/uEnv.txt

.. code-block:: text

name_overlays=ti/k3-am62x-sk-csi2-imx219.dtbo

Verifying the sensor is detected
---------------------------------

.. code-block:: console

$ cam -l
[...] INFO Camera camera_manager.cpp:223 Adding camera \
'/base/bus@f0000/i2c@20020000/i2c-mux@71/i2c@1/camera@10' for pipeline handler simple
Available cameras:
1: External camera 'imx219' (...)

The sensor is listed as ``External camera`` because the device tree sets
``orientation = <MEDIA_ORIENTATION_EXTERNAL>``. libcamera requires this
property to expose ``V4L2_CID_CAMERA_ORIENTATION`` and will refuse to open
the sensor without it.

.. note::

The ``Rotation control not available`` warning is expected and harmless.

Capturing frames
----------------

.. code-block:: console

$ export LIBCAMERA_SOFTISP_MODE=cpu
$ cam -c 1 --capture=1 --file

The frame is saved as ``frame-cam0-stream0-000000.bin`` in the current
directory in ``ABGR8888`` format.

libcamera capture to display
-----------------------------

.. code-block:: console

$ systemctl stop emptty
$ export LIBCAMERA_SOFTISP_MODE=cpu

# 1920x1080 — expected ~15 fps
$ gst-launch-1.0 libcamerasrc ! video/x-raw,format=RGBA,width=1920,height=1080 ! kmssink driver-name=tidss

# 640x480 — expected ~30 fps
$ gst-launch-1.0 libcamerasrc ! video/x-raw,format=RGBA,width=640,height=480 ! kmssink driver-name=tidss

Embedded Data
=============

Expand Down
3 changes: 3 additions & 0 deletions ...e/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/_CSI2RX_common.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,7 @@ I2C2 and CSI-RX Port 0:
/plugin/;

#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/media/video-interface-devices.h>

&{/} {
clk_sensor_fixed: sensor-xclk {
Expand All @@ -176,6 +177,8 @@ I2C2 and CSI-RX Port 0:
clocks = <&clk_sensor_fixed>;
clock-names = "xclk";

orientation = <MEDIA_ORIENTATION_EXTERNAL>;

port {
csi2_cam0: endpoint {
remote-endpoint = <&csi2rx0_in_sensor>;
Expand Down
Loading