Rpi/embedded data support/rpi 6.6.y #6437

The v4l2_pipeline_pm_get() and v4l2_pipeline_pm_put() functions were needed to control sub-devices' power states before runtime PM. These functions should no longer be used, and instead sub-device drivers should use runtime PM. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 6d7ca3c9c1de48c627769602af9cf841459c1828)

Drop an unneeded double colon, and use 'shall' instead of 'must' for consistency with the rest of the file. Signed-off-by: Laurent Pinchart <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 5adf57f8a9ddf1540a27c132af47bb1349bfe170) Signed-off-by: Jacopo Mondi <[email protected]>

The Documentation/devicetree/bindings/clock/clock-bindings.txt file is deprecated and points to clock-bindings.yaml, which is not hosted in the kernel source tree. Use an HTTPS link to refer to the YAML binding document. While at it, drop "currently" from the paragraph, as the whole file refers to the current recommended practices except where explicitly noted. Signed-off-by: Laurent Pinchart <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit a7acee9965d914202386ff438799feed74bd504c) Signed-off-by: Jacopo Mondi <[email protected]>

Move the power management section up, just after clocks, as it relates to internal system resources and not features exposed to applications. The text itself is otherwise unchanged. Signed-off-by: Laurent Pinchart <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 3a8b77f735ce0c17a57025426d79764d74c41047) Signed-off-by: Jacopo Mondi <[email protected]>

…tation Camera sensor drivers are highly subject to cargo cult programming, with back practices being copied from old to new drivers. In particular, many drivers implement system and runtime PM incorrectly. As a first step towards fixing this situation, refactor and expand the power management documentation to detail correct usage of system and runtime PM. Signed-off-by: Laurent Pinchart <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 072278810ca27421fb75d5ad8ce5b310ad09a267) Signed-off-by: Jacopo Mondi <[email protected]>

Add a definition for MIPI CSI-2 generic long packet types. The generic long packet types are numbered from 1 to 4. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 2d21fef)

The try_fmt, try_crop and try_compose fields of the v4l2_subdev_pad_config structure are misnamed (for historical reason) as they also store data for the subdev active configuration. Rename them to format, crop and compose respectively and update the accessor helpers. Signed-off-by: Laurent Pinchart <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Mauro Carvalho Chehab <[email protected]> (cherry picked from commit 36f2cd3)

The 'try_*' fields of the v4l2_subdev_pad_config structure are meant to be accessed through helper functions. Replace direct access with usage of the v4l2_subdev_get_pad_format(), v4l2_subdev_get_pad_crop() and v4l2_subdev_get_pad_compose() helpers. Signed-off-by: Laurent Pinchart <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Mauro Carvalho Chehab <[email protected]> (cherry picked from commit fd17e3a) Signed-off-by: Jacopo Mondi <[email protected]>

The v4l2_subdev_pad_config structure is not passed to subdev operations anymore. Drop an outdated comment to refers to that old mechanism. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Tomi Valkeinen <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Mauro Carvalho Chehab <[email protected]> (cherry picked from commit 6078b2b)

V4L2 subdev operations have moved from operating on a v4l2_subdev_pad_config to a v4l2_subdev_state a long time ago. Fix the few remaining incorrect references to pad config in the documentation. Signed-off-by: Laurent Pinchart <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Mauro Carvalho Chehab <[email protected]> (cherry picked from commit c1ac529)

Store the sub-device in the sub-device state. This will be needed in e.g. validating pad number when retrieving information for non-stream-aware users. There are expected to be more needs for this in the future. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Reviewed-by: Tomi Valkeinen <[email protected]> Signed-off-by: Mauro Carvalho Chehab <[email protected]> (cherry picked from commit 52c2575)

…tions There are two sets of functions that return information from sub-device state, one for stream-unaware users and another for stream-aware users. Add support for stream-aware functions to return format, crop and compose information from pad-based array that are functionally equivalent to the old, stream-unaware ones. Also check state is non-NULL, in order to guard against old drivers potentially calling this with NULL state for active formats or selection rectangles. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Reviewed-by: Tomi Valkeinen <[email protected]> Signed-off-by: Mauro Carvalho Chehab <[email protected]> (cherry picked from commit 049fa16)

Rename the sub-devices state information access functions, removing "_stream" from them and replacing "format" by "ffmt". This makes them shorter and so more convenient to use. No other sets of functions will be needed to access this information. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Reviewed-by: Tomi Valkeinen <[email protected]> Signed-off-by: Mauro Carvalho Chehab <[email protected]> (cherry picked from commit d0fde6a)

…t now Now that v4l2_subdev_state_get_format() always returns format, don't call alternative v4l2_subdev_get_pad_format() anymore. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Reviewed-by: Tomi Valkeinen <[email protected]> Signed-off-by: Mauro Carvalho Chehab <[email protected]> (cherry picked from commit 8824170)

…ctions The sub-device state access functions take three arguments: sub-device state, pad and stream. The stream is not relevant for the majority of drivers and having to specify 0 for the stream is considered a nuisance. Provide a two-argument macros for these state access functions to cover the needs of stream-unaware users. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Mauro Carvalho Chehab <[email protected]> (cherry picked from commit 791765b)

Compile sub-device state information access functions v4l2_subdev_state_get_{format,crop,compose} unconditionally as they are now also used by plain V4L2 drivers. Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Mauro Carvalho Chehab <[email protected]> (cherry picked from commit 981e0d4)

Remove stream-unaware sub-device state access functions and macros. These are no longer used. [Sakari Ailus: Resolve a minor conflict in removed code.] Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Mauro Carvalho Chehab <[email protected]> (cherry picked from commit 34dfd1d)

…cess" This reverts commit 064468855c423bfea9299af66239e6fd96709502.

Return NULL from sub-device pad state access functions (v4l2_subdev_state_get_{format,crop,compose}) for non-existent pads. While this behaviour differs from older set of pad state information access functions, we've had a WARN_ON() there for a long time and callers also do validate the pad index nowadays. Therefore problems are not expected. Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Mauro Carvalho Chehab <[email protected]> (cherry picked from commit 3591c53)

…bdev Graciously handle an uninitialised (but still zeroed) sub-device in v4l2_subdev_cleanup(). The list_empty() check there is unnecessary, too, so replace that by cheking whether the lists's next field is NULL. Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit ab8d719)

…MEDIA_CONTROLLER The subdev state locking macros and macros to get the active state are currently behind CONFIG_MEDIA_CONTROLLER. This makes sense, as there can be no subdev state without MC. However, we have code paths common to MC and non-MC cases which call subdev operations that have subdev state as a parameter. In the non-MC case the state parameter would always be NULL. Thus it makes sense to allow, e.g.: v4l2_subdev_call_state_active(sd, pad, get_fmt, fmt) which for non-MC case would call the subdev passing NULL as the state. This currently fails: https://lore.kernel.org/oe-kbuild-all/[email protected]/ Fix the issue by moving the related macros to be outside CONFIG_MEDIA_CONTROLLER. The v4l2_subdev_lock_state() and v4l2_subdev_unlock_state() macros will crash if given NULL as the state, but the other macros behave correctly even when there's no active state, and they will only call the lock/unlock macros if there is a state. An alternative fix would be to make another version of v4l2_subdev_call_state_try() with ifdefs, which would not use any state macros and would always pass NULL as the state. But having two version of a macro/function is always more confusing than having just one, so I went this way. So, this fixes the v4l2_subdev_call_state_active() macro. But we also have v4l2_subdev_call_state_try(). It would be possible to fix that macro by additionally creating "no-op" variants of the state alloc and free functions. However, v4l2_subdev_call_state_try() is only used by a single driver (stm32-dcmi), which selects MC, and the macro is supposed to be removed as soon as the users have been converted away from the macro. Thus I have not touched the state alloc/free functions, and I think it makes sense to keep alloc/free functions available only if there's actually something that can be allocated or freed. Signed-off-by: Tomi Valkeinen <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Reviewed-by: Hans Verkuil <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Laurent Pinchart <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 08e5c36)

Align lines for numbered list so that Sphinx produces an uniform output for all list entries. Also indent paragraphs of such list entries for consistency. Also use ReST numbered list syntax for the entries. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 580139f)

Clear frame descriptor before calling transmitter's get_frame_desc() op. Also remove the corresponding memset() calls from drivers. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Reviewed-by: Tomi Valkeinen <[email protected]> (cherry picked from commit 92b224d588792669348777b304f2c88570b0898d) Signed-off-by: Jacopo Mondi <[email protected]>

Print debug level information on returned frame descriptors. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> (cherry picked from commit 703cfceaa5e253ecb5311ed19445753dbfc604bc) Signed-off-by: Jacopo Mondi <[email protected]>

Split camera sensor documentation into user and kernel portions. This should make it easier for the user space developers to find the relevant documentation. Also add a list of exemplary drivers and add imx219 driver to it, besides those that were already mentioned. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Reviewed-by: Jacopo Mondi <[email protected]> (cherry picked from commit 0fa78064f6e3a354616fb24462864900c0db3191) Signed-off-by: Jacopo Mondi <[email protected]>

Check the validity of pad flags on entity init. Exactly one of the flags must be set. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit deb866f)

Add generic serial metadata mbus formats. These formats describe data width and packing but not the content itself. The reason for specifying such formats is that the formats as such are fairly device specific but they are still handled by CSI-2 receiver drivers that should not be aware of device specific formats. What makes generic metadata formats possible is that these formats are parsed by software only, after capturing the data to system memory. Also add a definition for "Data Unit" to cover what is essentially a pixel but is not image data. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit f28bdda) Signed-off-by: Jacopo Mondi <[email protected]>

Now that metadata mbus formats have been added, it is necessary to define which fields in struct v4l2_mbus_format are applicable to them (not many). Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Tomi Valkeinen <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit ac5214a) Signed-off-by: Jacopo Mondi <[email protected]>

Generic 8-bit metadata formats define the in-memory data layout but not the format of the data itself. The reasoning for having such formats is to allow CSI-2 receiver drivers to receive and DMA drivers to write the data to memory without knowing a large number of device-specific formats. These formats may be used only in conjunction with a Media Controller pipeline where the internal pad of the source sub-device defines the specific format of the data (using an mbus code). Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 1d92152) Signed-off-by: Jacopo Mondi <[email protected]>

Many camera sensors, among other devices, transmit embedded data and image data for each CSI-2 frame. This embedded data typically contains register configuration of the sensor that has been used to capture the image data of the same frame. The embedded data is received by the CSI-2 receiver and has the same properties as the image data, including that it is line based: it has width, height and bytesperline (stride). Add these fields to struct v4l2_meta_format and document them. Also add V4L2_FMT_FLAG_META_LINE_BASED to tell a given format is line-based i.e. these fields of struct v4l2_meta_format are valid for it. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 89345c2) Signed-off-by: Jacopo Mondi <[email protected]>

Set (and unset) the V4L2_FMT_FLAG_META_LINE_BASED flag in struct v4l2_fmtdesc based on the format after returning the driver callback for enumerating formats. This way the drivers don't need to care about the flag. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 2182860) Signed-off-by: Jacopo Mondi <[email protected]>

Add two new functions, v4l2_subdev_lock_states() and v4l2_subdev_unclock_states(), to acquire and release the state of two sub-devices. They differ from calling v4l2_subdev_{un,}lock_state() so that if the two states share the same lock, the lock is acquired only once. Also use the new functions in v4l2_subdev_link_validate(). Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Julien Massot <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 72364b9) Signed-off-by: Jacopo Mondi <[email protected]>

As the user needs to know what went wrong for S_ROUTING, copy array arguments back to the user. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Julien Massot <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 38c8493) Signed-off-by: Jacopo Mondi <[email protected]>

The len_routes field is used to tell the size of the routes array in struct v4l2_subdev_routing. This way the number of routes returned from S_ROUTING IOCTL may be larger than the number of routes provided, in case there are more routes returned by the driver. Note that this uAPI is still disabled in the code, so this change can safely be done. Anyone who manually patched the code to enable this uAPI must update their code. The patch also increases the number of reserved fields in struct v4l2_subdev_routing. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 83a22a0) Signed-off-by: Jacopo Mondi <[email protected]>

Return the routes set using S_ROUTING back to the user. Also reflect this in documentation. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 91e99e5) Signed-off-by: Jacopo Mondi <[email protected]>

Add trivial S_ROUTING IOCTL support for drivers where routing is static. Essentially this means returning the same information G_ROUTING call would have done. Signed-off-by: Sakari Ailus <[email protected]> Co-developed-by: Laurent Pinchart <[email protected]> Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Julien Massot <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 1bfef49) Signed-off-by: Jacopo Mondi <[email protected]>

The generic metadata pixel formats (V4L2_META_FMT_GENERIC_*) are meant to be used in conjunction with device-specific media bus codes. Those codes are work in progress and not available in the upstream kernel yet. To make sure the generic metadata pixel formats won't be used by userspace until we have the full infrastructure in place, keep their definition private to the kernel for now. Signed-off-by: Laurent Pinchart <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit d69c842) Signed-off-by: Jacopo Mondi <[email protected]>

Add helper functions to enable and disable the privacy led. This moves the #if from the call site to the privacy led functions, and makes adding privacy led support to v4l2_subdev_enable/disable_streams() cleaner. Reviewed-by: Umang Jain <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Tested-by: Umang Jain <[email protected]> Signed-off-by: Tomi Valkeinen <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 4e628f9) Signed-off-by: Jacopo Mondi <[email protected]>

…_streams() Use v4l2_subdev_has_op() in v4l2_subdev_enable/disable_streams() instead of open coding the same. Reviewed-by: Umang Jain <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Tested-by: Umang Jain <[email protected]> Signed-off-by: Tomi Valkeinen <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 68e3624) Signed-off-by: Jacopo Mondi <[email protected]>

Add some checks to verify that the subdev driver implements required features. A subdevice that supports streams (V4L2_SUBDEV_FL_STREAMS) must do one of the following: - Implement neither .enable/disable_streams() nor .s_stream(), if the subdev is part of a video driver that uses an internal method to enable the subdev. - Implement only .enable/disable_streams(), if support for legacy sink-side subdevices is not needed. - Implement .enable/disable_streams() and .s_stream(), if support for legacy sink-side subdevices is needed. At the moment the framework doesn't check this requirement. Add the check. Reviewed-by: Umang Jain <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Tested-by: Umang Jain <[email protected]> Signed-off-by: Tomi Valkeinen <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit e003fd9) Signed-off-by: Jacopo Mondi <[email protected]>

call_s_stream() uses sd->enabled_streams to track whether streaming has already been enabled. However, v4l2_subdev_enable/disable_streams_fallback(), which was the original user of this field, already uses it, and v4l2_subdev_enable/disable_streams_fallback() will call call_s_stream(). This leads to a conflict as both functions set the field. Afaics, both functions set the field to the same value, so it won't cause a runtime bug, but it's still wrong and if we, e.g., change how v4l2_subdev_enable/disable_streams_fallback() operates we might easily cause bugs. Fix this by adding a new field, 's_stream_enabled', for call_s_stream(). Reviewed-by: Umang Jain <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Tested-by: Umang Jain <[email protected]> Signed-off-by: Tomi Valkeinen <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 1d78042) Signed-off-by: Jacopo Mondi <[email protected]>

v4l2_subdev_enable/disable_streams_fallback() supports falling back to .s_stream() for subdevs with a single source pad. It also tracks the enabled streams for that one pad in the sd->enabled_streams field. Tracking the enabled streams with sd->enabled_streams does not make sense, as with .s_stream() there can only be a single stream per pad. Thus, as the v4l2_subdev_enable/disable_streams_fallback() only supports a single source pad, all we really need is a boolean which tells whether streaming has been enabled on this pad or not. However, as we only need a true/false state for a pad (instead of tracking which streams have been enabled for a pad), we can easily extend the fallback mechanism to support multiple source pads as we only need to keep track of which pads have been enabled. Change the sd->enabled_streams field to sd->enabled_pads, which is a 64-bit bitmask tracking the enabled source pads. With this change we can remove the restriction that v4l2_subdev_enable/disable_streams_fallback() only supports a single source pad. Reviewed-by: Laurent Pinchart <[email protected]> Tested-by: Umang Jain <[email protected]> Signed-off-by: Tomi Valkeinen <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 61d6c8c) Signed-off-by: Jacopo Mondi <[email protected]>

Add a helper function which returns whether the subdevice is streaming, i.e. if .s_stream or .enable_streams has been called successfully. Reviewed-by: Umang Jain <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Tested-by: Umang Jain <[email protected]> Signed-off-by: Tomi Valkeinen <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 5f3ce14) Signed-off-by: Jacopo Mondi <[email protected]>

…ams() We support camera privacy leds with the .s_stream() operation, in call_s_stream(), but we don't have that support when the subdevice implements .enable/disable_streams() operations. Add the support by enabling the led when the first stream for a subdevice is enabled, and disabling the led then the last stream is disabled. Reviewed-by: Umang Jain <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Tested-by: Umang Jain <[email protected]> Signed-off-by: Tomi Valkeinen <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 585d8fd) Signed-off-by: Jacopo Mondi <[email protected]>

Add two internal helper functions, v4l2_subdev_collect_streams() and v4l2_subdev_set_streams_enabled(), which allows us to refactor v4l2_subdev_enable/disable_streams() functions. This (I think) makes the code a bit easier to read, and lets us more easily add new functionality in the helper functions in the following patch. Reviewed-by: Laurent Pinchart <[email protected]> Tested-by: Umang Jain <[email protected]> Signed-off-by: Tomi Valkeinen <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 8686230) Signed-off-by: Jacopo Mondi <[email protected]>

…le_streams() At the moment the v4l2_subdev_enable/disable_streams() functions call fallback helpers to handle the case where the subdev only implements .s_stream(), and the main function handles the case where the subdev implements streams (V4L2_SUBDEV_FL_STREAMS, which implies .enable/disable_streams()). What is missing is support for subdevs which do not implement streams support, but do implement .enable/disable_streams(). Example cases of these subdevices are single-stream cameras, where using .enable/disable_streams() is not required but helps us remove the users of the legacy .s_stream(), and subdevices with multiple source pads (but single stream per pad), where .enable/disable_streams() allows the subdevice to control the enable/disable state per pad. The two single-streams cases (.s_stream() and .enable/disable_streams()) are very similar, and with small changes we can change the v4l2_subdev_enable/disable_streams() functions to support all three cases, without needing separate fallback functions. A few potentially problematic details, though: - For the single-streams cases we use sd->enabled_pads field, which limits the number of pads for the subdevice to 64. For simplicity I added the check for this limitation to the beginning of the function, and it also applies to the streams case. - The fallback functions only allowed the target pad to be a source pad. It is not very clear to me why this check was needed, but it was not needed in the streams case. However, I doubt the v4l2_subdev_enable/disable_streams() code has ever been tested with sink pads, so to be on the safe side, I added the same check to the v4l2_subdev_enable/disable_streams() functions. Reviewed-by: Laurent Pinchart <[email protected]> Tested-by: Umang Jain <[email protected]> Signed-off-by: Tomi Valkeinen <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit b62949d) Signed-off-by: Jacopo Mondi <[email protected]>

…lper() At the moment v4l2_subdev_s_stream_helper() only works for subdevices that support routing. As enable/disable_streams now also works for subdevices without routing, improve v4l2_subdev_s_stream_helper() to do the same. Reviewed-by: Laurent Pinchart <[email protected]> Tested-by: Umang Jain <[email protected]> Signed-off-by: Tomi Valkeinen <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 93c726f) Signed-off-by: Jacopo Mondi <[email protected]>

The v4l2_subdev_link_validate_get_format() function warns if the pad given as argument belongs to a non-subdev entity. This can't happen, as the function is called from v4l2_subdev_link_validate() only (indirectly through v4l2_subdev_link_validate_locked()), and that function ensures that both ends of the link are subdevs. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Tomi Valkeinen <[email protected]> (cherry picked from commit 18a8f4c) Signed-off-by: Jacopo Mondi <[email protected]>

The v4l2_subdev_link_validate() function prints a one-time warning if it gets called on a link whose source or sink is not a subdev. As links get validated in the context of their sink, a call to the helper when the link's sink is not a subdev indicates that the driver has set its .link_validate() handler to v4l2_subdev_link_validate() on a non-subdev entity, which is a clear driver bug. On the other hand, the link's source not being a subdev indicates that the helper is used for a subdev connected to a video output device, which is a lesser issue, if only because this is currently common practice. There are no drivers left in the kernel that use v4l2_subdev_link_validate() in a context where it may get called on a non-subdev sink. Replace the pr_warn_once() with a WARN_ON_ONCE() in this case to make sure that new offenders won't be introduced. A subsequent change will improve the v4l2_subdev_link_validate() helper to properly support validating video device to subdev links. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Tomi Valkeinen <[email protected]> Acked-by: Sakari Ailus <[email protected]> (cherry picked from commit d130767) Signed-off-by: Jacopo Mondi <[email protected]>

The v4l2_subdev_link_validate() helper function is meant to be used as a drop-in implementation of a V4L2 subdev entity .link_validate() handler. It supports subdev-to-subdev links only, and complains if one end of the link is not a subdev. This forces drivers that have video output devices connected to subdevs to implement a custom .link_validate() handler, calling v4l2_subdev_link_validate() for the subdev-to-subdev links, and performing manual link validation for the video-to-subdev links. Video devices embed a media entity, and therefore also have a .link_validate() operation. For video capture devices, the operation should be manually implemented by drivers for validate the subdev-to-video links. For video output devices, on the other hand, that operation is never called, as link validation is performed in the context of the sink entity. As a result, we end up forcing drivers to implement a custom .link_validate() handler for subdevs connected to video output devices, when the video devices provide an operation that could be used for that purpose. To improve that situation, make v4l2_subdev_link_validate() delegate link validation to the source's .link_validate() operation when the link source is a video device and the link sink is a subdev. This allows broader usage of v4l2_subdev_link_validate(), and simplifies drivers by making video device link validation easy to implement in the video device .link_validate(), regardless of whether the video device is an output device or a capture device. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Tomi Valkeinen <[email protected]> Acked-by: Sakari Ailus <[email protected]> (cherry picked from commit 5fd3e24) Signed-off-by: Jacopo Mondi <[email protected]>

Since commit 7d3c7d2 ("media: i2c: Add a camera sensor top level menu") the CONFIG_MEDIA_CONTROLLER and CONFIG_VIDEO_V4L2_SUBDEV_API are selected by the top-level VIDEO_CAMERA_SENSOR menu. Remove all ifdefferies from camera sensor drivers to simplify the code. Compile-tested only. Signed-off-by: Jacopo Mondi <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit b4e9a2d) Signed-off-by: Jacopo Mondi <[email protected]>

The 'try_*' fields of the v4l2_subdev_pad_config structure are meant to be accessed through helper functions. Replace direct access with usage of the v4l2_subdev_get_pad_format(), v4l2_subdev_get_pad_crop() and v4l2_subdev_get_pad_compose() helpers. Signed-off-by: Laurent Pinchart <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Mauro Carvalho Chehab <[email protected]> (cherry picked from commit f4b7c07) Signed-off-by: Jacopo Mondi <[email protected]>

The 'try_*' fields of the v4l2_subdev_pad_config structure are meant to be accessed through helper functions. Replace direct access with usage of the v4l2_subdev_get_pad_format(), v4l2_subdev_get_pad_crop() and v4l2_subdev_get_pad_compose() helpers. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Luca Ceresoli <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Mauro Carvalho Chehab <[email protected]> (cherry picked from commit 0623979) Signed-off-by: Jacopo Mondi <[email protected]>

The 'try_*' fields of the v4l2_subdev_pad_config structure are meant to be accessed through helper functions. Replace direct access with usage of the v4l2_subdev_get_pad_format(), v4l2_subdev_get_pad_crop() and v4l2_subdev_get_pad_compose() helpers. Signed-off-by: Laurent Pinchart <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Mauro Carvalho Chehab <[email protected]> (cherry picked from commit f9c12d6) Signed-off-by: Jacopo Mondi <[email protected]>

The 'try_*' fields of the v4l2_subdev_pad_config structure are meant to be accessed through helper functions. Replace direct access with usage of the v4l2_subdev_get_pad_format(), v4l2_subdev_get_pad_crop() and v4l2_subdev_get_pad_compose() helpers. Signed-off-by: Laurent Pinchart <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Mauro Carvalho Chehab <[email protected]> (cherry picked from commit 098a1ee) Signed-off-by: Jacopo Mondi <[email protected]>

The 'try_*' fields of the v4l2_subdev_pad_config structure are meant to be accessed through helper functions. Replace direct access with usage of the v4l2_subdev_get_pad_format(), v4l2_subdev_get_pad_crop() and v4l2_subdev_get_pad_compose() helpers. Signed-off-by: Laurent Pinchart <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Mauro Carvalho Chehab <[email protected]> (cherry picked from commit 9295e7e) Signed-off-by: Jacopo Mondi <[email protected]>

ccs_register_subdev() did not clean up the media entity in error case, do that now. Also switch to goto based error handling. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit f055e53) Signed-off-by: Jacopo Mondi <[email protected]>

Use init_cfg() instead of manually setting up defaults in file handle open. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 3935665) Signed-off-by: Jacopo Mondi <[email protected]>

The ccs_create_subdev() function initialises a sub-device in the CCS driver, including CCS specific needs. Rename it as ccs_init_subdev() as it better reflects what the function does. Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit ac84b72) Signed-off-by: Jacopo Mondi <[email protected]>

The media entity will soon need to be initialised before the sub-device init finalisation that allocates memory for sub-device state. Do that now. Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 77ab9dc) Signed-off-by: Jacopo Mondi <[email protected]>

The available mbus codes will soon be needed earlier, at the time sub-devices are initialisaed. This is due to calling init_cfg() op via the v4l2_subdev_init_finalize(). Move ccs_get_mbus_formats() before ccs_init_subdev() calls. Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit f00a157) Signed-off-by: Jacopo Mondi <[email protected]>

Make use of sub-device active state. In most cases the effect on need for acquiring the mutex is non-existent as access to the driver's core data structure still needs to be serialised. This still removes a lot of code as the code paths for active and try state are the same in many cases. Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit d8bca3e) Signed-off-by: Jacopo Mondi <[email protected]>

The s_stream is called to enable and to disable streaming on a sub-device. The caller may only call it to change the state, enabling streaming is not allowed when it is already disabled, and similarly for disabling streaming. Remove the check from the CCS driver. Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 8a695a2) Signed-off-by: Jacopo Mondi <[email protected]>

Initialise sub-device state in init_cfg callback using ccs_propagate() to the extent it covers of the initialisation. This fixes a bug where the driver configuration was incorrectly initialised. Fixes: d8bca3e ("media: ccs: Use sub-device active state") Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 256b776) Signed-off-by: Jacopo Mondi <[email protected]>

The v4l2_subdev_init_finalize() is a macro that creates an unique lockdep key and name. As the CCS driver initialises all three of its sub-devices using the same call site, this creates a lockdep warning. Address it. Fixes: d8bca3e ("media: ccs: Use sub-device active state") Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 827804d) Signed-off-by: Jacopo Mondi <[email protected]>

If the sensor remains powered on after probe when it is needed again and the runtime PM usage_count is incremented, the control handler setup for neither control handler is run. As this is, in most cases, a needless operation in probe, track the status of sensor power handling after probe and set up control handlers even if the device was already active right after probe. Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit a274f4d) Signed-off-by: Jacopo Mondi <[email protected]>

Call media_entity_cleanup() in probe error path and remove to make sure the media entity is cleaned up properly. Suggested-by: Laurent Pinchart <[email protected]> Signed-off-by: Pratyush Yadav <[email protected]> Tested-by: Julien Massot <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Reviewed-by: Tomi Valkeinen <[email protected]> Reviewed-by: Maxime Ripard <[email protected]> Signed-off-by: Jai Luthra <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit aee5b41) Signed-off-by: Jacopo Mondi <[email protected]>

The format is needed to calculate the link speed for the external DPHY configuration. It is not right to query the format from the source subdev. Add get_fmt and set_fmt pad operations so that the format can be configured and correct bpp be selected. Initialize and use the v4l2 subdev active state to keep track of the active formats. Also propagate the new format from the sink pad to all the source pads. Signed-off-by: Pratyush Yadav <[email protected]> Tested-by: Julien Massot <[email protected]> Reviewed-by: Maxime Ripard <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Reviewed-by: Tomi Valkeinen <[email protected]> Co-developed-by: Jai Luthra <[email protected]> Signed-off-by: Jai Luthra <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit dbca7b3) Signed-off-by: Jacopo Mondi <[email protected]>

Use the V4L2 subdev active state API to store the active format and crop rectangle. This simplifies the driver not only by dropping the state stored in the rkisp1_csi structure, but also by replacing the ops_lock with the state lock. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Paul Elder <[email protected]> Reviewed-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit c7e2621) Signed-off-by: Jacopo Mondi <[email protected]>

Use the V4L2 subdev active state API to store the active format and crop rectangle. This simplifies the driver not only by dropping the state stored in the rkisp1_isp structure, but also by replacing the ops_lock with the state lock. The rkisp1_isp.sink_fmt field needs to be kept, as it is accessed from the stats interrupt handler. To simplify the rkisp1_isp_set_sink_fmt() implementation, the field is now set when starting the ISP, instead of when setting the format. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Paul Elder <[email protected]> Reviewed-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 2cce0a3) Signed-off-by: Jacopo Mondi <[email protected]>

Use the V4L2 subdev active state API to store the active format and crop rectangle. This simplifies the driver not only by dropping the state stored in the rkisp1_resizer structure, but also by replacing the ops_lock with the state lock. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Paul Elder <[email protected]> Reviewed-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 88f870f) Signed-off-by: Jacopo Mondi <[email protected]>

Pointers to v4l2_mbus_framefmt and v4l2_rect instances don't need to be modified when configuring the resizer. Make them const. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 1a6ae62) Signed-off-by: Jacopo Mondi <[email protected]>

The rkisp1_rsz_config() and rkisp1_rsz_config_regs() functions use a v4l2_rect to pass frame sizes, leaving the top and left members unused and uninitialized. Use v4l2_area instead. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 603fbdc) Signed-off-by: Jacopo Mondi <[email protected]>

The sink_y local variable in rkisp1_rsz_config() stores a copy of the sink_crop crop rectangle. Drop it, and rename sink_crop to sink_y. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 1126d89) Signed-off-by: Jacopo Mondi <[email protected]>

Switch all drivers accessing sub-device state to use the stream-aware functions. We will soon remove the old ones. This patch has been generated using the following Coccinelle script: ---------8<------------ @@ expression E1, E2, E3; @@ - v4l2_subdev_get_pad_format(E1, E2, E3) + v4l2_subdev_state_get_format(E2, E3) @@ expression E1, E2, E3; @@ - v4l2_subdev_get_pad_crop(E1, E2, E3) + v4l2_subdev_state_get_crop(E2, E3) @@ expression E1, E2, E3; @@ - v4l2_subdev_get_pad_compose(E1, E2, E3) + v4l2_subdev_state_get_compose(E2, E3) @@ expression E1, E2, E3; @@ - v4l2_subdev_get_try_format(E1, E2, E3) + v4l2_subdev_state_get_format(E2, E3) @@ expression E1, E2, E3; @@ - v4l2_subdev_get_try_crop(E1, E2, E3) + v4l2_subdev_state_get_crop(E2, E3) @@ expression E1, E2, E3; @@ - v4l2_subdev_get_try_compose(E1, E2, E3) + v4l2_subdev_state_get_compose(E2, E3) ---------8<------------ Additionally drivers/media/i2c/s5k5baf.c and drivers/media/platform/samsung/s3c-camif/camif-capture.c have been manually changed as Coccinelle didn't. Further local variables have been removed as they became unused as a result of the other changes. Also Coccinelle introduced indentation by space in files drivers/media/i2c/st-mipid02.c and drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c. This has been also corrected. The diff from Coccinelle-generated changes are: > diff --git b/drivers/media/i2c/imx319.c a/drivers/media/i2c/imx319.c > index e549692ff478..420984382173 100644 > --- b/drivers/media/i2c/imx319.c > +++ a/drivers/media/i2c/imx319.c > @@ -2001,7 +2001,6 @@ static int imx319_do_get_pad_format(struct imx319 *imx319, > struct v4l2_subdev_format *fmt) > { > struct v4l2_mbus_framefmt *framefmt; > - struct v4l2_subdev *sd = &imx319->sd; > > if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) { > framefmt = v4l2_subdev_state_get_format(sd_state, fmt->pad); > diff --git b/drivers/media/i2c/imx355.c a/drivers/media/i2c/imx355.c > index 96bdde685d65..e1b1d2fc79dd 100644 > --- b/drivers/media/i2c/imx355.c > +++ a/drivers/media/i2c/imx355.c > @@ -1299,7 +1299,6 @@ static int imx355_do_get_pad_format(struct imx355 *imx355, > struct v4l2_subdev_format *fmt) > { > struct v4l2_mbus_framefmt *framefmt; > - struct v4l2_subdev *sd = &imx355->sd; > > if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) { > framefmt = v4l2_subdev_state_get_format(sd_state, fmt->pad); > diff --git b/drivers/media/i2c/ov08x40.c a/drivers/media/i2c/ov08x40.c > index ca799bbcfdb7..abbb0b774d43 100644 > --- b/drivers/media/i2c/ov08x40.c > +++ a/drivers/media/i2c/ov08x40.c > @@ -2774,7 +2774,6 @@ static int ov08x40_do_get_pad_format(struct ov08x40 *ov08x, > struct v4l2_subdev_format *fmt) > { > struct v4l2_mbus_framefmt *framefmt; > - struct v4l2_subdev *sd = &ov08x->sd; > > if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) { > framefmt = v4l2_subdev_state_get_format(sd_state, fmt->pad); > diff --git b/drivers/media/i2c/ov13858.c a/drivers/media/i2c/ov13858.c > index 7816d9787c61..09387e335d80 100644 > --- b/drivers/media/i2c/ov13858.c > +++ a/drivers/media/i2c/ov13858.c > @@ -1316,7 +1316,6 @@ static int ov13858_do_get_pad_format(struct ov13858 *ov13858, > struct v4l2_subdev_format *fmt) > { > struct v4l2_mbus_framefmt *framefmt; > - struct v4l2_subdev *sd = &ov13858->sd; > > if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) { > framefmt = v4l2_subdev_state_get_format(sd_state, fmt->pad); > diff --git b/drivers/media/i2c/ov13b10.c a/drivers/media/i2c/ov13b10.c > index 268cd4b03f9c..c06411d5ee2b 100644 > --- b/drivers/media/i2c/ov13b10.c > +++ a/drivers/media/i2c/ov13b10.c > @@ -1001,7 +1001,6 @@ static int ov13b10_do_get_pad_format(struct ov13b10 *ov13b, > struct v4l2_subdev_format *fmt) > { > struct v4l2_mbus_framefmt *framefmt; > - struct v4l2_subdev *sd = &ov13b->sd; > > if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) { > framefmt = v4l2_subdev_state_get_format(sd_state, fmt->pad); > diff --git b/drivers/media/i2c/s5c73m3/s5c73m3-core.c a/drivers/media/i2c/s5c73m3/s5c73m3-core.c > index 47605e36bc60..8f9b5713daf7 100644 > --- b/drivers/media/i2c/s5c73m3/s5c73m3-core.c > +++ a/drivers/media/i2c/s5c73m3/s5c73m3-core.c > @@ -819,7 +819,6 @@ static void s5c73m3_oif_try_format(struct s5c73m3 *state, > struct v4l2_subdev_format *fmt, > const struct s5c73m3_frame_size **fs) > { > - struct v4l2_subdev *sd = &state->sensor_sd; > u32 code; > > switch (fmt->pad) { > diff --git a/drivers/media/i2c/s5k5baf.c b/drivers/media/i2c/s5k5baf.c > index 67da2045f543..03ccfb0e1e11 100644 > --- a/drivers/media/i2c/s5k5baf.c > +++ b/drivers/media/i2c/s5k5baf.c > @@ -1472,14 +1472,11 @@ static int s5k5baf_set_selection(struct v4l2_subdev *sd, > > if (sel->which == V4L2_SUBDEV_FORMAT_TRY) { > rects = (struct v4l2_rect * []) { > - &s5k5baf_cis_rect, > - v4l2_subdev_get_try_crop(sd, sd_state, > - PAD_CIS), > - v4l2_subdev_get_try_compose(sd, sd_state, > - PAD_CIS), > - v4l2_subdev_get_try_crop(sd, sd_state, > - PAD_OUT) > - }; > + &s5k5baf_cis_rect, > + v4l2_subdev_state_get_crop(sd_state, PAD_CIS), > + v4l2_subdev_state_get_compose(sd_state, PAD_CIS), > + v4l2_subdev_state_get_crop(sd_state, PAD_OUT) > + }; > s5k5baf_set_rect_and_adjust(rects, rtype, &sel->r); > return 0; > } > diff --git b/drivers/media/platform/samsung/s3c-camif/camif-capture.c a/drivers/media/platform/samsung/s3c-camif/camif-capture.c > index 295e083f38e8..be58260ea67e 100644 > --- b/drivers/media/platform/samsung/s3c-camif/camif-capture.c > +++ a/drivers/media/platform/samsung/s3c-camif/camif-capture.c > @@ -1216,7 +1216,7 @@ static int s3c_camif_subdev_get_fmt(struct v4l2_subdev *sd, > struct v4l2_mbus_framefmt *mf = &fmt->format; > > if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) { > - mf = v4l2_subdev_get_try_format(sd, sd_state, fmt->pad); > + mf = v4l2_subdev_state_get_format(sd_state, fmt->pad); > fmt->format = *mf; > return 0; > } > @@ -1305,7 +1305,7 @@ static int s3c_camif_subdev_set_fmt(struct v4l2_subdev *sd, > __camif_subdev_try_format(camif, mf, fmt->pad); > > if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) { > - mf = v4l2_subdev_get_try_format(sd, sd_state, fmt->pad); > + mf = v4l2_subdev_state_get_format(sd_state, fmt->pad); > *mf = fmt->format; > mutex_unlock(&camif->lock); > return 0; > diff --git b/drivers/media/platform/ti/cal/cal-camerarx.c a/drivers/media/platform/ti/cal/cal-camerarx.c > index cea454ed9c20..61433744c6c4 100644 > --- b/drivers/media/platform/ti/cal/cal-camerarx.c > +++ a/drivers/media/platform/ti/cal/cal-camerarx.c > @@ -621,8 +621,6 @@ static int cal_camerarx_sd_enum_mbus_code(struct v4l2_subdev *sd, > struct v4l2_subdev_state *state, > struct v4l2_subdev_mbus_code_enum *code) > { > - struct cal_camerarx *phy = to_cal_camerarx(sd); > - > /* No transcoding, source and sink codes must match. */ > if (cal_rx_pad_is_source(code->pad)) { > struct v4l2_mbus_framefmt *fmt; > diff --git b/drivers/staging/media/imx/imx-ic-prp.c a/drivers/staging/media/imx/imx-ic-prp.c > index dd558fac6477..61d69f19657e 100644 > --- b/drivers/staging/media/imx/imx-ic-prp.c > +++ a/drivers/staging/media/imx/imx-ic-prp.c > @@ -82,8 +82,6 @@ static struct v4l2_mbus_framefmt * > __prp_get_fmt(struct prp_priv *priv, struct v4l2_subdev_state *sd_state, > unsigned int pad, enum v4l2_subdev_format_whence which) > { > - struct imx_ic_priv *ic_priv = priv->ic_priv; > - > if (which == V4L2_SUBDEV_FORMAT_TRY) > return v4l2_subdev_state_get_format(sd_state, pad); > else > diff --git b/drivers/staging/media/imx/imx-ic-prpencvf.c a/drivers/staging/media/imx/imx-ic-prpencvf.c > index 02db7dbb884b..ec73c901079e 100644 > --- b/drivers/staging/media/imx/imx-ic-prpencvf.c > +++ a/drivers/staging/media/imx/imx-ic-prpencvf.c > @@ -790,8 +790,6 @@ static struct v4l2_mbus_framefmt * > __prp_get_fmt(struct prp_priv *priv, struct v4l2_subdev_state *sd_state, > unsigned int pad, enum v4l2_subdev_format_whence which) > { > - struct imx_ic_priv *ic_priv = priv->ic_priv; > - > if (which == V4L2_SUBDEV_FORMAT_TRY) > return v4l2_subdev_state_get_format(sd_state, pad); > else > diff --git a/drivers/media/i2c/st-mipid02.c b/drivers/media/i2c/st-mipid02.c > index 9c9361354c00..b08a249b5fdd 100644 > --- a/drivers/media/i2c/st-mipid02.c > +++ b/drivers/media/i2c/st-mipid02.c > @@ -751,7 +751,7 @@ static void mipid02_set_fmt_source(struct v4l2_subdev *sd, > format->format = bridge->fmt; > else > format->format = *v4l2_subdev_state_get_format(sd_state, > - MIPID02_SINK_0); > + MIPID02_SINK_0); > > /* but code may need to be converted */ > format->format.code = serial_to_parallel_code(format->format.code); > diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c > index 117912d3bfbd..96353648c032 100644 > --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c > +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c > @@ -319,7 +319,7 @@ static void rkisp1_isp_start(struct rkisp1_isp *isp, > rkisp1_write(rkisp1, RKISP1_CIF_ISP_CTRL, val); > > src_fmt = v4l2_subdev_state_get_format(sd_state, > - RKISP1_ISP_PAD_SOURCE_VIDEO); > + RKISP1_ISP_PAD_SOURCE_VIDEO); > src_info = rkisp1_mbus_info_get_by_code(src_fmt->code); > > if (src_info->pixel_enc != V4L2_PIXEL_ENC_BAYER) > @@ -475,9 +475,9 @@ static void rkisp1_isp_set_src_fmt(struct rkisp1_isp *isp, > sink_fmt = v4l2_subdev_state_get_format(sd_state, > RKISP1_ISP_PAD_SINK_VIDEO); > src_fmt = v4l2_subdev_state_get_format(sd_state, > - RKISP1_ISP_PAD_SOURCE_VIDEO); > + RKISP1_ISP_PAD_SOURCE_VIDEO); > src_crop = v4l2_subdev_state_get_crop(sd_state, > - RKISP1_ISP_PAD_SOURCE_VIDEO); > + RKISP1_ISP_PAD_SOURCE_VIDEO); > > /* > * Media bus code. The ISP can operate in pass-through mode (Bayer in, Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Mauro Carvalho Chehab <[email protected]> (cherry picked from commit 5bae63e) Signed-off-by: Jacopo Mondi <[email protected]> --- Conflicts: drivers/media/platform/rockchip/rkisp1/rkisp1-csi.c drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c drivers/media/platform/rockchip/rkisp1/rkisp1-resizer.c Patch not applied on the above entries Signed-off-by: Jacopo Mondi <[email protected]>

For the purpose of setting old non-pad based sub-device try format as a basis for VIDIOC_TRY_FMT implementation, there is no need to set the format in the sub-device state. Drop the assignment to the state, which would result in a NULL pointer dereference. Fixes: fd17e3a ("media: i2c: Use accessors for pad config 'try_*' fields") Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit c692696) Signed-off-by: Jacopo Mondi <[email protected]>

For the purpose of setting old non-pad based sub-device try format as a basis for VIDIOC_TRY_FMT implementation, there is no need to set the format in the sub-device state. Drop the assignment to the state, which would result in a NULL pointer dereference. Fixes: fd17e3a ("media: i2c: Use accessors for pad config 'try_*' fields") Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit dff1eeb) Signed-off-by: Jacopo Mondi <[email protected]>

For the purpose of setting old non-pad based sub-device try format as a basis for VIDIOC_TRY_FMT implementation, there is no need to set the format in the sub-device state. Drop the assignment to the state, which would result in a NULL pointer dereference. Fixes: fd17e3a ("media: i2c: Use accessors for pad config 'try_*' fields") Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 72c8cb4) Signed-off-by: Jacopo Mondi <[email protected]>

For the purpose of setting old non-pad based sub-device try format as a basis for VIDIOC_TRY_FMT implementation, there is no need to set the format in the sub-device state. Drop the assignment to the state, which would result in a NULL pointer dereference. Fixes: fd17e3a ("media: i2c: Use accessors for pad config 'try_*' fields") Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 09aee39) Signed-off-by: Jacopo Mondi <[email protected]>

For the purpose of setting old non-pad based sub-device try format as a basis for VIDIOC_TRY_FMT implementation, there is no need to set the format in the sub-device state. Drop the assignment to the state, which would result in a NULL pointer dereference. Fixes: fd17e3a ("media: i2c: Use accessors for pad config 'try_*' fields") Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 843750f) Signed-off-by: Jacopo Mondi <[email protected]>

For the purpose of setting old non-pad based sub-device try format as a basis for VIDIOC_TRY_FMT implementation, there is no need to set the format in the sub-device state. Drop the assignment to the state, which would result in a NULL pointer dereference. Fixes: fd17e3a ("media: i2c: Use accessors for pad config 'try_*' fields") Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit e55a948) Signed-off-by: Jacopo Mondi <[email protected]>

The subdev .s_stream() operation shall not be called to start streaming on an already started subdev, or stop streaming on a stopped subdev. Remove the check that guards against that condition. Signed-off-by: Laurent Pinchart <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 58e7ab2) Signed-off-by: Jacopo Mondi <[email protected]>

Stopping streaming on a camera pipeline at system suspend time, and restarting it at system resume time, requires coordinated action between the bridge driver and the camera sensor driver. This is handled by the bridge driver calling the sensor's .s_stream() handler at system suspend and resume time. There is thus no need for the sensor to independently implement system sleep PM operations. Drop them. The streaming field of the driver's private structure is now unused, drop it as well. Signed-off-by: Laurent Pinchart <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 55e941d) Signed-off-by: Jacopo Mondi <[email protected]>

Enable runtime PM before registering the async subdev as the driver UAPI may become accessible immediately after the registration. Runtime PM needs to be enabled by that time. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 1cfe77a) Signed-off-by: Jacopo Mondi <[email protected]>

Use sub-device active state. Rely on control handler lock to serialise access to the active state. Also clean up locking on s_stream handler. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 289c259) Signed-off-by: Jacopo Mondi <[email protected]>

With ipu bridge, endpoints may only be created when ipu bridge has initialised. This may happen after the sensor driver has first probed. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 07d81b5) Signed-off-by: Jacopo Mondi <[email protected]>

The subdev .s_stream() operation shall not be called to start streaming on an already started subdev, or stop streaming on a stopped subdev. Remove the check that guards against that condition. The streaming field of the driver's private structure is now unused, drop it as well. Signed-off-by: Laurent Pinchart <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 36cb37a) Signed-off-by: Jacopo Mondi <[email protected]>

V4L2 subdev operations have moved from operating on a v4l2_subdev_pad_config to a v4l2_subdev_state a long time ago. Fix the few remaining incorrect references to pad config in the I2C drivers. Signed-off-by: Laurent Pinchart <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Mauro Carvalho Chehab <[email protected]> (cherry picked from commit aeb18af) Signed-off-by: Jacopo Mondi <[email protected]>

The subdev .init_cfg() operation is affected by two issues: - It has long been extended to initialize a whole v4l2_subdev_state instead of just a v4l2_subdev_pad_config, but its name has stuck around. - Despite operating on a whole subdev state and not being directly exposed to the subdev users (either in-kernel or through the userspace API), .init_cfg() is categorized as a subdev pad operation. This participates in making the subdev API confusing for new developers. Fix it by renaming the operation to .init_state(), and make it a subdev internal operation. Signed-off-by: Laurent Pinchart <[email protected]> Acked-by: Michael Riesch <[email protected]> # for imx415 Acked-by: Shuah Khan <[email protected]> # for vimc Reviewed-by: Philipp Zabel <[email protected]> Reviewed-by: Tomi Valkeinen <[email protected]> [Sakari Ailus: Resolved a conflict in Renesas vsp1 driver.] Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]> (cherry picked from commit 5755be5) Conflicts: drivers/media/i2c/gc2145.c drivers/media/i2c/imx219.c drivers/media/i2c/mt9m114.c drivers/media/i2c/mt9v111.c drivers/media/i2c/thp7312.c drivers/media/platform/cadence/cdns-csi2rx.c drivers/media/platform/renesas/vsp1/vsp1_entity.c drivers/media/platform/xilinx/xilinx-csi2rxss.c drivers/staging/media/starfive/camss/stf-isp.c Signed-off-by: Jacopo Mondi <[email protected]>

Having extra streams on the source end of the link that cannot be captured by the sink sub-device generally are not an issue, at least not on CSI-2 bus. Still document that there may be hardware-specific limitations. For example on parallel bus this might not work on all cases. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Tomi Valkeinen <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]>

Document internal sink pads, pads that have both SINK and INTERNAL flags set. Use the IMX219 camera sensor as an example. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by Julien Massot <[email protected]>

Add new MIPI CCS embedded data media bus code (MEDIA_BUS_FMT_CCS_EMBEDDED). Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Julien Massot <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]>

Document which routes are available for the CCS driver (source) sub-device and what configuration are possible. Also update copyright. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]>

Add a media bus code for ov2740 camera sensor embedded data and document it. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Julien Massot <[email protected]>

Document S_ROUTING behaviour for different devices. Generally in devices that produce streams the streams are static and some can be enabled and disabled, whereas in devices that just transport them or write them to memory, more configurability is allowed. Document this. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Julien Massot <[email protected]>

Move G_ROUTING IOCTL handling below that of S_ROUTING. G_ROUTING implementation will soon needed in handling S_ROUTING as well. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Julien Massot <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]>

Internal sink pads will be used as routing endpoints in V4L2 [GS]_ROUTING IOCTLs, to indicate that the stream begins in the entity. Internal sink pads are pads that have both SINK and INTERNAL flags set. Also prevent creating links to pads that have been flagged as internal and initialising SOURCE pads with INTERNAL flag set. Signed-off-by: Sakari Ailus <[email protected]> Reviewed-by: Tomi Valkeinen <[email protected]> Reviewed-by: Laurent Pinchart <[email protected]>

Add a flag to denote immutable routes, V4L2_SUBDEV_ROUTE_FL_IMMUTABLE. Such routes cannot be changed and they're always active. Signed-off-by: Sakari Ailus <[email protected]>

Signed-off-by: Tomi Valkeinen <[email protected]> Signed-off-by: Jacopo Mondi <[email protected]>

Add two meta formats for PiSP FE: V4L2_META_FMT_RPI_FE_CFG and V4L2_META_FMT_RPI_FE_STATS. The former is used to provide configuration for the FE and the latter is used to read the statistics from the FE. Signed-off-by: Tomi Valkeinen <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Mauro Carvalho Chehab <[email protected]> (cherry picked from commit 6390834c6f9b2c5e33f52f34579efa0d0df073db) Signed-off-by: Jacopo Mondi <[email protected]>

Add DT bindings for raspberrypi,rp1-cfe. Reviewed-by: Krzysztof Kozlowski <[email protected]> Signed-off-by: Tomi Valkeinen <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Mauro Carvalho Chehab <[email protected]> (cherry picked from commit 1358bb523949e2dd28f2396eb59707151fb79479) Signed-off-by: Jacopo Mondi <[email protected]>

Add support for Raspberry Pi CFE. The CFE is a hardware block that contains: - MIPI D-PHY - MIPI CSI-2 receiver - Front End ISP (FE) The driver has been upported from the Raspberry Pi kernel commit 88a681d ("ARM: dts: bcm2712-rpi: Add i2c<n>_pins labels"). Co-developed-by: Naushir Patuck <[email protected]> Signed-off-by: Naushir Patuck <[email protected]> Signed-off-by: Tomi Valkeinen <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Mauro Carvalho Chehab <[email protected]> (cherry picked from commit 6edb685abb2af445773876a326292b989dcb3c9f) Signed-off-by: Jacopo Mondi <[email protected]>

(cherry picked from commit b0667aaac1e485efe0c8d9461c60a46e0a0984ec) Signed-off-by: Jacopo Mondi <[email protected]>

Add documentation for rp1-cfe driver. Signed-off-by: Tomi Valkeinen <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Mauro Carvalho Chehab <[email protected]> (cherry picked from commit 40249b1d5b3cfea2e8eadd4b5777cf2d82b86d21) Signed-off-by: Jacopo Mondi <[email protected]>

The mainline driver has support for v4l2 streams, which is used to capture two streams from the sensor (image and embedded data) Replace the BSP driver in drivers/media/platform/bcm2835 with the mainline version, checked-out at revision 8771b7f ("media: bcm2835-unicam: Depend on COMMON_CLK"). Signed-off-by: Jai Luthra <[email protected]> Signed-off-by: Jacopo Mondi <[email protected]>

Fix a few minor issues to allow the mainline driver version to compile on the v6.6.y RaspberryPi BSP. Signed-off-by: Jacopo Mondi <[email protected]>

Ensure that the frame sequence counter is incremented only if a previous frame start interrupt has occurred, or a frame start + frame end has occurred simultaneously. This corresponds the sequence number with the actual number of frames produced by the sensor, not the number of frame buffers dequeued back to userland. Signed-off-by: Naushir Patuck <[email protected]> (cherry picked from commit ea9953f) Signed-off-by: Jacopo Mondi <[email protected]>

The legacy stack had an option to have a GPIO track frame start and end events to give basic synchronisation to the incoming image stream. https://forums.raspberrypi.com/viewtopic.php?t=190314 Replicate this in the kernel Unicam driver. Signed-off-by: Dave Stevenson <[email protected]> (cherry picked from commit 6a0ae67) Signed-off-by: Jacopo Mondi <[email protected]>

V4L2_CID_USER_BCM2835_ISP_BASE is defined in RPi BSP as (V4L2_CID_USER_BASE + 0x10e0) However in mainline this address has been occupied by other control bases. Align V4L2_CID_USER_BCM2835_ISP_BASE to the last available identifier available in mainline. Signed-off-by: Jai Luthra <[email protected]> Signed-off-by: Jacopo Mondi <[email protected]>

The unicam driver from mainline expects additional required properties: reg-names and brcm,num-data-lanes. Signed-off-by: Jai Luthra <[email protected]>

This reverts commit ccb3449.

…1232p" This reverts commit 24e8a7d.

…0 mode" This reverts commit caebe4f.

This reverts commit 41d7a5a.

This reverts commit 1ca6523.

This reverts commit 4b249d9.

The subdev .s_stream() operation shall not be called to start streaming on an already started subdev, or stop streaming on a stopped subdev. Remove the check that guards against that condition. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Dave Stevenson <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]>

Stopping streaming on a camera pipeline at system suspend time, and restarting it at system resume time, requires coordinated action between the bridge driver and the camera sensor driver. This is handled by the bridge driver calling the sensor's .s_stream() handler at system suspend and resume time. There is thus no need for the sensor to independently implement system sleep PM operations. Drop them. The streaming field of the driver's private structure is now unused, drop it as well. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Dave Stevenson <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]>

Drop a handful of macros that are not used and don't provide any value. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Dave Stevenson <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]>

The 640x480 mode specifies incorrect values for the TP_WINDOW_WIDTH and TP_WINDOW_HEIGHT registers, which likely got copied from the 1640x1232 mode. They should be identical to the X_OUTPUT_SIZE and Y_OUTPUT_SIZE registers as for all the other modes, to avoid cropping the test pattern. Fix them. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Dave Stevenson <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]>

Replace the per-mode register arrays with code that sets the same register values using the mode definitions. This avoids duplicating information in two different places. The error check for invalid formats in imx219_set_framefmt() is dropped as the format is guaranteed to be valid. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Dave Stevenson <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]>

The imx219_set_binning() function sets registers based on the bpp value, which is computed in imx219_set_framefmt(). As both functions are called from the same place consecutively, and set registers based on the selected mode, merge them together to simplify the code. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Dave Stevenson <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]>

While the ycbcr_enc field doesn't apply to raw formats, leaving it uninitialized makes the driver behave in a less deterministic way. Fix it by picking the default value for the colorspace. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Dave Stevenson <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]>

Configure the crop-related registers from the values stored in the active crop rectangle instead of the mode structure. This removes usage of the mode from the imx219_set_framefmt(). No functional change is intended. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Jacopo Mondi <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]>

Compare the format and crop rectangle dimensions to infer binning settings, instead of storing the binning mode in the imx219_mode structure. This removes duplicate information from the mode. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Jacopo Mondi <[email protected]> Reviewed-by: Dave Stevenson <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]>

Use the active format height instead of the mode height in imx219_set_ctrl(). This prepares for dropping the mode field from the imx219 structure. The state is retrieved using v4l2_subdev_get_locked_active_state() as the subdev active state and the control handler share the same lock. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Jacopo Mondi <[email protected]> Reviewed-by: Dave Stevenson <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]>

The mode field of the imx219 structure is only used in imx219_init_controls(), after the probe function sets it to point to the default mode. Use the default mode directly when initializing controls, and drop the mode field from the imx219 structure. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Jacopo Mondi <[email protected]> Reviewed-by: Dave Stevenson <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]>

The IMX219_VTS_* macros define default VTS values for the modes supported by the driver. They are used in a single place, and hinder readability compared to using the value directly as a decimal number. Drop them. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Dave Stevenson <[email protected]> Reviewed-by: Jacopo Mondi <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]>

Move functions around to group them by purpose, in order to improve readability. No functional change is intended. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Dave Stevenson <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]>

Simplify the imx219_set_stream() by removing an unneeded goto statement, and its corresponding error label. The natural flow of the function is correct. While at it, drop a useless comment. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Dave Stevenson <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]>

Instead of duplicating the logical implemented in the .set_fmt() operation in .init_cfg(), call .set_fmt() directly. This centralizes the format and crop rectangle calculations in a single place. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Dave Stevenson <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]>

The IMX219 has distinct binning registers for the horizontal and vertical directions. Calculate their value and write them separately. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Jacopo Mondi <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]>

Calculate the crop rectangle size and location dynamically when setting the format, instead of storing it in the imx219_mode structure. This removes duplicated information from the mode, to guarantee consistency. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Jacopo Mondi <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]>

Subdev state variables are named with a mix of 'state' and 'sd_state' through the driver. To improve consistency, name them all 'state'. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Dave Stevenson <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]>

The exposure_max, exposure_def and hblank variables are only used in an inner scope in the imx219_set_pad_format() function. Move them to that scope to keep them closer to their usage and improve readability. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Dave Stevenson <[email protected]> Signed-off-by: Sakari Ailus <[email protected]> Signed-off-by: Hans Verkuil <[email protected]>

The imx219_update_pad_format() is short and called from a single place, in imx219_set_pad_format(). Inline the code in the caller to keep all format adjustments grouped in a single place and improve readability. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Jacopo Mondi <[email protected]> Reviewed-by: Dave Stevenson <[email protected]>

Use the newly added internal pad API to expose the internal configuration of the sensor to userspace in a standard manner. This also paves the way for adding support for embedded data with an additional internal pad. To maintain compatibility with existing userspace that may operate on pad 0 unconditionally, keep the source pad numbered 0 and number the internal image pad 1. Signed-off-by: Laurent Pinchart <[email protected]> --- Changes since v7: - Set IMX219_PAD_SOURCE to 0 explicitly - Update comment to clarify the whole image pad format is fixed - Compare code->index with 0 - Drop unneeded parentheses

In preparation for embedded data stream support, introduce a new imx219_stream_ids enumeration for stream IDs, with a single value, IMX219_STREAM_IMAGE for the image data stream. Use it when accessing the formats, crop and compose rectangles on the source pad. This is meant to reduce the size of further commits, and doesn't introduce any functional change. Signed-off-by: Laurent Pinchart <[email protected]>

The imx219 driver implements the .s_stream() operation, which is not compatible with streams. In preparation for the addition of streams support in the driver, switch to the .enable_streams() and .disable_streams() operations. Signed-off-by: Laurent Pinchart <[email protected]>

Usage of internal pads creates a route internal to the subdev, and the V4L2 camera sensor API requires such routes to be reported to userspace. Create the route in the .init_state() operation. Internal routing support requires stream support, so set the V4L2_SUBDEV_FL_STREAMS flag. As the route is immutable, there's no need to implement the .set_routing() operation. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Jacopo Mondi <[email protected]> --- Changes since v7: - Update commit message to match the changes in v6 - Fix name of STREAMS flag in commit message - Use IMX219_STREAM_IMAGE Changes since v6: - Drop change to get format API in imx219_set_ctrl() - Fix function name in commit message - Set V4L2_SUBDEV_ROUTE_FL_IMMUTABLE flag on route

Implement the .get_frame_desc() subdev operation to report information about streams to the connected CSI-2 receiver. This is required to let the CSI-2 receiver driver know about virtual channels and data types for each stream. Signed-off-by: Laurent Pinchart <[email protected]> Reviewed-by: Jacopo Mondi <[email protected]> --- Changes since v7: - Use IMX219_STREAM_IMAGE - Drop unneeded memset() Changes since v6: - Replace v4l2_subdev_state_get_stream_format() with v4l2_subdev_state_get_format() - Make imx219_format_bpp() return an unsigned int

The IMX219 generates embedded data unconditionally. Report it as an additional stream, with a new internal embedded data pad, and update subdev operations accordingly. Signed-off-by: Laurent Pinchart <[email protected]> --- Changes since v7: - Compare fse->index with 0 in imx219_enum_frame_size() - Don't set route array size explicitly in imx219_init_state() - Implement imx219_format_edata() on top of imx219_format_bpp() - Handle streams in .{enable,disable}_streams() Changes since v6: - Get format from IMX219_STREAM_IMAGE in imx219_set_ctrl() - Fix mbus code for second stream in imx219_get_frame_desc() - Set V4L2_SUBDEV_ROUTE_FL_IMMUTABLE flag on route

The datasheet for this sensor documents the minimum vblanking as being 32 lines. It does fix some problems with occasional black lines at the bottom of images (tested on Raspberry Pi). Signed-off-by: David Plowman <[email protected]>

The HBLANK control was read-only, and always configured such that the sensor HTS register was 3448. This limited the maximum exposure time that could be achieved to around 1.26 secs. Make HBLANK read/write so that the line time can be extended, and thereby allow longer exposures (and slower frame rates). Retain the overall HTS setting when changing modes rather than resetting it to a default. Signed-off-by: Dave Stevenson <[email protected]> Signed-off-by: Jai Luthra <[email protected]>

The 640x480 mode uses a special binning mode for high framerate operation where the pixel rate is effectively doubled. Account for this when setting up the pixel clock rate, and applying the vblank and exposure controls. Signed-off-by: Naushir Patuck <[email protected]> Signed-off-by: Jai Luthra <[email protected]>

HTS was still using the raw register ID. Fixes: dd26d43 ("media: i2c: imx219: make HBLANK r/w to allow longer exposures") Signed-off-by: Dave Stevenson <[email protected]>

At a high FPS with RAW10, there is frame corruption for 480p because the rate_factor of 2 is used with the normal 2x2 bining [1]. This commit ties the rate_factor to the selected binning mode. For the 480p mode, analog 2x2 binning mode with a rate_factor of 2 is always used. For the 1232p mode the normal 2x2 binning mode is used for RAW10 while analog 2x2 binning mode is used for RAW8. [1] raspberrypi#5493 Signed-off-by: Vinay Varma <[email protected]> Reworked due to upstream changes Signed-off-by: Dave Stevenson <[email protected]> Reworked again due to upstream changes Signed-off-by: Jai Luthra <[email protected]>

Add the V4L2 streams API support for embedded data in the IMX477 device driver. This also updates the drier to use the V4L2 subdev state API. Signed-off-by: Naushir Patuck <[email protected]> Signed-off-by: Jacopo Mondi <[email protected]>

The .s_stream() operation is not compatible with streams, so switch to the .{enable,disable}_streams() operations. Signed-off-by: Jai Luthra <[email protected]>

Add the V4L2 streams API support for embedded/pdaf/hdr data in the IMX708 device driver. This also updates the drier to use the V4L2 subdev state API. Signed-off-by: Naushir Patuck <[email protected]> Signed-off-by: Jacopo Mondi <[email protected]>

The .s_stream() operation is not compatible with streams, so switch to the .{enable,disable}_streams() operations. Signed-off-by: Jai Luthra <[email protected]>

Add the V4L2 streams API support for embedded data in the IMX519 device driver. This also updates the drier to use the V4L2 subdev state API. Signed-off-by: Naushir Patuck <[email protected]> Signed-off-by: Jacopo Mondi <[email protected]>

The .s_stream() operation is not compatible with streams, so switch to the .{enable,disable}_streams() operations. Signed-off-by: Jai Luthra <[email protected]>

With the introduction of the streams API, the legacy embedded data support has been removed. Pull out all code associated with the legacy support. Note that this driver has not been converted to use the streams API, so embedded data will be unavailable. Signed-off-by: Naushir Patuck <[email protected]> Signed-off-by: Jacopo Mondi <[email protected]>

Remove v4l2_subdev_enable_streams_api variable that was used to easily enable streams API for development, and conditions that use the variable. This patch enables the streams API for V4L2 sub-device interface which allows transporting multiple streams on a single MC link. Signed-off-by: Sakari Ailus <[email protected]>

When matching formats via try_fmt/set_fmt ioctls, test for the unpacked formats as well as packed formats. This allows userland clients setup unpacking to 16-bits from the 10/12/14-packed CSI2 formats. Signed-off-by: Naushir Patuck <[email protected]>

The imx219/imx708 sensors frequently generate a single corrupt frame (image or embedded data) when the sensor first starts. This can either be a missing line, or invalid samples within the line. This only occurrs using the upstream Unicam kernel driver. Disabling trigger mode elimiates this corruption. Since trigger mode is a legacy feature copied from the firmware driver and not expected to be needed, remove it. Tested on the Raspberry Pi cameras and shows no ill effects. Signed-off-by: Naushir Patuck <[email protected]>

The Unicam hardware has been observed to cause a buffer overrun when using the dummy buffer as a circular buffer. The conditions that cause the overrun are not fully known, but it seems to occur when the memory bus is heavily loaded. To avoid the overrun, program the hardware with a buffer size of 0 when using the dummy buffer. This will cause overrun into the allocated dummy buffer, but avoid out of bounds writes. Signed-off-by: Naushir Patuck <[email protected]>

This change aligns the FS/FE interrupt handling with the Raspberry Pi kernel downstream Unicam driver. If we get a simultaneous FS + FE interrupt for the same frame, it cannot be marked as completed and returned to userland as the framebuffer will be refilled by Unicam on the next sensor frame. Additionally, the timestamp will be set to 0 as the FS interrupt handling code will not have run yet. To avoid these problems, the frame is considered dropped in the FE handler, and will be returned to userland on the subsequent sensor frame. Signed-off-by: Naushir Patuck <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rpi/embedded data support/rpi 6.6.y #6437

Rpi/embedded data support/rpi 6.6.y #6437

Commits on Nov 21, 2024

Rpi/embedded data support/rpi 6.6.y #6437

Are you sure you want to change the base?

Rpi/embedded data support/rpi 6.6.y #6437

Commits on Nov 21, 2024