[BUG] Koheesio 0.9.0 and Spark Connect: delta.connect present only on Databricks

[YevIgn]

Bug Description

When testing new version of Koheesio - #97

And issue is observed when using DeltaTableWriter.

--> 257 raise ValueError(
    258     "Provide `merge_builder` in "
    259     "DeltaTableWriter(output_mode_params={'merge_builder': DeltaMergeBuilder()}). "
    260     "See documentation for options."
    261 )

The problem stems from the code

koheesio/src/koheesio/spark/writers/delta/batch.py

Line 251 in 6d6ccbd

if isinstance(merge_builder, DeltaMergeBuilder):

which checks specifically for delta.tables.DeltaMergeBuilder instance.

As there is no delta.connect.tables module in open-source delta-io package releases at the moment (2024.11.11) the possible way to resolve this issue is to check for class name or fully-qualified class name without connect part instead of using ininstance check as connect classes aren't subclasses of their regular counterparts.

Expected behavior

Using DeltaMergeBuilder doesn't fail if code is executed in Spark Connect mode on Databricks and DBR 15.4+

Environment

• DBR 15.4
• Koheesio 0.9.0rc0
  -->

Additional context

Same problem can happen with other classes from same namespaces delta.tables and delta.connect.tables in case Pydantic or other ways of type checking will check for first classes.

[dannymeijer]

#100 should fix this

[YevIgn]

Validated on 0.9.0rc6, issue is no longer present.