Plugin Development Kit: Connectors

[marioortizmanero]

Pull request

Description

This Pull Request is my first approach to integrating the new Plugin Development Kit (#791) for the connectors artifact. It requires a ton of changes because it moves every single connector into a separate crate, and rewrites part of the internals so that they can be used safely through the FFI boundary.

Related

• RFC
• Related Issues: Plugin Development Kit ( PDK ) #791 Connectors Implementation  #1077
• Related docs: https://nullderef.com/series/rust-plugins/

Checklist

• The RFC, if required, has been submitted and approved
• Any user-facing impact of the changes is reflected in docs.tremor.rs
• The code is tested
• Use of unsafe code is reasoned about in a comment
• Update CHANGELOG.md appropriately, recording any changes, bug fixes, or other observable changes in behavior
• The performance impact of the change is measured (see below)

Custom checklist:

• Panicking or aborting?
• Cleaning up simd_json::Value in OpMeta
• Supporting StableAbi derive in value_trait

Performance

I do have a few benchmarks so far in https://github.com/marioortizmanero/pdk-experiments, which I will analyze in a new post on nullderef.com. These do not represent a real usage of Tremor, however, so I will make a few more once everything is working properly.

[marioortizmanero]

This is very WIP yet so I do not suggest reviewing the code at this moment. I will let the team know once it's at least working.

[marioortizmanero]

Closing in favor of marioortizmanero#1 (which I'm not sure how to make over this repo instead of over my fork)