Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add prefix matching expression #49

Closed
rdblue opened this issue Aug 8, 2018 · 5 comments
Closed

Add prefix matching expression #49

rdblue opened this issue Aug 8, 2018 · 5 comments

Comments

@rdblue
Copy link
Contributor

rdblue commented Aug 8, 2018

Some users have requested prefix matching or startsWith.

@Liorba
Copy link

Liorba commented Oct 21, 2018

@rdblue can please give a bit more context on this issue?
is this prefix matching on partition transformation?
or is it new kind of Expression?

@Liorba
Copy link

Liorba commented Oct 22, 2018

working on this!

@Liorba
Copy link

Liorba commented Oct 22, 2018

@rdblue I tried to implement the startWith expression and I struggle with the fact that there is not place in the code I can assume that I'm working with strings.
all other Operation implementation at least in BoundExpressionVisitor, seem to rely on the comparator function implemented in types. which imply order.
startsWith should have to know that the literal and field types are both strings and use implantation other than compare.
if I do it like this the code will not compile, how do you suggest i'll add this?

@rdblue
Copy link
Contributor Author

rdblue commented Oct 22, 2018

I'd start with adding a new factory method in Expressions that accepts a String. That will create an unbound predicate with a String literal, so you know at least that type. An unbound predicate may still be invalid.

Binding is the process of looking up the column in a schema to validate that it is there and then updating the expression to work with that column's type. For other predicates, the literal is coerced to the correct type in UnboundPredicate#bind by calling the literal's to method and passing in the column type. But in this case, you'd update bind to handle the startsWith expression and validate that both the column and the literal are types supported by the predicate. The bound predicate that is created by bind guarantees that types are coherent and that operations can be performed on those types.

To implement the startsWith predicate in an evaluator, you'd add it to the bound expression visitor and add eval implementations. Those implementations will operate on the guarantees from the binding process.

You should also implement at least inclusive projection in the transforms, but I can help you with that when you're close to that problem.

@rdblue
Copy link
Contributor Author

rdblue commented Dec 7, 2018

I've moved this issue to apache/iceberg#31. Please continue discussion there. Thanks!

@rdblue rdblue closed this as completed Dec 7, 2018
Parth-Brahmbhatt pushed a commit to Parth-Brahmbhatt/iceberg that referenced this issue Apr 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants