Add support for rules on search query rules

[soen]

Look into the way this is implemented in WFFM, this can also serve as inspiration: http://sitecoreblog.blogspot.dk/2013/03/in-sitecore-one-of-field-you-can-use-is.html?m=1

[soen]

It was suggested by @jammykam that the following blog post written by @cardinal252 could be of use: https://cardinalcore.co.uk/2014/07/02/driving-lucinq-or-lucene-net-search-with-the-sitecore-rules-engine/

[cardinal252]

I would be more than happy to collaborate on this if you want to?

[soen]

Thanks for reaching out @cardinal252, I'd very much appreciate collaborating on getting this feature implemented.

From the top of my mind, my originally thought was to let the different search query elements (search query groupings and search query rules) have a 'Rule' field on their template. The rules field would allow the configurator of the search query element(s) to set up rules that determines whether a given part of the search query should be included in the query or not - like any other configuration you can set on a search query element. As such, before a given part of a search query would be included in the final query, the rule should be executed (given it has one defined), in order to either include or exclude the query parts.

What are your thoughts on this, especially compared to the approach you described in your blog post?

[soen]

Recently, @zkniebel and I had a great conversation on Slack about how personalization could be implemented in Conjunction. In our conversation, @zkniebel presented some ideas that aligns very well with the ideas described in the blog post by @cardinal252, yet with some key differences. Since the conversation contains a great amount of details on how this can be achieved, and what needs to be changed in Conjunction, the main parts of the conversation has been added to this issue (see below).

If everyone agrees on the suggested ideas, I suggest that we sketch out the different parts that requires work, divide the work between us, and start up the job of making the great idea come alive.

Thoughts?

Excerpt of the Slack conversation

@zkniebel:

1. You mentioned that the search queries can support personalization, but I didn't see any documentation for doing this. Is this an upcoming feature or one that exists already? How does it work/how do you envision it working?
2. Looking at the Search Query Rule, I have to ask: what made you choose to create specifically defined fields for these settings, rather than a single "Rules" type field with a custom condition (leveraging a custom comparison operator and macros for the other fields)?

I know that the alternative approach (rules type field) in question 2 is a bit more up-front work, but the reason I ask is because going that route would mean that you could incorporate filtering based on personalization into your Search Query Rules, granting very granular control. In addition, I would also add a second "Rules" type field for to your Search Query Rules template for boosting results based on configurable rules (including personalization). By the same token, I would use a rule field on your Search Query Root and Search Query Grouping items to allow personalization/custom rules-based boosting at those levels, as well.

Note: my assumption is that personalization would be leveraged for boosting and filtering search results, only. If there are other uses that I haven't thought of, let me know - that could change the above, significantly.

@soen:
To answer the first of your question, then you’re absolutely right - there is little documentation on this in the docs. The reason for this is that it is planned to be implemented in the upcoming release v1.2, which should arrive next month (if time allows). For now, the only personalization enabled, is the fact that you can create different data sources (queries) and use component-personalization (poor mans choice, yes, but at least you can do some personalization compared to the standard way of working with the content search api). In terms of how I envisioned this working, my originally thought was to let the different search query elements (search query groupings and search query rules) have a 'Rule' field on their template. The rules field would allow the configurator of the search query element(s) to set up rules that determines whether a given part of the search query should be included in the query or not - like any other configuration you can set on a search query element. As such, before a given part of a search query would be included in the final query, the rule should be executed (given it has one defined), in order to either include or exclude the query parts.

Going on to your second question, that is actually a good question. When I started the project, I was focused on the idea of exposing the bare minimum needed to configure a search query rule (field name, comparison operator, value, etc.), even without using personalization - since personalization is an option, not something that should be mandatory to use - as mentioned in the above, the rule field was meant for mainly enable, disable query parts, not drive the actual query logic itself. Since then I’ve been looking at the work @cardinal252 did in his blog post, https://cardinalcore.co.uk/2014/07/02/driving-lucinq-or-lucene-net-search-with-the-sitecore-rules-engine/ based on a conversation I had with @jammykam , which somewhat resembles what you describe (if I am not mistaken?), and talked with him about this as well - actually there is an open issue on the repo, which centers around this whole thing: #2. Going back to the question it seems to me like, if you take the route of just using personalization to drive search queries exclusively, you have to go strictly down that path? I wanted the enable users to create simply queries as well, without the use of personalization - which is the reason why I express the search query rule, and then put a personalization rule on top of that (or a grouping for that matter), which should get evaluated to determine if that part of the search query structure should be include or not. That being said, maybe using a rule field for this, and simply configure the search query rule using custom rules may actually be a better approach. I’m very much open to the idea of making changes to the this, if it fits into the intensions behind what I wanted to achieve - and I think it’s great to discuss this 😃

@zkniebel:
Looking through the blog post it does look a lot like what I'm describing, yes. The idea that I had was that the configurator could use the rules engine with or without personalization to do what @cardinal252 does in his post. The only differences between what I conceptualized and @cardinal252 wrote in his blog post are:

1. I would have used Rule Operators rather than trying to add one of each operation to the query and stopping once a condition type is matched, as he did
2. I would have also written optional conditions that are based on the current user's pattern/profile card (for personalization)
3. I would have followed your hierarchical structure and put the Query Rules field (Rules field type; this is the field that acts similar to @cardinal252 's solution) on the Search Query Rules template, in order to control query nesting the same way that you are
4. I would have added a boost parameter to the conditions, so that certain conditions can boost results

@soen:
I really like your ideas, and I can see how that would fit into Conjunction. As I see it, we want to achieve the same thing, it just the means to end that are different. So we would be changing the way search query rules are configured, and rely on using the rules editor exclusively. This does make a lot of sense, given that you would define the query logic first, and then you could add optional conditions as well as you say. The added benefit would also be that the configurator works within the same context of doing the entire configuration of a single search query rule. Also, it would make Conjunction even more configurable, since you could essentially create any condition or operator and stuff that in into the rules engine. That said, we will have to make changes to the way search query rules are constructed, but the overall structure of building up an expression used by the contentsearch api would be the same as I see it.

[soen]

Following up on this issue, @cardinal252 is currently working on migrating over the core of his RuleQueries implementation - you can follow along the work here. Once done, we'll continue to discuss how it fits in the framework as a whole.