Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Federated tutorial and explanation #3002

Merged
merged 7 commits into from
Oct 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions assets/datasets/crm-chats.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
[
{
"id": 0,
"client_name": "Natasha Nguyen",
"message": "My email is [email protected]",
"time": 1727349362
},
{
"id": 1,
"client_name": "Riccardo Rotondo",
"message": "No, I changed my email, it's no longer [email protected], the one you see on my profile is the right one",
"time": 1726344418
}
]
12 changes: 12 additions & 0 deletions assets/datasets/crm-profiles.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
[
{
"id": 0,
"name": "Natasha Nguyen",
"email": "[email protected]"
},
{
"id": 1,
"name": "Riccardo Rotondo",
"email": "[email protected]"
}
]
16 changes: 16 additions & 0 deletions assets/datasets/crm-tickets.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
[
{
"id": 0,
"time": 1727349362,
"client_name": "Natasha Nguyen",
"type": "complaint",
"title": "I'm not receiving any emails"
},
{
"id": 1,
"time": 1701448312,
"client_name": "Riccardo Rotondo",
"type": "support",
"title": "Please remove my email from your mailing list"
}
]
16 changes: 16 additions & 0 deletions config/sidebar-learn.json
Original file line number Diff line number Diff line change
Expand Up @@ -253,6 +253,22 @@
}
]
},
{
"title": "Multi-search",
"slug": "multi_search",
"routes": [
{
"source": "learn/multi_search/performing_federated_search.mdx",
"label": "Using multi-search to perform a federated search",
"slug": "performing_federated_search"
},
{
"source": "learn/multi_search/multi_search_vs_federated_search.mdx",
"label": "Differences between multi-search and federated search",
"slug": "multi_search_vs_federated_search"
}
]
},
{
"title": "Update and migration",
"slug": "update_and_migration",
Expand Down
26 changes: 26 additions & 0 deletions learn/multi_search/multi_search_vs_federated_search.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
---
title: Differences between multi-search and federated search — Meilisearch API reference
description: This article defines multi-search and federated search and then describes the different uses of each.
---

# Differences between multi-search and federated search

This article defines multi-search and federated search and then describes the different uses of each.

## What is multi-search?

Multi-search, also called multi-index search, is a search operation that makes multiple queries at the same time. These queries may target different indexes. Meilisearch then returns a separate list results for each query. Use the `/multi-search` route to perform multi-searches.

Multi-search favors discovery scenarios, where users might not have a clear idea of what they need and searches might have many valid results.

## What is federated search?

Federated search is a type of multi-index search. This operation also makes multiple search requests at the same time, but returns a single list with the most relevant results from all queries. Use the `/multi-search` route and specify a non-null value for `federation` to perform a federated search.

Federated search favors scenarios where users have a clear idea of what they need and expect a single best top result.

## Use cases

Because multi-search groups results by query, it is often useful when the origin and type of document contain information relevant to your users. For example, a person searching for `shygirl` in a music streaming application is likely to appreciate seeing separate results for matching artists, albums, and individual tracks.

Federated search is a better approach when the source of the information is not relevant to your users. For example, a person searching for a client's email in a CRM application is unlikely to care whether this email comes from chat logs, support tickets, or other data sources.
143 changes: 143 additions & 0 deletions learn/multi_search/performing_federated_search.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
---
title: Using multi-search to perform a federated search — Meilisearch API reference
description: In this tutorial you will see how to perform a query searching multiple indexes at the same time to obtain a single list of results.
---

# Using multi-search to perform a federated search

Meilisearch allows you to make multiple search requests at the same time with the `/multi-search` endpoint. A federated search is a multi-search that returns results from multiple queries in a single list.

In this tutorial you will see how to create separate indexes containing different types of data from a CRM application. You will then perform a query searching all these indexes at the same time to obtain a single list of results.

## Requirements

- A running Meilisearch project
- A command-line console

## Create three indexes

Download the following datasets: <a href="/assets/datasets/crm-chats.json">`crm-chats.json`</a>, <a href="/assets/datasets/crm-profiles.json">`crm-profiles.json`</a>, and <a href="/assets/datasets/crm-tickets.json">`crm-tickets.json`</a> containing data from a fictional CRM application.

Add the datasets to Meilisearch and create three separate indexes, `profiles`, `chats`, and `tickets`:

```sh
curl -X POST 'http://localhost:7700/indexes/profiles' -H 'Content-Type: application/json' --data-binary @crm-profiles.json &&
curl -X POST 'http://localhost:7700/indexes/chats' -H 'Content-Type: application/json' --data-binary @crm-chats.json &&
curl -X POST 'http://localhost:7700/indexes/tickets' -H 'Content-Type: application/json' --data-binary @crm-tickets.json
```

[Use the tasks endpoint](/learn/async/working_with_tasks) to check the indexing status. Once Meilisearch successfully indexed all three datasets, you are ready to perform a federated search.

## Perform a federated search

When you are looking for Natasha Nguyen's email address in your CRM application, you may not know whether you will find it in a chat log, among the existing customer profiles, or in a recent support ticket. In this situation, you can use federated search to search across all possible sources and receive a single list of results.

Use the `/multi-search` endpoint with the `federation` parameter to query the three indexes simultaneously:

```sh
curl \
-X POST 'http://localhost:7700/multi-search' \
-H 'Content-Type: application/json' \
--data-binary '{
"federation": {},
"queries": [
{
"indexUid": "chats",
"q": "natasha"
},
{
"indexUid": "profiles",
"q": "natasha"
},
{
"indexUid": "tickets",
"q": "natasha"
}
]
}'
```

Meilisearch should respond with a single list of search results:

```json
{
"hits": [
{
"id": 0,
"client_name": "Natasha Nguyen",
"message": "My email is [email protected]",
"time": 1727349362,
"_federation": {
"indexUid": "chats",
"queriesPosition": 0
}
},
],
"processingTimeMs": 0,
"limit": 20,
"offset": 0,
"estimatedTotalHits": 3,
"semanticHitCount": 0
}
```

## Promote results from a specific index

Since this is a CRM application, users have profiles with their preferred contact information. If you want to search for Riccardo Rotondo's preferred email, you can boost documents in the `profiles` index.

Use the `weight` property of the `federation` parameter to boost results coming from a specific query:

```sh
curl \
-X POST 'http://localhost:7700/multi-search' \
-H 'Content-Type: application/json' \
--data-binary '{
"federation": {},
"queries": [
{
"indexUid": "chats",
"q": "rotondo"
},
{
"indexUid": "profiles",
"q": "rotondo",
"federationOptions": {
"weight": 1.2
}
},
{
"indexUid": "tickets",
"q": "rotondo"
}
]
}'
```

This request will lead to results from the query targeting `profile` ranking higher than documents from other queries:

```json
{
"hits": [
{
"id": 1,
"name": "Riccardo Rotondo",
"email": "[email protected]",
"_federation": {
"indexUid": "profiles",
"queriesPosition": 1
}
},
],
"processingTimeMs": 0,
"limit": 20,
"offset": 0,
"estimatedTotalHits": 3,
"semanticHitCount": 0
}
```

## Conclusion

You have created three indexes, then performed a federated multi-index search to receive all results in a single list. You then used `weight` to boost results from the index most likely to contain the information you wanted.