feat(integration-service): optimised the SDK/API calls and removed re… (

#878)

…dundant files

Closes #885 

<!-- ELLIPSIS_HIDDEN -->

----

> [!IMPORTANT]
> Optimize SDK/API calls, improve integration handling, and update
documentation in the integration service.
> 
>   - **Integration Handling**:
> - `execute_integration.py`: Refactor to handle provider and method
validation more efficiently.
> - `browserbase.py`, `llama_parse.py`, `spider.py`: Simplify API key
handling and improve retry logic.
>   - **Model and Tool Handling**:
> - `prompt_step.py`, `chat.py`: Add hotfixes for handling specific
models like Claude and Groq, and tool formatting.
>     - Remove redundant model definitions in `models.py`.
>   - **Serialization Optimization**:
> - `codec.py`: Improve serialization/deserialization logging and
handling.
>   - **Documentation Updates**:
> - `README.md`: Update example cookbook links to reflect new paths and
remove outdated references.
>   - **Miscellaneous**:
> - `web.py`: Add GZIP middleware and optimize event loop policy for
better performance.
> 
> <sup>This description was created by </sup>[<img alt="Ellipsis"
src="https://img.shields.io/badge/Ellipsis-blue?color=175173">](https://www.ellipsis.dev?ref=julep-ai%2Fjulep&utm_source=github&utm_medium=referral)<sup>
for 181dc96. It will automatically
update as commits are pushed.</sup>

<!-- ELLIPSIS_HIDDEN -->

---------

Co-authored-by: Vedantsahai18 <[email protected]>
Co-authored-by: Diwank Singh Tomer <[email protected]>

README.md

@@ -1143,8 +1143,6 @@ main:
 
 Whenever julep encounters a _user-defined function_, it pauses, giving control back to the client and waits for the client to run the function call and give the results back to julep.
 
-> [!TIP] > **Example cookbook**: [cookbooks/13-Error_Handling_and_Recovery.py](https://github.com/julep-ai/julep/blob/dev/cookbooks/13-Error_Handling_and_Recovery.py)
-
 ### `system` tools
 
 Built-in tools that can be used to call the julep APIs themselves, like triggering a task execution, appending to a metadata field, etc.
@@ -1224,15 +1222,15 @@ Additional operations available for some resources:
 
 Note: The availability of these operations may vary depending on the specific resource and implementation details.
 
-> [!TIP] > **Example cookbook**: [cookbooks/10-Document_Management_and_Search.py](https://github.com/julep-ai/julep/blob/dev/cookbooks/10-Document_Management_and_Search.py)
+> [!TIP] > **Example cookbook**: [cookbooks/06-browser-use.ipynb](https://github.com/julep-ai/julep/blob/dev/cookbooks/06-browser-use.ipynb)
 
 ### Built-in `integrations`
 
 Julep comes with a number of built-in integrations (as described in the section below). `integration` tools are directly executed on the julep backend. Any additional parameters needed by them at runtime can be set in the agent/session/user's `metadata` fields.
 
 See [Integrations](#integrations) for details on the available integrations.
 
-> [!TIP] > **Example cookbook**: [cookbooks/01-Website_Crawler_using_Spider.ipynb](https://github.com/julep-ai/julep/blob/dev/cookbooks/01-Website_Crawler_using_Spider.ipynb)
+> [!TIP] > **Example cookbook**: [cookbooks/01-website-crawler.ipynb](https://github.com/julep-ai/julep/blob/dev/cookbooks/01-website-crawler.ipynb)
 
 ### Direct `api_calls`
 
@@ -1290,7 +1288,7 @@ output:
 
 <td>
 
-**Example cookbook**: [cookbooks/03-SmartResearcher_With_WebSearch.ipynb](https://github.com/julep-ai/julep/blob/dev/cookbooks/03-SmartResearcher_With_WebSearch.ipynb)
+**Example cookbook**: [cookbooks/02-sarcastic-news-headline-generator.ipynb](https://github.com/julep-ai/julep/blob/dev/cookbooks/02-sarcastic-news-headline-generator.ipynb)
 
 </td>
 </tr>
@@ -1313,6 +1311,11 @@ output:
 
 </td>
 
+<td>
+
+**Example cookbook**: [cookbooks/06-browser-use.ipynb](https://github.com/julep-ai/julep/blob/dev/cookbooks/06-browser-use.ipynb)
+
+</td>
 </tr>
 <tr>
 <td> <b>Email</b> </td>
@@ -1364,7 +1367,7 @@ output:
 
 <td>
 
-**Example cookbook**: [cookbooks/01-Website_Crawler_using_Spider.ipynb](https://github.com/julep-ai/julep/blob/dev/cookbooks/01-Website_Crawler_using_Spider.ipynb)
+**Example cookbook**: [cookbooks/01-website-crawler.ipynb](https://github.com/julep-ai/julep/blob/dev/cookbooks/01-website-crawler.ipynb)
 
 </td>
 </tr>
@@ -1387,7 +1390,7 @@ output:
 
 <td>
 
-**Example cookbook**: [cookbooks/04-TripPlanner_With_Weather_And_WikiInfo.ipynb](https://github.com/julep-ai/julep/blob/dev/cookbooks/04-TripPlanner_With_Weather_And_WikiInfo.ipynb)
+**Example cookbook**: [cookbooks/03-trip-planning-assistant.ipynb](https://github.com/julep-ai/julep/blob/dev/cookbooks/03-trip-planning-assistant.ipynb)
 
 </td>
 </tr>
@@ -1409,10 +1412,87 @@ output:
 
 <td>
 
-**Example cookbook**: [cookbooks/04-TripPlanner_With_Weather_And_WikiInfo.ipynb](https://github.com/julep-ai/julep/blob/dev/cookbooks/04-TripPlanner_With_Weather_And_WikiInfo.ipynb)
+**Example cookbook**: [cookbooks/03-trip-planning-assistant.ipynb](https://github.com/julep-ai/julep/blob/dev/cookbooks/03-trip-planning-assistant.ipynb)
+
+</td>
+</tr>
+
+<tr>
+<td> <b>FFmpeg</b> </td>
+<td>
+
+```yaml
+arguments:
+  cmd: string # The FFmpeg command to execute
+  file: string # The base64 encoded file to process
+
+output:
+  fileoutput: string # The output file from the FFmpeg command in base64 encoding
+  result: boolean # Whether the FFmpeg command was executed successfully
+  mime_type: string # The MIME type of the output file
+```
 
 </td>
+
 </tr>
+
+<tr>
+<td> <b>Llama Parse</b> </td>
+<td>
+
+```yaml
+setup:
+  llamaparse_api_key: string # The API key for Llama Parse
+
+arguments:
+  file: string # The base64 encoded file to parse
+  filename: string # The filename of the file
+
+output:
+  documents: list # The parsed data from the document
+```
+
+</td>
+
+</tr>
+
+<tr>
+<td> <b>Cloudinary</b> </td>
+<td>
+
+```yaml
+
+method: media_upload | media_edit # The method to use for the Cloudinary integration
+
+setup:
+  cloudinary_cloud_name: string # Your Cloudinary cloud name
+  cloudinary_api_key: string # Your Cloudinary API key
+  cloudinary_api_secret: string # Your Cloudinary API secret
+
+arguments:
+  file: string # The URL of the file upload. Only available for media_upload method.
+  upload_params: dict # Additional parameters for the upload. Only available for media_upload method.
+  public_id: string # The public ID for the file. For media_edit method it is MANDATORY. For media_upload method it is optional.
+  transformation: list[dict] # The transformations to apply to the file
+  return_base64: boolean # Whether to return the file in base64 encoding
+
+output:
+  url: string # The URL of the uploaded file. Only available for media_upload method.
+  meta_data: dict # Additional metadata from the upload response. Only available for media_upload method.
+  public_id: string # The public ID of the uploaded file. Only available for media_upload method.
+  transformed_url: string # The transformed URL. Only available for media_edit method.
+  base64: string # The base64 encoded file. Only available for media_edit method.
+```
+
+</td>
+
+<td>
+
+**Example cookbook**: [cookbooks/05-video-processing-with-natural-language.ipynb](https://github.com/julep-ai/julep/blob/dev/cookbooks/05-video-processing-with-natural-language.ipynb)
+
+</td>
+</tr>
+
 </table>
 
 For more details, refer to our [Integrations Documentation](#integrations).

agents-api/agents_api/activities/task_steps/prompt_step.py

@@ -167,10 +167,25 @@ async def prompt_step(context: StepContext) -> StepOutcome:
                 },
             }
             formatted_tools.append(tool)
-
+    # For non-Claude models, we don't need to send tools
+    # FIXME: Enable formatted_tools once format-tools PR is merged.
     if not is_claude_model:
         formatted_tools = None
 
+    # HOTFIX: for groq calls, litellm expects tool_calls_id not to be in the messages
+    # FIXME: This is a temporary fix. We need to update the agent-api to use the new tool calling format
+    # FIXME: Enable formatted_tools once format-tools PR is merged.
+    is_groq_model = agent_model.lower().startswith("llama-3.1")
+    if is_groq_model:
+        prompt = [
+            {
+                k: v
+                for k, v in message.items()
+                if k not in ["tool_calls", "tool_call_id", "user", "continue_", "name"]
+            }
+            for message in prompt
+        ]
+
     # Use litellm for other models
     completion_data: dict = {
         "model": agent_model,

agents-api/agents_api/routers/sessions/chat.py

@@ -168,11 +168,24 @@ async def chat(
             }
             formatted_tools.append(tool)
 
-    # If not using Claude model,
-
+    # If not using Claude model
+    # FIXME: Enable formatted_tools once format-tools PR is merged.
     if not is_claude_model:
         formatted_tools = None
 
+    # HOTFIX: for groq calls, litellm expects tool_calls_id not to be in the messages
+    # FIXME: This is a temporary fix. We need to update the agent-api to use the new tool calling format
+    is_groq_model = settings["model"].lower().startswith("llama-3.1")
+    if is_groq_model:
+        messages = [
+            {
+                k: v
+                for k, v in message.items()
+                if k not in ["tool_calls", "tool_call_id", "user", "continue_", "name"]
+            }
+            for message in messages
+        ]
+
     # Use litellm for other models
     model_response = await litellm.acompletion(
         messages=messages,

integrations-service/integrations/utils/execute_integration.py

@@ -21,17 +21,13 @@ async def execute_integration(
     setup: ExecutionSetup | None = None,
     arguments: ExecutionArguments,
 ) -> ExecutionResponse:
-    provider_obj: BaseProvider | None = getattr(available_providers, provider, None)
-
-    if not provider_obj:
+    provider_obj = getattr(available_providers, provider, None)
+    if not provider_obj or not isinstance(provider_obj, BaseProvider):
         raise HTTPException(status_code=400, detail=f"Unknown provider: {provider}")
 
-    assert isinstance(provider_obj, BaseProvider)
-
-    if method is None:
-        method = provider_obj.methods[0].method
-
-    elif method not in [method.method for method in provider_obj.methods]:
+    method = method or provider_obj.methods[0].method
+    method_config = next((m for m in provider_obj.methods if m.method == method), None)
+    if not method_config:
         raise HTTPException(
             status_code=400, detail=f"Unknown method: {method} for provider: {provider}"
         )
@@ -41,28 +37,22 @@ async def execute_integration(
         package="integrations",
     )
 
-    execution_function = getattr(provider_module, method)
-
-    setup_obj = setup
-
-    if setup is not None:
-        setup_class = provider_obj.setup
-
-        if setup_class and not isinstance(setup, setup_class):
-            setup_obj = setup_class(**setup.model_dump())
-
-    arguments_class = next(
-        m for m in provider_obj.methods if m.method == method
-    ).arguments
+    if (
+        setup is not None
+        and provider_obj.setup
+        and not isinstance(setup, provider_obj.setup)
+    ):
+        setup = provider_obj.setup(**setup.model_dump())
+
+    arguments = (
+        method_config.arguments(**arguments.model_dump())
+        if not isinstance(arguments, method_config.arguments)
+        else arguments
+    )
 
-    if not isinstance(arguments, arguments_class):
-        parsed_arguments = arguments_class(**arguments.model_dump())
-    else:
-        parsed_arguments = arguments
     try:
-        if setup_obj:
-            return await execution_function(setup=setup_obj, arguments=parsed_arguments)
-        else:
-            return await execution_function(arguments=parsed_arguments)
+        return await getattr(provider_module, method)(
+            **({"setup": setup} if setup else {}), arguments=arguments
+        )
     except BaseException as e:
         return ExecutionError(error=str(e))

integrations-service/integrations/utils/integrations/browserbase.py

@@ -38,10 +38,14 @@
 
 
 def get_browserbase_client(setup: BrowserbaseSetup) -> Browserbase:
-    if setup.api_key == "DEMO_API_KEY":
-        setup.api_key = browserbase_api_key
-    if setup.project_id == "DEMO_PROJECT_ID":
-        setup.project_id = browserbase_project_id
+    setup.api_key = (
+        browserbase_api_key if setup.api_key == "DEMO_API_KEY" else setup.api_key
+    )
+    setup.project_id = (
+        browserbase_project_id
+        if setup.project_id == "DEMO_PROJECT_ID"
+        else setup.project_id
+    )
 
     return Browserbase(
         api_key=setup.api_key,

integrations-service/integrations/utils/integrations/llama_parse.py

@@ -26,23 +26,23 @@ async def parse(
     assert isinstance(setup, LlamaParseSetup), "Invalid setup"
     assert isinstance(arguments, LlamaParseFetchArguments), "Invalid arguments"
 
-    if setup.llamaparse_api_key == "DEMO_API_KEY":
-        setup.llamaparse_api_key = llama_api_key
+    # Use walrus operator to simplify assignment and condition
+    if (api_key := setup.llamaparse_api_key) == "DEMO_API_KEY":
+        api_key = llama_api_key
 
     parser = LlamaParse(
-        api_key=setup.llamaparse_api_key,
+        api_key=api_key,  # Use the local variable instead
         result_type=arguments.result_format,
         num_workers=arguments.num_workers,
         language=arguments.language,
     )
 
-    # Decode base64 file content
-    file_content = base64.b64decode(arguments.file)
-    extra_info = {
-        "file_name": arguments.filename if arguments.filename else str(uuid.uuid4())
-    }
+    # Simplify filename assignment using or operator
+    extra_info = {"file_name": arguments.filename or str(uuid.uuid4())}
 
-    # Parse the document
-    documents = await parser.aload_data(file_content, extra_info=extra_info)
+    # Parse the document (decode inline)
+    documents = await parser.aload_data(
+        base64.b64decode(arguments.file), extra_info=extra_info
+    )
 
     return LlamaParseFetchOutput(documents=documents)