Adding initial cloud Structure docs

.github/ISSUE_TEMPLATE/bug_report.md

@@ -13,11 +13,7 @@ assignees: ''
 A clear and concise description of what the bug is.
 
 **To Reproduce**
-Steps to reproduce the behavior:
-1. Go to '...'
-2. Click on '....'
-3. Scroll down to '....'
-4. See error
+A minimal, reproducible code example.
 
 **Expected behavior**
 A clear and concise description of what you expected to happen.

.github/workflows/docs-integration-tests.yml

@@ -124,6 +124,8 @@ jobs:
       ZENROWS_API_KEY: ${{ secrets.INTEG_ZENROWS_API_KEY }}
       QDRANT_CLUSTER_ENDPOINT: ${{ secrets.INTEG_QDRANT_CLUSTER_ENDPOINT }}
       QDRANT_CLUSTER_API_KEY: ${{ secrets.INTEG_QDRANT_CLUSTER_API_KEY }}
+      ASTRA_DB_API_ENDPOINT: ${{ secrets.INTEG_ASTRA_DB_API_ENDPOINT }}
+      ASTRA_DB_APPLICATION_TOKEN: ${{ secrets.INTEG_ASTRA_DB_APPLICATION_TOKEN }}
     services:
       postgres:
         image: ankane/pgvector:v0.5.0

CHANGELOG.md

@@ -6,6 +6,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## Unreleased
 
+### Added
+- `AstraDbVectorStoreDriver` to support DataStax Astra DB as a vector store.
+- Ability to set custom schema properties on Tool Activities via `extra_schema_properties`.
+
 ## [0.29.0] - 2024-07-30
 
 ### Added
@@ -22,6 +26,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - `@observable` decorator for selecting which functions/methods to provide observability for.
 - `GenericArtifact` for storing any data.
 - `BaseTextArtifact` for text-based Artifacts to subclass.
+- `HuggingFacePipelineImageGenerationDriver` for generating images locally with HuggingFace pipelines.
+- `BaseImageGenerationPipelineDriver` as the base class for drivers interfacing with HuggingFace image generation pipelines.
+- `StableDiffusion3ImageGenerationPipelineDriver` for local text-to-image generation using a Stable Diffusion 3 pipeline.
+- `StableDiffusion3Img2ImgImageGenerationPipelineDriver` for local image-to-image generation using a Stable Diffusion 3 pipeline.
+- `StableDiffusion3ControlNetImageGenerationPipelineDriver` for local ControlNet image generation using a Stable Diffusion 3 pipeline.
+- Optional `params` field to `WebSearch`'s `search` schema that the LLM can be steered into using.
 
 ### Changed
 - **BREAKING**: `BaseVectorStoreDriver.upsert_text_artifacts` optional arguments are now keyword-only arguments.
@@ -41,6 +51,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **BREAKING**: Renamed `drivers-vector-postgresql` extra to `drivers-vector-pgvector`.
 - **BREAKING**: Update `marqo` dependency to `^3.7.0`.
 - **BREAKING**: Removed `drivers-sql-postgresql` extra. Use `drivers-sql` extra and install necessary drivers (i.e. `psycopg2`) separately.
+- **BREAKING**: `api_key` and `search_id` are now required fields in `GoogleWebSearchDriver`.
+- **BREAKING**: `web_search_driver` is now required fields in the `WebSearch` Tool.
+- `GoogleWebSearchDriver` and `DuckDuckGoWebSearchDriver` now use `kwargs` passed to the `run` method.
 - Removed unnecessary `sqlalchemy-redshift` dependency in `drivers-sql-amazon-redshift` extra.
 - Removed unnecessary `transformers` dependency in `drivers-prompt-huggingface` extra.
 - Removed unnecessary `huggingface-hub` dependency in `drivers-prompt-huggingface-pipeline` extra.

docs/examples/query-webpage-astra-db.md

@@ -0,0 +1,78 @@
+The following example script ingests a Web page (a blog post),
+stores its chunked contents on Astra DB through the Astra DB vector store driver,
+and finally runs a RAG process to answer a question specific to the topic of the
+Web page.
+
+This script requires that a vector collection has been created in the Astra database
+(with name `"griptape_test_collection"` and vector dimension matching the embedding being used, i.e. 1536 in this case).
+
+_Note:_ Besides the [Astra DB](../griptape-framework/drivers/vector-store-drivers.md#astra-db) extra,
+this example requires the `drivers-web-scraper-trafilatura`
+Griptape extra to be installed as well.
+
+
+```python
+import os
+
+from griptape.drivers import (
+    AstraDbVectorStoreDriver,
+    OpenAiChatPromptDriver,
+    OpenAiEmbeddingDriver,
+)
+from griptape.engines.rag import RagEngine
+from griptape.engines.rag.modules import (
+    PromptResponseRagModule,
+    VectorStoreRetrievalRagModule,
+)
+from griptape.engines.rag.stages import ResponseRagStage, RetrievalRagStage
+from griptape.loaders import WebLoader
+from griptape.structures import Agent
+from griptape.tools import RagClient, TaskMemoryClient
+
+
+namespace = "datastax_blog"
+input_blogpost = (
+    "www.datastax.com/blog/indexing-all-of-wikipedia-on-a-laptop"
+)
+
+vector_store_driver = AstraDbVectorStoreDriver(
+    embedding_driver=OpenAiEmbeddingDriver(),
+    api_endpoint=os.environ["ASTRA_DB_API_ENDPOINT"],
+    token=os.environ["ASTRA_DB_APPLICATION_TOKEN"],
+    collection_name="griptape_test_collection",
+    astra_db_namespace=os.environ.get("ASTRA_DB_KEYSPACE"),
+)
+
+engine = RagEngine(
+    retrieval_stage=RetrievalRagStage(
+        retrieval_modules=[
+            VectorStoreRetrievalRagModule(
+                vector_store_driver=vector_store_driver,
+                query_params={
+                    "count": 2,
+                    "namespace": namespace,
+                },
+            )
+        ]
+    ),
+    response_stage=ResponseRagStage(
+        response_module=PromptResponseRagModule(
+            prompt_driver=OpenAiChatPromptDriver(model="gpt-4o")
+        )
+    )
+)
+
+vector_store_driver.upsert_text_artifacts(
+    {namespace: WebLoader(max_tokens=256).load(input_blogpost)}
+)
+
+vector_store_tool = RagClient(
+    description="A DataStax blog post",
+    rag_engine=engine,
+)
+agent = Agent(tools=[vector_store_tool, TaskMemoryClient(off_prompt=False)])
+agent.run(
+    "What engine made possible to index such an amount of data, "
+    "and what kind of tuning was required?"
+)
+```

docs/griptape-cloud/index.md

@@ -1,3 +1,5 @@
 # Griptape Cloud
 
-Griptape Cloud is a managed platform for running AI-powered agents, pipelines, and workflows.
+Griptape Cloud provides managed services for your AI app stack. Deploy and scale end-to-end solutions, from LLM-powered data prep and retrieval to AI agents, pipelines and workflows.
+
+To get started with AI Structures in the Cloud, check out the [managed-structure-template](https://github.com/griptape-ai/managed-structure-template) or deploy one of the [griptape-sample-structures](https://github.com/griptape-ai/griptape-sample-structures/tree/main).

docs/griptape-cloud/structures/structure-config.md

@@ -0,0 +1,56 @@
+## Overview
+
+Structure repositories require a configuration file which informs Griptape Cloud of your Managed Structure's dependencies and how it needs to build and run.
+
+## Structure Config File Schema
+
+The schema for the configuration file is as follows:
+
+```yaml
+version: 1.0
+runtime: python3
+runtime_version: 3.11
+build:
+  pre_build_install_script: scripts/my-pre-build-install-script.sh
+  post_build_install_script: scripts/my-post-build-install-script.sh
+  requirements_file: requirements.txt
+  cache_build_dependencies:
+    enabled: false
+    watched_files:
+      - requirements.txt
+      - scripts/my-pre-build-install-script.sh
+      - scripts/my-post-build-install-script.sh
+run:
+  main_file: structure.py
+```
+
+### Configuration Fields
+
+#### version
+
+The Structure Config schema version number.
+
+#### runtime
+
+The runtime environment to use for the Structure.
+
+#### runtime_version
+
+The specific version of the runtime environment for the Structure.
+
+#### build (OPTIONAL)
+
+The build-time configuration for the Structure.
+
+* **pre_build_install_script** - The path to your pre_build_install_script, for running during the Structure build prior to dependency installation. This path is relative to the structure configuration file. Or absolute from the repository root if a forward slash is used: `/my-pre-build-install-script.sh`.
+* **post_build_install_script** - The path to your post_build_install_script, for running during the Structure build after dependency installation. This path is relative to the structure configuration file. Or absolute from the repository root if a forward slash is used: `/my-post-build-install-script.sh`.
+* **requirements_file** - The path to your Structure's requirements.txt file.
+* **cache_build_dependencies** - Defines the configuration for caching build dependencies in order to speed up Deployments
+  * **enabled** - Defines whether the build dependency caching is on or off
+  * **watched_files** - Defines the particular files that will trigger cache invalidation, resulting in a full rebuild of the Structure and dependencies
+
+#### run (REQUIRED)
+
+The run-time configuration for the Structure.
+
+* **main_file** - Specifies the path to the entry point file of the Managed Structure. This path is relative to the structure_config.yaml. Or absolute from the repository root if a forward slash is used: `/structure.py`.

docs/griptape-framework/drivers/vector-store-drivers.md

@@ -482,3 +482,49 @@ values = [r.to_artifact().value for r in results]
 
 print("\n\n".join(values))
 ```
+
+### Astra DB
+
+!!! info
+    This Driver requires the `drivers-vector-astra-db` [extra](../index.md#extras).
+
+The AstraDbVectorStoreDriver supports [DataStax Astra DB](https://www.datastax.com/products/datastax-astra).
+
+The following example shows how to store vector entries and query the information using the driver:
+
+```python
+import os
+from griptape.drivers import AstraDbVectorStoreDriver, OpenAiEmbeddingDriver
+from griptape.loaders import WebLoader
+
+# Astra DB secrets and connection parameters
+api_endpoint = os.environ["ASTRA_DB_API_ENDPOINT"]
+token = os.environ["ASTRA_DB_APPLICATION_TOKEN"]
+astra_db_namespace = os.environ.get("ASTRA_DB_KEYSPACE")  # optional
+
+# Initialize an Embedding Driver.
+embedding_driver = OpenAiEmbeddingDriver(api_key=os.environ["OPENAI_API_KEY"])
+
+vector_store_driver = AstraDbVectorStoreDriver(
+    embedding_driver=embedding_driver,
+    api_endpoint=api_endpoint,
+    token=token,
+    collection_name="griptape_test_collection",
+    astra_db_namespace=astra_db_namespace,  # optional
+)
+
+# Load Artifacts from the web
+artifacts = WebLoader().load("https://www.griptape.ai")
+
+# Upsert Artifacts into the Vector Store Driver
+[
+    vector_store_driver.upsert_text_artifact(a, namespace="griptape")
+    for a in artifacts
+]
+
+results = vector_store_driver.query(query="What is griptape?")
+
+values = [r.to_artifact().value for r in results]
+
+print("\n\n".join(values))
+```

docs/griptape-tools/official-tools/web-search.md

@@ -110,3 +110,36 @@ agent.run("Tell me how photosynthesis works")
                              in the atmosphere and forms the basis of the food  
                              chain.      
 ```
+
+Extra schema properties can be added to the Tool to allow for more customization if the Driver supports it.
+In this example, we add a `sort` property to the `search` Activity which will be added as a [Google custom search query parameter](https://developers.google.com/custom-search/v1/reference/rest/v1/cse/list).
+
+```python
+import os
+import schema
+from griptape.structures import Agent
+from griptape.drivers import GoogleWebSearchDriver
+from griptape.tools import WebSearch
+
+
+agent = Agent(
+    tools=[
+        WebSearch(
+            web_search_driver=GoogleWebSearchDriver(
+                api_key=os.environ["GOOGLE_API_KEY"],
+                search_id=os.environ["GOOGLE_API_SEARCH_ID"],
+            ),
+            extra_schema_properties={
+                "search": {
+                    schema.Literal(
+                        "sort",
+                        description="Date range to search within. Format: date:r:YYYYMMDD:YYYYMMDD",
+                    ): str
+                }
+            },
+        )
+    ],
+)
+
+agent.run("Search for articles about the history of the internet from 1990 to 2000")
+```

griptape/drivers/__init__.py

@@ -41,6 +41,7 @@
 from .vector.azure_mongodb_vector_store_driver import AzureMongoDbVectorStoreDriver
 from .vector.dummy_vector_store_driver import DummyVectorStoreDriver
 from .vector.qdrant_vector_store_driver import QdrantVectorStoreDriver
+from .vector.astradb_vector_store_driver import AstraDbVectorStoreDriver
 from .vector.griptape_cloud_knowledge_base_vector_store_driver import GriptapeCloudKnowledgeBaseVectorStoreDriver
 
 from .sql.base_sql_driver import BaseSqlDriver
@@ -171,6 +172,7 @@
     "AmazonOpenSearchVectorStoreDriver",
     "PgVectorVectorStoreDriver",
     "QdrantVectorStoreDriver",
+    "AstraDbVectorStoreDriver",
     "DummyVectorStoreDriver",
     "GriptapeCloudKnowledgeBaseVectorStoreDriver",
     "BaseSqlDriver",