Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(agents-api): Remove auto_blob_store in favor of interceptor based system #977

Merged
merged 3 commits into from
Dec 21, 2024

Conversation

creatorrr
Copy link
Contributor

@creatorrr creatorrr commented Dec 19, 2024

User description

  • feat(agents-api): Add entry queries
  • refactor: Lint agents-api (CI)
  • chore: update the entyr queries
  • refactor: Lint agents-api (CI)
  • chore: inner join developer table with entry queries
  • refactor: Lint agents-api (CI)
  • wip(agents-api): Add session sql queries
  • refactor: Lint agents-api (CI)
  • chore: developers and user refactor + add test for entry queries + bug fixes
  • refactor: Lint agents-api (CI)
  • feat(agents-api): Fix tests for sessions
  • refactor: Lint agents-api (CI)
  • wip(agents-api): Entry queries
  • refactor: Lint agents-api (CI)
  • fix(agents-api): change modelname to model in BaseEntry
  • feat(agents-api): add agent queries tests
  • feat(agents-api): implement agent queries and tests
  • refactor: Lint agents-api (CI)
  • fix(agents-api): misc fixes
  • refactor: Lint agents-api (CI)
  • wip
  • refactor: Lint agents-api (CI)
  • chore: minor refactors
  • refactor: Lint agents-api (CI)
  • feat(memory-store,agents-api): Move is_leaf handling to postgres
  • fix(agents-api): fix sessions and agents queries / tests
  • refactor: Lint agents-api (CI)
  • feat(agents-api): Remove auto_blob_store in favor of interceptor based system

PR Type

Enhancement


Description

  • Replaced auto_blob_store decorator system with a new interceptor-based approach for handling large data objects
  • Simplified RemoteObject implementation with generic type support and improved serialization
  • Removed storage_handler.py module and consolidated functionality into interceptors.py
  • Updated blob store configuration for testing environment
  • Removed @auto_blob_store decorators from multiple activity and workflow files
  • Added new health check endpoint

Changes walkthrough 📝

Relevant files
Enhancement
interceptors.py
Implement new interceptor-based blob store system               

agents-api/agents_api/common/interceptors.py

  • Implemented new interceptor system for handling large data objects
  • Added functions for loading/offloading data to blob store
  • Added error handling for execution with common exceptions
  • +115/-74
    remote.py
    Simplify and improve remote object handling                           

    agents-api/agents_api/common/protocol/remote.py

  • Simplified RemoteObject class with generic type support
  • Added methods for serialization and deserialization
  • Removed BaseRemoteModel class
  • +20/-77 
    storage_handler.py
    Remove storage handler in favor of interceptors                   

    agents-api/agents_api/common/storage_handler.py

  • Removed entire storage handler module
  • Functionality moved to interceptors.py
  • +0/-226 
    Configuration changes
    env.py
    Update blob store configuration settings                                 

    agents-api/agents_api/env.py

    • Modified blob store configuration for testing environment
    +2/-2     

    💡 PR-Agent usage: Comment /help "your question" on any pull request to receive relevant information

    Copy link

    gitguardian bot commented Dec 19, 2024

    ️✅ There are no secrets present in this pull request anymore.

    If these secrets were true positive and are still valid, we highly recommend you to revoke them.
    While these secrets were previously flagged, we no longer have a reference to the
    specific commits where they were detected. Once a secret has been leaked into a git
    repository, you should consider it compromised, even if it was deleted immediately.
    Find here more information about risks.


    🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

    Copy link

    qodo-merge-pro-for-open-source bot commented Dec 19, 2024

    CI Failure Feedback 🧐

    (Checks updated until commit 76819e1)

    Action: Test

    Failed stage: Run tests [❌]

    Failed test name: test_agent_queries

    Failure summary:

    The tests failed because required database tables were not found in the PostgreSQL database:

  • The 'agents' table was missing, causing failures in three agent-related SQL tests
  • The 'developers' table was also missing, causing a fixture error
  • The error "migrate: not found" suggests that database migrations were not run before the tests,
    which would have created these required tables

  • Relevant error logs:
    1:  ##[group]Operating System
    2:  Ubuntu
    ...
    
    1258:  Container started: 7e995726c95d
    1259:  Waiting for container <Container: 7e995726c95d> with image timescale/timescaledb-ha:pg17 to be ready ...
    1260:  Waiting for container <Container: 7e995726c95d> with image timescale/timescaledb-ha:pg17 to be ready ...
    1261:  /bin/sh: 1: migrate: not found
    1262:  FAIL  test_agent_queries:28 query: create agent sql                          2%
    1263:  FAIL  test_agent_queries:44 query: create or update agent sql                4%
    1264:  FAIL  test_agent_queries:63 query: update agent sql                          7%
    1265:  ─────────────────────────── query: create agent sql ────────────────────────────
    1266:  Failed at tests/test_agent_queries.py                                         
    ...
    
    1285:  │                                                                          │  
    1286:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    1287:  │ python3.12/asyncio/runners.py:118 in run                                 │  
    1288:  │                                                                          │  
    1289:  │   115 │   │                                                              │  
    1290:  │   116 │   │   self._interrupt_count = 0                                  │  
    1291:  │   117 │   │   try:                                                       │  
    1292:  │ ❱ 118 │   │   │   return self._loop.run_until_complete(task)             │  
    1293:  │   119 │   │   except exceptions.CancelledError:                          │  
    ...
    
    1298:  │ │        context = <_contextvars.Context object at 0x7fb06328cf80>     │ │  
    1299:  │ │           coro = <coroutine object _ at 0x7fb06300d240>              │ │  
    1300:  │ │           self = <asyncio.runners.Runner object at 0x7fb064289e20>   │ │  
    1301:  │ │ sigint_handler = functools.partial(<bound method Runner._on_sigint   │ │  
    1302:  │ │                  of <asyncio.runners.Runner object at                │ │  
    1303:  │ │                  0x7fb064289e20>>, main_task=<Task finished          │ │  
    1304:  │ │                  name='Task-1' coro=<_() done, defined at            │ │  
    1305:  │ │                  /home/runner/work/julep/julep/agents-api/tests/tes… │ │  
    1306:  │ │                  exception=UndefinedTableError('relation "agents"    │ │  
    1307:  │ │                  does not exist')>)                                  │ │  
    1308:  │ │           task = <Task finished name='Task-1' coro=<_() done,        │ │  
    1309:  │ │                  defined at                                          │ │  
    1310:  │ │                  /home/runner/work/julep/julep/agents-api/tests/tes… │ │  
    1311:  │ │                  exception=UndefinedTableError('relation "agents"    │ │  
    ...
    
    1437:  │ │                    │   │   │   │   {},                               │ │  
    1438:  │ │                    │   │   │   │   {}                                │ │  
    1439:  │ │                    │   │   │   ],                                    │ │  
    1440:  │ │                    │   │   │   'timeout': 90.0                       │ │  
    1441:  │ │                    │   │   }                                         │ │  
    1442:  │ │                    │   )                                             │ │  
    1443:  │ │                    ]                                                 │ │  
    1444:  │ │             conn = <PoolConnectionProxy [released] 0x7fb064033580>   │ │  
    1445:  │ │ connection_error = False                                             │ │  
    ...
    
    1456:  │ │                    │   │   model='gpt-4o-mini',                      │ │  
    1457:  │ │                    │   │   instructions=[],                          │ │  
    1458:  │ │                    │   │   default_settings=None                     │ │  
    1459:  │ │                    │   )                                             │ │  
    1460:  │ │                    }                                                 │ │  
    1461:  │ │           method = <bound method Connection.fetch of                 │ │  
    1462:  │ │                    <PoolConnectionProxy [released] 0x7fb064033580>>  │ │  
    1463:  │ │      method_name = 'fetch'                                           │ │  
    1464:  │ │    only_on_error = False                                             │ │  
    ...
    
    1659:  │ │           statement = None                                           │ │  
    1660:  │ │           stmt_name = '__asyncpg_stmt_b__'                           │ │  
    1661:  │ │             timeout = 90.0                                           │ │  
    1662:  │ │           use_cache = True                                           │ │  
    1663:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    1664:  │                                                                          │  
    1665:  │ in prepare:165                                                           │  
    1666:  ╰──────────────────────────────────────────────────────────────────────────╯  
    1667:  UndefinedTableError: relation "agents" does not exist                         
    1668:  ────────────────────── query: create or update agent sql ───────────────────────
    1669:  Failed at tests/test_agent_queries.py                                         
    ...
    
    1688:  │                                                                          │  
    1689:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    1690:  │ python3.12/asyncio/runners.py:118 in run                                 │  
    1691:  │                                                                          │  
    1692:  │   115 │   │                                                              │  
    1693:  │   116 │   │   self._interrupt_count = 0                                  │  
    1694:  │   117 │   │   try:                                                       │  
    1695:  │ ❱ 118 │   │   │   return self._loop.run_until_complete(task)             │  
    1696:  │   119 │   │   except exceptions.CancelledError:                          │  
    ...
    
    1701:  │ │        context = <_contextvars.Context object at 0x7fb06330bcc0>     │ │  
    1702:  │ │           coro = <coroutine object _ at 0x7fb0630328a0>              │ │  
    1703:  │ │           self = <asyncio.runners.Runner object at 0x7fb0632ad640>   │ │  
    1704:  │ │ sigint_handler = functools.partial(<bound method Runner._on_sigint   │ │  
    1705:  │ │                  of <asyncio.runners.Runner object at                │ │  
    1706:  │ │                  0x7fb0632ad640>>, main_task=<Task finished          │ │  
    1707:  │ │                  name='Task-14' coro=<_() done, defined at           │ │  
    1708:  │ │                  /home/runner/work/julep/julep/agents-api/tests/tes… │ │  
    1709:  │ │                  exception=UndefinedTableError('relation "agents"    │ │  
    1710:  │ │                  does not exist')>)                                  │ │  
    1711:  │ │           task = <Task finished name='Task-14' coro=<_() done,       │ │  
    1712:  │ │                  defined at                                          │ │  
    1713:  │ │                  /home/runner/work/julep/julep/agents-api/tests/tes… │ │  
    1714:  │ │                  exception=UndefinedTableError('relation "agents"    │ │  
    ...
    
    1844:  │ │                    │   │   │   │   {},                               │ │  
    1845:  │ │                    │   │   │   │   {}                                │ │  
    1846:  │ │                    │   │   │   ],                                    │ │  
    1847:  │ │                    │   │   │   'timeout': 90.0                       │ │  
    1848:  │ │                    │   │   }                                         │ │  
    1849:  │ │                    │   )                                             │ │  
    1850:  │ │                    ]                                                 │ │  
    1851:  │ │             conn = <PoolConnectionProxy [released] 0x7fb0630d4ee0>   │ │  
    1852:  │ │ connection_error = False                                             │ │  
    ...
    
    1865:  │ │                    │   │   model='gpt-4o-mini',                      │ │  
    1866:  │ │                    │   │   instructions=['test instruction'],        │ │  
    1867:  │ │                    │   │   default_settings=None                     │ │  
    1868:  │ │                    │   )                                             │ │  
    1869:  │ │                    }                                                 │ │  
    1870:  │ │           method = <bound method Connection.fetch of                 │ │  
    1871:  │ │                    <PoolConnectionProxy [released] 0x7fb0630d4ee0>>  │ │  
    1872:  │ │      method_name = 'fetch'                                           │ │  
    1873:  │ │    only_on_error = False                                             │ │  
    ...
    
    2068:  │ │           statement = None                                           │ │  
    2069:  │ │           stmt_name = '__asyncpg_stmt_16__'                          │ │  
    2070:  │ │             timeout = 90.0                                           │ │  
    2071:  │ │           use_cache = True                                           │ │  
    2072:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    2073:  │                                                                          │  
    2074:  │ in prepare:165                                                           │  
    2075:  ╰──────────────────────────────────────────────────────────────────────────╯  
    2076:  UndefinedTableError: relation "agents" does not exist                         
    2077:  ─────────────────────────── query: update agent sql ────────────────────────────
    2078:  Failed at tests/test_agent_queries.py                                         
    ...
    
    2219:  │                                                                          │  
    2220:  │ /home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/ │  
    2221:  │ python3.12/asyncio/runners.py:118 in run                                 │  
    2222:  │                                                                          │  
    2223:  │   115 │   │                                                              │  
    2224:  │   116 │   │   self._interrupt_count = 0                                  │  
    2225:  │   117 │   │   try:                                                       │  
    2226:  │ ❱ 118 │   │   │   return self._loop.run_until_complete(task)             │  
    2227:  │   119 │   │   except exceptions.CancelledError:                          │  
    ...
    
    2233:  │ │           coro = <coroutine object test_developer at 0x7fb062ee8900> │ │  
    2234:  │ │           self = <asyncio.runners.Runner object at 0x7fb0630d6990>   │ │  
    2235:  │ │ sigint_handler = functools.partial(<bound method Runner._on_sigint   │ │  
    2236:  │ │                  of <asyncio.runners.Runner object at                │ │  
    2237:  │ │                  0x7fb0630d6990>>, main_task=<Task finished          │ │  
    2238:  │ │                  name='Task-27' coro=<test_developer() done, defined │ │  
    2239:  │ │                  at                                                  │ │  
    2240:  │ │                  /home/runner/work/julep/julep/agents-api/tests/fix… │ │  
    2241:  │ │                  exception=UndefinedTableError('relation             │ │  
    2242:  │ │                  "developers" does not exist')>)                     │ │  
    2243:  │ │           task = <Task finished name='Task-27'                       │ │  
    2244:  │ │                  coro=<test_developer() done, defined at             │ │  
    2245:  │ │                  /home/runner/work/julep/julep/agents-api/tests/fix… │ │  
    2246:  │ │                  exception=UndefinedTableError('relation             │ │  
    ...
    
    2265:  │ │          dsn = '***localhost:32769/test?sslmode=d… │ │  
    2266:  │ │         pool = <asyncpg.pool.Pool object at 0x7fb06306bc40>          │ │  
    2267:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    2268:  │                                                                          │  
    2269:  │ /home/runner/work/julep/julep/agents-api/agents_api/queries/utils.py:301 │  
    2270:  │ in async_wrapper                                                         │  
    2271:  │                                                                          │  
    2272:  │   298 │   │   │   │   result: T = await func(*args, **kwargs)            │  
    2273:  │   299 │   │   │   except BaseException as error:                         │  
    2274:  │   300 │   │   │   │   _check_error(error)                                │  
    2275:  │ ❱ 301 │   │   │   │   raise error                                        │  
    ...
    
    2289:  │                                                                          │  
    2290:  │ /home/runner/work/julep/julep/agents-api/agents_api/queries/utils.py:298 │  
    2291:  │ in async_wrapper                                                         │  
    2292:  │                                                                          │  
    2293:  │   295 │   │   @wraps(func)                                               │  
    2294:  │   296 │   │   async def async_wrapper(*args: P.args, **kwargs: P.kwargs) │  
    2295:  │   297 │   │   │   try:                                                   │  
    2296:  │ ❱ 298 │   │   │   │   result: T = await func(*args, **kwargs)            │  
    2297:  │   299 │   │   │   except BaseException as error:                         │  
    2298:  │   300 │   │   │   │   _check_error(error)                                │  
    2299:  │   301 │   │   │   │   raise error                                        │  
    ...
    
    2354:  │ │                    │   │   │   │                                     │ │  
    2355:  │ │                    '00000000-0000-0000-0000-000000000000'            │ │  
    2356:  │ │                    │   │   │   ],                                    │ │  
    2357:  │ │                    │   │   │   'timeout': 90.0                       │ │  
    2358:  │ │                    │   │   }                                         │ │  
    2359:  │ │                    │   )                                             │ │  
    2360:  │ │                    ]                                                 │ │  
    2361:  │ │             conn = <PoolConnectionProxy [released] 0x7fb0630d41f0>   │ │  
    2362:  │ │ connection_error = False                                             │ │  
    ...
    
    2364:  │ │            debug = None                                              │ │  
    2365:  │ │           kwargs = {                                                 │ │  
    2366:  │ │                    │   'developer_id':                               │ │  
    2367:  │ │                    UUID('00000000-0000-0000-0000-000000000000')      │ │  
    2368:  │ │                    }                                                 │ │  
    2369:  │ │           method = <bound method Connection.fetch of                 │ │  
    2370:  │ │                    <PoolConnectionProxy [released] 0x7fb0630d41f0>>  │ │  
    2371:  │ │      method_name = 'fetch'                                           │ │  
    2372:  │ │    only_on_error = False                                             │ │  
    ...
    
    2506:  │ │           statement = None                                           │ │  
    2507:  │ │           stmt_name = '__asyncpg_stmt_21__'                          │ │  
    2508:  │ │             timeout = 90.0                                           │ │  
    2509:  │ │           use_cache = True                                           │ │  
    2510:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    2511:  │                                                                          │  
    2512:  │ in prepare:165                                                           │  
    2513:  ╰──────────────────────────────────────────────────────────────────────────╯  
    2514:  UndefinedTableError: relation "developers" does not exist                     
    ...
    
    2810:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    2811:  │                                                                          │  
    2812:  │ /home/runner/work/julep/julep/agents-api/.venv/lib/python3.12/site-packa │  
    2813:  │ ges/ward/testing.py:637 in _resolve_single_arg                           │  
    2814:  │                                                                          │  
    2815:  │   634 │   │   │   else:                                                  │  
    2816:  │   635 │   │   │   │   fixture.resolved_val = arg(**args_to_inject)       │  
    2817:  │   636 │   │   except (Exception, SystemExit) as e:                       │  
    2818:  │ ❱ 637 │   │   │   raise FixtureError(f"Unable to resolve fixture '{fixtu │  
    ...
    
    2926:  │ │                     │   │   timer=<ward._testing._Timer object at    │ │  
    2927:  │ │                     0x7fb0630d5dc0>,                                 │ │  
    2928:  │ │                     │   │   tags=[]                                  │ │  
    2929:  │ │                     │   ),                                           │ │  
    2930:  │ │                     │   iteration=0                                  │ │  
    2931:  │ │                     )                                                │ │  
    2932:  │ ╰──────────────────────────────────────────────────────────────────────╯ │  
    2933:  ╰──────────────────────────────────────────────────────────────────────────╯  
    2934:  FixtureError: Unable to resolve fixture 'test_developer'                      
    2935:  ────────────────────────────────────────────────────────────────────────────────
    2936:  ╭──────────── Results ─────────────╮
    2937:  │  3  Tests Encountered            │
    2938:  │  3  Failures           (100.0%)  │
    2939:  ╰──────────────────────────────────╯
    2940:  ─────────────────────────── FAILED in 42.71 seconds ────────────────────────────
    2941:  ##[error]Process completed with exit code 1.
    

    ✨ CI feedback usage guide:

    The CI feedback tool (/checks) automatically triggers when a PR has a failed check.
    The tool analyzes the failed checks and provides several feedbacks:

    • Failed stage
    • Failed test name
    • Failure summary
    • Relevant error logs

    In addition to being automatically triggered, the tool can also be invoked manually by commenting on a PR:

    /checks "https://github.com/{repo_name}/actions/runs/{run_number}/job/{job_number}"
    

    where {repo_name} is the name of the repository, {run_number} is the run number of the failed check, and {job_number} is the job number of the failed check.

    Configuration options

    • enable_auto_checks_feedback - if set to true, the tool will automatically provide feedback when a check is failed. Default is true.
    • excluded_checks_list - a list of checks to exclude from the feedback, for example: ["check1", "check2"]. Default is an empty list.
    • enable_help_text - if set to true, the tool will provide a help message with the feedback. Default is true.
    • persistent_comment - if set to true, the tool will overwrite a previous checks comment with the new feedback. Default is true.
    • final_update_message - if persistent_comment is true and updating a previous checks message, the tool will also create a new message: "Persistent checks updated to latest commit". Default is true.

    See more information about the checks tool in the docs.

    Copy link

    qodo-merge-pro-for-open-source bot commented Dec 19, 2024

    PR Reviewer Guide 🔍

    (Review updated until commit 76819e1)

    Here are some key observations to aid the review process:

    ⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
    🧪 PR contains tests
    🔒 No security concerns identified
    ⚡ Recommended focus areas for review

    Error Handling

    The new interceptor-based system for handling large data objects needs careful review of error handling and edge cases, particularly around the blob store operations and remote object loading/saving.

    async def handle_execution_with_errors[I, T](
        execution_fn: Callable[[I], Awaitable[T]],
        input: I,
    ) -> T:
        """
        Common error handling logic for both activities and workflows.
    
        Args:
            execution_fn: Async function to execute with error handling
            input: Input to the execution function
    
        Returns:
            The result of the execution function
    
        Raises:
            ApplicationError: For non-retryable errors
            Any other exception: For retryable errors
        """
        try:
            return await execution_fn(input)
        except PASSTHROUGH_EXCEPTIONS:
            raise
        except BaseException as e:
            if not is_retryable_error(e):
                raise ApplicationError(
                    str(e),
                    type=type(e).__name__,
                    non_retryable=True,
                )
            raise
    Type Safety

    The RemoteObject implementation with generic types should be validated to ensure type safety is maintained when serializing/deserializing objects, especially with complex nested types.

    class RemoteObject(Generic[T]):
        _type: Type[T]
        key: str
        bucket: str
    
        @classmethod
        async def from_value(cls, x: T) -> Self:
            await async_s3.setup()
    
            serialized = serialize(x)
    
            key = await async_s3.add_object_with_hash(serialized)
            return RemoteObject[T](key=key, bucket=blob_store_bucket, _type=type(x))
    
        async def load(self) -> T:
            await async_s3.setup()
    
            fetched = await async_s3.get_object(self.key)
            return cast(self._type, deserialize(fetched))
    Configuration Change

    The modification to blob store configuration for testing environment could impact existing tests and deployments. Need to verify this change doesn't break existing functionality.

    use_blob_store_for_temporal: bool = testing or env.bool(
        "USE_BLOB_STORE_FOR_TEMPORAL", default=False
    )

    Copy link
    Contributor

    @ellipsis-dev ellipsis-dev bot left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    👍 Looks good to me! Reviewed everything up to e158f3a in 1 minute and 32 seconds

    More details
    • Looked at 5962 lines of code in 91 files
    • Skipped 0 files when reviewing.
    • Skipped posting 7 drafted comments based on config settings.
    1. agents-api/agents_api/activities/embed_docs.py:14
    • Draft comment:
      Ensure that the removal of @auto_blob_store(deep=True) is compensated by the new interceptor-based system to handle large data offloading.
    • Reason this comment was not posted:
      Confidence changes required: 50%
      The removal of the auto_blob_store decorator is consistent across multiple files. This change aligns with the PR's intent to replace it with an interceptor-based system. However, the decorator's removal should be verified to ensure that the new system handles the same functionality.
    2. agents-api/agents_api/activities/execute_system.py:36
    • Draft comment:
      Verify that the new interceptor system handles the functionality previously managed by load_from_blob_store_if_remote.
    • Reason this comment was not posted:
      Confidence changes required: 50%
      The load_from_blob_store_if_remote function call is removed, which was responsible for loading data from a remote blob store. Ensure that the new interceptor system handles this functionality.
    3. agents-api/agents_api/activities/task_steps/base_evaluate.py:62
    • Draft comment:
      Ensure that the removal of @auto_blob_store(deep=True) is consistent with the new interceptor-based system for data handling.
    • Reason this comment was not posted:
      Confidence changes required: 50%
      The auto_blob_store decorator is removed from multiple functions. This change should be consistent with the new interceptor-based system to ensure data handling is still managed correctly.
    4. agents-api/agents_api/activities/task_steps/cozo_query_step.py:10
    • Draft comment:
      Ensure that the removal of @auto_blob_store(deep=True) is consistent with the new interceptor-based system for data handling.
    • Reason this comment was not posted:
      Confidence changes required: 50%
      The auto_blob_store decorator is removed from multiple functions. This change should be consistent with the new interceptor-based system to ensure data handling is still managed correctly.
    5. agents-api/agents_api/activities/task_steps/evaluate_step.py:10
    • Draft comment:
      Ensure that the removal of @auto_blob_store(deep=True) is consistent with the new interceptor-based system for data handling.
    • Reason this comment was not posted:
      Confidence changes required: 50%
      The auto_blob_store decorator is removed from multiple functions. This change should be consistent with the new interceptor-based system to ensure data handling is still managed correctly.
    6. agents-api/agents_api/activities/task_steps/for_each_step.py:11
    • Draft comment:
      Ensure that the removal of @auto_blob_store(deep=True) is consistent with the new interceptor-based system for data handling.
    • Reason this comment was not posted:
      Confidence changes required: 50%
      The auto_blob_store decorator is removed from multiple functions. This change should be consistent with the new interceptor-based system to ensure data handling is still managed correctly.
    7. agents-api/agents_api/activities/task_steps/get_value_step.py:11
    • Draft comment:
      Ensure that the removal of @auto_blob_store(deep=True) is consistent with the new interceptor-based system for data handling.
    • Reason this comment was not posted:
      Confidence changes required: 50%
      The auto_blob_store decorator is removed from multiple functions. This change should be consistent with the new interceptor-based system to ensure data handling is still managed correctly.

    Workflow ID: wflow_AjDxfoccjl58OUog


    You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

    Copy link

    qodo-merge-pro-for-open-source bot commented Dec 19, 2024

    PR Code Suggestions ✨

    Latest suggestions up to 76819e1
    Explore these optional code suggestions:

    CategorySuggestion                                                                                                                                    Score
    General
    Return appropriate HTTP status code for health check failures

    Add status code handling to return 503 Service Unavailable when health check fails
    instead of returning a 200 OK with error message.

    agents-api/agents_api/routers/healthz/check_health.py [8-19]

    +from fastapi import HTTPException, status
     @router.get("/healthz", tags=["healthz"])
     async def check_health() -> dict:
         try:
             list_agents_query(
                 developer_id=UUID("00000000-0000-0000-0000-000000000000"),
             )
         except Exception as e:
             logging.error("An error occurred while checking health: %s", str(e))
    -        return {"status": "error", "message": "An internal error has occurred."}
    +        raise HTTPException(
    +            status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
    +            detail="Service is unhealthy"
    +        )
         return {"status": "ok"}
    • Apply this suggestion
    Suggestion importance[1-10]: 9

    Why: Using proper HTTP status codes is crucial for health check endpoints as it enables monitoring systems to correctly identify service health status, making this a high-impact improvement.

    9
    Possible issue
    Add error handling for blob store data loading failures to prevent silent failures

    Add error handling for the case when load_if_remote fails to load data from the blob
    store. This could happen if the blob store is unavailable or if the data is
    corrupted.

    agents-api/agents_api/common/interceptors.py [52-56]

     async def load_if_remote[T](arg: T | RemoteObject[T]) -> T:
         if use_blob_store_for_temporal and isinstance(arg, RemoteObject):
    -        return await arg.load()
    +        try:
    +            return await arg.load()
    +        except Exception as e:
    +            raise ApplicationError(f"Failed to load remote object: {str(e)}", non_retryable=True)
         return arg
    • Apply this suggestion
    Suggestion importance[1-10]: 8

    Why: Adding error handling for blob store loading failures is critical as it prevents silent failures and provides clear error messages when remote data cannot be accessed, which is essential for debugging and system reliability.

    8
    Add validation and error handling for blob store data storage operations

    Add size validation before attempting to offload data to blob store to prevent
    attempting to store empty or invalid data.

    agents-api/agents_api/common/interceptors.py [59-63]

     async def offload_if_large[T](result: T) -> T:
    +    if result is None:
    +        return result
         if use_blob_store_for_temporal and is_too_large(result):
    -        return await RemoteObject.from_value(result)
    +        try:
    +            return await RemoteObject.from_value(result)
    +        except Exception as e:
    +            raise ApplicationError(f"Failed to store remote object: {str(e)}", non_retryable=True)
         return result
    • Apply this suggestion
    Suggestion importance[1-10]: 7

    Why: The suggestion improves data integrity by validating input and handling storage errors explicitly, preventing potential issues with storing invalid or corrupted data.

    7

    Previous suggestions

    Suggestions up to commit e158f3a
    CategorySuggestion                                                                                                                                    Score
    Possible issue
    Fix SQL parameter count mismatch in entry relations query

    The SQL query for entry relations has an incorrect number of parameters. The VALUES
    clause has 5 placeholders but only 4 parameters are provided in the query string.

    agents-api/agents_api/queries/entries/create_entries.py [43-50]

     entry_relation_query = """
     INSERT INTO entry_relations (
         session_id,
         head,
         relation,
    -    tail,
    -) VALUES ($1, $2, $3, $4, $5)
    +    tail
    +) VALUES ($1, $2, $3, $4)
     RETURNING *;
     """
    Suggestion importance[1-10]: 10

    Why: Critical bug fix: The SQL query has 5 placeholders ($1-$5) but only 4 columns in the VALUES clause, which would cause a database error. The fix removes the trailing comma and extra placeholder.

    10
    Add timeout handling for remote object loading to prevent workflow hangs

    Add timeout handling for remote object loading operations to prevent indefinite
    hangs.

    agents-api/agents_api/common/interceptors.py [52-56]

     async def load_if_remote[T](arg: T | RemoteObject[T]) -> T:
         if use_blob_store_for_temporal and isinstance(arg, RemoteObject):
    -        return await arg.load()
    +        try:
    +            return await asyncio.wait_for(arg.load(), timeout=30.0)
    +        except asyncio.TimeoutError:
    +            raise ApplicationError("Remote object load timeout", non_retryable=True)
         return arg
    Suggestion importance[1-10]: 9

    Why: Adding timeout handling is critical for system stability as it prevents indefinite hangs in workflows when remote objects fail to load. The suggestion provides a concrete timeout value and proper error handling.

    9
    Fix incorrect error message and status code for unique constraint violations

    The error message in the UniqueViolationError handler incorrectly states "The
    specified developer does not exist" when it should indicate a duplicate developer
    error. Update the error message to accurately reflect the unique constraint
    violation.

    agents-api/agents_api/queries/developers/create_developer.py [39-43]

     asyncpg.UniqueViolationError: partialclass(
         HTTPException,
    -    status_code=404,
    -    detail="The specified developer does not exist.",
    +    status_code=409,
    +    detail="A developer with this ID or email already exists.",
     )
    Suggestion importance[1-10]: 9

    Why: The suggestion fixes a critical issue where the error handling provides misleading information and incorrect HTTP status code (404 instead of 409) for unique constraint violations, which could confuse API clients and make debugging harder.

    9
    Add error handling for blob store operations to prevent silent failures

    Add error handling for the blob store operations to handle potential
    storage/retrieval failures gracefully.

    agents-api/agents_api/common/interceptors.py [59-63]

     async def offload_if_large[T](result: T) -> T:
    -    if use_blob_store_for_temporal and is_too_large(result):
    -        return await RemoteObject.from_value(result)
    -    return result
    +    try:
    +        if use_blob_store_for_temporal and is_too_large(result):
    +            return await RemoteObject.from_value(result)
    +        return result
    +    except Exception as e:
    +        raise ApplicationError(f"Failed to offload large result: {str(e)}", non_retryable=True)
    Suggestion importance[1-10]: 8

    Why: Adding error handling for blob store operations is crucial for system reliability, as silent failures in storage operations could lead to data loss or corruption. The suggestion properly wraps errors in ApplicationError with non-retryable flag.

    8
    Remove incorrect database error handler for read operation

    The UniqueViolationError handler is incorrectly used for a GET operation - this
    error cannot occur during a SELECT query. Remove this unnecessary error handler.

    agents-api/agents_api/queries/users/get_user.py [34-38]

    -asyncpg.UniqueViolationError: partialclass(
    +asyncpg.NoDataFoundError: partialclass(
         HTTPException,
         status_code=404,
         detail="The specified user does not exist.",
     )
    Suggestion importance[1-10]: 8

    Why: The suggestion identifies and fixes an incorrect error handler that cannot occur during SELECT operations, replacing it with the appropriate NoDataFoundError handler for better error handling accuracy.

    8
    Prevent unintentional NULL overwrites in PATCH operations

    The SQL query uses COALESCE for all fields which could unintentionally overwrite
    existing values with NULL when fields are not provided in the patch request. Use a
    dynamic SET clause that only updates provided fields.

    agents-api/agents_api/queries/sessions/patch_session.py [17-25]

     SET
    -    situation = COALESCE($3, situation),
    -    system_template = COALESCE($4, system_template),
    +    situation = CASE WHEN $3 IS NOT NULL THEN $3 ELSE situation END,
    +    system_template = CASE WHEN $4 IS NOT NULL THEN $4 ELSE system_template END,
         metadata = sessions.metadata || $5,
    -    render_templates = COALESCE($6, render_templates),
    -    token_budget = COALESCE($7, token_budget),
    -    context_overflow = COALESCE($8, context_overflow),
    -    forward_tool_calls = COALESCE($9, forward_tool_calls),
    +    render_templates = CASE WHEN $6 IS NOT NULL THEN $6 ELSE render_templates END,
    +    token_budget = CASE WHEN $7 IS NOT NULL THEN $7 ELSE token_budget END,
    +    context_overflow = CASE WHEN $8 IS NOT NULL THEN $8 ELSE context_overflow END,
    +    forward_tool_calls = CASE WHEN $9 IS NOT NULL THEN $9 ELSE forward_tool_calls END,
         recall_options = sessions.recall_options || $10
    Suggestion importance[1-10]: 7

    Why: The suggestion improves the robustness of the PATCH operation by preventing unintentional NULL overwrites of existing values when fields are not provided in the request, using CASE statements instead of COALESCE.

    7
    Add type validation when loading remote objects to prevent runtime type errors

    Add type validation for the input arguments before loading from blob store to ensure
    type safety. The current implementation assumes all args can be remote objects
    without validation.

    agents-api/agents_api/common/interceptors.py [79-80]

     if use_blob_store_for_temporal:
    -    input.args = await asyncio.gather(*[load_if_remote(arg) for arg in args])
    +    input.args = await asyncio.gather(*[
    +        load_if_remote(arg) if isinstance(arg, (RemoteObject, T)) else arg 
    +        for arg in args
    +    ])
    Suggestion importance[1-10]: 5

    Why: The suggestion adds type validation to prevent potential runtime errors when loading remote objects, improving type safety. However, the impact is moderate since the existing code would likely handle most cases correctly.

    5

    Signed-off-by: Diwank Singh Tomer <[email protected]>
    Copy link
    Contributor

    @ellipsis-dev ellipsis-dev bot left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    👍 Looks good to me! Incremental review on ca5f4e2 in 50 seconds

    More details
    • Looked at 53 lines of code in 4 files
    • Skipped 0 files when reviewing.
    • Skipped posting 3 drafted comments based on config settings.
    1. agents-api/tests/fixtures.py:3
    • Draft comment:
      The import of time is unnecessary as it is not used in this file. Consider removing it to clean up the code.
    • Reason this comment was not posted:
      Confidence changes required: 50%
      The import of time in agents-api/tests/fixtures.py is unnecessary as it is not used anywhere in the file. Removing unused imports is a good practice to keep the code clean and efficient.
    2. agents-api/agents_api/queries/sessions/create_session.py:11
    • Draft comment:
      The import of ResourceCreatedResponse is unnecessary as it is not used in this file. Consider removing it to clean up the code.
    • Reason this comment was not posted:
      Confidence changes required: 50%
      The import of ResourceCreatedResponse in agents-api/agents_api/queries/sessions/create_session.py is unnecessary as it is not used anywhere in the file. Removing unused imports is a good practice to keep the code clean and efficient.
    3. agents-api/tests/test_session_queries.py:13
    • Draft comment:
      The import of ResourceCreatedResponse is unnecessary as it is not used in this file. Consider removing it to clean up the code.
    • Reason this comment was not posted:
      Confidence changes required: 50%
      The import of ResourceCreatedResponse in agents-api/tests/test_session_queries.py is unnecessary as it is not used anywhere in the file. Removing unused imports is a good practice to keep the code clean and efficient.

    Workflow ID: wflow_4DsKSc8XWp0obflW


    You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

    @creatorrr creatorrr marked this pull request as draft December 21, 2024 08:25
    @creatorrr creatorrr marked this pull request as ready for review December 21, 2024 15:39

    Persistent review updated to latest commit 76819e1

    Copy link
    Contributor

    @ellipsis-dev ellipsis-dev bot left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    👍 Looks good to me! Reviewed everything up to 76819e1 in 1 minute and 23 seconds

    More details
    • Looked at 1358 lines of code in 37 files
    • Skipped 0 files when reviewing.
    • Skipped posting 2 drafted comments based on config settings.
    1. agents-api/agents_api/routers/healthz/check_health.py:12
    • Draft comment:
      Consider using an asynchronous call for list_agents_query to avoid blocking behavior.
    • Reason this comment was not posted:
      Decided after close inspection that this draft comment was likely wrong and/or not actionable:
      This is a health check endpoint, so performance is not critical. The blocking nature of list_agents_query won't significantly impact the system since this endpoint is called infrequently. Making it async would be a nice-to-have optimization but not essential. The comment doesn't indicate a clear problem that needs fixing.
      The comment could be valid if this is part of a larger async system where blocking calls are strictly avoided. The blocking call could potentially cause issues under high load.
      For a health check endpoint, simplicity is more important than perfect async implementation. The blocking behavior is acceptable here given the endpoint's purpose and frequency of use.
      Delete the comment as it suggests an optimization that isn't clearly necessary for this health check endpoint.
    2. agents-api/agents_api/activities/sync_items_remote.py:14
    • Draft comment:
      Consider using return_exceptions=True in asyncio.gather to handle exceptions from individual coroutines without cancelling others.
    • Reason this comment was not posted:
      Confidence changes required: 50%
      In sync_items_remote.py, the save_inputs_remote_fn and load_inputs_remote_fn functions are using asyncio.gather without specifying return_exceptions=True. This means if one coroutine raises an exception, the others will be cancelled. Consider whether this behavior is desired.

    Workflow ID: wflow_0hHRxtRzosn0tTXv


    You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

    @creatorrr creatorrr merged commit 1982583 into f/switch-to-pg Dec 21, 2024
    7 of 11 checks passed
    @creatorrr creatorrr deleted the f/simplify-blob-store branch December 21, 2024 15:41
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    1 participant