[AttributeError]: Create with completion on list[Model] does not work

[pedroallenrevez]

• This is actually a bug report.
• I am not getting good LLM Results
• I have tried asking for help in the community on discord or discussions and have not received a response.
• I have tried searching the documentation and have not found an answer.

What Model are you using?

• gpt-3.5-turbo
• gpt-4-turbo
• gpt-4
• Other (please specify) (all of them)

Describe the bug
create_with_completion doesn't work with list[Model]. AttributeError

To Reproduce

import instructor
from pydantic import BaseModel
from openai import OpenAI


# Define your desired output structure
class UserInfo(BaseModel):
    name: str
    age: int


# Patch the OpenAI client
client = instructor.from_openai(OpenAI())

# Extract structured data from natural language
user_info = client.chat.completions.create_with_completion(
    model="gpt-4o",
    response_model=list[UserInfo],
    messages=[{"role": "user", "content": "John Doe is 30 years old.t"}],
)
user_info

Errors in

270 return model, model._raw_response

Expected behavior
Getting completion responses like when using normal create with list[Model] which runs fine.
Use create_iterable? I want with completion, and there is no documentation on it.
I would be expecting the returned pydantic objects to have _raw_response but they dont

[ivanbelenky]

I think this is a known issues for the core devs. Comments in the process_response from 7 months ago on the codebase (pasted below).

In your case list[UserInfo] implies an AdapterBase model.

def process_response(...) -> T_Model | ChatCompletion:
    ...
    # ? This really hints at the fact that we need a better way of
    # ? attaching usage data and the raw response to the model we return.
    if isinstance(model, IterableBase):
        logger.debug(f"Returning takes from IterableBase")
        return [task for task in model.tasks]

    if isinstance(response_model, ParallelBase):
        logger.debug(f"Returning model from ParallelBase")
        return model

    if isinstance(model, AdapterBase):
        logger.debug(f"Returning model from AdapterBase")
        return model.content

    model._raw_response = response

The error you are pointing out is not just reflecting the need for better way of attaching usage data, but showing that this function is broken, i.e. returned objects are dependents on model type. I lack knowledge of the entire design of this project, but judging by the function's return statement

def create_with_completion(...):
    ...
    return model, model._raw_response

and others alike, they will probably benefit from a wrapper that enables .content and .extra_api_data_and_stuff properties.

As a hacky solution for now I would suggest compounding your desired list return type, with another BaseModel

class UserInfo(BaseModel):
    name: str
    age: int

class UserInfoList(BaseModel):
    user_infos: list[UserInfo]
    

user_info = client.chat.completions.create_with_completion(
    model="gpt-3.5-turbo",
    response_model=UserInfoList,
    messages=[{"role": "user", "content": "John Doe is 30 years old.t"}],
)

[ivanbelenky]

Proposed Solution

I just created a very general solution for this known problem.

I failed to find documentation for testing locally. Did I miss it in the docs @ivanleomk?

For the sake of replication, I tested everything locally as is it is in the release workflow (except cohere)

$ poetry run pytest tests/ -k 'not llm and not openai and not gemini and not anthropic and not cohere and not vertexai'

The idea is to wrap create calls inside a very basic Creation object

T = TypeVar("T", bound=Union[BaseModel, "Iterable[Any]", "Partial[Any]"])


class Creation(Generic[T]):
    raw: Any # should be uniform completion type
    processed: T

    def __init__(self, raw: Any, processed: T) -> None:
        self.raw, self.processed = raw, processed

Then just altering retry_sync and retry_async return type. I.e. from T_Model to Creation[T_Model].