You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Microsoft.Extensions.AI.OllamaChatClientIntegrationTests.CompleteAsync_StructuredOutputEnum
Message:
Assert.Equal() Failure: Values differ
Expected: Arm64
Actual: Arm
Stack Trace:
ChatClientIntegrationTests.CompleteAsync_StructuredOutputEnum() line 665
--- End of stack trace from previous location ---
I ran this with "Run until failure" in VS and got 4 passes then a failure:
To be honest I think this is a fair pass rate given the way the test is set up. M2 chips are ARM-based, even though ARM64 is a more precise statement. It's totally reasonable for this model to return ARM instead of ARM64 sometimes.
Given that what we want from our tests is a reliable signal about whether things are working or not, I'll update the test so that it's a less ambiguous test case by defining a different enum type.
cc: @SteveSandersonMS
The text was updated successfully, but these errors were encountered: