-
Notifications
You must be signed in to change notification settings - Fork 38
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update validator to handle typebox literal values #154
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for crawling through the rats nest to try to make this better.
src/validator.ts
Outdated
const constValueErrors = propErrorGroups[property]; | ||
|
||
typeErrorsList.push( | ||
`property '${property}' is a constant which must be equal to one of: ` + |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we could make this slightly more concise.
`property '${property}' is a constant which must be equal to one of: ` + | |
`property '${property}' must be equal to one of: ` + |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good idea, thanks!
…pdate relevant typescript types, fix bug in vectorOperationsProvider, update related unit tests
2ad2a18
to
c655eae
Compare
@@ -27,7 +26,7 @@ export type IndexListDescription = { | |||
export type IndexList = Array<IndexListDescription>; | |||
|
|||
export const listIndexes = (api: IndexOperationsApi) => { | |||
return async (): Promise<Array<IndexListMeta>> => { | |||
return async (): Promise<IndexList> => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Snuck this update in after merging the last PR.
_This revision is dependent on a previous branches / PRs. These will need to be reviewed and merged first:_ - #153 - #154 ## Problem We want to be able to track usage specific to the spruce release of the client. ## Solution - Update `buildUserAgent` logic to check for `packageInfo.release`, and append to the user agent if present. - Update `release-spruce-dev` and `release-spruce` workflows to add `@spruce` to `version.json` during these specific releases. ## Type of Change - [X] New feature (non-breaking change which adds functionality) ## Test Plan We will need to release through either the `release-spruce` or `release-spruce-dev` workflows and validate the `version.json` file has the proper info, and the `release` is appended to the user agent header.
_This revision is dependent on a previous branches / PRs. These will need to be reviewed and merged first:_ - #153 ## Problem When replacing the OpenAPI generated core in a previous revision (#153), our `validator` module which handles errors emitted from typebox was not handling literal types correctly. We added new union of literal types to support `cloud` values: ``` export const CloudSchema = Type.Union([ Type.Literal('gcp'), Type.Literal('aws'), Type.Literal('azure'), ]); ``` When passing an incorrect value for the `cloud` field, validator would not handle things gracefully: ![Screenshot 2023-11-03 at 7 28 21 PM](https://github.com/pinecone-io/pinecone-ts-client/assets/119623786/c2485c24-c898-49eb-81cc-260e3532f299) The errors emitted from typebox look like this: ![Screenshot 2023-11-03 at 7 31 11 PM](https://github.com/pinecone-io/pinecone-ts-client/assets/119623786/2ebbeeca-8a6a-4867-b04e-3cbfb2d1150f) ## Solution This took a lot of trial and error and testing, but I think this is a reasonable approach. I'm sure there's a lot more bike shedding and refactoring we could do in `validator`, but for now it feels most sensible to try and cater to the typebox schemas we support. Previously, at the top level of `errorFormatter` we were pulling out all errors that contained `anyOf` in the schema. We return early with the assumption that these errors are specific to the shape of the top level arguments being validated. Generally, these types of errors will not have an `instancePath`. It is also possible to have `anyOf` errors relating to a specific property, which in our case is the `cloud` field on `createIndex`. - At the top-level of `errorFormatter()` rename `anyOfErrors` to `anyOfArgumentErrors`, and update the filter to exclude errors with keyword of `const` or `type`. - Update `typeErrors()` to handle possible `anyOf` errors with keyword `const` and `instancePath.length > 1` as these are errors specific to a property whose type is some set of literal values. - Update broken unit tests. ## Type of Change - [X] Bug fix (non-breaking change which fixes an issue) ## Test Plan Pull this PR down and test using the repl: ``` $ npm run repl $ await init() // call createIndex and test that passing invalid cloud values throws correctly $ await client.createIndex({ name: 'test-index', dimension: 10, cloud: 'gg', region: 'us-east4', capacityMode: 'pod' }) // Uncaught: // PineconeArgumentError: The argument to createIndex had type errors: property 'cloud' is a constant which must be equal to one of: 'gcp', 'aws', 'azure'. // at /Users/austin/workspace/pinecone-ts-client/dist/validator.js:239:19 // at /Users/austin/workspace/pinecone-ts-client/dist/control/createIndex.js:67:21 // at step (/Users/austin/workspace/pinecone-ts-client/dist/control/createIndex.js:33:23) // at Object.next (/Users/austin/workspace/pinecone-ts-client/dist/control/createIndex.js:14:53) // at /Users/austin/workspace/pinecone-ts-client/dist/control/createIndex.js:8:71 // at new Promise (<anonymous>) // at __awaiter (/Users/austin/workspace/pinecone-ts-client/dist/control/createIndex.js:4:12) // at Pinecone._createIndex (/Users/austin/workspace/pinecone-ts-client/dist/control/createIndex.js:62:40) // at Pinecone.createIndex (/Users/austin/workspace/pinecone-ts-client/dist/pinecone.js:243:21) ```
_This revision is dependent on a previous branches / PRs. These will need to be reviewed and merged first:_ - #153 - #154 ## Problem We want to be able to track usage specific to the spruce release of the client. ## Solution - Update `buildUserAgent` logic to check for `packageInfo.release`, and append to the user agent if present. - Update `release-spruce-dev` and `release-spruce` workflows to add `@spruce` to `version.json` during these specific releases. ## Type of Change - [X] New feature (non-breaking change which adds functionality) ## Test Plan We will need to release through either the `release-spruce` or `release-spruce-dev` workflows and validate the `version.json` file has the proper info, and the `release` is appended to the user agent header.
_This revision is dependent on a previous branches / PRs. These will need to be reviewed and merged first:_ - #153 ## Problem When replacing the OpenAPI generated core in a previous revision (#153), our `validator` module which handles errors emitted from typebox was not handling literal types correctly. We added new union of literal types to support `cloud` values: ``` export const CloudSchema = Type.Union([ Type.Literal('gcp'), Type.Literal('aws'), Type.Literal('azure'), ]); ``` When passing an incorrect value for the `cloud` field, validator would not handle things gracefully: ![Screenshot 2023-11-03 at 7 28 21 PM](https://github.com/pinecone-io/pinecone-ts-client/assets/119623786/c2485c24-c898-49eb-81cc-260e3532f299) The errors emitted from typebox look like this: ![Screenshot 2023-11-03 at 7 31 11 PM](https://github.com/pinecone-io/pinecone-ts-client/assets/119623786/2ebbeeca-8a6a-4867-b04e-3cbfb2d1150f) ## Solution This took a lot of trial and error and testing, but I think this is a reasonable approach. I'm sure there's a lot more bike shedding and refactoring we could do in `validator`, but for now it feels most sensible to try and cater to the typebox schemas we support. Previously, at the top level of `errorFormatter` we were pulling out all errors that contained `anyOf` in the schema. We return early with the assumption that these errors are specific to the shape of the top level arguments being validated. Generally, these types of errors will not have an `instancePath`. It is also possible to have `anyOf` errors relating to a specific property, which in our case is the `cloud` field on `createIndex`. - At the top-level of `errorFormatter()` rename `anyOfErrors` to `anyOfArgumentErrors`, and update the filter to exclude errors with keyword of `const` or `type`. - Update `typeErrors()` to handle possible `anyOf` errors with keyword `const` and `instancePath.length > 1` as these are errors specific to a property whose type is some set of literal values. - Update broken unit tests. ## Type of Change - [X] Bug fix (non-breaking change which fixes an issue) ## Test Plan Pull this PR down and test using the repl: ``` $ npm run repl $ await init() // call createIndex and test that passing invalid cloud values throws correctly $ await client.createIndex({ name: 'test-index', dimension: 10, cloud: 'gg', region: 'us-east4', capacityMode: 'pod' }) // Uncaught: // PineconeArgumentError: The argument to createIndex had type errors: property 'cloud' is a constant which must be equal to one of: 'gcp', 'aws', 'azure'. // at /Users/austin/workspace/pinecone-ts-client/dist/validator.js:239:19 // at /Users/austin/workspace/pinecone-ts-client/dist/control/createIndex.js:67:21 // at step (/Users/austin/workspace/pinecone-ts-client/dist/control/createIndex.js:33:23) // at Object.next (/Users/austin/workspace/pinecone-ts-client/dist/control/createIndex.js:14:53) // at /Users/austin/workspace/pinecone-ts-client/dist/control/createIndex.js:8:71 // at new Promise (<anonymous>) // at __awaiter (/Users/austin/workspace/pinecone-ts-client/dist/control/createIndex.js:4:12) // at Pinecone._createIndex (/Users/austin/workspace/pinecone-ts-client/dist/control/createIndex.js:62:40) // at Pinecone.createIndex (/Users/austin/workspace/pinecone-ts-client/dist/pinecone.js:243:21) ```
_This revision is dependent on a previous branches / PRs. These will need to be reviewed and merged first:_ - #153 - #154 We want to be able to track usage specific to the spruce release of the client. - Update `buildUserAgent` logic to check for `packageInfo.release`, and append to the user agent if present. - Update `release-spruce-dev` and `release-spruce` workflows to add `@spruce` to `version.json` during these specific releases. - [X] New feature (non-breaking change which adds functionality) We will need to release through either the `release-spruce` or `release-spruce-dev` workflows and validate the `version.json` file has the proper info, and the `release` is appended to the user agent header.
This revision is dependent on a previous branches / PRs. These will need to be reviewed and merged first:
Problem
When replacing the OpenAPI generated core in a previous revision (#153), our
validator
module which handles errors emitted from typebox was not handling literal types correctly.We added new union of literal types to support
cloud
values:When passing an incorrect value for the
cloud
field, validator would not handle things gracefully:The errors emitted from typebox look like this:
Solution
This took a lot of trial and error and testing, but I think this is a reasonable approach. I'm sure there's a lot more bike shedding and refactoring we could do in
validator
, but for now it feels most sensible to try and cater to the typebox schemas we support.Previously, at the top level of
errorFormatter
we were pulling out all errors that containedanyOf
in the schema. We return early with the assumption that these errors are specific to the shape of the top level arguments being validated. Generally, these types of errors will not have aninstancePath
. It is also possible to haveanyOf
errors relating to a specific property, which in our case is thecloud
field oncreateIndex
.errorFormatter()
renameanyOfErrors
toanyOfArgumentErrors
, and update the filter to exclude errors with keyword ofconst
ortype
.typeErrors()
to handle possibleanyOf
errors with keywordconst
andinstancePath.length > 1
as these are errors specific to a property whose type is some set of literal values.Type of Change
Test Plan
Pull this PR down and test using the repl: