Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adjust query to support ordering #107

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/run-tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ jobs:
- 5432:5432
- 8080:8080
- 8181:8181
- 8282:8282
steps:
- name: Checkout code
uses: actions/checkout@v2
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
"build": "npm run clean && npx tsc",
"start": "node build/src/index.js",
"dev": "cross-env NODE_NO_WARNINGS=1 npx nodemon src/index.ts",
"test": "npx tsc && for testfile in $(find build/tests -type f -name '*.test.js'); do node $testfile; done",
"test": "./run-tests.sh",
"clean": "rimraf ./build",
"gen-test-mocks": "cross-env NODE_NO_WARNINGS=1 node --loader ts-node/esm tests/mocked_sql/generate_mock_data.ts",
"benchmark": "cross-env NODE_NO_WARNINGS=1 node --loader ts-node/esm benchmark/setup.ts && npx artillery run benchmark/graphql.yaml --output benchmark/report.json",
Expand Down
11 changes: 11 additions & 0 deletions run-tests.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/usr/bin/env bash
set -e
shopt -s globstar # to expand '**' into nested directories./

npm run build

# find all unit tests in build and run them
for f in ./build/**/*test.js; do
echo "Running $f"
node --enable-source-maps --stack-trace-limit=1000 --test $f;
done
5 changes: 5 additions & 0 deletions schema.graphql
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
}

type EventData {
accountUpdateId: String!

Check notice on line 26 in schema.graphql

View workflow job for this annotation

GitHub Actions / GraphQL Inspector

Field 'accountUpdateId' was added to object type 'EventData'

Field 'accountUpdateId' was added to object type 'EventData'
transactionInfo: TransactionInfo
data: [String]!
}
Expand Down Expand Up @@ -50,6 +51,10 @@
hash: String!
memo: String!
authorizationKind: String!
sequenceNumber: Int! # TODO: Is it ok to make this required?

Check notice on line 54 in schema.graphql

View workflow job for this annotation

GitHub Actions / GraphQL Inspector

Field 'sequenceNumber' was added to object type 'TransactionInfo'

Field 'sequenceNumber' was added to object type 'TransactionInfo'
zkappAccountUpdateIds: [Int]!

Check notice on line 55 in schema.graphql

View workflow job for this annotation

GitHub Actions / GraphQL Inspector

Field 'zkappAccountUpdateIds' was added to object type 'TransactionInfo'

Field 'zkappAccountUpdateIds' was added to object type 'TransactionInfo'
zkappEventElementIds: [Int]!

Check notice on line 56 in schema.graphql

View workflow job for this annotation

GitHub Actions / GraphQL Inspector

Field 'zkappEventElementIds' was added to object type 'TransactionInfo'

Field 'zkappEventElementIds' was added to object type 'TransactionInfo'
zkappFieldArrayElementIds: [Int]!

Check notice on line 57 in schema.graphql

View workflow job for this annotation

GitHub Actions / GraphQL Inspector

Field 'zkappFieldArrayElementIds' was added to object type 'TransactionInfo'

Field 'zkappFieldArrayElementIds' was added to object type 'TransactionInfo'
}

type ActionStates {
Expand Down
5 changes: 5 additions & 0 deletions src/blockchain/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ export enum BlockStatusFilter {
}

export type Event = {
accountUpdateId: string;
transactionInfo: TransactionInfo;
data: string[];
};
Expand Down Expand Up @@ -41,6 +42,10 @@ export type TransactionInfo = {
hash: string;
memo: string;
authorizationKind: string;
sequenceNumber: number;
zkappAccountUpdateIds: number[];
zkappEventElementIds: number[];

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'll want another similar field for Events (this is for Actions) to include as part of the Events data, no?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need that, assuming events will not be sorted?

zkappFieldArrayElementIds: number[];
};

export type Events = {
Expand Down
6 changes: 6 additions & 0 deletions src/blockchain/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,14 +24,20 @@ export function createTransactionInfo(
hash: row.hash,
memo: row.memo,
authorizationKind: row.authorization_kind,
sequenceNumber: row.sequence_number,
zkappAccountUpdateIds: row.zkapp_account_updates_ids,
zkappEventElementIds: row.zkapp_event_element_ids,
zkappFieldArrayElementIds: row.zkapp_field_array_element_ids,
};
}

export function createEvent(
accountUpdateId: string,
data: string[],
transactionInfo: TransactionInfo
): Event {
return {
accountUpdateId,
data,
transactionInfo,
};
Expand Down
3 changes: 3 additions & 0 deletions src/db/sql/events-actions/queries.ts
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,7 @@ function emittedZkAppCommandsCTE(db_client: postgres.Sql) {
SELECT
blocks_accessed.*,
zkcu.id AS zkapp_account_update_id,
bzkc.sequence_no AS sequence_number,
zkapp_fee_payer_body_id,
zkapp_account_updates_ids,
authorization_kind,
Expand Down Expand Up @@ -136,6 +137,7 @@ function emittedEventsCTE(db_client: postgres.Sql) {
*,
zke.id AS zkapp_event_id,
zke.element_ids AS zkapp_event_element_ids,
zkfa.element_ids AS zkapp_field_array_element_ids,
zkfa.id AS zkapp_event_array_id
FROM
emitted_zkapp_commands
Expand All @@ -153,6 +155,7 @@ function emittedActionsCTE(db_client: postgres.Sql) {
*,
zke.id AS zkapp_event_id,
zke.element_ids AS zkapp_event_element_ids,
zkfa.element_ids AS zkapp_field_array_element_ids,
zkfa.id AS zkapp_event_array_id
FROM
emitted_zkapp_commands
Expand Down
11 changes: 11 additions & 0 deletions src/db/sql/events-actions/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,9 @@ export type ArchiveNodeDatabaseRow = {
// Current status of the block within the chain.
chain_status: string;

// Sequence number of the transaction within a block
sequence_number: number;

// Hash representing the ledger state.
ledger_hash: string;

Expand All @@ -42,6 +45,10 @@ export type ArchiveNodeDatabaseRow = {
// Unique identifier for the zkapp account update.
zkapp_account_update_id: number;

// TODO: Would this be more accurately named `zkapp_event_field_array_id`?
// Unique identifier for the event within an account update.
zkapp_event_id: number;

// List of identifiers inside a zkapp account update.
zkapp_account_updates_ids: number[];

Expand All @@ -57,9 +64,13 @@ export type ArchiveNodeDatabaseRow = {
// The unique identifier that maps events/actions to a specific zkApp.
zkapp_event_array_id: number;

// TODO: Would this be more accuratley named `zkapp_event_field_array_ids`?
// List of `element_ids` that are used to construct the zkApp event.
zkapp_event_element_ids: number[];

// List of `element_ids` that are used to construct the field array.
zkapp_field_array_element_ids: number[];

// `element_ids` represent a list of identifiers that map to specific field values.
// These are used to identify which field values are used in a zkApp transaction and construct the data returned to the user.
element_ids: number[];
Expand Down
22 changes: 22 additions & 0 deletions src/resolvers-types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ export { BlockStatusFilter };

export type EventData = {
__typename?: 'EventData';
accountUpdateId: Scalars['String']['output'];
data: Array<Maybe<Scalars['String']['output']>>;
transactionInfo?: Maybe<TransactionInfo>;
};
Expand Down Expand Up @@ -124,7 +125,11 @@ export type TransactionInfo = {
authorizationKind: Scalars['String']['output'];
hash: Scalars['String']['output'];
memo: Scalars['String']['output'];
sequenceNumber: Scalars['Int']['output'];
status: Scalars['String']['output'];
zkappAccountUpdateIds: Array<Maybe<Scalars['Int']['output']>>;
zkappEventElementIds: Array<Maybe<Scalars['Int']['output']>>;
zkappFieldArrayElementIds: Array<Maybe<Scalars['Int']['output']>>;
};

export type ResolverTypeWrapper<T> = Promise<T> | T;
Expand Down Expand Up @@ -386,6 +391,7 @@ export type EventDataResolvers<
ParentType extends
ResolversParentTypes['EventData'] = ResolversParentTypes['EventData'],
> = {
accountUpdateId?: Resolver<ResolversTypes['String'], ParentType, ContextType>;
data?: Resolver<
Array<Maybe<ResolversTypes['String']>>,
ParentType,
Expand Down Expand Up @@ -448,7 +454,23 @@ export type TransactionInfoResolvers<
>;
hash?: Resolver<ResolversTypes['String'], ParentType, ContextType>;
memo?: Resolver<ResolversTypes['String'], ParentType, ContextType>;
sequenceNumber?: Resolver<ResolversTypes['Int'], ParentType, ContextType>;
status?: Resolver<ResolversTypes['String'], ParentType, ContextType>;
zkappAccountUpdateIds?: Resolver<
Array<Maybe<ResolversTypes['Int']>>,
ParentType,
ContextType
>;
zkappEventElementIds?: Resolver<
Array<Maybe<ResolversTypes['Int']>>,
ParentType,
ContextType
>;
zkappFieldArrayElementIds?: Resolver<
Array<Maybe<ResolversTypes['Int']>>,
ParentType,
ContextType
>;
__isTypeOf?: IsTypeOfResolverFn<ParentType, ContextType>;
};

Expand Down
29 changes: 27 additions & 2 deletions src/services/actions-service/actions-service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ class ActionsService implements IActionsService {
);
for (let i = 0; i < blockTransactionEntries.length; i++) {
const transactions = blockTransactionEntries[i][1];
const transaction = transactions.values().next().value[0];
const transaction = transactions.values().next().value![0];
const blockInfo = createBlockInfo(transaction);
const {
action_state_value1,
Expand All @@ -143,7 +143,7 @@ class ActionsService implements IActionsService {
}
actions.push({
blockInfo,
actionData: actionsData.flat(),
actionData: this.sortActions(actionsData.flat()),
actionState: {
/* eslint-disable */
actionStateOne: action_state_value1!,
Expand All @@ -157,4 +157,29 @@ class ActionsService implements IActionsService {
}
return actions;
}

sortActions(actions: Action[]): Action[] {
return actions.sort((a, b) => {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since actions are returned grouped by block, this is focused on ordering the actions within a block.

I'm not sure the sort order is correct. Maybe it's reversed, but we should get it right, then we will have access to the unit test as a spec.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@deepthiskumar it's worth noting that you and gregor both mentioned a sub-order to the account update in o1-labs/o1js#1872

It is your point 4, and Gregor said "It's extremely important that when reordering the actions here, the order within the same account update is maintained".

This sort only looks at sequence number and account update id. But with my JSON comment on this PR and this code, we should be able to get on the same page more easily. If there's a third sorting lever we need here, then I can add it, but it's not clear to me what it is.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the third sorting lever is the element ID index from zkappEventElementIds

// Sort by sequence number
if (
a.transactionInfo.sequenceNumber !== b.transactionInfo.sequenceNumber
) {
return (
a.transactionInfo.sequenceNumber - b.transactionInfo.sequenceNumber
);
}

// Sort by account update index within the transaction
const aAccountUpdateIndex =
a.transactionInfo.zkappAccountUpdateIds.indexOf(
Number(a.accountUpdateId)
);
const bAccountUpdateIndex =
b.transactionInfo.zkappAccountUpdateIds.indexOf(
Number(b.accountUpdateId)
);

return aAccountUpdateIndex - bAccountUpdateIndex;
});
}
}
8 changes: 6 additions & 2 deletions src/services/data-adapters/database-row-adapters.ts
Original file line number Diff line number Diff line change
Expand Up @@ -199,15 +199,19 @@ function mapActionOrEvent(
) {
const data: (Event | Action)[] = [];
for (let i = 0; i < rows.length; i++) {
const { element_ids } = rows[i];
const { zkapp_account_update_id, element_ids } = rows[i];
const transactionInfo = createTransactionInfo(rows[i]);
const elementIdToFieldValues = getFieldValuesFromElementIds(
element_ids,
elementIdFieldValues
);

if (kind === 'event') {
const event = createEvent(elementIdToFieldValues, transactionInfo);
const event = createEvent(
zkapp_account_update_id.toString(),
elementIdToFieldValues,
transactionInfo
);
data.push(event);
} else {
const { zkapp_account_update_id } = rows[i];
Expand Down
66 changes: 66 additions & 0 deletions tests/makeActionsRequest.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
import { createYoga, createSchema } from 'graphql-yoga';
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This whole file may be better to remove. It's just a script to make a graphql request against the local API.

import { loadSchemaSync } from '@graphql-tools/load';
import { GraphQLFileLoader } from '@graphql-tools/graphql-file-loader';
import { buildHTTPExecutor } from '@graphql-tools/executor-http';
import { parse } from 'graphql';
import { resolvers } from '../src/resolvers.js';
import { buildContext, GraphQLContext } from '../src/context.js';

const PG_CONN = 'postgresql://postgres:postgres@localhost:5432/archive ';
const zkappAccount = 'B62qmBrPiukbHj4VnXdgMzjj2zpoQZeSBxZA6JDYMeeShApRAKaorto';

const actionsQuery = `
query getActions($input: ActionFilterOptionsInput!) {
actions(input: $input) {
blockInfo {
stateHash
timestamp
height
parentHash
chainStatus
distanceFromMaxBlockHeight
globalSlotSinceGenesis
}
actionState {
actionStateOne
actionStateTwo
actionStateThree
actionStateFour
actionStateFive
}
actionData {
data
accountUpdateId
transactionInfo {
status
hash
memo
sequenceNumber
zkappAccountUpdateIds
zkappEventElementIds
}
}
}
}
`;

const schema = createSchema<GraphQLContext>({
typeDefs: loadSchemaSync('./schema.graphql', {
loaders: [new GraphQLFileLoader()],
}),
resolvers,
});
const context = await buildContext(PG_CONN);
const yoga = createYoga<GraphQLContext>({ schema, context });
const executor = buildHTTPExecutor({
fetch: yoga.fetch,
});

const results = await executor({
variables: {
input: { address: zkappAccount },
},
document: parse(`${actionsQuery}`),
});

console.log(JSON.stringify(results));
Loading
Loading