Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to Neo4j #396

Open
wants to merge 81 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
81 commits
Select commit Hold shift + click to select a range
a013891
include TX history endpoint fetching from Neo4j
pedromtcosta Sep 15, 2022
8d0a215
add TX comming from certificates and withdrawals
pedromtcosta Sep 20, 2022
29f1e54
WIP: configure Neo4j and skip creating Postgres views
pedromtcosta Oct 7, 2022
3e38062
query enhancements
pedromtcosta Oct 11, 2022
818e432
fix env vars
pedromtcosta Oct 11, 2022
5a22fdb
close session
pedromtcosta Oct 11, 2022
9c2db41
fix bug
pedromtcosta Oct 12, 2022
9e16a14
dummy testing code
pedromtcosta Oct 12, 2022
696f80a
migrate filterUsed, bestblock, utxo for addresses and asset metadata
pedromtcosta Nov 3, 2022
3f89b65
added v3 txio endpoint
DoctorMarshall Nov 4, 2022
8ebd9bc
added v3 tx io by output index endpoint
DoctorMarshall Nov 7, 2022
995671c
adjusted endpoint path
DoctorMarshall Nov 7, 2022
5700916
added registration history endpoint
DoctorMarshall Nov 8, 2022
045360e
migrated endpoint /multiAsset/policyIdExists
DoctorMarshall Nov 8, 2022
8981507
add v3 validate NFT
pedromtcosta Nov 8, 2022
94a83ec
extracted common tx parsing logic
DoctorMarshall Nov 10, 2022
b4f612b
migrated txs get
DoctorMarshall Nov 10, 2022
b5684b1
migrated pool info
DoctorMarshall Nov 10, 2022
9da5d22
migrated mintTxs endpoint
DoctorMarshall Nov 14, 2022
96af2f9
extracted common pool fetching logic
DoctorMarshall Nov 14, 2022
c41190c
migrated /pool/delegationHistory
DoctorMarshall Nov 14, 2022
b58e071
Migrated /txs/utxoSumForAddresses
DoctorMarshall Nov 14, 2022
19437f9
Migrated /multiAsset/supply endpoint
DoctorMarshall Nov 15, 2022
75f3f1c
Migrated /v2/txs/utxoAtPoint
DoctorMarshall Nov 16, 2022
f0ab3f7
tip status endpoints
pedromtcosta Nov 15, 2022
81eabab
start utxo diff
pedromtcosta Nov 15, 2022
280c995
added pagination support
DoctorMarshall Nov 16, 2022
f737b76
finish UTxO diff since point
pedromtcosta Nov 16, 2022
b85b4d6
transfer neo4j endpoints to v2 and v2.1
pedromtcosta Nov 17, 2022
e65e606
filter by payment cred
pedromtcosta Nov 17, 2022
8d8566a
fix utxo diff since point
pedromtcosta Nov 17, 2022
5606e06
fix multiasset metadata endpoint
pedromtcosta Nov 21, 2022
518e2bb
get until tx recursively
pedromtcosta Nov 21, 2022
2a68e56
get assets metadata in parallel
pedromtcosta Nov 22, 2022
4440440
fix identation
pedromtcosta Nov 23, 2022
4b53692
Remove Postgres endpoints
pedromtcosta Nov 23, 2022
b9b61f9
convert from bech32 to base58 when needed
pedromtcosta Nov 23, 2022
c1f550a
support hex addresses
pedromtcosta Nov 23, 2022
8f90068
calculate testnet times
pedromtcosta Nov 24, 2022
2a7b32e
improve performance of Neo4j endpoints
pedromtcosta Nov 27, 2022
8ba4653
go back to previous utxo diff api
pedromtcosta Nov 29, 2022
a2c3a27
support new utxo diff API
pedromtcosta Nov 29, 2022
b7663c8
fix bugs for utxo manager
pedromtcosta Nov 30, 2022
2db9f66
return lastFoundSafeblock and lastFoundBestblock on early response
pedromtcosta Dec 1, 2022
db1fe0d
sync package lock
pedromtcosta Dec 8, 2022
3e8e10b
placeholder for txs summaries endpoint
pedromtcosta Dec 12, 2022
b084c23
make endpoints transactional
pedromtcosta Dec 12, 2022
a4d3266
wrap queries in try/finally
pedromtcosta Dec 13, 2022
f83f23e
order certificates
pedromtcosta Dec 20, 2022
5e6c631
order txs/get certificates
pedromtcosta Dec 20, 2022
1717d42
migrated lastBlockBySlot
DoctorMarshall Dec 21, 2022
0b3520c
support payment creds in utxo at point
pedromtcosta Dec 21, 2022
eab1c27
Merge branch 'master' into neo4j
pedromtcosta Dec 22, 2022
1dd6c7f
migrate txs summaries to neo4j
pedromtcosta Dec 22, 2022
1116620
group transactions by address
pedromtcosta Dec 22, 2022
e41c469
removing filtering only spent utxos
vsubhuman Dec 26, 2022
96c740f
Merge branch 'neo4j' of https://github.com/Emurgo/yoroi-graphql-migra…
DoctorMarshall Dec 27, 2022
557565a
Merge branch 'ruslan/filter-used-fix' into neo4j
pedromtcosta Dec 27, 2022
65e2422
use bech32 prefix
pedromtcosta Dec 27, 2022
a5b394b
use address from payload
pedromtcosta Dec 27, 2022
f19ffad
return empty result if no addresses
pedromtcosta Dec 27, 2022
e976302
log errors
pedromtcosta Jan 5, 2023
5466dc6
fix testnet timestamps
pedromtcosta Jan 11, 2023
6c64a7d
support preview network
pedromtcosta Jan 15, 2023
0a4173d
fixed issue, redesigned
DoctorMarshall Jan 20, 2023
31fa5dc
Merge branch 'master' into neo4j
vsubhuman Jan 26, 2023
0235468
fix policyIdExists bug
pedromtcosta Jan 31, 2023
9c0ffff
Add docs for /v2.1/lastBlockBySlot endpoint
neuodev Jan 31, 2023
408584b
Add tests for the /v2.1/lastBlockBySlot endpoint
neuodev Jan 31, 2023
5e981bd
Use ternary operator instead of if/else to get the block hash value
neuodev Jan 31, 2023
eb01aec
Merge pull request #415 from Emurgo/Ahmed/blockhash-by-slot-no
neuodev Feb 4, 2023
0284007
free wasm objects
pedromtcosta Feb 6, 2023
3b0fb43
add global slot to transaction data
pedromtcosta Feb 23, 2023
c09e525
wrap CSL objects
pedromtcosta Mar 29, 2023
725934d
assume metadatas only for specific policy/asset when 721 or 20
pedromtcosta Apr 3, 2023
7ec47d1
exit process when offset is out of bounds
pedromtcosta Apr 4, 2023
59b2c4b
run supply query for all assets in parallel
pedromtcosta Apr 6, 2023
8cc15ee
use formatNeo4jBigNumber for converting the supply
pedromtcosta Apr 11, 2023
fa73047
preview network shelley time fixes
vsubhuman Apr 13, 2023
4c6c29f
Merge pull request #421 from Emurgo/ruslan/fixing-shelley-time
pedromtcosta Apr 13, 2023
3d72d8b
check null port
pedromtcosta Apr 19, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .eslintrc.json
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
],
"@typescript-eslint/explicit-module-boundary-types": 0,
"@typescript-eslint/no-explicit-any": 0,
"@typescript-eslint/no-unused-vars": ["warn", { "argsIgnorePattern": "^_" }]
"@typescript-eslint/no-unused-vars": ["warn", { "argsIgnorePattern": "^_" }],
"@typescript-eslint/no-namespace": "off"
}
}
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
node_modules
*.swp
dist
.vscode
31 changes: 31 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1057,4 +1057,35 @@ We recommend querying using payment key hashes (`addr_vkh`) when possible (other
]
}
```
</details>

</details>

<details>
<summary> POST /v2.1/lastBlockBySlot</summary>

Resolve the latest block created at a slot equal to or lower than a given slot.

#### Request

The field slots are required to be a non-empty array. Each entry is a tuple of two numbers: epoch and slot.

```ts
type SlotNo = number;
type EpochNo = number;
type Request = {
slots: Array<[EpochNo, SlotNo]>;
};
```

#### Response

```ts
type Slot = [EpochNo, SlotNo];
type Response = {
blockHashes: {
[key: Slot]: string | null;
};
};
```
</details>
6 changes: 6 additions & 0 deletions config/default.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,5 +39,11 @@ export default {
accessKeyId: process.env.PRICE_DATA_S3_ACCESS_KEY_ID,
secretAccessKey: process.env.PRICE_DATA_S3_SECRET_ACCESS_KEY,
},
},
network: process.env.NETWORK || "mainnet",
neo4j: {
url: process.env.NEO4J_URL || "neo4j://dus-01.emurgo-rnd.com:7687",
username: process.env.NEO4J_USERNAME || "neo4j",
password: process.env.NEO4J_PASSWORD || "neo4j"
}
};
71 changes: 70 additions & 1 deletion package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 3 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@
"dev": "tsc-watch --onSuccess \"node ./dist/index.js\"",
"eslint": "eslint src tests",
"test": "mocha -r ts-node/register tests/**/*.test.ts --slow 0",
"testtxhist": "mocha -r ts-node/register tests/txHistory.test.ts",
"test:file": "mocha -r ts-node/register",
"testtxhist": "npm run test:file -- tests/txHistory.test.ts",
"prettier": "npx prettier --write src"
},
"author": "",
Expand All @@ -38,6 +39,7 @@
"hash-wasm": "^4.9.0",
"lodash": "^4.17.21",
"mocha": "^10.0.0",
"neo4j-driver": "^5.7.0",
"pg": "^8.8.0",
"pm2": "^5.2.0",
"semver-compare": "^1.0.0",
Expand Down
14 changes: 11 additions & 3 deletions src/coin-price/sign.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import {
PublicKey,
} from "@emurgo/cardano-serialization-lib-nodejs";
import { serializeTicker, sign, verify } from "./sign";
import { createCslContext } from "../utils/csl";

const TICKER = {
from: "ADA",
Expand All @@ -18,18 +19,22 @@ const PUBLIC_KEY =
"4711516aac6adccd06054939c45e2487df239ba86cc277ae56aaac0a83f0bf96";

it("sign and verify a ticker", () => {
const ctx = createCslContext();

const signature = sign(
TICKER,
serializeTicker,
PrivateKey.from_extended_bytes(Buffer.from(PRIVATE_KEY, "hex"))
ctx.wrap(PrivateKey.from_extended_bytes(Buffer.from(PRIVATE_KEY, "hex"))),
ctx
);

expect(
verify(
TICKER,
serializeTicker,
signature,
PublicKey.from_bytes(Buffer.from(PUBLIC_KEY, "hex"))
ctx.wrap(PublicKey.from_bytes(Buffer.from(PUBLIC_KEY, "hex"))),
ctx
)
).to.equal(true);

Expand All @@ -39,7 +44,10 @@ it("sign and verify a ticker", () => {
TICKER,
serializeTicker,
invalidSig,
PublicKey.from_bytes(Buffer.from(PUBLIC_KEY, "hex"))
ctx.wrap(PublicKey.from_bytes(Buffer.from(PUBLIC_KEY, "hex"))),
ctx
)
).to.equal(false);

ctx.freeAll();
});
11 changes: 7 additions & 4 deletions src/coin-price/sign.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import CardanoWasm from "@emurgo/cardano-serialization-lib-nodejs";
import { CslContext } from "../utils/csl";
import type { Ticker } from "./types";

export function serializeTicker(ticker: Ticker): Buffer {
Expand All @@ -16,19 +17,21 @@ export function serializeTicker(ticker: Ticker): Buffer {
export function sign(
obj: any,
serializer: (arg0: any) => Buffer,
privateKey: CardanoWasm.PrivateKey
privateKey: CardanoWasm.PrivateKey,
ctx: CslContext
): string {
return privateKey.sign(serializer(obj)).to_hex();
return ctx.wrap(privateKey.sign(serializer(obj))).to_hex();
}

export function verify(
obj: any,
serializer: (arg0: any) => Buffer,
signatureHex: string,
publicKey: CardanoWasm.PublicKey
publicKey: CardanoWasm.PublicKey,
ctx: CslContext
): boolean {
return publicKey.verify(
serializer(obj),
CardanoWasm.Ed25519Signature.from_bytes(Buffer.from(signatureHex, "hex"))
ctx.wrap(CardanoWasm.Ed25519Signature.from_bytes(Buffer.from(signatureHex, "hex")))
);
}
Loading