Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support for access/authorize and update #392

Merged
merged 6 commits into from
Feb 1, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions packages/access-api/migrations/0004_add_accounts_table.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
-- Migration number: 0004 2023-01-24T15:09:12.316Z
CREATE TABLE
IF NOT EXISTS accounts (
did TEXT NOT NULL PRIMARY KEY,
inserted_at TEXT NOT NULL DEFAULT (strftime ('%Y-%m-%dT%H:%M:%fZ', 'now')),
updated_at TEXT NOT NULL DEFAULT (strftime ('%Y-%m-%dT%H:%M:%fZ', 'now')),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is updated_at relevant?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not really just a standard columns lol maybe in the future

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you prefer that i remove both updated_at ?

UNIQUE (did)
);

CREATE TABLE
IF NOT EXISTS delegations (
cid TEXT NOT NULL PRIMARY KEY,
bytes BLOB NOT NULL,
audience TEXT NOT NULL,
issuer TEXT NOT NULL,
expiration TEXT,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

number?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its a date iso string so we can use sqlite Date functions on it, kysely plugin handles transformations for this. JS Date > ISO String on insert and back on select.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It appears that sqlite doesn't support full ISO-8601, only a subset of its string syntax, so we might get more sqlite compatability by storing as numbers https://www.sqlite.org/lang_datefunc.html

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you check section "2. Time Values" in that link you can see sqlite support date.toISOString

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: maybe rename to expire_at to be consistent with other date rows?

inserted_at TEXT NOT NULL DEFAULT (strftime ('%Y-%m-%dT%H:%M:%fZ', 'now')),
updated_at TEXT NOT NULL DEFAULT (strftime ('%Y-%m-%dT%H:%M:%fZ', 'now')),
UNIQUE (cid),
FOREIGN KEY (audience) REFERENCES accounts (did) ON UPDATE CASCADE ON DELETE CASCADE
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does ON UPDATE CASCADE mean here? Why would we update the audience? Does it mean it updates the did in accounts?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we change something in accounts, delegation stays in sync. Its just a good default so we dont store dangling delegations because these are not issued by us.
Updates any change to PK/FK pair (should never happen) and deletes delegations if we delete account.

);
10 changes: 9 additions & 1 deletion packages/access-api/src/bindings.d.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,16 @@
import type { Logging } from '@web3-storage/worker-utils/logging'
import type { SpaceTable } from '@web3-storage/access/types'
import type {
AccountTable,
DelegationTable,
SpaceTable,
} from '@web3-storage/access/types'
import type { Handler as _Handler } from '@web3-storage/worker-utils/router'
import { Email } from './utils/email.js'
import { Spaces } from './models/spaces.js'
import { Validations } from './models/validations.js'
import { loadConfig } from './config.js'
import { Signer as EdSigner } from '@ucanto/principal/ed25519'
import { Accounts } from './models/accounts.js'

export {}

Expand Down Expand Up @@ -53,6 +58,7 @@ export interface RouteContext {
models: {
spaces: Spaces
validations: Validations
accounts: Accounts
}
uploadApi: {
production?: URL
Expand Down Expand Up @@ -98,4 +104,6 @@ export interface D1ErrorRaw extends Error {

export interface D1Schema {
spaces: SpaceTable
accounts: AccountTable
delegations: DelegationTable
}
112 changes: 112 additions & 0 deletions packages/access-api/src/models/accounts.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
// eslint-disable-next-line no-unused-vars
import * as Ucanto from '@ucanto/interface'
Comment on lines +1 to +2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we import types needed like below DelegationRecord instead of importing directly in source and ignoring eslint error?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps we could disable no-unused-vars since it is not ts-aware and instead only use @typescript-eslint/no-unused-vars. https://stackoverflow.com/a/61555310

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vasco-santos the interface package is made with an empty .js file to enable imports like this, because generic types in jsdoc typedefs are a pain.

@gobengo that could work and it looks cleaner

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fwiw what I proposed before doesn't work, but this does https://stackoverflow.com/a/66261985

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

import {
delegationsToBytes,
expirationToDate,
} from '@web3-storage/access/encoding'
import { Kysely } from 'kysely'
import { D1Dialect } from 'kysely-d1'
import { GenericPlugin } from '../utils/d1.js'

/**
* @typedef {import('@web3-storage/access/src/types.js').DelegationRecord} DelegationRecord
*/

/**
* Accounts
*/
export class Accounts {
/**
*
* @param {D1Database} d1
*/
constructor(d1) {
/** @type {GenericPlugin<DelegationRecord>} */
const objectPlugin = new GenericPlugin({
// eslint-disable-next-line unicorn/no-null
expires_at: (v) => (typeof v === 'string' ? new Date(v) : null),
inserted_at: (v) => new Date(v),
updated_at: (v) => new Date(v),
})
this.d1 = /** @type {Kysely<import('../bindings').D1Schema>} */ (
new Kysely({
dialect: new D1Dialect({ database: d1 }),
plugins: [objectPlugin],
})
)
}

/**
* @param {Ucanto.URI<"did:">} did
*/
async create(did) {
const result = await this.d1
.insertInto('accounts')
.values({
did,
})
.onConflict((oc) => oc.column('did').doNothing())
.returning('accounts.did')
.execute()
return { data: result }
}

/**
*
* @param {Ucanto.Delegation} del
*/
async addDelegation(del) {
gobengo marked this conversation as resolved.
Show resolved Hide resolved
const result = await this.d1
.insertInto('delegations')
.values({
cid: del.cid.toV1().toString(),
audience: del.audience.did(),
issuer: del.issuer.did(),
bytes: delegationsToBytes([del]),
expires_at: expirationToDate(del.expiration),
})
.onConflict((oc) => oc.column('cid').doNothing())
.returningAll()
.executeTakeFirst()
return result
}

/**
* @param {Ucanto.URI<"did:">} did
*/
async get(did) {
return await this.d1
.selectFrom('accounts')
.selectAll()
.where('accounts.did', '=', did)
.executeTakeFirst()
}

/**
* @param {Ucanto.URI<"did:">} did
*/
async getDelegations(did) {
return await this.d1
.selectFrom('delegations')
.selectAll()
.where('delegations.audience', '=', did)
.execute()
}

/**
* @param {string} cid
*/
async getDelegationsByCid(cid) {
return await this.d1
.selectFrom('delegations')
.selectAll()
.where('delegations.cid', '=', cid)
.where((qb) =>
qb
.where('delegations.expires_at', '>=', new Date())
// eslint-disable-next-line unicorn/no-null
gobengo marked this conversation as resolved.
Show resolved Hide resolved
.orWhere('delegations.expires_at', 'is', null)
)
.executeTakeFirst()
}
}
20 changes: 14 additions & 6 deletions packages/access-api/src/models/spaces.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,11 @@ import * as Ucanto from '@ucanto/interface'
import { delegationToString } from '@web3-storage/access/encoding'
import { Kysely } from 'kysely'
import { D1Dialect } from 'kysely-d1'
import { D1Error, SpacePlugin } from '../utils/d1.js'
import { D1Error, GenericPlugin } from '../utils/d1.js'

const spacePlugin = new SpacePlugin()
/**
* @typedef {import('@web3-storage/access/src/types.js').SpaceRecord} SpaceRecord
*/

/**
* Spaces
Expand All @@ -17,8 +19,17 @@ export class Spaces {
* @param {D1Database} d1
*/
constructor(d1) {
/** @type {GenericPlugin<SpaceRecord>} */
const objectPlugin = new GenericPlugin({
metadata: (v) => JSON.parse(v),
inserted_at: (v) => new Date(v),
updated_at: (v) => new Date(v),
})
this.d1 = /** @type {Kysely<import('../bindings').D1Schema>} */ (
new Kysely({ dialect: new D1Dialect({ database: d1 }) })
new Kysely({
dialect: new D1Dialect({ database: d1 }),
plugins: [objectPlugin],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice we can use plugins here directly 👌🏼

})
)
}

Expand All @@ -34,7 +45,6 @@ export class Spaces {
/** @type {unknown} */ (invocation.facts[0])
)
const result = await this.d1
.withPlugin(spacePlugin)
.insertInto('spaces')
.values({
agent: invocation.issuer.did(),
Expand Down Expand Up @@ -64,7 +74,6 @@ export class Spaces {
*/
async get(did) {
const space = await this.d1
.withPlugin(spacePlugin)
.selectFrom('spaces')
.selectAll()
.where('spaces.did', '=', did)
Expand All @@ -80,7 +89,6 @@ export class Spaces {
*/
async getByEmail(email) {
const spaces = await this.d1
.withPlugin(spacePlugin)
.selectFrom('spaces')
.selectAll()
.where('spaces.email', '=', email.replace('mailto:', ''))
Expand Down
12 changes: 12 additions & 0 deletions packages/access-api/src/models/validations.js
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,18 @@ export class Validations {
return delegation
}

/**
* @template {import('@ucanto/interface').Capabilities} T = import('@ucanto/interface').Capabilities
* @param {import('@web3-storage/access/src/types').EncodedDelegation<T>} ucan
* @param {import('@ucanto/interface').DID} agent
* @param {number} expirationTtl - Expiration in second from now. Defaults to 5 mins.
*/
async putSession(ucan, agent, expirationTtl = 60 * 5) {
return await this.kv.put(agent, ucan, {
expirationTtl,
})
}

/**
* @param {string} did
*/
Expand Down
51 changes: 49 additions & 2 deletions packages/access-api/src/routes/validate-email.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
/* eslint-disable no-unused-vars */
import { stringToDelegation } from '@web3-storage/access/encoding'
import QRCode from 'qrcode'
import { toEmail } from '../utils/did-mailto.js'
import {
HtmlResponse,
ValidateEmail,
Expand All @@ -14,6 +16,10 @@ export async function validateEmail(req, env) {
if (req.query && req.query.ucan && req.query.mode === 'recover') {
return recover(req, env)
}

if (req.query && req.query.ucan && req.query.mode === 'session') {
return session(req, env)
}
if (req.query && req.query.ucan) {
try {
const delegation = await env.models.validations.put(
Expand All @@ -25,7 +31,11 @@ export async function validateEmail(req, env) {
return new HtmlResponse(
(
<ValidateEmail
delegation={delegation}
email={delegation.capabilities[0].nb.identity.replace(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could be good to define a utility function for extracting the email from the ability, because this logic using String#replace won't work for did:mailto using the syntax we changed to once realizing @ was not a valid DID character.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and then it could be used on line 84 below as well

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

after reading more, I also don't really understand where nb.identity is coming from or what type it would be (I don't see nb.identity in the access capabilities). If I'm reading right, it might be type any when the replace method is called on it here. We might avoid runtime bugs by clarifying what the return type of env.models.validations.put is expecting (e.g. specific Capability types within the delegation, not generic allowing any Capabilities tuple) and/or doing more explicit parsing of req.query.ucan instead of using the type assertion on line 26

Copy link
Contributor

@gobengo gobengo Jan 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it would be good to add a test for what should happen if someone tries to inject malicious req.query.ucan and IMO it's best to ensure we don't create db records unless the req.query.ucan parses into a well-known ability can

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is from the old recover cap it will go away soon, regarding ucan validation i agree we should validate the caps for the specific mode and the signature.

#411

'mailto:',
''
)}
audience={delegation.audience.did()}
ucan={req.query.ucan}
qrcode={await QRCode.toString(req.query.ucan, {
type: 'svg',
Expand Down Expand Up @@ -71,7 +81,8 @@ async function recover(req, env) {
return new HtmlResponse(
(
<ValidateEmail
delegation={delegation}
email={delegation.capabilities[0].nb.identity.replace('mailto:', '')}
audience={delegation.audience.did()}
ucan={req.query.ucan}
qrcode={await QRCode.toString(req.query.ucan, {
type: 'svg',
Expand All @@ -96,3 +107,39 @@ async function recover(req, env) {
)
}
}

/**
* @param {import('@web3-storage/worker-utils/router').ParsedRequest} req
* @param {import('../bindings.js').RouteContext} env
*/
async function session(req, env) {
/** @type {import('@ucanto/interface').Delegation<[import('@web3-storage/capabilities/src/types.js').AccessSession]>} */
const delegation = stringToDelegation(req.query.ucan)
await env.models.validations.putSession(
req.query.ucan,
delegation.capabilities[0].nb.key
)

try {
return new HtmlResponse(
(
<ValidateEmail
email={toEmail(delegation.audience.did())}
audience={delegation.audience.did()}
ucan={req.query.ucan}
qrcode={await QRCode.toString(req.query.ucan, {
type: 'svg',
errorCorrectionLevel: 'M',
margin: 10,
})}
/>
)
)
} catch (error) {
const err = /** @type {Error} */ (error)
env.log.error(err)
return new HtmlResponse(
<ValidateEmailError msg={'Oops something went wrong.'} />
)
}
}
45 changes: 45 additions & 0 deletions packages/access-api/src/service/access-authorize.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
// @ts-ignore
// eslint-disable-next-line no-unused-vars
import * as Ucanto from '@ucanto/interface'
import * as Server from '@ucanto/server'
import * as Access from '@web3-storage/capabilities/access'
import * as Mailto from '../utils/did-mailto.js'
import * as DID from '@ipld/dag-ucan/did'
import { delegationToString } from '@web3-storage/access/encoding'

/**
* @param {import('../bindings').RouteContext} ctx
*/
export function accessAuthorizeProvider(ctx) {
return Server.provide(
Access.authorize,
async ({ capability, invocation }) => {
const session = await Access.session
.invoke({
issuer: ctx.signer,
audience: DID.parse(capability.nb.as),
with: ctx.signer.did(),
lifetimeInSeconds: 86_400 * 7, // 7 days
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be spec'ed? I could not find this anywhere

Copy link
Contributor

@gobengo gobengo Jan 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also might be good to use an absolute datetime here instead of a duration (so that the expiry datetime is unambiguous)

I realized that comment only made sense because I thought lifetimeInSeconds was in the nb of an invocation, but just realized it's a ucanto argument

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nb: {
key: capability.with,
},
})
.delegate()

const encoded = delegationToString(session)

await ctx.models.accounts.create(capability.nb.as)

const url = `${ctx.url.protocol}//${ctx.url.host}/validate-email?ucan=${encoded}&mode=session`
// For testing
if (ctx.config.ENV === 'test') {
return url
}

await ctx.email.sendValidation({
to: Mailto.toEmail(capability.nb.as),
url,
})
}
)
}
Loading