Skip to content

Commit

Permalink
feat: replaced serialize: true option with "auto"
Browse files Browse the repository at this point in the history
  • Loading branch information
oliver-oloughlin committed Nov 27, 2023
1 parent 7bff34b commit 4564c7e
Show file tree
Hide file tree
Showing 11 changed files with 80 additions and 76 deletions.
16 changes: 8 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ const kv = await Deno.openKv()
const db = kvdex(kv, {
numbers: collection(model<number>()),
serializedStrings: collection(model<string>(), {
serialize: true
serialize: "auto"
}),
users: collection(UserSchema, {
idGenerator: () => crypto.randomUUID(),
Expand Down Expand Up @@ -665,11 +665,11 @@ db.users.listenQueue(async (data) => {
Serialized collections can store much larger sized data by serializaing,
compresing and splitting the data across multiple KV entries. There is a
tradeoff between speed and storage efficiency. Custom serialize and compress
functions can be set through the collection options. The default serializer
picks between the Deno core serializer and custom json serializer depending on
being run in the standard Deno runtime or on Deploy (uses the slower json
serializer on Deploy as Deno core is not exposed). Alternatively, the serializer
can be specified to always use core or json.
functions can be set through the collection options. Setting the serialize
option to "auto" picks between the Deno core serializer and custom json
serializer depending on being run in the standard Deno runtime or on Deploy
(uses the slower json serializer on Deploy as Deno core is not exposed).
Alternatively, the serialize option can be set to specify between core or json.

```ts
import { collection, kvdex, model } from "https://deno.land/x/kvdex/mod.ts"
Expand All @@ -683,8 +683,8 @@ type LargeData = {
const kv = await Deno.openKv()
const db = kvdex(kv, {
users: collection(model<LargeData>(), {
// Use default serialize/compress
serialize: true,
// Picks default serialize/compress
serialize: "auto",

// Use Deno core serializer
serialize: "core",
Expand Down
2 changes: 1 addition & 1 deletion src/collection.ts
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,7 @@ export class Collection<
const defaultSerialize = isDeploy ? jsonSerialize : denoCoreSerialize
const defaultDeserialize = isDeploy ? jsonDeserialize : denoCoreDeserialize

if (opts?.serialize === true) {
if (opts?.serialize === "auto") {
this._serializer = {
serialize: defaultSerialize,
deserialize: defaultDeserialize,
Expand Down
2 changes: 1 addition & 1 deletion src/kvdex.ts
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ import { AtomicWrapper } from "./atomic_wrapper.ts"
* const db = kvdex(kv, {
* numbers: collection(model<number>()),
* u64s: collection(model<Deno.KvU64>()),
* serializedStrings: collection(model<string>(), { serialize: true }),
* serializedStrings: collection(model<string>(), { serialize: "auto" }),
* users: collection(model<User>(), {
* indices: {
* username: "primary",
Expand Down
2 changes: 1 addition & 1 deletion src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -308,7 +308,7 @@ export type SerializedEntry = {
ids: KvId[]
}

export type SerializeOptions = true | "core" | "json" | Partial<Serializer>
export type SerializeOptions = "auto" | "core" | "json" | Partial<Serializer>

/***************************/
/* */
Expand Down
2 changes: 1 addition & 1 deletion tests/ext/zod.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ Deno.test("ext - zod", async (t) => {
age: "secondary",
},
}),
s_users: collection(zodModel(UserSchema), { serialize: true }),
s_users: collection(zodModel(UserSchema), { serialize: "auto" }),
})

const cr1 = await db.users.add({
Expand Down
2 changes: 1 addition & 1 deletion tests/serialized_collection/enqueue.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Deno.test("serialized_collection - enqueue", async (t) => {
const undeliveredId = "undelivered"

const db = kvdex(kv, {
s_users: collection(model<User>(), { serialize: true }),
s_users: collection(model<User>(), { serialize: "auto" }),
})

const sleeper = createResolver()
Expand Down
2 changes: 2 additions & 0 deletions tests/serialized_collection/properties.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,11 @@ Deno.test("serialized_collection - properties", async (t) => {
await useKv((kv) => {
const db = kvdex(kv, {
users1: collection(model<User>(), {
serialize: "auto",
idGenerator: () => Math.random(),
}),
users2: collection(model<User>(), {
serialize: "auto",
idGenerator: (data) => data.username,
}),
})
Expand Down
52 changes: 27 additions & 25 deletions tests/serialized_collection/types.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,12 @@ Deno.test("serialized_collection - types", async (t) => {
async () => {
await useKv(async (kv) => {
const db = kvdex(kv, {
nulls: collection(model<null>(), { serialize: true }),
undefineds: collection(model<undefined>(), { serialize: true }),
strings: collection(model<string>(), { serialize: true }),
numbers: collection(model<number>(), { serialize: true }),
bigints: collection(model<bigint>(), { serialize: true }),
u64s: collection(model<Deno.KvU64>(), { serialize: true }),
nulls: collection(model<null>(), { serialize: "auto" }),
undefineds: collection(model<undefined>(), { serialize: "auto" }),
strings: collection(model<string>(), { serialize: "auto" }),
numbers: collection(model<number>(), { serialize: "auto" }),
bigints: collection(model<bigint>(), { serialize: "auto" }),
u64s: collection(model<Deno.KvU64>(), { serialize: "auto" }),
})

const cr1 = await db.nulls.add(null)
Expand Down Expand Up @@ -49,12 +49,12 @@ Deno.test("serialized_collection - types", async (t) => {
async () => {
await useKv(async (kv) => {
const db = kvdex(kv, {
dates: collection(model<Date>(), { serialize: true }),
sets: collection(model<Set<string>>(), { serialize: true }),
maps: collection(model<Map<string, number>>(), { serialize: true }),
regExps: collection(model<RegExp>(), { serialize: true }),
dataVeiws: collection(model<DataView>(), { serialize: true }),
errors: collection(model<Error>(), { serialize: true }),
dates: collection(model<Date>(), { serialize: "auto" }),
sets: collection(model<Set<string>>(), { serialize: "auto" }),
maps: collection(model<Map<string, number>>(), { serialize: "auto" }),
regExps: collection(model<RegExp>(), { serialize: "auto" }),
dataVeiws: collection(model<DataView>(), { serialize: "auto" }),
errors: collection(model<Error>(), { serialize: "auto" }),
})

const cr1 = await db.dates.add(new Date())
Expand Down Expand Up @@ -103,19 +103,21 @@ Deno.test("serialized_collection - types", async (t) => {
async () => {
await useKv(async (kv) => {
const db = kvdex(kv, {
arrs: collection(model<Array<string>>(), { serialize: true }),
i8arrs: collection(model<Int8Array>(), { serialize: true }),
i16arrs: collection(model<Int16Array>(), { serialize: true }),
i32arrs: collection(model<Int32Array>(), { serialize: true }),
i64arrs: collection(model<BigInt64Array>(), { serialize: true }),
u8arrs: collection(model<Uint8Array>(), { serialize: true }),
u16arrs: collection(model<Uint16Array>(), { serialize: true }),
u32arrs: collection(model<Uint32Array>(), { serialize: true }),
u64arrs: collection(model<BigUint64Array>(), { serialize: true }),
u8carrs: collection(model<Uint8ClampedArray>(), { serialize: true }),
f32arrs: collection(model<Float32Array>(), { serialize: true }),
f64arrs: collection(model<Float64Array>(), { serialize: true }),
buffers: collection(model<ArrayBuffer>(), { serialize: true }),
arrs: collection(model<Array<string>>(), { serialize: "auto" }),
i8arrs: collection(model<Int8Array>(), { serialize: "auto" }),
i16arrs: collection(model<Int16Array>(), { serialize: "auto" }),
i32arrs: collection(model<Int32Array>(), { serialize: "auto" }),
i64arrs: collection(model<BigInt64Array>(), { serialize: "auto" }),
u8arrs: collection(model<Uint8Array>(), { serialize: "auto" }),
u16arrs: collection(model<Uint16Array>(), { serialize: "auto" }),
u32arrs: collection(model<Uint32Array>(), { serialize: "auto" }),
u64arrs: collection(model<BigUint64Array>(), { serialize: "auto" }),
u8carrs: collection(model<Uint8ClampedArray>(), {
serialize: "auto",
}),
f32arrs: collection(model<Float32Array>(), { serialize: "auto" }),
f64arrs: collection(model<Float64Array>(), { serialize: "auto" }),
buffers: collection(model<ArrayBuffer>(), { serialize: "auto" }),
})

const cr1 = await db.arrs.add(["str1", "str2", "str3"])
Expand Down
4 changes: 2 additions & 2 deletions tests/serialized_indexable_collection/listenQueue.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Deno.test("serialized_indexable_collection - listenQueue", async (t) => {
const sleeper = createResolver()

const db = kvdex(kv, {
is_users: collection(model<User>(), { indices: {}, serialize: true }),
is_users: collection(model<User>(), { indices: {}, serialize: "auto" }),
})

const handlerId = createHandlerId(db.is_users._keys.base, undefined)
Expand Down Expand Up @@ -61,7 +61,7 @@ Deno.test("serialized_indexable_collection - listenQueue", async (t) => {
await t.step("Should not receive db queue message", async () => {
await useKv(async (kv) => {
const db = kvdex(kv, {
is_users: collection(model<User>(), { indices: {}, serialize: true }),
is_users: collection(model<User>(), { indices: {}, serialize: "auto" }),
})

let assertion = true
Expand Down
Loading

0 comments on commit 4564c7e

Please sign in to comment.