Skip to content

Commit

Permalink
feat(keyv-mysql): use internal scheduler to delete expired keys
Browse files Browse the repository at this point in the history
  • Loading branch information
johaven committed Jan 26, 2025
1 parent a09f473 commit 721e48f
Show file tree
Hide file tree
Showing 3 changed files with 21 additions and 4 deletions.
4 changes: 3 additions & 1 deletion packages/mysql/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ keyv.on('error', handleConnectionError);
```

You can specify a custom table with the `table` option and the primary key size with `keySize`.
If you want to use native MySQL scheduler to delete expired keys, you can enable `useInternalScheduler`.

e.g:

Expand All @@ -35,7 +36,8 @@ import KeyvMysql from '@keyv/mysql';

const keyv = new Keyv(new KeyvMysql('mysql://user:pass@localhost:3306/dbname'), {
table: 'cache',
keySize: 255
keySize: 255,
useInternalScheduler: true
});
```

Expand Down
20 changes: 17 additions & 3 deletions packages/mysql/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ import {
} from './types';
import {endPool, pool} from './pool';

const keyvMysqlKeys = new Set(['adapter', 'compression', 'connect', 'dialect', 'keySize', 'table', 'ttl', 'uri']);
const keyvMysqlKeys = new Set(['adapter', 'compression', 'connect', 'dialect', 'keySize', 'useInternalScheduler',

Check failure on line 9 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 20

There should be a linebreak after this element.

Check failure on line 9 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 20

There should be a linebreak after this element.

Check failure on line 9 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 20

There should be a linebreak after this element.

Check failure on line 9 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 20

There should be a linebreak after this element.

Check failure on line 9 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 20

There should be a linebreak after this element.

Check failure on line 9 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 20

There should be a linebreak after this element.

Check failure on line 9 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 20

There should be a linebreak after this element.

Check failure on line 9 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 20

There should be a linebreak after this element.

Check failure on line 9 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 20

There should be a linebreak after this element.

Check failure on line 9 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 20

There should be a linebreak after this element.

Check failure on line 9 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 22

There should be a linebreak after this element.

Check failure on line 9 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 22

There should be a linebreak after this element.

Check failure on line 9 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 22

There should be a linebreak after this element.

Check failure on line 9 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 22

There should be a linebreak after this element.

Check failure on line 9 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 22

There should be a linebreak after this element.
'table', 'ttl', 'uri']);

Check failure on line 10 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 20

Expected indentation of 1 tab but found 4 spaces.

Check failure on line 10 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 20

There should be a linebreak after this element.

Check failure on line 10 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 20

There should be a linebreak after this element.

Check failure on line 10 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 20

Expected indentation of 1 tab but found 4 spaces.

Check failure on line 10 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 20

There should be a linebreak after this element.

Check failure on line 10 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 20

There should be a linebreak after this element.

Check failure on line 10 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 22

Expected indentation of 1 tab but found 4 spaces.

Check failure on line 10 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 22

There should be a linebreak after this element.

Check failure on line 10 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 22

There should be a linebreak after this element.

type QueryType<T> = Promise<T extends
mysql.RowDataPacket[][] |
Expand All @@ -18,13 +19,12 @@ mysql.ResultSetHeader
: never>;

export class KeyvMysql extends EventEmitter implements KeyvStoreAdapter {
ttlSupport: boolean;
ttlSupport = false;
opts: KeyvMysqlOptions;
namespace?: string;
query: <T>(sqlString: string) => QueryType<T>;
constructor(keyvOptions?: KeyvMysqlOptions | string) {
super();
this.ttlSupport = false;

let options: KeyvMysqlOptions = {
dialect: 'mysql',
Expand All @@ -46,6 +46,9 @@ export class KeyvMysql extends EventEmitter implements KeyvStoreAdapter {
),
);

this.ttlSupport = !!mysqlOptions.useInternalScheduler;

Check failure on line 49 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 20

Expected indentation of 2 tabs but found 8 spaces.

Check failure on line 49 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 20

use `Boolean(mysqlOptions.useInternalScheduler)` instead.

Check failure on line 49 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 20

Expected indentation of 2 tabs but found 8 spaces.

Check failure on line 49 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 20

use `Boolean(mysqlOptions.useInternalScheduler)` instead.

Check failure on line 49 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 22

Expected indentation of 2 tabs but found 8 spaces.

Check failure on line 49 in packages/mysql/src/index.ts

View workflow job for this annotation

GitHub Actions / Node 22

use `Boolean(mysqlOptions.useInternalScheduler)` instead.

delete mysqlOptions.useInternalScheduler;
delete mysqlOptions.namespace;
delete mysqlOptions.serialize;
delete mysqlOptions.deserialize;
Expand All @@ -63,10 +66,21 @@ export class KeyvMysql extends EventEmitter implements KeyvStoreAdapter {
keySize: 255, ...options,
};

const ttlScheduler: string[] = [
`SET GLOBAL event_scheduler = ON;`,
`DROP EVENT IF EXISTS delete_expired_keys;`,
`CREATE EVENT delete_expired_keys ON SCHEDULE EVERY 30 SECOND
DO DELETE FROM ${this.opts.table!} WHERE JSON_EXTRACT( value, '$.expiration' ) < UNIX_TIMESTAMP();`]

const createTable = `CREATE TABLE IF NOT EXISTS ${this.opts.table!}(id VARCHAR(${Number(this.opts.keySize!)}) PRIMARY KEY, value TEXT )`;

const connected = connection().then(async query => {
await query(createTable);
if (this.ttlSupport) {
for (const ttlCmd of ttlScheduler) {
await query(ttlCmd);
}
}
return query;
}).catch(error => this.emit('error', error));

Expand Down
1 change: 1 addition & 0 deletions packages/mysql/src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,6 @@ export type KeyvMysqlOptions = {
uri?: string;
table?: string;
keySize?: number;
useInternalScheduler?: boolean;
iterationLimit?: string | number;
} & ConnectionOptions;

0 comments on commit 721e48f

Please sign in to comment.