-
Notifications
You must be signed in to change notification settings - Fork 77
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
9dfd0b2
commit 97b9209
Showing
2 changed files
with
223 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,113 @@ | ||
# Gorm Sharding | ||
|
||
Gorm Plugin for sharding table. | ||
|
||
This plugin base on SQL Parser, catch raw SQL before execute db, and then replace table name by sharding rules. | ||
|
||
## Translations | ||
|
||
For chinese, see [README_zh.md](./README_zh.md) | ||
|
||
## Features | ||
|
||
- Non-intrusive design. Load the plugin, specify the config, and all done. | ||
- Lighting-fast. No network based middlewares, as fast as Go. | ||
- Multiple database support. PostgreSQL tested, MySQL and SQLite is coming. | ||
|
||
## Install | ||
|
||
```bash | ||
go get -u github.com/longbridgeapp/gorm-sharding | ||
``` | ||
|
||
## Usage | ||
|
||
After the database connection opened, use the sharding plugin that registered the tables which you want to shard. | ||
|
||
The `Register` function takes a map, the key is the **original table name** and the value is a **resolver** which is composed by five configurable fields that described in [Config description](config-description). | ||
|
||
## Usage Example | ||
|
||
```go | ||
sharding := gormsharding.Register(map[string]gormsharding.Resolver{ | ||
"orders": { | ||
EnableFullTable: true, | ||
ShardingColumn: "user_id", | ||
ShardingAlgorithm: func(value interface{}) (suffix string, err error) { | ||
switch user_id := value.(type) { | ||
case int64: | ||
return fmt.Sprintf("_%02d", user_id % 64), nil | ||
default: | ||
return "", errors.New("invalid user_id") | ||
} | ||
}, | ||
ShardingAlgorithmByPrimaryKey: func(id int64) (suffix string) { | ||
return fmt.Sprintf("_%02d", keygen.TableIdx(id)) | ||
}, | ||
PrimaryKeyGenerate: func(tableIdx int64) int64 { | ||
return keygen.Next(tableIdx) | ||
} | ||
}, | ||
}) | ||
db.Use(sharding) | ||
``` | ||
|
||
## Config description | ||
|
||
- [EnableFullTable](#EnableFullTable) | ||
- [ShardingColumn](#ShardingColumn) | ||
- [ShardingAlgorithm](#ShardingAlgorithm) | ||
- [ShardingAlgorithmByPrimaryKey](#ShardingAlgorithmByPrimaryKey) | ||
- [PrimaryKeyGenerate](#PrimaryKeyGenerate) | ||
|
||
#### EnableFullTable | ||
|
||
Whether to enable a full table. | ||
|
||
When full table enabled, sharding plugin will double write data to both main table and sharding table. | ||
|
||
#### ShardingColumn | ||
|
||
Which table column you want to used for sharding the table rows. | ||
|
||
For example, for a product order table, you may want to split the rows by `user_id`. | ||
|
||
### ShardingAlgorithm | ||
|
||
A function to generate the sharding table's suffix by the column value. | ||
|
||
It's signature is `func(columnValue interface{}) (suffix string, err error)`. | ||
|
||
For an example, see the [usage example](#usage-example) above. | ||
|
||
#### ShardingAlgorithmByPrimaryKey | ||
|
||
A function to generate the sharding table's suffix by the primary key. | ||
|
||
It's signature is `func(id int64) (suffix string)`. | ||
|
||
For an example, see the [usage example](#usage-example) above. | ||
|
||
Note, when the record contains an id field, ShardingAlgorithmByPrimaryKey will preferred than ShardingAlgorithm. | ||
|
||
#### PrimaryKeyGenerate | ||
|
||
A function to generate the primary key. | ||
|
||
Used only when insert and the record does not contains an id field. | ||
|
||
It's signature is `func(tableIdx int64) int64`. | ||
|
||
For an example, see the [usage example](#usage-example) above. | ||
|
||
We recommend you use the [keygen](https://github.com/longbridgeapp/gorm-sharding) component, it is a distributed primary key generator. | ||
|
||
When use auto-increment like generator, the tableIdx argument could ignored. | ||
|
||
## References | ||
|
||
https://gorm.io/docs/write_plugins.html#Plugin | ||
|
||
## License | ||
|
||
This project under MIT license. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,110 @@ | ||
# Gorm Sharding | ||
|
||
Gorm 分表插件。 | ||
|
||
基于 SQL 解析器,执行前捕获原始 SQL,根据分表规则替换表名。 | ||
|
||
## 特色 | ||
|
||
- 非侵入式设计。加载插件,指定配置,完成。 | ||
- 快。没有网络中间件,跟 Go 一样快。 | ||
- 多数据库支持。PostgreSQL 已测试,MySQL 和 SQLite 进行中。 | ||
|
||
## 安装 | ||
|
||
```bash | ||
go get -u github.com/longbridgeapp/gorm-sharding | ||
``` | ||
|
||
## 用法 | ||
|
||
数据库连接打开后,使用分表插件注册需要拆分的表。 | ||
|
||
`Register` 函数接收一个 map,键是**原始表名**,值是一个 **resolver** ,由可配置的字段组成,见配置描述。 | ||
|
||
## 用法示例 | ||
|
||
```go | ||
sharding := gormsharding.Register(map[string]gormsharding.Resolver{ | ||
"orders": { | ||
EnableFullTable: true, | ||
ShardingColumn: "user_id", | ||
ShardingAlgorithm: func(value interface{}) (suffix string, err error) { | ||
switch user_id := value.(type) { | ||
case int64: | ||
return fmt.Sprintf("_%02d", user_id % 64), nil | ||
default: | ||
return "", errors.New("invalid user_id") | ||
} | ||
}, | ||
ShardingAlgorithmByPrimaryKey: func(id int64) (suffix string) { | ||
return fmt.Sprintf("_%02d", keygen.TableIdx(id)) | ||
}, | ||
PrimaryKeyGenerate: func(tableIdx int64) int64 { | ||
return keygen.Next(tableIdx) | ||
} | ||
}, | ||
}) | ||
db.Use(sharding) | ||
``` | ||
|
||
## 配置描述 | ||
|
||
- [EnableFullTable](#EnableFullTable) | ||
- [ShardingColumn](#ShardingColumn) | ||
- [ShardingAlgorithm](#ShardingAlgorithm) | ||
- [ShardingAlgorithmByPrimaryKey](#ShardingAlgorithmByPrimaryKey) | ||
- [PrimaryKeyGenerate](#PrimaryKeyGenerate) | ||
|
||
#### EnableFullTable | ||
|
||
Whether to enable a full table. | ||
是否开启全表。 | ||
|
||
开启后,分表插件将双写数据到主表和分表。 | ||
|
||
#### ShardingColumn | ||
|
||
用来分表的表字段。 | ||
|
||
如,对于一个订单表,你可能想用 `user_id` 拆分数据。 | ||
|
||
### ShardingAlgorithm | ||
|
||
使用列值生成分表后缀的函数。 | ||
|
||
签名是 `func(columnValue interface{}) (suffix string, err error)`。 | ||
|
||
可以参考上面的用法示例。 | ||
|
||
#### ShardingAlgorithmByPrimaryKey | ||
|
||
使用主键生成分表后缀的函数。 | ||
|
||
签名是 `func(id int64) (suffix string)`。 | ||
|
||
可以参考上面的用法示例。 | ||
|
||
注意,当记录包含 id 字段时,ShardingAlgorithmByPrimaryKey 优先于 ShardingAlgorithm。 | ||
|
||
#### PrimaryKeyGenerate | ||
|
||
生成主键的函数。 | ||
|
||
只有当插入数据并且记录不包含 id 字段时使用。 | ||
|
||
签名是 `func(tableIdx int64) int64`。 | ||
|
||
可以参考上面的用法示例。 | ||
|
||
推荐使用 [keygen](https://github.com/longbridgeapp/gorm-sharding) 组件,它是一个分布式的主键生成器。 | ||
|
||
当使用自增类的生成器时,tableIdx 参数可以忽略。 | ||
|
||
## 参考 | ||
|
||
https://gorm.io/docs/write_plugins.html#Plugin | ||
|
||
## 许可证 | ||
|
||
本项目使用 MIT 许可证。 |