Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hasql version 2.0 #15

Merged
merged 20 commits into from
Nov 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,6 @@ jobs:
steps:
- uses: actions/checkout@v2
- name: golangci-lint
uses: golangci/golangci-lint-action@v3
uses: golangci/golangci-lint-action@v6
with:
version: v1.52.2
version: v1.62.0
8 changes: 4 additions & 4 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,24 +11,24 @@ jobs:
test:
strategy:
matrix:
go-version: [1.19.x, 1.20.x]
go-version: [1.22.x, 1.23.x]
os: [ubuntu-latest, macos-latest, windows-latest]
env:
OS: ${{ matrix.os }}
GO: ${{ matrix.go-version }}
runs-on: ${{ matrix.os }}
steps:
- name: Install Go
uses: actions/setup-go@v2
uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}
- name: Checkout code
uses: actions/checkout@v2
uses: actions/checkout@v4
- name: Test
run: go test -race ./... -coverprofile=coverage.txt -covermode=atomic
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v1
if: matrix.os == 'ubuntu-latest' && matrix.go-version == '1.20.x'
if: matrix.os == 'ubuntu-latest' && matrix.go-version == '1.23.x'
with:
file: ./coverage.txt
flags: unittests
Expand Down
7 changes: 1 addition & 6 deletions .golangci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -100,25 +100,20 @@ linters-settings:
linters:
disable-all: true
enable:
- deadcode
- depguard
- errcheck
- goconst
- gofmt # On why gofmt when goimports is enabled - https://github.com/golang/go/issues/21476
- goimports
- golint
- gosimple
- govet
- ineffassign
- maligned
- misspell
- revive
- staticcheck
- structcheck
- typecheck
- unconvert
- unparam
- unused
- varcheck

issues:
# List of regexps of issue texts to exclude, empty list by default.
Expand Down
77 changes: 42 additions & 35 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
With [Go module](https://github.com/golang/go/wiki/Modules) support, simply add the following import

```go
import "golang.yandex/hasql"
import "golang.yandex/hasql/v2"
```

to your code, and then `go [build|run|test]` will automatically fetch the
Expand All @@ -30,82 +30,94 @@ necessary dependencies.
Otherwise, to install the `hasql` package, run the following command:

```console
$ go get -u golang.yandex/hasql
$ go get -u golang.yandex/hasql/v2
```

## How does it work
`hasql` operates using standard `database/sql` connection pool objects. User creates `*sql.DB` objects for each node of database cluster and passes them to constructor. Library keeps up to date information on state of each node by 'pinging' them periodically. User is provided with a set of interfaces to retrieve `*sql.DB` object suitable for required operation.
`hasql` operates using standard `database/sql` connection pool objects. User creates `*sql.DB`-compatible objects for each node of database cluster and passes them to constructor. Library keeps up to date information on state of each node by 'pinging' them periodically. User is provided with a set of interfaces to retrieve database node object suitable for required operation.

```go
dbFoo, _ := sql.Open("pgx", "host=foo")
dbBar, _ := sql.Open("pgx", "host=bar")
cl, err := hasql.NewCluster(
[]hasql.Node{hasql.NewNode("foo", dbFoo), hasql.NewNode("bar", dbBar) },
checkers.PostgreSQL,

discoverer := NewStaticNodeDiscoverer(
NewNode("foo", dbFoo),
NewNode("bar", dbBar),
)

cl, err := hasql.NewCluster(discoverer, hasql.PostgreSQLChecker)
if err != nil { ... }

node := cl.Primary()
if node == nil {
err := cl.Err() // most recent errors for all nodes in the cluster
}

// Do anything you like
fmt.Println("Node address", node.Addr)
fmt.Printf("got node %s\n", node)

ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()

if err = node.DB().PingContext(ctx); err != nil { ... }
```

`hasql` does not configure provided connection pools in any way. It is user's job to set them up properly. Library does handle their lifetime though - pools are closed when `Cluster` object is closed.

### Concepts and entities

**Cluster** is a set of `database/sql`-compatible nodes that tracks their lifespan and provides access to each individual nodes.

**Node** is a single database instance in high-availability cluster.

**Node discoverer** provides nodes objects to cluster. This abstraction allows user to dynamically change set of cluster nodes, for example collect nodes list via Service Discovery (etcd, Consul).

**Node checker** collects information about current state of individual node in cluster, such as: cluster role, network latency, replication lag, etc.

**Node picker** picks node from cluster by given criterion using predefined algorithm: random, round-robin, lowest latency, etc.

### Supported criteria
_Alive primary_|_Alive standby_|_Any alive_ node, or _none_ otherwise
_Alive primary_|_Alive standby_|_Any alive_ node or _none_ otherwise
```go
node := c.Primary()
if node == nil { ... }
node := c.Node(hasql.Alive)
```

_Alive primary_|_Alive standby_, or _any alive_ node, or _none_ otherwise
_Alive primary_ or _none_ otherwise
```go
node := c.PreferPrimary()
if node == nil { ... }
node := c.Node(hasql.Primary)
```

### Ways of accessing nodes
Any of _currently alive_ nodes _satisfying criteria_, or _none_ otherwise
_Alive standby_ or _none_ otherwise
```go
node := c.Primary()
if node == nil { ... }
node := c.Node(hasql.Standby)
```

Any of _currently alive_ nodes _satisfying criteria_, or _wait_ for one to become _alive_
_Alive primary_|_Alive standby_ or _none_ otherwise
```go
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
node, err := c.WaitForPrimary(ctx)
if err == nil { ... }
node := c.Node(hasql.PreferPrimary)
```

_Alive standby_|_Alive primary_ or _none_ otherwise
```go
node := c.Node(hasql.PreferStandby)
```

### Node pickers
When user asks `Cluster` object for a node a random one from a list of suitable nodes is returned. User can override this behavior by providing a custom node picker.

Library provides a couple of predefined pickers. For example if user wants 'closest' node (with lowest latency) `PickNodeClosest` picker should be used.
Library provides a couple of predefined pickers. For example if user wants 'closest' node (with lowest latency) `LatencyNodePicker` should be used.

```go
cl, err := hasql.NewCluster(
[]hasql.Node{hasql.NewNode("foo", dbFoo), hasql.NewNode("bar", dbBar) },
checkers.PostgreSQL,
hasql.WithNodePicker(hasql.PickNodeClosest()),
hasql.NewStaticNodeDiscoverer(hasql.NewNode("foo", dbFoo), hasql.NewNode("bar", dbBar)),
hasql.PostgreSQLChecker,
hasql.WithNodePicker(new(hasql.LatencyNodePicker[*sql.DB])),
)
if err != nil { ... }
```

## Supported databases
Since library works over standard `database/sql` it supports any database that has a `database/sql` driver. All it requires is a database-specific checker function that can tell if node is primary or standby.
Since library requires `Querier` interface, which describes a subset of `database/sql.DB` methods, it supports any database that has a `database/sql` driver. All it requires is a database-specific checker function that can provide node state info.

Check out `golang.yandex/hasql/checkers` package for more information.
Check out `node_checker.go` file for more information.

### Caveats
Node's state is transient at any given time. If `Primary()` returns a node it does not mean that node is still primary when you execute statement on it. All it means is that it was primary when it was last checked. Nodes can change their state at a whim or even go offline and `hasql` can't control it in any manner.
Expand All @@ -115,8 +127,3 @@ This is one of the reasons why nodes do not expose their perceived state to user
## Extensions
### Instrumentation
You can add instrumentation via `Tracer` object similar to [httptrace](https://godoc.org/net/http/httptrace) in standard library.

### sqlx
`hasql` can operate over `database/sql` pools wrapped with [sqlx](https://github.com/jmoiron/sqlx). It works the same as with standard library but requires user to import `golang.yandex/hasql/sqlx` instead.

Refer to `golang.yandex/hasql/sqlx` package for more information.
162 changes: 0 additions & 162 deletions check_nodes.go

This file was deleted.

Loading
Loading