Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Static Host Catalog #51

Merged
merged 92 commits into from
May 27, 2020
Merged
Show file tree
Hide file tree
Changes from 89 commits
Commits
Show all changes
92 commits
Select commit Hold shift + click to select a range
7785b9f
database: add wt_public_id domain type
mgaffney May 5, 2020
327fd42
static-hosts: add migrations
mgaffney May 5, 2020
3ce5832
Merge branch 'master' into mgaffney/static-hosts-domain
mgaffney May 7, 2020
4e2d9d5
Rename migrations
mgaffney May 7, 2020
897777b
migrations: add description column and fix foreign keys
mgaffney May 7, 2020
5846c2e
Add protobuf for static host
mgaffney May 7, 2020
1ad0acd
Merge remote-tracking branch 'origin/master' into mgaffney/static-hos…
mgaffney May 8, 2020
b9870ac
Add HostCatalog and HostSet
mgaffney May 11, 2020
efb8690
Add Options
mgaffney May 12, 2020
f4e17d5
Add foreign key constraint
mgaffney May 12, 2020
8e30e7b
Add package doc
mgaffney May 12, 2020
90b8c43
Add copied scanner code
mgaffney May 12, 2020
02aeadb
Add NewHostCatalog with tests
mgaffney May 12, 2020
fbff7a6
Merge remote-tracking branch 'origin/master' into mgaffney/static-hos…
mgaffney May 12, 2020
8dcc6fc
Override default table name for static host catalog
mgaffney May 12, 2020
1e20732
WIP: save static host catalog to database
mgaffney May 12, 2020
05693d1
Merge remote-tracking branch 'origin/master' into mgaffney/static-hos…
mgaffney May 12, 2020
1fd467c
WIP: Print db url for debugging
mgaffney May 12, 2020
a7c6020
Add iam test helper function
mgaffney May 13, 2020
6f0f607
Make gorm default to null for names and descriptions
mgaffney May 13, 2020
23df827
Add foreign key constraint from host catalog to scope
mgaffney May 13, 2020
e93bc4c
Names are not needed for the test org and project
mgaffney May 13, 2020
da6abb6
Avoid an ordering dependency in down migrations
mgaffney May 13, 2020
65bc134
Merge remote-tracking branch 'origin/master' into mgaffney/static-hos…
mgaffney May 13, 2020
b570191
Use wt_timestamp
mgaffney May 13, 2020
432a704
Add check constraint on host address column
mgaffney May 13, 2020
cf990d0
Add Host
mgaffney May 13, 2020
272979c
Add HostSet
mgaffney May 13, 2020
8e6bb8e
Add HostSetMember
mgaffney May 14, 2020
28b0408
Test a HostSetMember can be written to the database
mgaffney May 14, 2020
43424ea
Move the getters and setters for table name to a new file
mgaffney May 14, 2020
699bbc8
Add repository
mgaffney May 18, 2020
311560e
Merge branch 'master' into mgaffney/static-hosts-domain
mgaffney May 18, 2020
65d4e87
Add NewRepository
mgaffney May 19, 2020
9c1f9d5
Close connection before cleanup of database
mgaffney May 19, 2020
1eb5a57
Add comments to protobuf file and regenerate
mgaffney May 19, 2020
b3b31fc
Define repository methods for HostCatalog
mgaffney May 19, 2020
a953a94
Assign PublicId when resource is inserted into the database
mgaffney May 19, 2020
2aa9a4c
Wrap a nil parameter error
mgaffney May 19, 2020
66662a1
Add oplog tickets for static host types
mgaffney May 19, 2020
5726422
WIP: first cut at CreateCatalog
mgaffney May 19, 2020
3f907cb
CreateHostCatalog: more tests
mgaffney May 19, 2020
de05b9b
Detect unique constraint violation
mgaffney May 19, 2020
cda3faf
Merge remote-tracking branch 'origin/master' into mgaffney/static-hos…
mgaffney May 19, 2020
fb8864c
Use const for PublicId prefixes
mgaffney May 20, 2020
00fb6eb
Use PostgreSQL error name instead of error code
mgaffney May 20, 2020
7678e90
Add TODO comments
mgaffney May 20, 2020
6e0daef
Fix comments
mgaffney May 20, 2020
26ef138
Use PublicId not PublicID
mgaffney May 20, 2020
65aac56
Refactor: move error variables from host/static to db
mgaffney May 20, 2020
e59ca3e
Create a new repository for each test
mgaffney May 20, 2020
9870015
Validate CreateCatalog returns a new HostCatalog
mgaffney May 20, 2020
4495581
Do not modify the HostCatalog passed in
mgaffney May 20, 2020
85707e3
Implement UpdateCatalog
mgaffney May 21, 2020
39c9e82
Merge remote-tracking branch 'origin/master' into mgaffney/static-hos…
mgaffney May 21, 2020
3ee7fcf
Merge remote-tracking branch 'origin/master' into mgaffney/static-hos…
mgaffney May 21, 2020
9e968a9
Minor cleanup to make folds work
mgaffney May 23, 2020
cbb55b9
Implement LookupCatalog
mgaffney May 23, 2020
b2687af
Refactor: extract allocCatalog function
mgaffney May 23, 2020
241b383
Add comment
mgaffney May 23, 2020
03ed3fd
Refactor: extract newCatalogMetadata function
mgaffney May 23, 2020
053e1ff
Make error strings more consistent
mgaffney May 23, 2020
930b630
Move database specific error check to internal/db package
mgaffney May 23, 2020
12b4135
Change signature of DeleteCatalog and update docs
mgaffney May 23, 2020
e363439
Implement DeleteCatalog
mgaffney May 23, 2020
33a0766
Make error strings more consistent
mgaffney May 23, 2020
7fdf761
Add examples
mgaffney May 23, 2020
e9e5dcb
Update docs
mgaffney May 23, 2020
b407bdb
Add package level doc and example for repository
mgaffney May 23, 2020
04e5727
No need for using reflect.DeepEqual
mgaffney May 26, 2020
31de34e
Remove redundant comment
mgaffney May 26, 2020
5eecfd5
Clarify ErrMultipleRecords results in the transaction being rolled back
mgaffney May 26, 2020
0f440c3
Remove redundant error checks
mgaffney May 26, 2020
8ac88fd
Remove unnecessary check
mgaffney May 26, 2020
0d85ee3
Return early instead of using nested 'else' statements
mgaffney May 26, 2020
1fe3f7e
Update and Delete must return a row count
mgaffney May 27, 2020
1db4ccd
catch err on defers (#82)
jimlambrt May 27, 2020
6bca56f
Remove 'if assert' pattern
mgaffney May 27, 2020
a723919
Use assert package in test helper function
mgaffney May 27, 2020
a02bc24
Add test for updating a non existent catalog
mgaffney May 27, 2020
4fb5032
Use a single assert when a specific error is expected
mgaffney May 27, 2020
a464981
Use assert package when it clarifies the check
mgaffney May 27, 2020
b6eeb6c
Catch errors in cleanup functions
mgaffney May 27, 2020
26feffe
Update comments in protobuf file
mgaffney May 27, 2020
2716de4
Update doc for UpdateCatalog to reflect the new behavior
mgaffney May 27, 2020
032d54a
Add common error values for field mask errors
mgaffney May 27, 2020
82e8088
Require a field mask when calling update methods
mgaffney May 27, 2020
dc82593
Add TODO for removing direct call to gorm
mgaffney May 27, 2020
112d2b2
Export Public ID prefixes
mgaffney May 27, 2020
dbeff1a
Merge remote-tracking branch 'origin/master' into mgaffney/static-hos…
mgaffney May 27, 2020
9487e74
Add db timestamp guards
mgaffney May 27, 2020
50f82b1
Remove unused option
mgaffney May 27, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ protobuild:
@protoc-go-inject-tag -input=./internal/oplog/oplog_test/oplog_test.pb.go
@protoc-go-inject-tag -input=./internal/iam/store/scope.pb.go
@protoc-go-inject-tag -input=./internal/db/db_test/db_test.pb.go
@protoc-go-inject-tag -input=./internal/host/static/store/static.pb.go
@rm -R ${TMP_DIR}

protolint:
Expand Down
36 changes: 35 additions & 1 deletion internal/db/error.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
package db

import "errors"
import (
"errors"

"github.com/lib/pq"
)

// Errors returned from this package may be tested against these errors
// with errors.Is.
Expand All @@ -12,6 +16,14 @@ var (
// an attribute on a struct contains illegal or invalid values.
ErrInvalidParameter = errors.New("invalid parameter")

// ErrInvalidFieldMask is returned by update methods if the field mask
// contains unknown fields or fields that cannot be updated.
ErrInvalidFieldMask = errors.New("invalid field mask")

// ErrEmptyFieldMask is returned by update methods if the field mask is
// empty.
ErrEmptyFieldMask = errors.New("empty field mask")

// ErrNotUnique is returned by create and update methods when a write
// to the repository resulted in a unique constraint violation.
ErrNotUnique = errors.New("unique constraint violation")
Expand All @@ -23,4 +35,26 @@ var (
// when attempting to read from the database into struct.
// When reading into a slice it won't return this error.
ErrRecordNotFound = errors.New("record not found")

// ErrMultipleRecords is returned by update and delete methods when a
// write to the repository would result in more than one record being
// changed resulting in the transaction being rolled back.
ErrMultipleRecords = errors.New("multiple records")
)

// IsUnique returns a boolean indicating whether the error is known to
// report a unique constraint violation.
func IsUnique(err error) bool {
if err == nil {
return false
}

var pqError *pq.Error
if errors.As(err, &pqError) {
if pqError.Code.Name() == "unique_violation" {
return true
}
}

return false
}
45 changes: 45 additions & 0 deletions internal/db/error_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
package db

import (
"testing"

"github.com/lib/pq"
"github.com/stretchr/testify/assert"
)

func TestError_IsUnique(t *testing.T) {
var tests = []struct {
name string
in error
want bool
}{
{
name: "nil-error",
in: nil,
want: false,
},
{
name: "postgres-not-unique",
in: &pq.Error{
Code: pq.ErrorCode("23503"),
},
want: false,
},
{
name: "postgres-is-unique2",
in: &pq.Error{
Code: pq.ErrorCode("23505"),
},
want: true,
},
}
for _, tt := range tests {
tt := tt
t.Run(tt.name, func(t *testing.T) {
assert := assert.New(t)
err := tt.in
got := IsUnique(err)
assert.Equal(tt.want, got)
})
}
}
88 changes: 88 additions & 0 deletions internal/db/migrations/postgres.gen.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 8 additions & 0 deletions internal/db/migrations/postgres/10_static_host.down.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
begin;

drop table static_host_set_member cascade;
drop table static_host_set cascade;
drop table static_host cascade;
drop table static_host_catalog cascade;

commit;
68 changes: 68 additions & 0 deletions internal/db/migrations/postgres/10_static_host.up.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
begin;

create table static_host_catalog (
public_id wt_public_id primary key,
scope_id wt_public_id not null
references iam_scope (public_id)
on delete cascade
on update cascade,
name text,
description text,
create_time wt_timestamp,
update_time wt_timestamp,
unique(scope_id, name)
);

create table static_host (
public_id wt_public_id primary key,
static_host_catalog_id wt_public_id not null
references static_host_catalog (public_id)
on delete cascade
on update cascade,
name text,
description text,
address text not null
check(
length(trim(address)) > 7
and
length(trim(address)) < 256
),
create_time wt_timestamp,
update_time wt_timestamp,
unique(static_host_catalog_id, name)
);

create table static_host_set (
public_id wt_public_id primary key,
static_host_catalog_id wt_public_id not null
references static_host_catalog (public_id)
on delete cascade
on update cascade,
name text,
description text,
create_time wt_timestamp,
update_time wt_timestamp,
unique(static_host_catalog_id, name)
);

create table static_host_set_member (
static_host_set_id wt_public_id
references static_host_set (public_id)
on delete cascade
on update cascade,
static_host_id wt_public_id
references static_host (public_id)
on delete cascade
on update cascade,
primary key(static_host_set_id, static_host_id)
);

insert into oplog_ticket (name, version)
values
('static_host_catalog', 1),
('static_host', 1),
('static_host_set', 1),
('static_host_set_member', 1);

commit;

34 changes: 34 additions & 0 deletions internal/host/static/doc.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
// Package static provides a host, a host catalog, and a host set suitable
// for hosts with a static address.
//
// A host catalog contains a collection of hosts with static addresses.
// These hosts can be grouped into host sets allowing them to be used in a
// target. Hosts and host sets are owned by a single host catalog. If a
// host catalog is deleted, all hosts and host sets owned by it are also
// deleted. A host set contains references to zero or more hosts but does
// not own them. Deleting a host set does not effect any of the hosts it
// referenced. A host set can only reference hosts from the host catalog
// that owns it. Host addresses must be unique within a host catalog.
//
// Repository
//
// A repository provides methods for creating, updating, retrieving, and
// deleting host catalogs, host sets, and hosts. A new repository should be
// created for each transaction. For example:
//
// var wrapper wrapping.Wrapper
// ... init wrapper...
//
// // db implements both the reader and writer interfaces.
// db, _ := db.Open(db.Postgres, url)
//
// var repo *static.Repository
//
// repo, _ = static.NewRepository(db, db, wrapper)
// catalog, _ := repo.LookupCatalog(ctx, catalogId)
//
// catalog.Name = "new name"
//
// repo, _ = static.NewRepository(db, db, wrapper)
// catalog, _ := repo.UpdateCatalog(ctx, catalog, []string{"Name"})
package static
39 changes: 39 additions & 0 deletions internal/host/static/example_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
package static_test

import (
"fmt"

"github.com/hashicorp/watchtower/internal/host/static"
)

func ExampleNewHostCatalog() {
projectPublicId := "p_1234"
catalog, _ := static.NewHostCatalog(projectPublicId, static.WithName("my catalog"))
fmt.Println(catalog.Name)
// Output:
// my catalog
}

func ExampleNewHost() {
catalogPublicId := "sthc_1234"
host, _ := static.NewHost(catalogPublicId, "127.0.0.1")
fmt.Println(host.Address)
// Output:
// 127.0.0.1
}

func ExampleNewHostSet() {
catalogPublicId := "sthc_1234"
set, _ := static.NewHostSet(catalogPublicId, static.WithName("my host set"))
fmt.Println(set.Name)
// Output:
// my host set
}

func ExampleNewHostSetMember() {
setPublicId := "sths_11111"
hostPublicId := "sth_22222"
member, _ := static.NewHostSetMember(setPublicId, hostPublicId)
fmt.Println(member.StaticHostSetId)
fmt.Println(member.StaticHostId)
}
Loading