Skip to content
This repository has been archived by the owner on Sep 20, 2022. It is now read-only.

Commit

Permalink
Merge branch release-0.3.0
Browse files Browse the repository at this point in the history
  • Loading branch information
albrow committed May 15, 2015
2 parents 7c1e840 + 344edcd commit 3286495
Show file tree
Hide file tree
Showing 23 changed files with 812 additions and 281 deletions.
29 changes: 29 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
Contributing
------------

Feedback, bug reports, and pull requests are greatly appreciated :)

### Issues

The following are all great reasons to submit an issue:

1. You found a bug in the code.
2. Something is missing from the documentation or the existing documentation is unclear.
3. You have an idea for a new feature.

If you are thinking about submitting an issue please remember to:

1. Describe the issue in detail.
2. If applicable, describe the steps to reproduce the error, which probably should include some example code.
3. Mention details about your platform: OS, version of Go and Redis, etc.

### Pull Requests

Jobs uses semantic versioning and the [git branching model described here](http://nvie.com/posts/a-successful-git-branching-model/).
If you plan on submitting a pull request, you should:

1. Fork the repository.
2. Create a new "feature branch" with a descriptive name (e.g. fix-database-error).
3. Make your changes in the feature branch.
4. Run the tests to make sure that they still pass. Updated the tests if needed.
5. Submit a pull request to merge your feature branch into the *develop* branch. Please do not request to merge directly into master.
65 changes: 31 additions & 34 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ A persistent and flexible background jobs library for go.

[![GoDoc](https://godoc.org/github.com/albrow/jobs?status.svg)](https://godoc.org/github.com/albrow/jobs)

Version: 0.2.2
Version: 0.3.0

Jobs is powered by Redis and supports the following features:

Expand Down Expand Up @@ -39,6 +39,19 @@ the email might never be sent. Jobs guarantees that the email will be sent at so
and allows you to spread the work between different machines.


Development Status
------------------

Jobs is being actively developed and is well-tested. However, since it is still a young library,
it is not recommended for use in mission-critical production applications at this time. It is probably
fine to use for low-traffic hobby sites, and in fact we encourage its use in those settings to help
polish the API and find missing features and hidden bugs. Pull requests and issue reports are much
appreciated :)

Jobs follows semantic versioning but offers no guarantees of backwards compatibility until version
1.0. Keep in mind that breaking changes might occur. We will do our best to make the community aware
of any non-trivial breaking changes beforehand.

Installation
------------

Expand All @@ -48,15 +61,15 @@ Jobs requires Go version >= 1.2. If you do not already have it, follow these ins
- Follow the instructions for [setting up your go workspace](https://golang.org/doc/code.html)

Jobs requires access to a Redis database. If you plan to have multiple worker pools spread
out accross different machines, they should all connect to the same Redis database. If you
out across different machines, they should all connect to the same Redis database. If you
only want to run one worker pool, it is safe to install Redis locally and run it on the same
machine. In either case, if you need to install Redis, follow these instructions:

- [Install Redis](http://redis.io/download).
- Follow the instructions in the section called
[Installing Redis more properly](http://redis.io/topics/quickstart#installing-redis-more-properly).
- Make sure you understand how [Redis Persistence](http://redis.io/topics/persistence) works and have
edited your config file to get your desired persistence. I recommend using both RDB and AOF and setting
edited your config file to get your desired persistence. We recommend using both RDB and AOF and setting
fsync to either "always" or "everysec".

After that, you can install Jobs like you would any other go package: `go get github.com/albrow/jobs`.
Expand Down Expand Up @@ -137,14 +150,17 @@ or ScheduleRecurring to check on the status of the job or cancel it manually.
You can schedule any number of worker pools across any number of machines, provided every machine
agrees on the definition of the job types. If you want, you can start a worker pool on the same
machines that are scheduling jobs, or you can have each worker pool running on a designated machine.
It is technically safe to start multiple pools on a single machine, but typically there is no reason
to and you should only start one pool per machine.
Since each pool is assigned an id based on a unique hardware identifier, you must only run one
worker pool per machine.

To create a new pool with the [default configuration](http://godoc.org/github.com/albrow/jobs#pkg-variables),
just pass in nil:

``` go
pool := jobs.NewPool(nil)
pool, err := jobs.NewPool(nil)
if err != nil {
// Handle err
}
```

You can also specify a different configuration by passing in
Expand All @@ -153,10 +169,13 @@ in will fallback to the default values. So here's how you could start a pool wit
size of 10, while letting the other options remain the default.

``` go
pool := jobs.NewPool(&jobs.PoolConfig{
pool, err := jobs.NewPool(&jobs.PoolConfig{
NumWorkers: 10,
BatchSize: 10,
})
if err != nil {
// Handle err
}
```

After you have created a pool, you can start it with the Start method. Once started, the pool will
Expand All @@ -166,7 +185,10 @@ Wait in a defer statement. Typical usage looks something like this:

``` go
func main() {
pool := jobs.NewPool(nil)
pool, err := jobs.NewPool(nil)
if err != nil {
// Handle err
}
defer func() {
pool.Close()
if err := pool.Wait(); err != nil {
Expand Down Expand Up @@ -196,32 +218,7 @@ To run the tests just run `go test .` If anything fails, please report an issue
Contributing
------------

Feedback, bug reports, and pull requests are greatly appreciated :)

### Issues

The following are all great reasons to submit an issue:

1. You found a bug in the code.
2. Something is missing from the documentation or the existing documentation is unclear.
3. You have an idea for a new feature.

If you are thinking about submitting an issue please remember to:

1. Describe the issue in detail.
2. If applicable, describe the steps to reproduce the error, which probably should include some example code.
3. Mention details about your platform: OS, version of Go and Redis, etc.

### Pull Requests

Jobs uses semantic versioning and the [git branching model described here](http://nvie.com/posts/a-successful-git-branching-model/).
If you plan on submitting a pull request, you should:

1. Fork the repository.
2. Create a new "feature branch" with a descriptive name (e.g. fix-database-error).
3. Make your changes in the feature branch.
4. Run the tests to make sure that they still pass. Updated the tests if needed.
5. Submit a pull request to merge your feature branch into the *develop* branch. Please do not request to merge directly into master.
See [Contributing.md](https://github.com/albrow/jobs/blob/master/CONTRIBUTING.md)


Guarantees
Expand Down
4 changes: 3 additions & 1 deletion doc.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@

// package jobs is a persistent and flexible background jobs library.
//
// Version: 0.2.2
//
// Version: 0.3.0
//
//
// Jobs is powered by redis and supports the following features:
//
Expand Down
24 changes: 12 additions & 12 deletions job.go
Original file line number Diff line number Diff line change
Expand Up @@ -67,21 +67,21 @@ func (j *Job) Duration() time.Duration {
return j.Finished().Sub(j.Started())
}

// key returns the key used for the hash in redis which stores all the
// Key returns the key used for the hash in redis which stores all the
// fields for this job.
func (j *Job) key() string {
func (j *Job) Key() string {
return "jobs:" + j.id
}

// isRecurring returns true iff the job is recurring
func (j *Job) isRecurring() bool {
// IsRecurring returns true iff the job is recurring
func (j *Job) IsRecurring() bool {
return j.freq != 0
}

// nextTime returns the time (unix UTC with nanosecond precision) that the
// NextTime returns the time (unix UTC with nanosecond precision) that the
// job should execute next, if it is a recurring job, and 0 if it is not.
func (j *Job) nextTime() int64 {
if !j.isRecurring() {
func (j *Job) NextTime() int64 {
if !j.IsRecurring() {
return 0
}
// NOTE: is this the proper way to handle rescheduling?
Expand Down Expand Up @@ -128,7 +128,7 @@ func (t *transaction) saveJob(job *Job) {
// addJobToTimeIndex adds commands to the transaction which will, when executed,
// add the job id to the time index with a score equal to the job's time field.
func (t *transaction) addJobToTimeIndex(job *Job) {
t.command("ZADD", redis.Args{keys.jobsTimeIndex, job.time, job.id}, nil)
t.command("ZADD", redis.Args{Keys.JobsTimeIndex, job.time, job.id}, nil)
}

// Refresh mutates the job by setting its fields to the most recent data
Expand Down Expand Up @@ -161,7 +161,7 @@ func (j *Job) enqueue() error {
func (j *Job) Reschedule(time time.Time) error {
t := newTransaction()
unixNanoTime := time.UTC().UnixNano()
t.command("HSET", redis.Args{j.key(), "time", unixNanoTime}, nil)
t.command("HSET", redis.Args{j.Key(), "time", unixNanoTime}, nil)
t.setStatus(j, StatusQueued)
j.time = unixNanoTime
t.addJobToTimeIndex(j)
Expand All @@ -186,7 +186,7 @@ func (j *Job) Cancel() error {
func (j *Job) setError(err error) error {
j.err = err
t := newTransaction()
t.command("HSET", redis.Args{j.key(), "error", j.err.Error()}, nil)
t.command("HSET", redis.Args{j.Key(), "error", j.err.Error()}, nil)
if err := t.exec(); err != nil {
return err
}
Expand Down Expand Up @@ -232,7 +232,7 @@ func (j *Job) setStatus(status Status) error {

// mainHashArgs returns the args for the hash which will store the job data
func (j *Job) mainHashArgs() []interface{} {
hashArgs := []interface{}{j.key(),
hashArgs := []interface{}{j.Key(),
"data", string(j.data),
"type", j.typ.name,
"time", j.time,
Expand Down Expand Up @@ -394,7 +394,7 @@ func scanBytes(reply interface{}, v *[]byte) error {
// the transaction.
func (t *transaction) scanJobById(id string, job *Job) {
job.id = id
t.command("HGETALL", redis.Args{job.key()}, newScanJobHandler(job))
t.command("HGETALL", redis.Args{job.Key()}, newScanJobHandler(job))
}

// FindById returns the job with the given id or an error if the job cannot be found
Expand Down
8 changes: 4 additions & 4 deletions job_status.go
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ const (

// key returns the key used for the sorted set in redis which will hold
// all jobs with this status.
func (status Status) key() string {
func (status Status) Key() string {
return "jobs:" + string(status)
}

Expand All @@ -41,23 +41,23 @@ func (status Status) key() string {
func (status Status) Count() (int, error) {
conn := redisPool.Get()
defer conn.Close()
return redis.Int(conn.Do("ZCARD", status.key()))
return redis.Int(conn.Do("ZCARD", status.Key()))
}

// JobIds returns the ids of all jobs that have the given status, ordered by
// priority or an error if there was a problem connecting to the database.
func (status Status) JobIds() ([]string, error) {
conn := redisPool.Get()
defer conn.Close()
return redis.Strings(conn.Do("ZREVRANGE", status.key(), 0, -1))
return redis.Strings(conn.Do("ZREVRANGE", status.Key(), 0, -1))
}

// Jobs returns all jobs that have the given status, ordered by priority or
// an error if there was a problem connecting to the database.
func (status Status) Jobs() ([]*Job, error) {
t := newTransaction()
jobs := []*Job{}
t.getJobsByIds(status.key(), newScanJobsHandler(&jobs))
t.getJobsByIds(status.Key(), newScanJobsHandler(&jobs))
if err := t.exec(); err != nil {
return nil, err
}
Expand Down
2 changes: 1 addition & 1 deletion job_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -290,7 +290,7 @@ func TestScanJob(t *testing.T) {
}
conn := redisPool.Get()
defer conn.Close()
replies, err := conn.Do("HGETALL", job.key())
replies, err := conn.Do("HGETALL", job.Key())
if err != nil {
t.Errorf("Unexpected error in HGETALL: %s", err.Error())
}
Expand Down
2 changes: 1 addition & 1 deletion job_type_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ func TestTypeSchedule(t *testing.T) {
if job.id == "" {
t.Errorf("After Type.Schedule, job.id was empty.")
}
expectKeyExists(t, job.key())
expectKeyExists(t, job.Key())
expectJobFieldEquals(t, job, "priority", testJobPriority, intConverter)
expectJobFieldEquals(t, job, "time", encodedTime, int64Converter)
expectJobFieldEquals(t, job, "data", encodedData, bytesConverter)
Expand Down
Loading

0 comments on commit 3286495

Please sign in to comment.