Skip to content

Commit

Permalink
doc: add offset management section
Browse files Browse the repository at this point in the history
  • Loading branch information
fraidev committed Dec 10, 2024
1 parent 5fca44c commit f238756
Show file tree
Hide file tree
Showing 3 changed files with 10 additions and 7 deletions.
13 changes: 9 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,22 +159,26 @@ In this case, additional transformation will be performed before records are sen
Read more about [JSON to JSON transformations](https://www.fluvio.io/smartmodules/certified/jolt/).

### Offset Management

Fluvio Consumer Offset feature allows for a connector to store the offset in the Fluvio cluster and use it on restart.
To activate it, you need to provide the `consumer` name and set the `strategy: auto`.
See the example below:

```yaml
apiVersion: 0.2.0
apiVersion: 0.1.0
meta:
version: 0.2.11
name: my-http-sink
type: http-sink
topic:
meta:
name: http-sink-topic
topic: http-sink-topic
consumer:
id: my-http-sink
offset:
strategy: auto
start: beginning
flush-period:
secs: 10
nanos: 0
http:
endpoint: "http://127.0.0.1/post"
```
Expand All @@ -185,6 +189,7 @@ $ fluvio consumer list
CONSUMER TOPIC PARTITION OFFSET LAST SEEN
my-http-sink http-sink-topic 0 0 3s
```

## Contributing

Follow on the conventional `CONTRIBUTING.md` file to setup your environment and
Expand Down
2 changes: 1 addition & 1 deletion crates/http-sink/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ pub(crate) struct HttpConfig {
/// Http connect timeout in milliseconds
#[serde(with = "humantime_serde", default = "default_http_connect_timeout")]
pub http_connect_timeout: Duration,

/// Maximum backoff duration to reconnect to the database
#[serde(with = "humantime_serde", default = "default_backoff_max")]
pub backoff_max: Duration,
Expand Down
2 changes: 0 additions & 2 deletions crates/http-sink/src/sink.rs
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,6 @@ impl Sink<String> for HttpSink {
mod test {
use std::time::Duration;

use adaptive_backoff::prelude::{BackoffBuilder, ExponentialBackoffBuilder};

use super::*;

#[test]
Expand Down

0 comments on commit f238756

Please sign in to comment.