Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[-] Cleanup README with better clarity and more info #2

Merged
merged 1 commit into from
Oct 5, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 24 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,21 +3,37 @@
[![](https://github.com/naughtygopher/pocache/actions/workflows/go.yml/badge.svg?branch=main)](https://github.com/naughtygopher/pocache/actions)
[![](https://godoc.org/github.com/nathany/looper?status.svg)](http://godoc.org/github.com/naughtygopher/pocache)

# **P**remeptive **O**ptimistic Cache
# Pocache

Pocache is a minimal in-app cache package which uses `github.com/hashicorp/golang-lru/v2` for the underlying [LRU](https://en.wikipedia.org/wiki/Cache_replacement_policies#LRU) and storage.
Pocache is a lightweight in-app caching solution. It introduces preemptive cache updates, optimizing performance in concurrent environments by reducing redundant database calls while maintaining fresh data. It uses [Hashicorp's Go LRU package](https://github.com/hashicorp/golang-lru/tree/v2) as the default storage.

It implements **_preemptive_** updates of the cache. The trigger for the preemptive update is based on the threshold window which is provided as a configuration. Threshold window is the duration in which the cache is about to expire at the time of fetching the value. A debounced update is initiated when a key is fetched within the configured threshold window.
## Key Features

1. **Preemptive Cache Updates:** Automatically updates cache entries nearing expiration.
2. **Threshold Window:** Configurable time window before cache expiration to trigger updates.
3. **Debounced Updates:** Prevents excessive I/O calls by debouncing concurrent requests for the same key.

## How does it work?

Given a cache expiration time and a threshold window, Pocache triggers a preemptive cache update when a value is accessed within the threshold window.
Example:

- Cache expiration: 10 minutes
- Threshold window: 1 minute

```
|_____________________ ____threshold window____ ________|
0min 9mins 10mins
add key here get key key expires
|______________________ __threshold window__________ ______________|
0 min 9 mins 10 mins
Add key here Get key within window Key expires
```

Consider a cache expiry of 10 minutes, and a threshold window of 1 minute. In the timeline above, we add a new key to the cache at `0min`, and `10mins` is when the key is set to expire. If you try to **Get** a key between `9mins` & `10mins`, then it is within the _threshold_ window. At this point, it would initiate a cache update _preemptively_, _optimistic_ about its near future usage again. Thereby maintaining the freshness of the data in the cache.
When a key is fetched between 9-10 minutes (within the threshold window), Pocache initiates an update for that key. This ensures fresh data availability, anticipating future usage (_optimistic_).

## Why Use Preemptive Updates?

In highly concurrent environments (e.g., web servers), multiple requests might try to access the same cache entry simultaneously. Without preemptive updates, the system would query the underlying database multiple times until the cache is refreshed.

In a highly concurrent environment, preemptive and controlled updates help significantly reduce the number of I/O calls to the respective database, without compromising the freshness of cache. If not for a preemptive update, the application would have called the underlying database N times where N is the number of concurrent requests (e.g. in a web application), until the cache is updated locally. This would also increase the pressure on the application to handle the concurrent reads and writes for the same key within the app cache.
Additionally by debouncing these requests, Pocache ensures only a single update is triggered, reducing load on both the underlying storage and the application.

## The gopher

Expand Down
Loading