Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unique value or methods to check if item already exists #130

Open
VictorKozachek opened this issue Mar 24, 2020 · 2 comments
Open

Unique value or methods to check if item already exists #130

VictorKozachek opened this issue Mar 24, 2020 · 2 comments
Labels

Comments

@VictorKozachek
Copy link

Hi,

Is there any way to control unique value during PUT method or any method to check if item already exists?

Thanks

@imidoriya
Copy link
Collaborator

Here is something that I do as I start my application to clear any duplicates where I have an associated event and a job type. It could also be used to keep track of entries and check against it. I'm using a SQLiteAckQueue q.ack in this example to clear it, but you could just get(id=row.get("id")) to clear it in the other Queues. Note this uses the PR fork #153 which is being reviewed for merge.

from collections import defaultdict

q.clear_acked_data(max_delete=0, keep_latest=0, clear_ack_failed=True)
q.shrink_disk_usage()

duplicates = defaultdict(set)

rows = q.queue()
for row in rows:
    item = row.get("data")
    if item.get("tool_name") not in duplicates[item.get("event_id")]:
        duplicates[item.get("event_id")].add(item.get("tool_name"))
    else:
        print(
            "Removing duplicate job: "
            + str(item.get("event_id"))
            + " ("
            + item.get("tool_name")
            + ")"
        )
        q.ack(id=row.get("id"))

@imidoriya
Copy link
Collaborator

The UniqueQ or UniqueAckQ only allows for unique items. I should be using that myself. lol

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants