Skip to content

v0.4.11: adding s3 presigning tools

Compare
Choose a tag to compare
@trojblue trojblue released this 30 Sep 18:06
· 3 commits to main since this release

feat:

  • ub.presigns(s3_uri): presigning a s3 uri to create an accessible url. useful for working with s3 uris in transformers pipelines:
import unibox as ub
from transformers import pipeline

# More models in the model hub.
model_name = "openai/clip-vit-large-patch14"
classifier = pipeline("zero-shot-image-classification", model = model_name, device="cuda")

# s3 uri to url
image_to_classify = 's3://bucket-external/dataset/dataset_qft/qft_v5c_twitter-logfav_9.6_60k/100006176_p0.webp'
image_url = ub.presigns(image_to_classify)

# get results
labels = ["a girl", "a boy"]
scores = classifier(image_url, candidate_labels = labels)
scores
# [{'score': 0.9802619218826294, 'label': 'a girl'},
# {'score': 0.0197380892932415, 'label': 'a boy'}]

tweak:

  • removing unused methods in s3_client.py