Skip to content

Commit

Permalink
Update example
Browse files Browse the repository at this point in the history
  • Loading branch information
turboderp committed Dec 1, 2024
1 parent 48e6306 commit fa7e89c
Showing 1 changed file with 5 additions and 5 deletions.
10 changes: 5 additions & 5 deletions examples/multimodal_grounding_qwen.py
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ def inference(self, settext_fn, update_fn):

job = ExLlamaV2DynamicJob(
input_ids = input_ids,
max_new_tokens = 500,
max_new_tokens = 1000,
decode_special_tokens = True,
stop_conditions = [self.tokenizer.eos_token_id],
gen_settings = ExLlamaV2Sampler.Settings.greedy(),
Expand Down Expand Up @@ -162,18 +162,18 @@ def get_grounding_bb(self, start, end) -> tuple:
enclosed in the special tokens that Qwen would emit when prompted for grounding. Qwen is then strongly biased
towards completing the bounding box.
Since we're using the same description as the model original generated, all keys/values for the system prompt,
image and generated description up to the selection will be reused from the cache.
Since we're using the same description as the model originally generated, all keys/values for the system
prompt, image and generated description up to the selection will be reused from the cache.
"""

if start >= end:
return
return None, None

# Including leading space
if start > 0 and self.current_description[start - 1] == " ":
start -= 1

# Repeat the same
# Repeat the same prompt up to the selection, with grounding tokens added
prompt = self.get_prompt()
prompt += self.current_description[:start]
prompt += "<|object_ref_start|>"
Expand Down

0 comments on commit fa7e89c

Please sign in to comment.