Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Order prompts within probes for max bag attack success rate #1077

Open
leondz opened this issue Jan 15, 2025 · 0 comments
Open

Order prompts within probes for max bag attack success rate #1077

leondz opened this issue Jan 15, 2025 · 0 comments
Labels
architecture Architectural upgrades probes Content & activity of LLM probes

Comments

@leondz
Copy link
Collaborator

leondz commented Jan 15, 2025

Summary

For probes that don't compose their prompts, we can get info on which prompts are more or less successful in the bag of models used for calibration. This can be used to order prompts within a probe.

Once we have this infrastructure, we can then apply it to probes that compose their prompts out of components, as long as the component choice is logged in the attempt (not in this feature).

Process

  • update the calibration process to record (probe, detector, prompt, asr) tuples
  • from this derive (probe, prompt, asr) tuples taking the mean (not median) over all detectors used
  • work out a way of storing this artefact (HF? compressed and local? use a small hash like crc32? cut out everything below a certain ASR?)
  • work out where to add this function (probes.base.Probe() ?)
  • implement the sort
  • add a config variable for whether to do this optimisation
@leondz leondz added architecture Architectural upgrades probes Content & activity of LLM probes labels Jan 15, 2025
@leondz leondz added this to the 25.02 Efficiency milestone Jan 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
architecture Architectural upgrades probes Content & activity of LLM probes
Projects
None yet
Development

No branches or pull requests

1 participant