PackageHallucinationProbe
shouldn't show in --list_probes
#904
Labels
architecture
Architectural upgrades
cli
Command-line interface functions
probes
Content & activity of LLM probes
PackageHallucinationProbe
is intended as an abstract class and shouldn't show in--list_probes
, but does.One option is to refactor it to be a mixin, a pattern we use some places.
Another pattern we use, that I'm not a huge fan of, is a "lead" probe that other, similar probes inherit from. My reservations with this pattern are (1) that the hierarchy is not always clear, i.e. it's uncertain which probe is the "lead" one when it has a name like
PackageHallucinationRuby
, for example; (2) that it doesn't make sense to inherit from a lead probe with one purpose and then to remove that purpose and add another.Drawbacks with the mixin pattern - mixed inheritance isn't always so clean in python, and getting parent constructors to behave as intended can be non-trivial.
One alternative could be to use the
_
-prefix convention to denote plugin classes that shouldn't be shown in--list
or loaded withload_plugin()
, etc. E.g. in this instance we'd haveprobes.packagehallucination._PackageHallucinationProbe(Probe)
.The text was updated successfully, but these errors were encountered: