Joint mechanisms #105
Joint mechanisms #105
Conversation
| @@ -484,3 +485,228 @@ def test_norm_clipping_only(self, order, clipping_bound, expected_arrays, | |||
| is_local_privacy=True) | |||
| assert get_overall_value( | |||
| metrics[name]) == pytest.approx(expected_clip_fraction) | |||
|
|
|||
| @pytest.mark.parametrize('ops_module', framework_fixtures) | |||
There was a problem hiding this comment.
Since the mechanisms themselves are tested in other unittests, you can simplify this a lot by just using MagicMock as mechanisms input to JointMechanism.
and the test will be much faster, as i recall these statistical tests are not super fast.
There was a problem hiding this comment.
I've rewritten the test as suggested
| statistics keys. As such JointMechanism can only be applied to client | ||
| statistics of type MappedVectorStatistics. | ||
|
|
||
| :param mechanisms_and_keys: |
There was a problem hiding this comment.
is it possible to add a feature to also allow specify keys by sub path. e.g. if i have an LLM (which produce many keys) and then some other stats algo/value1, algo/value2, ... etc i want to put through a mechanism, it would be great if you can achieve this by:
mechanisms_and_keys = {
'algo_mechanism': (m0, ['algo/']),
'model_mechanism': (m1, [])
}
algo_mechanism takes all keys which start with algo/ and model_mechanism take all the rest of the keys.
There was a problem hiding this comment.
Nice, thanks for the suggestion. I've added the feature that allows keys to be matched by subpathing.
I felt like it was safer not to implement the: [] matches all remaining keys, since this could make it easier to accidentally pass the wrong key to model_mechanism e.g. if I just forget to add a particular key to another mechanism. The way it is currently implemented this would be caught by the check that all keys have been noised. I think this particular model example should be easy to handle in the current implementation by providing list(model.get_parameters().keys()) to mechanisms_and_keys?
|
Thanks for the feedback :) |
Implementing a privacy mechanism that takes in multiple existing mechanisms and applies them to disjoint portions of user statistics. This can be used in the situation that you want to make a multiple queries to a user simultaneously and each query should be noised differently e.g. because the queries have differing sensitivities or different noise distributions should be used.