You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
torchao's smoothquant recently broke after a change to PyTorch core: pytorch/pytorch#145733 . We should make the updates suggested by @anijain2305 in that issue to our code. I actually think we should go a bit farther and go with something like
## before#class_ActQuantizer:
def__init__(self, target_dtype, quant_min=-127):
self.target_dtype=target_dtypeself.quant_min=quant_mindefdynamic_quantize(self, input):
returnto_affine_quantized_intx(
input,
MappingType.SYMMETRIC,
_get_per_token_block_size(input),
self.target_dtype,
self.quant_min,
)
defstatic_quantize(self, input, scale, zero_point):
returnto_affine_quantized_intx_static(
input,
scale,
zero_point,
list(input.shape),
self.target_dtype,
self.quant_min,
)
## after#@dataclassclass_ActQuantConfig:
target_dtype: torch.dtypequant_min: int=-127# then, logic elsewhere chooses whether to call static or dynamic quant based on the contents of an instance of `_ActQuantConfig`
My feedback here is similar in spirit to #1595 - IMO it's simpler and safer to pass around dumb config objects and use them to choose which function to call, instead of encoding the "which function to call" information in the config as a callable object.
The text was updated successfully, but these errors were encountered:
torchao's smoothquant recently broke after a change to PyTorch core: pytorch/pytorch#145733 . We should make the updates suggested by @anijain2305 in that issue to our code. I actually think we should go a bit farther and go with something like
My feedback here is similar in spirit to #1595 - IMO it's simpler and safer to pass around dumb config objects and use them to choose which function to call, instead of encoding the "which function to call" information in the config as a callable object.
The text was updated successfully, but these errors were encountered: