New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Models got fail accuracy on BMG but passed on PVC #1305

Open

mengfei25 opened this issue Jan 20, 2025 · 1 comment

Assignees

Labels

Milestone

Contributor

mengfei25 commented Jan 20, 2025 •

edited

Loading

🐛 Describe the bug

python benchmarks/dynamo/timm_models.py --accuracy --float16 -d xpu -n10 --training --only botnet26t_256 --backend=inductor

Category	Model	Accuracy
timm_models_float16_training	botnet26t_256	fail_accuracy
torchbench_float32_inference	detectron2_maskrcnn_r_50_fpn	fail_accuracy
timm_models_amp_fp16_training	fbnetv3_b	fail_accuracy
timm_models_bfloat16_training	fbnetv3_b	fail_accuracy
timm_models_amp_bf16_training	gernet_l	fail_accuracy
torchbench_float16_training	hf_Longformer	fail_accuracy
torchbench_bfloat16_training	hf_Longformer	fail_accuracy

Versions

Environments:

Name	Version
Device	BMG
OS	Ubuntu 24.10
Driver	24.45.31740
Torch	v2.6.0-rc5
Torch-xpu-ops	release/2.6
Triton	3.2.0
Transformers	243e186efbf7fb93328dd6b34927a4e8c8f24395
Torchvision	0.21.0+xpu
Torchaudio	2.6.0+xpu
Torchbench	766a5e3a189384659fd35a68c3b17b88c761aaac
Timms	ac3470188b914c5d7a5058a7e28b9eb685a62427
Bundle	2025.0.4.20241205

The text was updated successfully, but these errors were encountered:

mengfei25 added E2E Accuracy labels

Contributor

daisyden commented Feb 20, 2025

@mengfei25 please help to update the latest status and what is accuracy gap for these tests.

daisyden added this to the PT2.7 milestone

daisyden assigned retonym

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment