Skip to content

Commit

Permalink
Merge pull request #308 from rod409/master
Browse files Browse the repository at this point in the history
add automotive rules to policy
  • Loading branch information
mrmhodak authored Jan 28, 2025
2 parents 8e94ff0 + 132a27a commit 3e9b53a
Showing 1 changed file with 3 additions and 0 deletions.
3 changes: 3 additions & 0 deletions inference_rules.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,7 @@ Each sample has the following definition:
|Mixtral-8x7B |one sequence
|RGAT |one node id
|Llama3.1-405B |one sequence
|PointPainting | five images and one lidar point cloud
|===

== Benchmarks
Expand Down Expand Up @@ -289,6 +290,7 @@ The Edge suite includes the following benchmarks:
|Language |Language processing |BERT |SQuAD v1.1 (max_seq_len=384) | 10833 | 99% of FP32 and 99.9% of FP32(f1_score=90.874%)
|Language |Summarization |GPT-J |CNN Dailymail (v3.0.0, max_seq_len=2048) | 13368 | 99% of FP32 and 99.9% of FP32 (rouge1=42.9865, rouge2=20.1235, rougeL=29.9881). Additionally, for both cases the generation length should be more than 90% of the reference (gen_len=4016878)
|Generative |Text to image |SDXL |Subset of coco-2014 val | 5000 |FID range: [23.01085758, 23.95007626] and CLIP range: [31.68631873, 31.81331801]
|Automotive | 3D Object Detection | PointPainting | Waymo Open Dataset | 1024 | 99.9% of FP32 (0.5425 mAP)
|===

Each Edge benchmark *requires* the following scenarios, and sometimes permit an optional scenario:
Expand All @@ -301,6 +303,7 @@ Each Edge benchmark *requires* the following scenarios, and sometimes permit an
|Language |Language processing |Single Stream, Offline
|Generative |Text to image |Single Stream, Offline
|Language |Summarization |Single Stream, Offline
|Automotive | 3D object detection | Single Stream with 99.9% tail latency
|===


Expand Down

0 comments on commit 3e9b53a

Please sign in to comment.