Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dma-perf can only get half of the bandwidth of a x8 PCIe Gen 5 card. Any suggestions? #273

Open
hjiang2018 opened this issue Apr 8, 2024 · 0 comments

Comments

@hjiang2018
Copy link

Running dma-perf with c2h/h2c/bi directions on a x8 PCIe Gen 5 card, but we can only get to half (~14GB/s) of limitation of the card's bandwidth (~32GB/s in theory). Below is a table of config file contents and test results:

image

We also noticed that with the following register writing sequence, the bandwidth of dma-perf may reach ~26GB/s, with offset_ch1=0x60000000000, but we don't understand the register settings:

    # register      noc 0 and 1     plaxi 0 and 1
    [ '0x2c8',      '0x0',          '0x0'           ],
    [ '0x308',      '0x0',          '0x0'           ],
    [ '0x824',      '0xffffffff',   '0xffffffff'    ],
    [ '0x828',      '0xffffffff',   '0xffffffff'    ],
    [ '0x82C',      '0xffffffff',   '0xffffffff'    ],
    [ '0x830',      '0xffffffff',   '0xffffffff'    ],
    [ '0x834',      '0xffffffff',   '0xffffffff'    ],
    [ '0x838',      '0xffffffff',   '0xffffffff'    ],
    [ '0x83C',      '0xffffffff',   '0xffffffff'    ],
    [ '0x840',      '0xffffffff',   '0xffffffff'    ],
    [ '0x804',      '0x0',          '0x0'           ],
    [ '0x808',      '0x0',          '0x0'           ],
    [ '0x80C',      '0x00000000',   '0x00000000'    ],
    [ '0x810',      '0x0',          '0x9'           ],
    [ '0x814',      '0x0',          '0x0'           ],
    [ '0x818',      '0x00000000',   '0x00900000'    ],
    [ '0x81C',      '0x0',          '0x0'           ],
    [ '0x820',      '0x0',          '0x0'           ],
    [ '0x844',      '0x34',         '0x34'          ],
    [ '0x2c8',      '0x0',          '0x0'           ],
    [ '0x308',      '0x0',          '0x0'           ],
    [ '0x804',      '0x0',          '0x0'           ],
    [ '0x808',      '0x0',          '0x0'           ],
    [ '0x80C',      '0x40000000',   '0x40000000'    ],
    [ '0x810',      '0x0',          '0x9'           ],
    [ '0x814',      '0x0',          '0x0'           ],
    [ '0x818',      '0x00040000',   '0x00940000'    ],
    [ '0x81C',      '0x0',          '0x0'           ],
    [ '0x820',      '0x0',          '0x0'           ],
    [ '0x844',      '0xb4',         '0xb4'          ]]

name=mm_01
pci_bus=01
pci_device=00
mode=mm
dir=c2h
pf_range=0:0
q_range=0:31
wb_acc=5
tmw_idx=9
cntr_idx=7
trig_mode=cntr_tmr
rngidx=0xF
ram_width=15 #31bits - 2^31 = 2GB
runtime=30 #sec
num_threads=2
bidir_en=1
num_pkt=1
pkt_sz=16777216
mm_chnl=2
offset=0x00000000000
offset_ch1=0x60000000000
dump_en=0
marker_en=0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant