VI Benchmarking

VI Benchmarking

Some notes on benchmarking a couple of VIs, back on an older commit circa 08/13.

Spreadsheet of results: https://docs.google.com/spreadsheet/ccc?key=0Ajz-75u_7nEydEdkNVlXZEVBcDRNaWdSM3ZqMDhDZHc#gid=0

Detailed stats:

• The total throughput from CAN is ~31KB/s (kilobytes) - our current firmware and hardware is missing about .9% of the messages at the moment. I hadn't noticed this before so it could be some non-optimal part of our code (something we can fix) or that we are actually hitting the technical limits of the hardware. I'd guess it's just inefficient code, because we haven't spent any time or effort trying to optimize it for throughput.
• As predicted, encoding JSON strings for the output interface does add some overhead and causes a 0.2% increase in the dropped message rate. Splitting out each CAN message into multiple signals will of course increase this overhead.
• Sending data over USB also adds overhead and increases the dropped message rate by ~18% and decreases the total throughput by ~12%.
• Even with no extra encoding, sending plain binary CAN data out would overwhelm the Bluetooth interface (31KB/s of data into a 23KB/s Bluetooth pipe).

These are the detailed statistics after running in a C-Max with the high speed and medium speed buses enabled for a few minutes:

Full passthrough with only INFO from c-max

CAN messages received since startup on bus 2: 44763
CAN messages dropped since startup on bus 2: 386
Data received on bus 2 since startup: 409.000000 KB
Aggregate throughput on bus 2 since startup: 4.543788 KB / s
Total CAN messages dropped since startup on all buses: 386
Aggregate message drop rate across all buses since startup: 4.287745 msgs / s
Dropped message ratio on all buses since startup: 0.008549
Total CAN messages received since startup on all buses: 44763
Aggregate message rate across all buses since startup: 497.140192 msgs / s
Aggregate throughput across all buses since startup: 4.542022 KB / s
USB messages sent: 44732
USB messages dropped: 31
USB message drop ratio: 0.000693
Aggregate USB sent message rate since startup: 496.685580 msgs / s
Average USB data rate since startup: 18.508444 KB / s
UART messages sent: 0
UART messages dropped: 0
UART message drop ratio: 0.000000
Aggregate UART sent message rate since startup: 0.000000 msgs / s
Average UART data rate since startup: 0.000000 KB / s
Network messages sent: 0
Network messages dropped: 0
Network message drop ratio: 0.000000
Aggregate Network sent message rate since startup: 0.000000 msgs / s
Average Network data rate since startup: 0.000000 KB / s

Cmax normal build, with the usual filters

CAN messages received since startup on bus 1: 61120
CAN messages dropped since startup on bus 1: 736
Data received on bus 1 since startup: 559.000000 KB
Aggregate throughput on bus 1 since startup: 4.360374 KB / s
CAN messages received since startup on bus 2: 4395
CAN messages dropped since startup on bus 2: 0
Data received on bus 2 since startup: 40.000000 KB
Aggregate throughput on bus 2 since startup: 0.311966 KB / s
Total CAN messages dropped since startup on all buses: 736
Aggregate message drop rate across all buses since startup: 5.739731 msgs / s
Dropped message ratio on all buses since startup: 0.011109
Total CAN messages received since startup on all buses: 65515
Aggregate message rate across all buses since startup: 510.850156 msgs / s
Aggregate throughput across all buses since startup: 4.670456 KB / s
USB messages sent: 6458
USB messages dropped: 65
USB message drop ratio: 0.009965
Aggregate USB sent message rate since startup: 50.348102 msgs / s
Average USB data rate since startup: 2.120477 KB / s
UART messages sent: 0
UART messages dropped: 0
UART message drop ratio: 0.000000
Aggregate UART sent message rate since startup: 0.000000 msgs / s
Average UART data rate since startup: 0.000000 KB / s
Network messages sent: 0
Network messages dropped: 0
Network message drop ratio: 0.000000
Aggregate Network sent message rate since startup: 0.000000 msgs / s
Average Network data rate since startup: 0.000000 KB / s

chipKIT VI

Passthrough with handler disabled

CAN messages received since startup on bus 1: 187487
CAN messages dropped since startup on bus 1: 2061
Data received on bus 1 since startup: 1716.000000 KB
Aggregate throughput on bus 1 since startup: 24.184001 KB / s
CAN messages received since startup on bus 2: 44561
CAN messages dropped since startup on bus 2: 142
Data received on bus 2 since startup: 407.000000 KB
Aggregate throughput on bus 2 since startup: 5.734414 KB / s
Total CAN messages dropped since startup on all buses: 2222
Aggregate message drop rate across all buses since startup: 1002828 msgs / s
Total CAN messages received since startup on all buses: 232048
Aggregate message rate across all buses since startup: 624831 msgs / s
Aggregate throughput across all buses since startup: 29.899724 KB / s

Full passthrough over USB

CAN messages received since startup on bus 1: 22256
CAN messages dropped since startup on bus 1: 59533
Data received on bus 1 since startup: 203.000000 KB
Aggregate throughput on bus 1 since startup: 6.671925 KB / s
CAN messages received since startup on bus 2: 18972
CAN messages dropped since startup on bus 2: 310
Data received on bus 2 since startup: 173.000000 KB
Aggregate throughput on bus 2 since startup: 5.682378 KB / s
Total CAN messages dropped since startup on all buses: 59887
Aggregate message drop rate across all buses since startup: 1966.344891 msgs / s
Total CAN messages received since startup on all buses: 41228
Aggregate message rate across all buses since startup: 1353.157411 msgs / s
Aggregate throughput across all buses since startup: 12.337982 KB / s