perf: shallow copy instead of deepcopy #197

ralwing · 2024-10-30T11:58:58Z

What

Performance optimization, along with the changes in driving_log_replayer allows realtime processing (5x speedup) of frames processing.

Type of change

Refactoring

Test performed

Running the driving log replayer and...

test.sensing_lsim


[2024-10-30 11:58:31,043] [INFO] [dataset.py:96 load_all_datasets] Start to load dataset ['test/sample_data/']
[2024-10-30 11:58:31,043] [INFO] [dataset.py:105 load_all_datasets] config: load_raw_data: True, evaluation_task: sensing, frame_id: ['base_link']
  0%|                                                                                                                                       | 0/1 [00:00 bicycle): 0 
vehicle.bicycle (-> bicycle): 0 
car (-> car): 0 
vehicle.car (-> car): 0 
vehicle.construction (-> car): 0 
vehicle.emergency (ambulance & police) (-> car): 0 
vehicle.police (-> car): 0 
vehicle.fire (-> car): 0 
vehicle.ambulance (-> car): 0 
pedestrian (-> pedestrian): 0 
stroller (-> pedestrian): 0 
pedestrian.adult (-> pedestrian): 0 
pedestrian.child (-> pedestrian): 0 
pedestrian.construction_worker (-> pedestrian): 0 
pedestrian.personal_mobility (-> pedestrian): 0 
pedestrian.police_officer (-> pedestrian): 0 
pedestrian.stroller (-> pedestrian): 0 
pedestrian.wheelchair (-> pedestrian): 0 
construction_worker (-> pedestrian): 0 
animal (-> unknown): 0 
unknown (-> unknown): 0 
movable_object.barrier (-> unknown): 0 
movable_object.debris (-> unknown): 0 
movable_object.pushable_pullable (-> unknown): 0 
movable_object.trafficcone (-> unknown): 0 
movable_object.traffic_cone (-> unknown): 0 
static_object.bicycle rack (-> unknown): 0 
static_object.bollard (-> unknown): 0 
forklift (-> unknown): 0 
false_positive (-> false_positive): 0 
bus (-> bus): 0 
vehicle.bus (bendy & rigid) (-> bus): 0 
vehicle.bus (-> bus): 0 
truck (-> truck): 0 
vehicle.truck (-> truck): 0 
trailer (-> truck): 0 
vehicle.trailer (-> truck): 0 
motorbike (-> motorbike): 0 
motorcycle (-> motorbike): 0 
vehicle.motorcycle (-> motorbike): 0 
[2024-10-30 11:58:31,124] [INFO] [sensing_lsim.py:141 visualize] all detections were succeeded.

[2024-10-30 11:58:31,124] [INFO] [sensing_lsim.py:144 visualize] [SUCCESS] Inside points: 1, Is detected: True, Nearest point: [1.8647348e+01 5.9553642e+01 5.0377835e-02]
[2024-10-30 11:58:31,124] [WARNING] [sensing_lsim.py:118 get_final_result] 0 fail results.

[2024-10-30 11:58:31,124] [INFO] [sensing_lsim.py:209 ] Frame result example (frame_results[0]: 1 success, 0 fail

[2024-10-30 11:58:31,124] [INFO] [sensing_lsim.py:215 ] Failed to be Non-detected pointcloud example (frame_results[0]): 0

[2024-10-30 11:58:31,124] [INFO] [sensing_lsim.py:221 ] Start visualizing sensing results

Visualize results for each frame: 100%|█████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 22.47it/s

test.perception_lsim


==================================================Start Detection==================================================
[2024-10-30 11:57:07,600] [INFO] [dataset.py:96 load_all_datasets] Start to load dataset ['test/sample_data/']
[2024-10-30 11:57:07,600] [INFO] [dataset.py:105 load_all_datasets] config: load_raw_data: True, evaluation_task: detection, frame_id: ['base_link']
  0%|                                                                                                                                       | 0/1 [00:00 bicycle): 1 
vehicle.bicycle (-> bicycle): 0 
car (-> car): 1 
vehicle.car (-> car): 0 
vehicle.construction (-> car): 0 
vehicle.emergency (ambulance & police) (-> car): 0 
vehicle.police (-> car): 0 
vehicle.fire (-> car): 0 
vehicle.ambulance (-> car): 0 
pedestrian (-> pedestrian): 1 
stroller (-> pedestrian): 0 
pedestrian.adult (-> pedestrian): 0 
pedestrian.child (-> pedestrian): 0 
pedestrian.construction_worker (-> pedestrian): 0 
pedestrian.personal_mobility (-> pedestrian): 0 
pedestrian.police_officer (-> pedestrian): 0 
pedestrian.stroller (-> pedestrian): 0 
pedestrian.wheelchair (-> pedestrian): 0 
construction_worker (-> pedestrian): 0 
animal (-> unknown): 0 
unknown (-> unknown): 0 
movable_object.barrier (-> unknown): 0 
movable_object.debris (-> unknown): 0 
movable_object.pushable_pullable (-> unknown): 0 
movable_object.trafficcone (-> unknown): 0 
movable_object.traffic_cone (-> unknown): 0 
static_object.bicycle rack (-> unknown): 0 
static_object.bollard (-> unknown): 0 
forklift (-> unknown): 0 
false_positive (-> false_positive): 0 
bus (-> bus): 0 
vehicle.bus (bendy & rigid) (-> bus): 0 
vehicle.bus (-> bus): 0 
truck (-> truck): 0 
vehicle.truck (-> truck): 0 
trailer (-> truck): 0 
vehicle.trailer (-> truck): 0 
motorbike (-> motorbike): 1 
motorcycle (-> motorbike): 0 
vehicle.motorcycle (-> motorbike): 0 
[2024-10-30 11:57:07,717] [INFO] [perception_lsim.py:153 visualize] 0 TP objects, 0 FP objects, 0 FN objects

[2024-10-30 11:57:07,717] [INFO] [perception_lsim.py:142 get_final_result] Number of fails for critical objects: 0

[2024-10-30 11:57:07,718] [INFO] [perception_lsim.py:146 get_final_result] final metrics result
Frame:

Total Num: 1

Skipped Frames: []

Skipped Frames Count: 0
Ground Truth Num: 0
mAP: inf, mAPH: inf (Center Distance)



Label
car(1.0)
bicycle(1.0)
pedestrian(1.0)
motorbike(1.0)




Predict_num
0
0
0
0


AP
inf
inf
inf
inf


APH
inf
inf
inf
inf



mAP: inf, mAPH: inf (Center Distance)



Label
car(2.0)
bicycle(2.0)
pedestrian(2.0)
motorbike(2.0)




Predict_num
0
0
0
0


AP
inf
inf
inf
inf


APH
inf
inf
inf
inf



mAP: inf, mAPH: inf (IoU 2D)



Label
car(0.5)
bicycle(0.5)
pedestrian(0.5)
motorbike(0.5)




Predict_num
0
0
0
0


AP
inf
inf
inf
inf


APH
inf
inf
inf
inf



mAP: inf, mAPH: inf (IoU 3D)



Label
car(0.5)
bicycle(0.5)
pedestrian(0.5)
motorbike(0.5)




Predict_num
0
0
0
0


AP
inf
inf
inf
inf


APH
inf
inf
inf
inf



mAP: inf, mAPH: inf (Plane Distance)



Label
car(2.0)
bicycle(2.0)
pedestrian(2.0)
motorbike(2.0)




Predict_num
0
0
0
0


AP
inf
inf
inf
inf


APH
inf
inf
inf
inf



mAP: inf, mAPH: inf (Plane Distance)



Label
car(3.0)
bicycle(3.0)
pedestrian(3.0)
motorbike(3.0)




Predict_num
0
0
0
0


AP
inf
inf
inf
inf


APH
inf
inf
inf
inf



[2024-10-30 11:57:07,718] [INFO] [perception_lsim.py:210 ] Frame result example (frame_results[0]):

{'frame_ground_truth': {'frame_name': '0',

'objects': [],

'raw_data': {<FrameID.CAM_FRONT: 'cam_front'>: ' --- length of element 900 ---,',

<FrameID.LIDAR_TOP: 'lidar_top'>: ' --- length of element 34688 ---,'},

'transforms': ' --- length of element 25 ---,',

'unix_time': 1532402927647951},

'frame_name': '0',

'metrics_score': {'classification_config': None,

'classification_scores': [],

'detection_config': {'center_distance_thresholds': ' --- length of element 2 ---,',

'iou_2d_thresholds': [' --- length of element 4 ---,'],

'iou_3d_thresholds': [' --- length of element 4 ---,'],

'plane_distance_thresholds': ' --- length of element 2 ---,',

'target_labels': ' --- length of element 4 ---,'},

'evaluation_task': 'detection',

'maps': ' --- length of element 6 ---,',

'prediction_config': None,

'prediction_scores': [],

'tracking_config': None,

'tracking_scores': []},

'object_results': [],

'pass_fail_result': {'critical_object_filter_config': {'confidence_threshold_list': None,

'filtering_params': {'confidence_threshold_list': None,

'ignore_attributes': ['cycle_state.without_rider'],

'max_distance_list': None,

'max_x_position_list': ' --- length of '

'element 4 ---,',

'max_y_position_list': ' --- length of '

'element 4 ---,',

'min_distance_list': None,

'min_point_numbers': None,

'target_labels': ' --- length of element '

'4 ---,',

'target_uuids': None},

'ignore_attributes': ['cycle_state.without_rider'],

'max_distance_list': None,

'max_x_position_list': ' --- length of element 4 ---,',

'max_y_position_list': ' --- length of element 4 ---,',

'min_distance_list': None,

'min_point_numbers': None,

'target_labels': ' --- length of element 4 ---,',

'target_uuids': None},

'fn_objects': [],

'fp_object_results': [],

'frame_number': '0',

'frame_pass_fail_config': {'confidence_threshold_list': None,

'evaluation_task': 'detection',

'matching_threshold_list': ' --- length of element 4 ---,',

'target_labels': ' --- length of element 4 ---,'},

'tn_objects': [],

'tp_object_results': [],

'transforms': ' --- length of element 25 ---,',

'unix_time': 1532402927647951},

'target_labels': ' --- length of element 4 ---,',

'unix_time': 1532402927647951}
[2024-10-30 11:57:07,719] [INFO] [perception_lsim.py:222 ] Detection Metrics example (final_metric_score):

{'classification_config': None,

'classification_scores': [],

'detection_config': {'center_distance_thresholds': [[1.0, 1.0, 1.0, 1.0], [2.0, 2.0, 2.0, 2.0]],

'iou_2d_thresholds': [[0.5, 0.5, 0.5, 0.5]],

'iou_3d_thresholds': [[0.5, 0.5, 0.5, 0.5]],

'plane_distance_thresholds': [[2.0, 2.0, 2.0, 2.0], [3.0, 3.0, 3.0, 3.0]],

'target_labels': ['car', 'bicycle', 'pedestrian', 'motorbike']},

'evaluation_task': 'detection',

'maps': ' --- length of element 6 ---,',

'prediction_config': None,

'prediction_scores': [],

'tracking_config': None,

'tracking_scores': []}
[2024-10-30 11:57:07,719] [INFO] [perception_lsim.py:228 ] mAP result example (final_metric_score.maps[0].aps[0]):

{'aphs': [{'ap': inf,

'fp_list': [],

'matching_average': None,

'matching_mode': 'Center Distance',

'matching_standard_deviation': None,

'matching_threshold_list': [1.0],

'num_ground_truth': 0,

'objects_results_num': 0,

'target_labels': ['car'],

'tp_list': [],

'tp_metrics': {}},

{'ap': inf,

'fp_list': [],

'matching_average': None,

'matching_mode': 'Center Distance',

'matching_standard_deviation': None,

'matching_threshold_list': [1.0],

'num_ground_truth': 0,

'objects_results_num': 0,

'target_labels': ['bicycle'],

'tp_list': [],

'tp_metrics': {}},

{'ap': inf,

'fp_list': [],

'matching_average': None,

'matching_mode': 'Center Distance',

'matching_standard_deviation': None,

'matching_threshold_list': [1.0],

'num_ground_truth': 0,

'objects_results_num': 0,

'target_labels': ['pedestrian'],

'tp_list': [],

'tp_metrics': {}},

{'ap': inf,

'fp_list': [],

'matching_average': None,

'matching_mode': 'Center Distance',

'matching_standard_deviation': None,

'matching_threshold_list': [1.0],

'num_ground_truth': 0,

'objects_results_num': 0,

'target_labels': ['motorbike'],

'tp_list': [],

'tp_metrics': {}}],

'aps': [{'ap': inf,

'fp_list': [],

'matching_average': None,

'matching_mode': 'Center Distance',

'matching_standard_deviation': None,

'matching_threshold_list': [1.0],

'num_ground_truth': 0,

'objects_results_num': 0,

'target_labels': ['car'],

'tp_list': [],

'tp_metrics': {}},

{'ap': inf,

'fp_list': [],

'matching_average': None,

'matching_mode': 'Center Distance',

'matching_standard_deviation': None,

'matching_threshold_list': [1.0],

'num_ground_truth': 0,

'objects_results_num': 0,

'target_labels': ['bicycle'],

'tp_list': [],

'tp_metrics': {}},

{'ap': inf,

'fp_list': [],

'matching_average': None,

'matching_mode': 'Center Distance',

'matching_standard_deviation': None,

'matching_threshold_list': [1.0],

'num_ground_truth': 0,

'objects_results_num': 0,

'target_labels': ['pedestrian'],

'tp_list': [],

'tp_metrics': {}},

{'ap': inf,

'fp_list': [],

'matching_average': None,

'matching_mode': 'Center Distance',

'matching_standard_deviation': None,

'matching_threshold_list': [1.0],

'num_ground_truth': 0,

'objects_results_num': 0,

'target_labels': ['motorbike'],

'tp_list': [],

'tp_metrics': {}}],

'is_detection_2d': False,

'map': inf,

'maph': inf,

'matching_mode': 'Center Distance',

'matching_threshold_list': [1.0, 1.0, 1.0, 1.0],

'target_labels': ['car', 'bicycle', 'pedestrian', 'motorbike']}
[2024-10-30 11:57:07,719] [INFO] [perception_lsim.py:235 ] Start visualizing detection results

Visualize results for each frame: 100%|█████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 22.12it/s]

Updating DataFrame: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 737.65it/s]

[2024-10-30 11:57:07,785] [WARNING] [perception_analyzer3d.py:415 analyze] There is no DataFrame to be able to analyze.

==================================================Start Tracking==================================================

[2024-10-30 11:57:07,785] [INFO] [dataset.py:96 load_all_datasets] Start to load dataset ['test/sample_data/']

[2024-10-30 11:57:07,785] [INFO] [dataset.py:105 load_all_datasets] config: load_raw_data: True, evaluation_task: tracking, frame_id: ['map']

0%|                                                                                                                                       | 0/1 [00:00<?, ?it/s][2024-10-30 11:57:07,802] [WARNING] [label.py:256 convert_label] Label human.pedestrian.adult is not registered.

100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 63.44it/s]

[2024-10-30 11:57:07,803] [INFO] [dataset.py:120 load_all_datasets] Finish loading dataset

bicycle (-> bicycle): 1

vehicle.bicycle (-> bicycle): 0

car (-> car): 1

vehicle.car (-> car): 0

vehicle.construction (-> car): 0

vehicle.emergency (ambulance & police) (-> car): 0

vehicle.police (-> car): 0

vehicle.fire (-> car): 0

vehicle.ambulance (-> car): 0

pedestrian (-> pedestrian): 1

stroller (-> pedestrian): 0

pedestrian.adult (-> pedestrian): 0

pedestrian.child (-> pedestrian): 0

pedestrian.construction_worker (-> pedestrian): 0

pedestrian.personal_mobility (-> pedestrian): 0

pedestrian.police_officer (-> pedestrian): 0

pedestrian.stroller (-> pedestrian): 0

pedestrian.wheelchair (-> pedestrian): 0

construction_worker (-> pedestrian): 0

animal (-> unknown): 0

unknown (-> unknown): 0

movable_object.barrier (-> unknown): 0

movable_object.debris (-> unknown): 0

movable_object.pushable_pullable (-> unknown): 0

movable_object.trafficcone (-> unknown): 0

movable_object.traffic_cone (-> unknown): 0

static_object.bicycle rack (-> unknown): 0

static_object.bollard (-> unknown): 0

forklift (-> unknown): 0

false_positive (-> false_positive): 0

bus (-> bus): 0

vehicle.bus (bendy & rigid) (-> bus): 0

vehicle.bus (-> bus): 0

truck (-> truck): 0

vehicle.truck (-> truck): 0

trailer (-> truck): 0

vehicle.trailer (-> truck): 0

motorbike (-> motorbike): 1

motorcycle (-> motorbike): 0

vehicle.motorcycle (-> motorbike): 0
[2024-10-30 11:57:07,840] [INFO] [dataset.py:340 get_interpolated_now_frame] Only before frame is available for interpolation

[2024-10-30 11:57:07,842] [INFO] [perception_lsim.py:153 visualize] 0 TP objects, 0 FP objects, 0 FN objects

[2024-10-30 11:57:07,842] [INFO] [perception_lsim.py:142 get_final_result] Number of fails for critical objects: 0

[2024-10-30 11:57:07,843] [INFO] [perception_lsim.py:146 get_final_result] final metrics result
Frame:

Total Num: 1

Skipped Frames: []

Skipped Frames Count: 0
Ground Truth Num: 0
mAP: inf, mAPH: inf (Center Distance)



Label
car(1.0)
bicycle(1.0)
pedestrian(1.0)
motorbike(1.0)




Predict_num
0
0
0
0


AP
inf
inf
inf
inf


APH
inf
inf
inf
inf



mAP: inf, mAPH: inf (Center Distance)



Label
car(2.0)
bicycle(2.0)
pedestrian(2.0)
motorbike(2.0)




Predict_num
0
0
0
0


AP
inf
inf
inf
inf


APH
inf
inf
inf
inf



mAP: inf, mAPH: inf (IoU 2D)



Label
car(0.5)
bicycle(0.5)
pedestrian(0.5)
motorbike(0.5)




Predict_num
0
0
0
0


AP
inf
inf
inf
inf


APH
inf
inf
inf
inf



mAP: inf, mAPH: inf (IoU 3D)



Label
car(0.5)
bicycle(0.5)
pedestrian(0.5)
motorbike(0.5)




Predict_num
0
0
0
0


AP
inf
inf
inf
inf


APH
inf
inf
inf
inf



mAP: inf, mAPH: inf (Plane Distance)



Label
car(2.0)
bicycle(2.0)
pedestrian(2.0)
motorbike(2.0)




Predict_num
0
0
0
0


AP
inf
inf
inf
inf


APH
inf
inf
inf
inf



mAP: inf, mAPH: inf (Plane Distance)



Label
car(3.0)
bicycle(3.0)
pedestrian(3.0)
motorbike(3.0)




Predict_num
0
0
0
0


AP
inf
inf
inf
inf


APH
inf
inf
inf
inf



[TOTAL] MOTA: inf, MOTP: inf, ID switch: 0 (Center Distance)



Label
car(1.0)
bicycle(1.0)
pedestrian(1.0)
motorbike(1.0)




predict_num
0
0
0
0


MOTA
inf
inf
inf
inf


MOTP
inf
inf
inf
inf


id_switch
0
0
0
0


tp
0.000
0.000
0.000
0.000


fp
0.000
0.000
0.000
0.000


tp_matching_score
0.000
0.000
0.000
0.000



[TOTAL] MOTA: inf, MOTP: inf, ID switch: 0 (Center Distance)



Label
car(2.0)
bicycle(2.0)
pedestrian(2.0)
motorbike(2.0)




predict_num
0
0
0
0


MOTA
inf
inf
inf
inf


MOTP
inf
inf
inf
inf


id_switch
0
0
0
0


tp
0.000
0.000
0.000
0.000


fp
0.000
0.000
0.000
0.000


tp_matching_score
0.000
0.000
0.000
0.000



[TOTAL] MOTA: inf, MOTP: inf, ID switch: 0 (IoU 2D)



Label
car(0.5)
bicycle(0.5)
pedestrian(0.5)
motorbike(0.5)




predict_num
0
0
0
0


MOTA
inf
inf
inf
inf


MOTP
inf
inf
inf
inf


id_switch
0
0
0
0


tp
0.000
0.000
0.000
0.000


fp
0.000
0.000
0.000
0.000


tp_matching_score
0.000
0.000
0.000
0.000



[TOTAL] MOTA: inf, MOTP: inf, ID switch: 0 (IoU 3D)



Label
car(0.5)
bicycle(0.5)
pedestrian(0.5)
motorbike(0.5)




predict_num
0
0
0
0


MOTA
inf
inf
inf
inf


MOTP
inf
inf
inf
inf


id_switch
0
0
0
0


tp
0.000
0.000
0.000
0.000


fp
0.000
0.000
0.000
0.000


tp_matching_score
0.000
0.000
0.000
0.000



[TOTAL] MOTA: inf, MOTP: inf, ID switch: 0 (Plane Distance)



Label
car(2.0)
bicycle(2.0)
pedestrian(2.0)
motorbike(2.0)




predict_num
0
0
0
0


MOTA
inf
inf
inf
inf


MOTP
inf
inf
inf
inf


id_switch
0
0
0
0


tp
0.000
0.000
0.000
0.000


fp
0.000
0.000
0.000
0.000


tp_matching_score
0.000
0.000
0.000
0.000



[TOTAL] MOTA: inf, MOTP: inf, ID switch: 0 (Plane Distance)



Label
car(3.0)
bicycle(3.0)
pedestrian(3.0)
motorbike(3.0)




predict_num
0
0
0
0


MOTA
inf
inf
inf
inf


MOTP
inf
inf
inf
inf


id_switch
0
0
0
0


tp
0.000
0.000
0.000
0.000


fp
0.000
0.000
0.000
0.000


tp_matching_score
0.000
0.000
0.000
0.000



[2024-10-30 11:57:07,844] [INFO] [perception_lsim.py:285 ] Frame result example (frame_results[0]):

{'frame_ground_truth': {'frame_name': '0',

'objects': [],

'raw_data': {<FrameID.CAM_FRONT: 'cam_front'>: ' --- length of element 900 ---,',

<FrameID.LIDAR_TOP: 'lidar_top'>: ' --- length of element 34688 ---,'},

'transforms': ' --- length of element 25 ---,',

'unix_time': 1532402927647951},

'frame_name': '0',

'metrics_score': {'classification_config': None,

'classification_scores': [],

'detection_config': {'center_distance_thresholds': ' --- length of element 2 ---,',

'iou_2d_thresholds': [' --- length of element 4 ---,'],

'iou_3d_thresholds': [' --- length of element 4 ---,'],

'plane_distance_thresholds': ' --- length of element 2 ---,',

'target_labels': ' --- length of element 4 ---,'},

'evaluation_task': 'tracking',

'maps': ' --- length of element 6 ---,',

'prediction_config': None,

'prediction_scores': [],

'tracking_config': {'center_distance_thresholds': ' --- length of element 2 ---,',

'iou_2d_thresholds': [' --- length of element 4 ---,'],

'iou_3d_thresholds': [' --- length of element 4 ---,'],

'plane_distance_thresholds': ' --- length of element 2 ---,',

'target_labels': ' --- length of element 4 ---,'},

'tracking_scores': ' --- length of element 6 ---,'},

'object_results': [],

'pass_fail_result': {'critical_object_filter_config': {'confidence_threshold_list': None,

'filtering_params': {'confidence_threshold_list': None,

'ignore_attributes': ['cycle_state.without_rider'],

'max_distance_list': None,

'max_x_position_list': ' --- length of '

'element 4 ---,',

'max_y_position_list': ' --- length of '

'element 4 ---,',

'min_distance_list': None,

'min_point_numbers': None,

'target_labels': ' --- length of element '

'4 ---,',

'target_uuids': None},

'ignore_attributes': ['cycle_state.without_rider'],

'max_distance_list': None,

'max_x_position_list': ' --- length of element 4 ---,',

'max_y_position_list': ' --- length of element 4 ---,',

'min_distance_list': None,

'min_point_numbers': None,

'target_labels': ' --- length of element 4 ---,',

'target_uuids': None},

'fn_objects': [],

'fp_object_results': [],

'frame_number': '0',

'frame_pass_fail_config': {'confidence_threshold_list': None,

'evaluation_task': 'tracking',

'matching_threshold_list': ' --- length of element 4 ---,',

'target_labels': ' --- length of element 4 ---,'},

'tn_objects': [],

'tp_object_results': [],

'transforms': ' --- length of element 25 ---,',

'unix_time': 1532402927647951},

'target_labels': ' --- length of element 4 ---,',

'unix_time': 1532402927647951}
[2024-10-30 11:57:07,844] [INFO] [perception_lsim.py:297 ] Tracking Metrics example (tracking_final_metric_score):

{'classification_config': None,

'classification_scores': [],

'detection_config': {'center_distance_thresholds': [[1.0, 1.0, 1.0, 1.0], [2.0, 2.0, 2.0, 2.0]],

'iou_2d_thresholds': [[0.5, 0.5, 0.5, 0.5]],

'iou_3d_thresholds': [[0.5, 0.5, 0.5, 0.5]],

'plane_distance_thresholds': [[2.0, 2.0, 2.0, 2.0], [3.0, 3.0, 3.0, 3.0]],

'target_labels': ['car', 'bicycle', 'pedestrian', 'motorbike']},

'evaluation_task': 'tracking',

'maps': ' --- length of element 6 ---,',

'prediction_config': None,

'prediction_scores': [],

'tracking_config': {'center_distance_thresholds': [[1.0, 1.0, 1.0, 1.0], [2.0, 2.0, 2.0, 2.0]],

'iou_2d_thresholds': [[0.5, 0.5, 0.5, 0.5]],

'iou_3d_thresholds': [[0.5, 0.5, 0.5, 0.5]],

'plane_distance_thresholds': [[2.0, 2.0, 2.0, 2.0], [3.0, 3.0, 3.0, 3.0]],

'target_labels': ['car', 'bicycle', 'pedestrian', 'motorbike']},

'tracking_scores': ' --- length of element 6 ---,'}
[2024-10-30 11:57:07,845] [INFO] [perception_lsim.py:303 ] mAP result example (tracking_final_metric_score.maps[0].aps[0]):

{'aphs': [{'ap': inf,

'fp_list': [],

'matching_average': None,

'matching_mode': 'Center Distance',

'matching_standard_deviation': None,

'matching_threshold_list': [1.0],

'num_ground_truth': 0,

'objects_results_num': 0,

'target_labels': ['car'],

'tp_list': [],

'tp_metrics': {}},

{'ap': inf,

'fp_list': [],

'matching_average': None,

'matching_mode': 'Center Distance',

'matching_standard_deviation': None,

'matching_threshold_list': [1.0],

'num_ground_truth': 0,

'objects_results_num': 0,

'target_labels': ['bicycle'],

'tp_list': [],

'tp_metrics': {}},

{'ap': inf,

'fp_list': [],

'matching_average': None,

'matching_mode': 'Center Distance',

'matching_standard_deviation': None,

'matching_threshold_list': [1.0],

'num_ground_truth': 0,

'objects_results_num': 0,

'target_labels': ['pedestrian'],

'tp_list': [],

'tp_metrics': {}},

{'ap': inf,

'fp_list': [],

'matching_average': None,

'matching_mode': 'Center Distance',

'matching_standard_deviation': None,

'matching_threshold_list': [1.0],

'num_ground_truth': 0,

'objects_results_num': 0,

'target_labels': ['motorbike'],

'tp_list': [],

'tp_metrics': {}}],

'aps': [{'ap': inf,

'fp_list': [],

'matching_average': None,

'matching_mode': 'Center Distance',

'matching_standard_deviation': None,

'matching_threshold_list': [1.0],

'num_ground_truth': 0,

'objects_results_num': 0,

'target_labels': ['car'],

'tp_list': [],

'tp_metrics': {}},

{'ap': inf,

'fp_list': [],

'matching_average': None,

'matching_mode': 'Center Distance',

'matching_standard_deviation': None,

'matching_threshold_list': [1.0],

'num_ground_truth': 0,

'objects_results_num': 0,

'target_labels': ['bicycle'],

'tp_list': [],

'tp_metrics': {}},

{'ap': inf,

'fp_list': [],

'matching_average': None,

'matching_mode': 'Center Distance',

'matching_standard_deviation': None,

'matching_threshold_list': [1.0],

'num_ground_truth': 0,

'objects_results_num': 0,

'target_labels': ['pedestrian'],

'tp_list': [],

'tp_metrics': {}},

{'ap': inf,

'fp_list': [],

'matching_average': None,

'matching_mode': 'Center Distance',

'matching_standard_deviation': None,

'matching_threshold_list': [1.0],

'num_ground_truth': 0,

'objects_results_num': 0,

'target_labels': ['motorbike'],

'tp_list': [],

'tp_metrics': {}}],

'is_detection_2d': False,

'map': inf,

'maph': inf,

'matching_mode': 'Center Distance',

'matching_threshold_list': [1.0, 1.0, 1.0, 1.0],

'target_labels': ['car', 'bicycle', 'pedestrian', 'motorbike']}
[2024-10-30 11:57:07,846] [INFO] [perception_lsim.py:309 ] CLEAR result example (tracking_final_metric_score.tracking_scores[0].clears[0]):

{'clears': [{'fp': 0.0,

'id_switch': 0,

'mota': inf,

'motp': inf,

'objects_results_num': 0,

'tp': 0.0,

'tp_matching_score': 0.0},

{'fp': 0.0,

'id_switch': 0,

'mota': inf,

'motp': inf,

'objects_results_num': 0,

'tp': 0.0,

'tp_matching_score': 0.0},

{'fp': 0.0,

'id_switch': 0,

'mota': inf,

'motp': inf,

'objects_results_num': 0,

'tp': 0.0,

'tp_matching_score': 0.0},

{'fp': 0.0,

'id_switch': 0,

'mota': inf,

'motp': inf,

'objects_results_num': 0,

'tp': 0.0,

'tp_matching_score': 0.0}],

'matching_mode': 'Center Distance',

'target_labels': ['car', 'bicycle', 'pedestrian', 'motorbike']}
[2024-10-30 11:57:07,846] [INFO] [perception_lsim.py:316 ] Start visualizing tracking results

Visualize results for each frame: 100%|█████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 23.21it/s]

Updating DataFrame: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 751.80it/s]

[2024-10-30 11:57:07,906] [WARNING] [perception_analyzer3d.py:415 analyze] There is no DataFrame to be able to analyze.

Reference

Notes for reviewer

YoshiRi · 2024-11-06T05:19:30Z

perception_eval/perception_eval/tool/utils.py

@@ -475,7 +475,7 @@ def filter_frame_by_distance(
    Returns:
        PerceptionFrameResult: Filtered frame results.
    """
-    ret_frame = deepcopy(frame)
+    ret_frame = copy(frame)


@ktro2828 Do you remember why you used deepcopy here?

I've used deepcopy in order to guarantee copying the all objects contained in PerceptionFrameResult.
But I think it is OK to use copy instead if it is OK.

ralwing · 2024-11-08T18:52:02Z

I've just noticed, that there is a bug: for more than one criterion, it only returns the last one.

ralwing · 2024-11-20T08:01:43Z

@YoshiRi
Ok, I've double tested this solution, and seems to be ready for a review.

ktro2828

LGTM

ralwing · 2024-11-22T12:11:57Z

There is much bigger problem with this code, than i previously though.
The estimated_object and ground_truth_object are copied multiple times in the code, in fact the class DynamicObjectWithPerceptionResult has a few copies of it under the hood. So when there is a call to deepcopy(perceptionframeresult) these estimated_object ground_truth_object are unnecessarily copied multiple times. e.g. every MatchingMethod implementation contains a copy of them.

ralwing · 2024-11-25T11:47:25Z

I hope this time I have considered all cases, however I have a request that T4 checks this PR on your data. This solution is not 2x speedup, but still it significantly shortens the processing time.
Thanks in advance.

ralwing · 2024-11-25T11:50:03Z

perception_eval/perception_eval/common/deepcopy.py

@@ -0,0 +1,38 @@
+from copy import deepcopy


I don't know what's the copyright procedure here? I could be either stackoverlow, Autonomous Systems or TIER4, or all together.

shallow copy instead of deepcopy

c7d6a50

ralwing changed the title ~~shallow copy instead of deepcopy~~ perf: shallow copy instead of deepcopy Oct 30, 2024

ralwing mentioned this pull request Oct 30, 2024

perf: 2x processing speedup tier4/driving_log_replayer#581

Merged

3 tasks

YoshiRi reviewed Nov 6, 2024

View reviewed changes

fixes for isort

c7cbc48

ralwing marked this pull request as draft November 8, 2024 18:50

ralwing added 2 commits November 20, 2024 07:52

shallow copy of required fields.

e0c5c4b

unused import

968d12f

ralwing marked this pull request as ready for review November 20, 2024 08:00

ktro2828 approved these changes Nov 21, 2024

View reviewed changes

ralwing marked this pull request as draft November 21, 2024 11:37

deepcopy with sharing

ce04d9f

ralwing marked this pull request as ready for review November 25, 2024 11:47

ralwing commented Nov 25, 2024

View reviewed changes

ralwing marked this pull request as draft December 3, 2024 07:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

perf: shallow copy instead of deepcopy #197

perf: shallow copy instead of deepcopy #197

ralwing commented Oct 30, 2024 •

edited

Loading

YoshiRi Nov 6, 2024

ktro2828 Nov 8, 2024

ralwing commented Nov 8, 2024

ralwing commented Nov 20, 2024 •

edited

Loading

ktro2828 left a comment

ralwing commented Nov 22, 2024

ralwing commented Nov 25, 2024

ralwing Nov 25, 2024

Label	car(1.0)	bicycle(1.0)	pedestrian(1.0)	motorbike(1.0)
predict_num	0	0	0	0
MOTA	inf	inf	inf	inf
MOTP	inf	inf	inf	inf
id_switch	0	0	0	0
tp	0.000	0.000	0.000	0.000
fp	0.000	0.000	0.000	0.000
tp_matching_score	0.000	0.000	0.000	0.000

perf: shallow copy instead of deepcopy #197

Are you sure you want to change the base?

perf: shallow copy instead of deepcopy #197

Conversation

ralwing commented Oct 30, 2024 • edited Loading

Category

What

Type of change

Test performed

Reference

Notes for reviewer

YoshiRi Nov 6, 2024

Choose a reason for hiding this comment

ktro2828 Nov 8, 2024

Choose a reason for hiding this comment

ralwing commented Nov 8, 2024

ralwing commented Nov 20, 2024 • edited Loading

ktro2828 left a comment

Choose a reason for hiding this comment

ralwing commented Nov 22, 2024

ralwing commented Nov 25, 2024

ralwing Nov 25, 2024

Choose a reason for hiding this comment

ralwing commented Oct 30, 2024 •

edited

Loading

ralwing commented Nov 20, 2024 •

edited

Loading