Load time improvements

[franzlst]

This issue is to collect current loading time measurements and have a platform for discussion about performance topics.

Some Stats about loading times performed on 25. June 2018.

sm stats

see unit_test_state_machines/deep_libraries

sm_with_deeper_libraries:

• 492 states (Max hierarchy level 7) / 699 transitions.
• only 2 states in root state

sm_with_deepest_libraries:

• has 1474 states / (Max hierarchy level 7) / 2095 transitions
• only 6 states in root state

measurements

core state machine loading

ssd:

deeper: ~ 1.0
deepest: ~ 3.0

local drive (e.g. /home_local):

deeper: ~ 1.2
deepest: ~ 3.6

USERSTORE:

deeper: 1.7
deepest: ~ 5.0

sdd: 100%
home_local: 120%
USERSTORE: 166%

model generation

with meta data loading:
deepest: ~ 4.0
without meta data loading:
deepest: ~ 3,5

gaphas canvas setup

deepest: ~ 6.5
deeper: ~2.2

For more information also have a look at the meeting minutes from 2018-05-28

Originally created by @sebastian-brunner ([email protected]) at 2018-07-03 09:17:30+00:00 (moved from RMC internal repository)

[franzlst]

Here the coarse results of my measuring of the core:

• ~80-90% of the time is spent in loading and creating the states (core_data.json to State object)
• of that time 20-30% is used for reading the file (core_data.json=> dict), the rest for creating the State object from that dict

So most of the time is needed for creating the core objects from dicts. @sebastian-brunner found out that the reason for this is the heavy verification of all items.

This is why we decided on

• implement a unsafe_init method (in states and state elements), which creates core objects without most verifications
• use that method in from_dict
• verify that a library interface hasn't changed:
  • calculate a hash of the library interface when loading a library
  • store a hash of the library when saving a library state (use the version property for this)
  • compare those two hashes when loading a library state

Originally created by @franzlst ([email protected]) at 2018-07-04 14:41:09+00:00 (moved from RMC internal repository)

[franzlst]

I also thought about to adapt the hash update:
It could be reasonable to update the hashes when the libraries are changed or loaded.

Originally created by @Rbelder at 2018-07-16 09:59:13+00:00 (moved from RMC internal repository)

[franzlst]

With the port to gtk3 we have significant speedups when loading states machines (up to 25% for the omnirob state machines). During the next omnirob integration days I will care about the unsafe_init stuff.

What still needs to be found out is why the creation of the models is also quite slow (next to the loading times of the state machines themselves). E.g. for models there are no validity checks ...

Originally created by @sebastian-brunner ([email protected]) at 2018-12-06 09:29:59+00:00 (moved from RMC internal repository)

[franzlst]

What still needs to be found out is why the creation of the models is also quite slow (next to the loading times of the state machines themselves). E.g. for models there are no validity checks ...

Yeah here we need to dig into gtkmvc. :-(

Is there anything new about gaphas and the spent time in gaphas all over? The setup canvas is mostly not all. Afterwards a wait_for_gui has to be performed to get the full gaphas time consumption. If operated by the user it is still only included in the time duration shown in the logger output if restore session is used. Normal loading miss the time after setup canvas.

Originally created by @Rbelder at 2018-12-06 16:50:08+00:00 (moved from RMC internal repository)

[franzlst]

Concerning the models, have a look at: #22

In this issue I will only focus on the unsafe_init and close it afterwards.

Originally created by @sebastian-brunner ([email protected]) at 2019-07-09 15:25:59+00:00 (moved from RMC internal repository)