fix crash due to session disconnect #332
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The issue was described initially here:
unispeech/asterisk-unimrcp#67 (comment)
In case of an asr server crash (mrcp server) asterisk-unimrcp and mrcp_client does not terminate properly the mpf_engine / stream and all the relevant to mpf resources. So the following crash is takes place.
From logs perspective we can see that the session has been marked as disconnected:
Finally I detected the issue that leads to the above crash.
mrcp_client of unimrcp (implements mrcp protocol and is used by asterisk-unimrcp asterisk module) in case of a disconnection in any of its’ connections, didn’t destroy the mrcp channel and its’ resources (mpf topology, engine, etc). Without this destroy action mpf engine was trying to read frames from the speech channel even after its termination from the asterisk-unimrcp module (pointers pointed to garbage).
After this realisation the solution was 1 line of code
Stresstesting for 40 minutes, ~4000 calls processed per pjsip/distributor, 10 asr server crashes, no asterisk taskprocessor lock and no asterisk crash.