Force exit CMUX does not actually work (IDFGH-13724)

[Ant2000]

Answers checklist.

• I have read the documentation for esp-protocols components and the issue is not addressed there.
• I have updated my esp-protocols branch (master or release) to the latest version and checked that the issue is present there.
• I have searched the issue tracker for a similar issue and not found a similar issue.

What component are you using? If you choose Other, provide details in More Information.

esp_modem

component version

v1.1.0

IDF version.

v5.2.2

More Information.

I had an issue with the esp-modem library being stuck in CMUX mode after I hard reset my modem using the power key. I referred to the issues #416 and #320 which described a method to fix this problem, but it didn't work. My transition was CMUX mode -> UNDEF mode -> CMUX Manual Exit mode -> UNDEF mode -> command mode.

Using printf statements, I verified that the device was in command mode, but the messages incoming from the modem were falsely getting interpreted as CMUX messages causing an infinite loop of CMUX recovery until I reset the ESP.

I found the issue was in the file esp_modem_dte.cpp in the function DTE::exit_cmux().

bool DTE::exit_cmux()
{
    if (!cmux_term) {
        return false;
    }
    if (!cmux_term->deinit()) {
        return false;
    }
    exit_cmux_internal();
    cmux_term.reset();
    return true;
}

The function cmux_term->deinit() was returning false which was expected since the modem has rebooted and is no longer in CMUX mode. But since the function returned immediately, it meant that exit_cmux_internal(); was never executed keeping the dte stuck interpreting incoming data as CMUX messages. So I changed the code a little bit.

bool DTE::exit_cmux()
{
    if (!cmux_term) {
        return false;
    }
    const bool success = cmux_term->deinit();
    exit_cmux_internal();
    cmux_term.reset();
    return success;
}

Now even if the CMUX terminal exit fails, it still clears the internal implementation, but returns false, putting the esp_modem into the UNDEF state and the transition to command mode to work.

[david-cermak]

Hi @Ant2000

Thank you for reporting this and suggesting the fix.
You're right, this transition doesn't really work and the only workaround is to destroy and recreate the DCE.

I've checked your fix and this works for me too, so if you feel like helping a little more, you can submit a PR with the changes you proposed. I'd be happy to accept it and merge it.

[david-cermak]

Fixed in #658