libc: Add support for stack smash canaries in debug variants.

[asiekierka]

I have checked the resulting ELFs, but haven't actually tried to smash the stack - however, I followed the ideas of retarget_lock_crash.

(Stack smash protection will work in non-debug variants, too, albeit without libnds functions themselves being protected.)

[asiekierka]

Solves half of blocksds/sdk#150 .

[AntonioND]

I can't test this, I've tried for about an hour and it just won't work for me. For example, I create this function:

void function(void)
{
    char buffer[5];
    strcpy(buffer, "ABCDEFGHIJK");
    printf("%s\n",buffer);
}

int main(int argc, char *argv[])
{
    defaultExceptionHandler();

    consoleDemoInit();
    function();

    printf("You shouldn't see this message.\n");

    while (1)
       swiWaitForVBlank();
}

Whenever I call it the ROM crashes, but not because of the thumb undefined instruction in the helper you've added.

I've built this ROM with:

LIBS := -lnds9d

CFLAGS := -fstack-protector-strong

[AntonioND]

I've merged this PR, but can you take a look at it whenever you have time? If you manage to get it working, add an example to "examples/debug" so that we can use it as reference.

[profi200]

Looking at your test there could be some stack variables in the way. The stack guard is the first one that goes on the stack on function call and you have to overwrite everything else that is also on the stack for this function.
Otherwise did you check the disassembly? If gcc recognizes that you are writing out of bounds it can optimize this out because it's undefined behavior. Worst case you have to make some inline assembly function that writes X bytes to sp.

[asiekierka]

@AntonioND The exact example you've provided works perfectly for me, though. It crashes at the opcode location for __stack_chk_fail.

[AntonioND]

Huh, indeed, your example works. I wonder what was wrong when I tried it...