socketpair support for posix compliance (IDFGH-12794)

[sramrajkar]

Answers checklist.

• I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
• I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
• I have searched the issue tracker for a similar issue and not found a similar issue.

General issue report

I am trying to compile the following library for ESP32S3 using 5.x.x branches
https://github.com/libplctag/libplctag
and need some implementation for int socketpair(int domain, int type, int protocol, int sv[2])
Looks like ESP IDF is missing this posix API.

[ESP-Marius]

Thanks for the request, we do indeed not support his. It is something we can consider adding support for in the future.

From looking at their github I could see some discussion about porting the library though, and it seems that even if we had socketpair() you might still run into problems trying to run this.

[david-cermak]

we most likely will not support socketpari() in any near future (we could maybe implement some simplified version using localhost sockets or other signalling mechanism, but not fully fledged implementation, at least not until it's supported in LWIP)

as for porting high-leverl libraries, we typically provide alternative implementation if pipe(), signals, and interprocess communication come into consideration and it's usually sufficient. you can check our asio port for your reference:

https://github.com/espressif/esp-protocols/tree/master/components/asio

Regarding the libplctag library, I think it's just easier to supply IDF specific implementation of sock_create_event_wakeup_channel(), maybe something similar to the windows platform signalling:

https://github.com/libplctag/libplctag/blob/4793d92893906e80fcfc67d4c4ed87fd37d81df8/src/platform/windows/platform.c#L2169-L2176

[sramrajkar]

@david-cermak @ESP-Marius thank you getting back on this issue.
@david-cermak and idf specific function implementation would be a great start for me if you guys can help out. I tried a method looking at some literature online and it does not really work. The code gets blocked in that function trying to send even the first packet out the ethernet netif.
Here is what I have tried

static int socketpair(int domain, int type, int protocol, int sv[2]) {
    int listener = socket(domain, type, protocol);
    if (listener < 0) return -1;

    struct sockaddr_in addr;
    memset(&addr, 0, sizeof(addr));
    addr.sin_family = AF_INET;
    addr.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
    addr.sin_port = 0;  // Let the OS choose the port

    if (bind(listener, (struct sockaddr *)&addr, sizeof(addr)) < 0) return -1;

    socklen_t len = sizeof(addr);
    if (getsockname(listener, (struct sockaddr *)&addr, &len) < 0) return -1;

    if (listen(listener, 1) < 0) return -1;

    sv[0] = socket(domain, type, protocol);
    if (sv[0] < 0) return -1;

    if (connect(sv[0], (struct sockaddr *)&addr, len) < 0) return -1;

    sv[1] = accept(listener, NULL, NULL);
    if (sv[1] < 0) return -1;

    close(listener);  // No longer needed
    return 0;
}

@ESP-Marius I have some concerns about the RAM usage as well and I had started looking into one issue at a time. Initially the logging code using 1.5k buffer on the stack which was blowing up the stack very quickly. I think I can handle some RAM related issues if the networking is set up and there are packets on the wire. We also have 2MB PSRAM and can increase it on the design if needed just for this stack.

[david-cermak]

@sramrajkar I think you're just missing the non-blocking mode, everything else looks good.
here's the updated code, I've only briefly tested:

#include <netdb.h>
#include <sys/socket.h>
#include "esp_check.h"

#define INVALID_SOCKET (-1)

static const char *TAG = "socket_helpers";

static int set_nonblocking(int sock)
{
    int opt;
    opt = fcntl(sock, F_GETFL, 0);
    if (opt == -1){
        return -1;
    }
    if (fcntl(sock, F_SETFL, opt | O_NONBLOCK) == -1) {
        return -1;
    }
    return 0;
}

int socketpair(int domain, int type, int protocol, int sv[2])
{
    struct sockaddr_storage ss;
    struct sockaddr_in *sa = (struct sockaddr_in *)&ss;
    socklen_t ss_len = sizeof(struct sockaddr_in);
    int fd1 = INVALID_SOCKET;
    int fd2 = INVALID_SOCKET;
    int listenfd = INVALID_SOCKET;
    int ret = 0; // Success

    sa->sin_family = AF_INET;
    sa->sin_addr.s_addr = htonl(INADDR_LOOPBACK);
    sa->sin_port = 0;
    listenfd = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
    ESP_GOTO_ON_FALSE(listenfd != INVALID_SOCKET, -1, err, TAG, "Cannot create listening socket");
    ESP_GOTO_ON_FALSE(listen(listenfd, 1) == 0, -1, err, TAG, "Failed to listen");

    memset(&ss, 0, sizeof(ss));
    ss_len = sizeof(ss);
    ESP_GOTO_ON_FALSE(getsockname(listenfd, (struct sockaddr *)&ss, &ss_len) >= 0, -1, err, TAG, "getsockname failed");

    sa->sin_family = AF_INET;
    sa->sin_addr.s_addr = htonl(INADDR_LOOPBACK);

    fd1 = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
    ESP_GOTO_ON_FALSE(fd1 != INVALID_SOCKET, -1, err, TAG, "Cannot create read side socket");
    ESP_GOTO_ON_FALSE(set_nonblocking(fd1) == 0, -1, err, TAG, "Failed to set socket to nonblocking mode");
    if (connect(fd1, (struct sockaddr *)&ss, ss_len) < 0) {
        ESP_GOTO_ON_FALSE(errno == EINPROGRESS || errno == EWOULDBLOCK, -1, err, TAG, "Failed to connect fd1");
    }
    fd2 = accept(listenfd, NULL, 0);
    if (fd2 == -1) {
        ESP_GOTO_ON_FALSE(errno == EINPROGRESS || errno == EWOULDBLOCK, -1, err, TAG, "Failed to accept fd2");
    }
    ESP_GOTO_ON_FALSE(set_nonblocking(fd2) == 0, -1, err, TAG, "Failed to set socket to nonblocking mode");

    close(listenfd);
    sv[0] = fd1;
    sv[1] = fd2;
    return ret;

err:
    if (listenfd != INVALID_SOCKET) {
        close(listenfd);
    }
    if (fd1 != INVALID_SOCKET) {
        close(fd1);
    }
    if (fd2 != INVALID_SOCKET) {
        close(fd2);
    }
    return ret;
}

[david-cermak]

@sramrajkar We will soon publish socket-helper component containing the above implementation of socketpair and some other utilities: espressif/esp-protocols#671
You can check if you're able to compile (and run) some libplctag project with this.

[sramrajkar]

@david-cermak yes I can try this over the weekend with the new module. Thank you for looking into this.

[david-cermak]

Here's an example of using sock-utils with libplctag library: https://github.com/david-cermak/esp-network-examples/tree/main/plctag/simple

There's still one problem with declaration of socketpair() from IDF -- I will address this in espressif/esp-protocols#713 and close this issue. Please comment if you have any concern.