Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Times out after 10s #6

Open
jason-cso opened this issue Jul 19, 2022 · 11 comments
Open

Times out after 10s #6

jason-cso opened this issue Jul 19, 2022 · 11 comments
Assignees
Labels
documentation Improvements or additions to documentation question Further information is requested

Comments

@jason-cso
Copy link

Seems like RTDE expects you to poll it continuously after you connect if you want fresh data. It'd be good if that was clearly called out in the docs.

If I do this:

rtde.connect()
time.sleep(11)
rtde.receive()

I get errors like received 0 bytes from Controller, probable cause: Controller has stopped or received 0 bytes from Controller. Also, I'll often get two stale messages before it notices the disconnect.

@gsim-inbolt
Copy link

gsim-inbolt commented Sep 8, 2022

Got the same issue here, ended up calling rtde.receive() at 500 Hz in a thread.
I wonder if this is by design of the RTDE protocol or caused by a mistake in this implementation though

@michal-milkowski
Copy link
Member

You're both right, after starting connection controller will send rt data, and python client is expected to read it. I'll add a line in documentation (as minimum).

@michal-milkowski michal-milkowski self-assigned this Dec 14, 2022
@michal-milkowski
Copy link
Member

I'm thinking about either adding an example with thread specifically for reading socket, or adding a method to library that clears receive buffer. Method would need to be called periodically.
What would be easier to use?

@michal-milkowski michal-milkowski added question Further information is requested documentation Improvements or additions to documentation labels Dec 20, 2022
@jason-cso
Copy link
Author

They both sound pretty useful. Could we have both? BTW this is how I solved it. Not very elegant, but I'm not latency sensitive.

    def receive(self):
        for i in range(10):
            now = time.monotonic()
            try:
                # RTDE seems to time out after 10s, so we'll use 8s for a little margin
                if now - self.last_receive > 8.0:
                    log.info("Calling RTDE receive() 2 extra times to notice likely disconnect")
                    results = self.rtde.receive(binary=False)
                    results = self.rtde.receive(binary=False)

                results = self.rtde.receive(binary=False)
                self.last_receive = time.monotonic()
                return results
            except rtde.RTDEException as e:
                log.error("RTDE error on attempt %d: %s", i, e)
                self.rtde.disconnect()
                self._setup()

        raise Exception("Couldn't receive")

@michal-milkowski
Copy link
Member

That makes a lot of sense. What would be the use case for ignoring incoming data?

@jason-cso
Copy link
Author

Some things are only accessible by rtde (like the registers and joints), so I have to use it even though I don't need its realtime nature. I did consider using a thread to keep it drained, but python's global interpreter lock and making sure I got setup/teardown right dissuaded me.

@gsim-inbolt
Copy link

A method clearing the whole buffer would be quite interesting, but the thread fulfills a different purpose imo so as @jason-cso mentioned having both would not feel redundant. Doing it cleanly with a thread certainly takes more effort though.

@sri331
Copy link

sri331 commented Mar 3, 2023

Got the same issue here, ended up calling rtde.receive() at 500 Hz in a thread. I wonder if this is by design of the RTDE protocol or caused by a mistake in this implementation though

I am trying to add a thread to handle connection lost issue but not successful ... Could you share sample code for the thread you implemented?

@gsim-inbolt
Copy link

Something similar to what we had would be:

def get_state():
    while not stop_event.is_set():
        state = rtde.receive()
        if state is None:
            logger.error("No data received, shutting down connection...")
            rtde.send_pause()
            rtde.disconnect()
            raise RTDEException
        if state_queue.full():
            state_queue.get_nowait()
        state_queue.put(state)
    return

robot_thread = th.Thread(target=get_state)
robot_thread.start()

@tomek2311
Copy link

I am running into smiliar (but different) 10 sec timeout issue. Here is the scenario:
I contenatized RTDE solution (based example provided in this repo) so that I can get access to UR states. When I run it from command line line (say: docker run --rm -dit -e ROBOT_IP="192.168.254.171" --name rtdeplugin rtdeplugin:latest) everything is fine - programs stays up.
On the other hand, when I try to run it as a service (using systemd), executng same command, it runs for 10 seconds and exists, systemd starts it again (since I specified to run forever), and the process continues forrever.

Did anyone tried to run rtde controller solution as a service, and was sucessful ?
Thanks, Tomasz

@sri331
Copy link

sri331 commented Mar 28, 2023

Thanks gsim-inbolt . The thread is useful. I am not sure if rtde class is thread safe not resulting in race condition. I am using a thread to make the connection active but some times it is giving below issue and going to infinite loop.
“Unknown package command: 0”

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation question Further information is requested
Projects
None yet
Development

No branches or pull requests

5 participants