control real-time processes with ROS/ROS2 on multicore system
I was wondering if it would be possible to replace a PLC running real-time processes completely by an IPC running Linux and ROS or ROS2.
We have already experience in running an ethercat master at 1ms on ROS Melodic and this works. (at least for one day long) But the processes we should run on ROS should be running on threads of 250us. We would also need to run different ROS nodes on different cores.
Does anyone have experience in running ROS on 250us threads in a multicore system? What is the stability and robustness of the system? Can a PLC really replaced by an IPC running ROS?
Thanks!
I don't believe adding "ROS" here really matters.
250 microseconds is not something you'd easily achieve with low jitter on a regular Linux machine. I guess you already have the kernel which supports the "ROS Melodic ethercat master" you mention patched with an RT patch, otherwise even 1 ms periods are hard to achieve reliably.
All of this is not even really affected by using ROS or not: you cannot implement a hard real-time system with ROS 1 (or: it's rather difficult and involved). The ethercat master you mention is most likely a single process with RT priority and the determinism guarantees are between the master and the ethercat slaves. ROS is likely not involved there.
There is something unclear in your description: is 250 us the total time budget for a certain task or operation, is it just one slice of a longer running task (ie ...(more)
Thanks for your reply. The ethercat master we got running is indeed running on a Linux machine that is patched with PREEMPT_RT. We have given the ethercat master highest priority and we see a maximum jitter now of 127us. ROS is involved as the ethercat master is publishing and subscribing data to the ROs layer at a pace of 1ms. The ROS nodes in that network are typically running at 10ms and 100ms.
For another project we would need ROS nodes that are running at 250us (critical real-time system), if we would want to use the same structure. So this is indeed the question: can we use the same ROS-structure or will we need to do everything inside the Linux environment and schedule processes immediately on the processor?
I'm a bit confused.
Outside of the ethercat master process, there is nothing RT any more. ROS 1 cannot guarantee deterministic message delivery nor processing. So the jitter you mention, is that jitter between the master and the slaves, or the jitter on the period of the master update cycle (ie: the process itself)?
In any case it seems like you're asking whether it would be possible to make this run "fast enough".
In my experience that would be difficult, especially if you start using pub-sub in your critical (messaging) path.
The infrastructure was just not designed nor implemented for this kind of use.
Even your current setup is essentially a "cheat": only the ethercat master process is real-time, everything outside of it using ROS msging is best effort (at best).
But as always with real-time systems: if you can't prove it (ie: exploit predictability of your ...(more)
this seems like a bit of a strange question.
ROS is not "an environment", it's a framework which includes a middleware.
"do everything inside the Linux environment" reads as-if you're considering ROS is doing something special or does something different from your base OS. It doesn't. It's all just regular Linux (patched with PREEMPT_RT in your case), so regular schedulers, regular process management and regular execution.
ROS nodes are also not special; they're just regular processes which happen to use network connections to exchange information instead of files or pipes.
and here we could "stop reading": ROS 1 is ...(more)
.. Or at least: if you include ROS 1 nodes in the safety system. If you have something else to which you can delegate that responsibility then it becomes a different question, and best-effort on the ROS side may be good enough.
Perhaps it would be good if you could clarify whether you intend to use ROS messaging between RT processes, and whether you then require deterministic behaviour between your ROS nodes as well.
In ROS 1, that's impossible, which would make the answer to your question: no.
Thanks again for the clarification. I know safety systems are out of the question here. We indeed were wondering if ROS nodes could run fast enough. (<1ms) The way to go then here is use RT Linux for all 'RT' control systems and use ROS nodes for higher level planning, management and steering. We do this indeed already with the ethercat master.