ROS Resources: Documentation | Support | Discussion Forum | Index | Service Status | Q&A answers.ros.org
Ask Your Question
0

control real-time processes with ROS/ROS2 on multicore system

asked 2020-10-06 02:01:10 -0600

Glenn_VTC gravatar image

I was wondering if it would be possible to replace a PLC running real-time processes completely by an IPC running Linux and ROS or ROS2.

We have already experience in running an ethercat master at 1ms on ROS Melodic and this works. (at least for one day long) But the processes we should run on ROS should be running on threads of 250us. We would also need to run different ROS nodes on different cores.

Does anyone have experience in running ROS on 250us threads in a multicore system? What is the stability and robustness of the system? Can a PLC really replaced by an IPC running ROS?

Thanks!

edit retag flag offensive close merge delete

Comments

I don't believe adding "ROS" here really matters.

250 microseconds is not something you'd easily achieve with low jitter on a regular Linux machine. I guess you already have the kernel which supports the "ROS Melodic ethercat master" you mention patched with an RT patch, otherwise even 1 ms periods are hard to achieve reliably.

All of this is not even really affected by using ROS or not: you cannot implement a hard real-time system with ROS 1 (or: it's rather difficult and involved). The ethercat master you mention is most likely a single process with RT priority and the determinism guarantees are between the master and the ethercat slaves. ROS is likely not involved there.

There is something unclear in your description: is 250 us the total time budget for a certain task or operation, is it just one slice of a longer running task (ie ...(more)

gvdhoorn gravatar image gvdhoorn  ( 2020-10-06 12:33:35 -0600 )edit

Thanks for your reply. The ethercat master we got running is indeed running on a Linux machine that is patched with PREEMPT_RT. We have given the ethercat master highest priority and we see a maximum jitter now of 127us. ROS is involved as the ethercat master is publishing and subscribing data to the ROs layer at a pace of 1ms. The ROS nodes in that network are typically running at 10ms and 100ms.

For another project we would need ROS nodes that are running at 250us (critical real-time system), if we would want to use the same structure. So this is indeed the question: can we use the same ROS-structure or will we need to do everything inside the Linux environment and schedule processes immediately on the processor?

Glenn_VTC gravatar image Glenn_VTC  ( 2020-10-07 00:58:36 -0600 )edit

I'm a bit confused.

Outside of the ethercat master process, there is nothing RT any more. ROS 1 cannot guarantee deterministic message delivery nor processing. So the jitter you mention, is that jitter between the master and the slaves, or the jitter on the period of the master update cycle (ie: the process itself)?

In any case it seems like you're asking whether it would be possible to make this run "fast enough".

In my experience that would be difficult, especially if you start using pub-sub in your critical (messaging) path.

The infrastructure was just not designed nor implemented for this kind of use.

Even your current setup is essentially a "cheat": only the ethercat master process is real-time, everything outside of it using ROS msging is best effort (at best).

But as always with real-time systems: if you can't prove it (ie: exploit predictability of your ...(more)

gvdhoorn gravatar image gvdhoorn  ( 2020-10-07 04:04:18 -0600 )edit

So this is indeed the question: can we use the same ROS-structure or will we need to do everything inside the Linux environment and schedule processes immediately on the processor?

this seems like a bit of a strange question.

ROS is not "an environment", it's a framework which includes a middleware.

"do everything inside the Linux environment" reads as-if you're considering ROS is doing something special or does something different from your base OS. It doesn't. It's all just regular Linux (patched with PREEMPT_RT in your case), so regular schedulers, regular process management and regular execution.

ROS nodes are also not special; they're just regular processes which happen to use network connections to exchange information instead of files or pipes.

For another project we would need ROS nodes that are running at 250us (critical real-time system)

and here we could "stop reading": ROS 1 is ...(more)

gvdhoorn gravatar image gvdhoorn  ( 2020-10-07 04:08:00 -0600 )edit

.. Or at least: if you include ROS 1 nodes in the safety system. If you have something else to which you can delegate that responsibility then it becomes a different question, and best-effort on the ROS side may be good enough.

Perhaps it would be good if you could clarify whether you intend to use ROS messaging between RT processes, and whether you then require deterministic behaviour between your ROS nodes as well.

In ROS 1, that's impossible, which would make the answer to your question: no.

gvdhoorn gravatar image gvdhoorn  ( 2020-10-07 04:10:02 -0600 )edit

Thanks again for the clarification. I know safety systems are out of the question here. We indeed were wondering if ROS nodes could run fast enough. (<1ms) The way to go then here is use RT Linux for all 'RT' control systems and use ROS nodes for higher level planning, management and steering. We do this indeed already with the ethercat master.

Glenn_VTC gravatar image Glenn_VTC  ( 2020-10-09 00:48:01 -0600 )edit

1 Answer

Sort by ยป oldest newest most voted
0

answered 2020-10-09 03:15:06 -0600

gvdhoorn gravatar image

We indeed were wondering if ROS nodes could run fast enough. (<1ms)

Well I'm not saying it's impossible.

What is impossible is to make blanket statements about "ROS" (in quotes, as what would be possible is almost completely determined by how things are used, so saying something "about ROS" in general is rather difficult).

So depending on how you use ROS infrastructure it's possible to "run nodes" at kHz frequencies. It gets difficult when node<->node communication gets involved. Not impossible (see MoveIt's servo component fi), but difficult, as determinism is gone, and jitter will increase.

The way to go then here is use RT Linux for all 'RT' control systems and use ROS nodes for higher level planning, management and steering.

That is how this is done often in ROS 1, yes.

If you have RT requirements, I would suggest to take a look at ROS 2. The real-time working group and safety working group may be interesting and perhaps even interested.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

2 followers

Stats

Asked: 2020-10-06 02:01:10 -0600

Seen: 501 times

Last updated: Oct 09 '20