Ask Your Question
0

how to best run a service that many clients connect to every few seconds?

asked 2011-08-11 08:07:44 -0600

this post is marked as community wiki

This post is a wiki. Anyone with karma >75 is welcome to improve it.

I'm running a simulator backend that allows clients to connect to it using the rosservice calls. Everything works fine with a few clients connected (<20 or so) but if I try a large-scale test with 64 clients, it will fail to connect on occasion. I'm not sure exactly what the issue is. I worked around it by implementing retries and only after trying a bunch of times failing.

Is the problem with the callback queue getting overloaded? Is there a buffer of service requests that is hidden away somewhere? (I had trouble with small buffers on my data topics already, but that was a simple fix!) Maybe it's a threading thing, although the backend is currently single-threaded, and the clients are multi-threaded but just to run a spinner on a completely disjoint messaging queue for client-client communications. If the global callback queue is involved then it is only getting touched by one thread ever per process.

Thanks for any help/pointers!

edit retag flag offensive close merge delete

Comments

Are the client - service connections persistent? That might help. I think they are non-persistent by default.
dornhege gravatar imagedornhege ( 2011-08-11 08:50:42 -0600 )edit
Yeah this was my first thought but they design someone else chose made that hard. If they are persistent, they actually keep the TCP connection live? That would probably make the "collision" on reconnect go away though. I just had hoped not to rewrite that much code of other people.
Devin Grady gravatar imageDevin Grady ( 2011-08-12 01:04:18 -0600 )edit

1 Answer

Sort by ยป oldest newest most voted
0

answered 2011-08-11 18:45:03 -0600

this post is marked as community wiki

This post is a wiki. Anyone with karma >75 is welcome to improve it.

Any specific service callback will only be called one at a time, even if there are several requests, they will be queued up and possibly overflow if you're calling it quickly.

If you do require each request to be sequential @dornhege 's suggestion of persistent connections will help.

However this sounds like you might want to consider a different interface design than a single service call. I suggest that you look into actionlib You could setup your simulator to be an action server which queues all the incoming requests and does whatever locking is necessary for thread saftey to interact with the simulator. With this technique you can allow it to interact with multiple clients at once and with concurrent requests in flight you can get better throughput in each cycle.

edit flag offensive delete link more

Comments

sounds like that'll just delay the issue, rather than solve it. I assume that multiple connections made all at the same time would still hit some sort of internal buffer of connections, and the problem is that they are dropped before they can interact at all. I do not require them to be sequential
Devin Grady gravatar imageDevin Grady ( 2011-08-12 01:01:49 -0600 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

Stats

Asked: 2011-08-11 08:07:44 -0600

Seen: 227 times

Last updated: Aug 11 '11