Revision history [back]

tl;dr: no, I don't believe this is supported.

Longer: looking at the genmsg_py.py script's help, this could be possible:

$ ./genmsg_py.py --help
Usage: genmsg_py.py file

Options:
  -h, --help      show this help message and exit
  --initpy        
  -p PACKAGE      
  -o OUTDIR       
  -I INCLUDEPATH

notice the -o OUTDIR command line argument.

genpy is the package providing the rospy compatible code generator used to convert your .msg defs into code.

gencpp.py supports a similar option, and the other message code generators might as well, I haven't checked.

So on the generation side, this could be possible.

However, I see two possible problems/challenges (although I haven't checked whether they are problems/challenges):

1. the default implementation of the CMake-side to genpy, gencpp et al. hard-codes the output directory to ${ARG_GEN_OUTPUT_DIR}/msg (see here for genpy fi). To change this, you'd have to fork that package and make it do something else, which will not be very scalable.
2. none of the consumers of messages will be 'aware' they have to add additional directories to their msg/srv include paths, leading to lookup failures in downstream consumer code-paths.

While technically you could fork the relevant packages and update them, I'm not sure that would be the kind of overhead you'd find acceptable for what you're trying to do.

tl;dr: no, I don't believe this is supported.

Longer: looking at the genmsg_py.py script's help, this could be possible:

$ ./genmsg_py.py --help
Usage: genmsg_py.py file

Options:
  -h, --help      show this help message and exit
  --initpy        
  -p PACKAGE      
  -o OUTDIR       
  -I INCLUDEPATH

notice the -o OUTDIR command line argument.

genpy is the package providing the rospy compatible code generator used to convert your .msg defs into code.

gencpp.py supports a similar option, and the other message code generators might as well, I haven't checked.

So on the generation side, this could be possible.

However, I see two possible problems/challenges (although I haven't checked whether they are problems/challenges):

1. the default implementation of the CMake-side to genpy, gencpp et al. hard-codes the output directory to ${ARG_GEN_OUTPUT_DIR}/msg (see here for genpy fi). To change this, you'd have to fork that package and make it do something else, which will not be very scalable.
2. none of the consumers of messages will be 'aware' they have to add additional directories to their msg/srv include paths, leading to lookup failures in downstream consumer code-paths.

While technically you could fork the relevant packages and update them, I'm not sure that would be the kind of overhead you'd find acceptable for what you're trying to do.

Edit: I'm not entirely sure, but from this:

I'm trying to create a flexible way of defining enumerations for different datatypes, such that adding a new enum doesn't invalidate the message hash, thereby ruining the bag.

it might be this is an xy-problem.

While changing a .msg does change its hash, it does not immediately "ruin" .bags which contain the old definition.

rosbag supports migration of .bags containing one version of a .msg to another. This requires bag migration rules to be provided, which tell rosbag how to transform old instances of a message type into the new one. The linked page has information on how to write those rules.

Such a migration does require processing the entire .bag, but it is a one-time process, and keeps the old .bag as-is.

On modern systems, with high throughput IO systems and high performance CPUs, a moderately sized bag should not take too long to migrate.

Original answer: tl;dr: no, I don't believe this is supported.

Longer: looking at the genmsg_py.py script's help, this could be possible:

$ ./genmsg_py.py --help
Usage: genmsg_py.py file

Options:
  -h, --help      show this help message and exit
  --initpy        
  -p PACKAGE      
  -o OUTDIR       
  -I INCLUDEPATH

notice the -o OUTDIR command line argument.

genpy is the package providing the rospy compatible code generator used to convert your .msg defs into code.

gencpp.py supports a similar option, and the other message code generators might as well, I haven't checked.

So on the generation side, this could be possible.

However, I see two possible problems/challenges (although I haven't checked whether they are problems/challenges):

1. the default implementation of the CMake-side to genpy, gencpp et al. hard-codes the output directory to ${ARG_GEN_OUTPUT_DIR}/msg (see here for genpy fi). To change this, you'd have to fork that package and make it do something else, which will not be very scalable.
2. none of the consumers of messages will be 'aware' they have to add additional directories to their msg/srv include paths, leading to lookup failures in downstream consumer code-paths.

While technically you could fork the relevant packages and update them, I'm not sure that would be the kind of overhead you'd find acceptable for what you're trying to do.

Edit: I'm not entirely sure, but from this:

I'm trying to create a flexible way of defining enumerations for different datatypes, such that adding a new enum doesn't invalidate the message hash, thereby ruining the bag.

it might be this is an xy-problem.

While changing a .msg does change its hash, it does not immediately "ruin" .bags which contain the old definition.

rosbag supports migration of .bags containing one version of a .msg to another. This requires bag migration rules to be provided, which tell rosbag how to transform old instances of a message type into the new one. The linked page has information on how to write those rules.

Such a migration does require processing the entire .bag, but it is a one-time process, and keeps the old .bag as-is.

On modern systems, with high throughput IO systems and high performance CPUs, a moderately sized bag should not take too long to migrate.

Original answer: tl;dr: no, I don't believe this is supported.

Longer: looking at the genmsg_py.py script's help, this could be possible:

$ ./genmsg_py.py --help
Usage: genmsg_py.py file

Options:
  -h, --help      show this help message and exit
  --initpy        
  -p PACKAGE      
  -o OUTDIR       
  -I INCLUDEPATH

notice the -o OUTDIR command line argument.

genpy is the package providing the rospy compatible code generator used to convert your .msg defs into code.

gencpp.py supports a similar option, and the other message code generators might as well, I haven't checked.

So on the generation side, this could be possible.

However, As you never really invoke any of the message code generation scripts directly, but use the message_generation and message_runtime CMake infrastructure, I see two possible problems/challenges (although I haven't checked whether they are problems/challenges):

1. the default implementation of the CMake-side to genpy, gencpp et al. hard-codes the output directory to ${ARG_GEN_OUTPUT_DIR}/msg (see here for genpy fi). To change this, you'd have to fork that package and make it do something else, which will not be very scalable.
2. none of the consumers of messages will be 'aware' they have to add additional directories to their msg/srv include paths, leading to lookup failures in downstream consumer code-paths.

While technically you could fork the relevant packages and update them, I'm not sure that would be the kind of overhead you'd find acceptable for what you're trying to do.

Edit: I'm not entirely sure, but from this:

I'm trying to create a flexible way of defining enumerations for different datatypes, such that adding a new enum doesn't invalidate the message hash, thereby ruining the bag.

it might be this is an xy-problem.

While changing a .msg does change its hash, it does not immediately "ruin" .bags which contain the old definition.

rosbag supports migration of .bags containing one version of a .msg to another. This requires bag migration rules to be provided, which tell rosbag how to transform old instances of a message type into the new one. The linked page has information on how to write those rules.

Such a migration does require processing the entire .bag, but it is a one-time process, and keeps the old .bag as-is.

On modern systems, with high throughput IO systems and high performance CPUs, a moderately sized bag should not take too long to migrate.

Original answer: tl;dr: no, I don't believe this is supported.

Longer: looking at the genmsg_py.py script's help, this could be possible:

$ ./genmsg_py.py --help
Usage: genmsg_py.py file

Options:
  -h, --help      show this help message and exit
  --initpy        
  -p PACKAGE      
  -o OUTDIR       
  -I INCLUDEPATH

notice the -o OUTDIR command line argument.

genpy is the package providing the rospy compatible code generator used to convert your .msg defs into code.

code. gencpp.py supports a similar option, and the other message code generators might as well, I haven't checked.

So on the generation side, this could be possible.

However, As as you never really invoke any of the message code generation scripts directly, but use the message_generation and message_runtime CMake infrastructure, I see two possible problems/challenges (although I haven't checked whether they are problems/challenges):

1. the default implementation of the CMake-side to genpy, gencpp et al. hard-codes the output directory to ${ARG_GEN_OUTPUT_DIR}/msg (see here for genpy fi). To change this, you'd have to fork that package and make it do something else, which will not be very scalable.
2. none of the consumers of messages will be 'aware' they have to add additional directories to their msg/srv include paths, leading to lookup failures in downstream consumer code-paths.

While technically you could fork the relevant packages and update them, I'm not sure that would be the kind of overhead you'd find acceptable for what you're trying to do.

Edit: I'm not entirely sure, but from this:

I'm trying to create a flexible way of defining enumerations for different datatypes, such that adding a new enum doesn't invalidate the message hash, thereby ruining the bag.

it might be this is an xy-problem.

While changing a .msg does change its hash, it does not immediately "ruin" .bags which contain the old definition.

rosbag supports migration of .bags containing one version of a .msg to another. This requires bag migration rules to be provided, which tell rosbag how to transform old instances of a message type into the new one. The linked page has information on how to write those rules.

Such a migration does require processing the entire .bag, but it is a one-time process, and keeps the old .bag as-is.

On modern systems, with high throughput IO systems and high performance CPUs, a moderately sized bag should not take too long to migrate.

Original answer: tl;dr: no, I don't believe this is supported.

Longer: looking at the genmsg_py.py script's help, this could be possible:

$ ./genmsg_py.py --help
Usage: genmsg_py.py file

Options:
  -h, --help      show this help message and exit
  --initpy        
  -p PACKAGE      
  -o OUTDIR       
  -I INCLUDEPATH

notice the -o OUTDIR command line argument.

genpy is the package providing the rospy compatible code generator used to convert your .msg defs into code. gencpp.py supports a similar option, and the other message code generators might as well, I haven't checked.

So on the generation side, this could be possible.

However, as you never really invoke any of the message code generation scripts directly, but use the message_generation and message_runtime CMake infrastructure, I see two possible problems/challenges (although I haven't checked whether they are problems/challenges):

1. the default implementation of the CMake-side to genpy, gencpp et al. hard-codes the output directory to ${ARG_GEN_OUTPUT_DIR}/msg (see here for genpy fi). To change this, you'd have to fork that package and make it do something else, which will not be very scalable.
2. none of the consumers of messages will be 'aware' they have to add additional directories to their msg/srv include paths, leading to lookup failures in downstream consumer code-paths.

While technically you could fork the relevant packages and update them, I'm not sure that would be the kind of overhead you'd find acceptable for what you're trying to do.

Edit 2:

Warning: long piece of text ahead about semantics of messages and why what we have is actually not that bad.

Thank you for the time and in-depth answer! Not the answer I was hoping for, but what I feared... I was hoping that hardcoded /msg in genpy wouldn't be there.

Thanks for the alternative solution, I was aware of the rosbag migration, just dealing with dozens of bags with 10s of thousands of messages makes adding any new enumeration extremely cumbersome.

With the risk of "gold plating" this, but I feel there is actually a big advantage to the way things are implemented now.

ROS messages (and services and actions) are supposed to be stand-alone. That is: their contents should be interpretable without requiring any external sources of information (that's of course difficult to achieve in reality, but that's the idea).

This is important both for archival reasons (ie: when stored in .bags from say 10 years ago) as well as to help ensure that consumers will be able to figure out what is being communicated right now. Both at the syntax level (ie: form of the data) as well as semantics (ie: meaning).

The MD5 hash of message structures ensures that the form of .msgs expected by consumers is exactly the same as those produced by publishers. This not only guarantees that consumers will be able to deserialise the data (ie: they know in what form the data is), but it will also go a long way to make sure that how they interpret the meaning of that data is as the producer expected them to (ie: their responses and behaviour match with what the intentions of the producer were).

What you propose in your example (using a plain uint16 to contain the value of an enum value and then storing the possible (legal?) values of that uint16 in a separate message type) goes against this (if it doesn't completely make it impossible).

With the proposed approach, it would be possible to change the members of the "enum" (in quotes, as the msg spec does not really support enums) without any producer or consumer being able to detect this, as there is no direct coupling between the enum and the uint16. None of the hashes would change, as the "enum message" is not stored anywhere.

The proper way to do this would be to either keep the enumeration values in the same .msg, or use the type of the enumeration .msg as the type of the field containing the value (ie: instead of uint16).

I realise this doesn't help make things any easier, but I believe the current system does help avoid all sorts of problems.

I was aware of the rosbag migration, just dealing with dozens of bags with 10s of thousands of messages makes adding any new enumeration extremely cumbersome.

This is easy for me to say (as I don't have to do it), but automation should go a long way here. Migration rules should be almost generatable themselves if we're only talking about adding values to enumeration fields. With the rule in place, batch processing .bags should also be relatively straightforward.

Edit: I'm not entirely sure, but from this:

I'm trying to create a flexible way of defining enumerations for different datatypes, such that adding a new enum doesn't invalidate the message hash, thereby ruining the bag.

it might be this is an xy-problem.

While changing a .msg does change its hash, it does not immediately "ruin" .bags which contain the old definition.

rosbag supports migration of .bags containing one version of a .msg to another. This requires bag migration rules to be provided, which tell rosbag how to transform old instances of a message type into the new one. The linked page has information on how to write those rules.

Such a migration does require processing the entire .bag, but it is a one-time process, and keeps the old .bag as-is.

On modern systems, with high throughput IO systems and high performance CPUs, a moderately sized bag should not take too long to migrate.

Original answer: tl;dr: no, I don't believe this is supported.

Longer: looking at the genmsg_py.py script's help, this could be possible:

$ ./genmsg_py.py --help
Usage: genmsg_py.py file

Options:
  -h, --help      show this help message and exit
  --initpy        
  -p PACKAGE      
  -o OUTDIR       
  -I INCLUDEPATH

notice the -o OUTDIR command line argument.

genpy is the package providing the rospy compatible code generator used to convert your .msg defs into code. gencpp.py supports a similar option, and the other message code generators might as well, I haven't checked.

So on the generation side, this could be possible.

However, as you never really invoke any of the message code generation scripts directly, but use the message_generation and message_runtime CMake infrastructure, I see two possible problems/challenges (although I haven't checked whether they are problems/challenges):

1. the default implementation of the CMake-side to genpy, gencpp et al. hard-codes the output directory to ${ARG_GEN_OUTPUT_DIR}/msg (see here for genpy fi). To change this, you'd have to fork that package and make it do something else, which will not be very scalable.
2. none of the consumers of messages will be 'aware' they have to add additional directories to their msg/srv include paths, leading to lookup failures in downstream consumer code-paths.

While technically you could fork the relevant packages and update them, I'm not sure that would be the kind of overhead you'd find acceptable for what you're trying to do.