Some Notes on Cyclone DDS

One of the biggest differences between ROS1 and ROS2 is the replacement of the single middleware with a plugin-based architecture. This allows ROS2 to use various Robotic Middle Ware (RMW) implementations. All these RMW implementations are currently based on DDS. You can read all about the details in the ROS2 Design Docs.

Over time, the supported RMW implementations have shifted and new ones have been introduced. The default is currently FastRTPS (which apparently has been renamed to FastDDS, but after the Foxy release). The newest option is CycloneDDS which uses Eclipse Cyclone DDS. Cyclone DDS has gotten a lot of praise lately, so let’s take a closer look.

RMW Implementations

Choosing between RMW implementations is still a bit of a challenge since ROS2 is still very much under active development. There are multiple tickets about FastDDS service discovery issues. CycloneDDS is less than two years old, which means it is still under very active development and might not be fully featured, but it is supposed to be really highly performant. Mixing multiple implementations at runtime has noted caveats.

Luckily, it’s very easy to switch between implementations by simply setting the RMW_IMPLEMENTATION environment variable (assuming the selected implementation is built/installed).

When switching between implementations, be sure to stop the ros2 dameon so that it gets restarted with the proper RMW implementation:

ros2 daemon stop

First you’ve heard of the ROS2 daemon? Check out this ROS Answers post which contains the best description I’ve seen.

Debugging Issues

While FastDDS was mostly working out of the box, the whole service problem was wreaking havoc on setting/getting parameters – and I’ve been tuning parameters frequently. I went ahead and set the RMW_IMPLEMENTATION to rmw_cyclonedds_cpp, or so I thought.

I noticed that service discovery wasn’t much better. Then I noticed on the robot I had set RMW_IMPLEMTATION - so I fixed the spelling mistake. Now everything should totally work great!

Wrong.

On the robot, discovery worked fine and services worked great - but half or more of the nodes couldn’t be seen by my laptop. Restarting launch files resulted in different nodes often missing!

I started to debug and came across the ddsperf tool. If you’re using ROS2 on MacOSX you’ll want to check out this issue on how to install ddsperf.

Multiple Network Interfaces

Running ddsperf sanity gave an interesting warning on the robot:

ddsperf: using network interface enp3s0 (udp/10.42.0.1) selected arbitrarily from: enp3s0, wlp2s0

The UBR-1 has two network interfaces: wlp2s0 is a wifi connection to the outside world and enp3s0 is an internal ethernet port which only talks to the robot hardware. Apparently, my nodes were frequently using the wrong network interface. The upstream Cyclone DDS README does mention, way down the page, that “proper use of multiple network interfaces simultaneously will come, but is not there yet.”

The configuration guide states that the selection of network adapter prefers non-link-local interfaces, but apparently something is tripping it up in detecting that the ethernet interface is configured that way.

The work around is to set a NetworkInterfaceAddress in the CYCLONEDDS_URI environment variable:

export CYCLONEDDS_URI='<CycloneDDS><Domain><General><NetworkInterfaceAddress>wlp2s0</NetworkInterfaceAddress></General></Domain></CycloneDDS>'

If you’re prone to typos, and want to make sure you’re actually running the expected RMW interface, I’d recommend this command:

ros2 doctor --report | grep middleware

After a few seconds, you should see:

middleware name    : rmw_cyclonedds_cpp

I actually setup an alias in my bashrc so that which_rmw runs that command. Once I settled on using Cyclone DDS as my new default, I also added the RMW_IMPLEMENTATION and CYCLONEDDS_URI settings to the bashrc on the robot.

Final Thoughts

Once I worked through the configuration issue, CycloneDDS appears to be the most stable of the few RMW implementations I’ve tried. I haven’t actually tested the performance head-to-head, but others have.

I would recommend looking at the Configuration section of the upstream Eclipse CycloneDDS project. This contains a bunch of useful information about what you can specify in the CYCLONEDDS_URI. The Guide to Configuring is also very worth reading. It’s honestly a great resource for simply understanding all those things you hoped you’d never need to learn about DDS.