@christophebedard well, except for the fact that most of us will have wasted hours of frustration every time to find that information… :-/
I’m going to take 3 packages that I maintain as examples:
- a pure ‘msgs’ package, hri_msgs: ROS Package: hri_msgs
Last I heard we are still working on this. I thought I heard about some progress on this front but I can’t seem to find the resource saying as much. The good news is that CLI does pull up message-level documentation which does make life a lot easier, but we really do need to get message documentation out the door.
- a C++ library, hri: ROS Package: hri
- a Python library, pyhri: ROS Package: pyhri
A How-To guide is available here. I can confirm that it works for Python, however there are certainly a lot of rough edges that need to be sanded off and we could use a bit of help. Per-package documentation once generated is landing on index.ros.org and is also available here. I would like to find a way to have that page better organized and I have a few ideas on how to get there, but it will take some developer resources.
I’ll try to get a PR out today to cross reference this documentation page, with this How-To guide.
We’re bumping up against what I call the “library problem” right now, most of the resources developers want are on docs.ros.org, but getting the right resource in front of the right person at the right time is still a problem. For anything package related your best bet is to go through index.ros.org. If you have ideas for what would help we’re all ears (and reviews). If you are willing to be the guinea pig I am more than happy to fold what you learn getting your docs squared away back into docs.ros.org
1 Like
Some support for message type documentation and message-only packages went into rosdoc2 not that long ago via Support msgs, services, and actions by rkent · Pull Request #76 · ros-infrastructure/rosdoc2 · GitHub and Support only messages by rkent · Pull Request #78 · ros-infrastructure/rosdoc2 · GitHub (although i have not tried using that myself yet)
That functionality might not be documented yet.
Also add me to the list of people initially confused about the rosdoc2 installation. It’s easy enough to do and also nice that it’s python-only without specific ros dependencies, but then again i would have expected it to be a ros package, buildable with colcon and installable via apt install ros-DISTRO-rosdoc2
(which reminds me that colcon doc would be nice to have, i think i saw some ideas about that before 
)
4 Likes
I was hoping to get in a few more features, then do a complete documentation. One important feature that has still not landed is the use of jinja templates with the index.rst and conf.py files. That will have a big impact on how customization is done.
Right now, the focus has been on getting documentation added that works by default. The number of packages that attempt to customize rosdoc2 is pretty small. That of course might change if the documentation was clearer on how to do that, but really I think for most packages, you are better off to make sure that your package.xml is complete, and adding actual Doxygen or Sphinx documentation to your code.
The pace of change is mostly review limited at this point.
@rkent thanks again so much! I knew we had discussed the message-level documentation previously but I couldn’t recall your name. I’ll see if I can get you unblocked on PR reviews. I am not a maintainer for rosdoc2 but perhaps that should change sooner rather than latter.
@severin-lemaignan I put together a quick band-aid / bread crumb update to the The Developer Guide on docs.ros.org. After re-reading it, I feel like we might want to be more prescriptive in that section of the document but that’s something I would need to discuss with the other maintainers. If I can get @rkent’s pull requests merged in a timely manner would you be willing to be a guinea pig for the updates to rosdoc2? Once we run through the new process once I can write the docs for the new feature and add the resulting docs where appropriate.
1 Like
hi @Katherine_Scott & @rkent ! First, thanks a lot for the prompt response. Much appreciated!! @Katherine_Scott , I’ve commented on your doc PR → this change would certainly improve discoverability. Nice!
@rkent: any chance you could release rosdoc2 in humble/iron/jazzy/rolling?
and @Katherine_Scott: happy to be a guinea pig: we are in a big ‘documentation sprint’ right now at PAL, and that’s the right time for me to iron out these issues.
Regarding specifically the generation of hri_msgs documentation on index.ros.org: running rosdoc2 locally on my machine seems to be working fine. So maybe it is simply a matter of re-running the Jenkins job? (according to the jenkins job log, it was running rosdoc2-0.1.0 last time it tried) → is there a way to re-trigger a documentation job?
@rkent: one of the reason why the doc of pyhri is not generated might be thatit is a Python module entirely written with pybind (eg C++), and the resulting module is named hri, not pyhri. I don’t know if/how rosdoc2 supports this configuration.
@Katherine_Scott : on the broader question of how to improve docs.ros.org for general documentation. It is a tough one, obviously. What I’m missing the most at the moment is the ability to document not just one package, but a ‘functionality’ that can be spread out over several packages. That was quite easy and natural to do that on the ROS wiki (and I have the impression that many people are still using the ROS wiki with ROS2, for that purpose).
Maybe the main docs.ros.org is the place to do that (but, compared to the ROS wiki, it does feel a bit intimidating to submit PR to modify ‘the main docs site’ – should everything go there? only ‘important’ functionalities?..), maybe not.
For ROS4HRI, for instance, we host most/all the ROS2 documentation on the PAL documentation server: 👥 Social perception - PAL OS 24.9 documentation
That’s ok, but it is not ideal (not very discoverable, and gives the impression that it is a ‘PAL only’ framework, which is certainly not the intention).
…not sure what is the correct route forward!
The quick and dirty rule for ROS 2 package documentation is that, “package docs live with the package.”
What I’m missing the most at the moment is the ability to document not just one package, but a ‘functionality’ that can be spread out over several packages. That was quite easy and natural to do that on the ROS wiki (and I have the impression that many people are still using the ROS wiki with ROS2, for that purpose).
I would need to know more specifics here, but if you have a meta-package repo I would use that. Alternatively you can create a top-level readme pointing to your docs on one of the pages and have all the other package repos point back to that.
A third option, that larger projects like MoveIt and Nav2 have taken, is to just stand up your own ReadTheDocs page.
I am hoping to work on a REP this summer that should at least make it easier to find individuals package docs by adding tags to the package.xml file. That won’t exactly fix your problem but I hope it fixes the discoverability problem.
1 Like
@severin-lemaignan: I looked over your pyhri. Yes, python bindings written in C++ are not “python code” from the perspective of rosdoc2 and sphinx, so you are not going to get a python API from that. You should remove that from rosdoc2.yaml, really you don’t need rosdoc2.yaml.
Some specific suggestions that would give you a little more interesting rosdoc2 output:
- remove conf.py in doc/, this is not helping you. Just use the defaults.
- in package.xml, add a url which links to your external documentation. That will show up under “Links” on the main page:
<package>
...
<url type="documentation">https://docs.pal-robotics.com/sdk-dev/social-perception.html</url>
</package>
“documentation” is not an official url type for package.xml, but rosdoc2 does not care. (I don’t think the rest of ros2 cares either, but I could be wrong there.)
-
in your doc/ subdirectory, use an index.rst file that simply includes a link to your external documentation. Then you will also have a Documentation first-level item in your index. (Maybe the link under Links is enough though). You could also include other instructions there of how to access the actual documentation, if that is helpful.
-
You can’t use a relative reference to a parent document in your README if you want to show this in rosdoc2.
That is I agree a big issue and problem.
I came into working on rosdoc2 because I was trying to develop for myself a “reference” rosdoc2 package that would show best-practices for layout and features, including documentation, testing, github actions usage, etc. I found that rosdoc2 was very under-developed, so I started working on it.
But really for a set of interacting packages, you typically want the interface (messages) definitions in one package, and the nodes that use them in another. Best practice seems to be to include interacting packages in a single repository, and have the documentation at that level. But there is no concept of a repo-level document in rosdoc2. rosindex does a better job here, but only for the readme and repo metatdata. I’m not sure what the right approach would be,
Including rosdoc2 as a ROS2 package is probably a good idea, though the place to make that suggestion is under rosdoc2 issues.
Unfortunately installation is getting more difficult now that newer python really doesn’t allow --user packages anymore in for example Ubuntu 24. IIUC the choices are;
- use pipx instead of pip
- use a venv
- use the PIP_BREAK_SYSTEM_PACKAGES environment variable.
I don’t know what the ros2 maintainers intend to recommend, it looks to me though that PIP_BREAK_SYSTEM_PACKAGES is what is being used in ros infrastructure.
It looks like the actual documentation of rosdoc2 commands and usage are still lacking. @rkent is your intention to fill this out once you finish your set of pull requests? If that’s not the case, I might be able to string together a short update to the README along with a cross reference between docs.ros.org and rosdoc2 github repo.
The README is incomplete and in some cases simply wrong. I started to see if I could fix it yesterday, and I think it would be better if I did it at this point. That does not really need to wait for further landings.
I did get a little stuck on the install problem, which I mentioned in my previous reply. Do you know what the current recommendation is for pip install in ros generally? This issue is the disabling of user installs in python in Ubuntu noble, see for example @tfoote’s https://github.com/ros-infrastructure/rosdoc2/pull/96 as some of the issues.
I assume the recommendation is use a venv?
:rkent
The intention is that index.ros.org provides the documentation for a package, group of packages, or even a whole repository of packages. If it doesn’t yet do that, then I think the best approach would be to add that capability. Unfortunately I’ve no experience developing index.ros.org myself so I don’t know how much work that would be.
I did get a little stuck on the install problem, which I mentioned in my previous reply. Do you know what the current recommendation is for
pip installin ros generally? This issue is the disabling of user installs in python in Ubuntu noble, see for example @tfoote’s https://github.com/ros-infrastructure/rosdoc2/pull/96 as some of the issues.I assume the recommendation is use a venv?
@rkent I don’t have a definitive answer for you at this time, but I am going to check in with the infrastructure team on Monday morning to figure out what we’re currently doing on the build farm and what our plans are going forward with Noble. I’ll get back to you with an answer after we meet.
index.ros.org is written using Jekyll and Ruby, which is pretty far from the experience of most people interested in Robotics, who are typically working in C++ or Python. I initially avoided it for that reason, but I spent a little time understanding it enough to do a couple of patches. It’s doable, and I am willing to put some effort there as well as rosdoc2.
I’ve approached rosdoc2 with the understanding that it is package-level documentation, so we should provide in the rosdoc2-generated website useful information about a single ROS 2 package that can be gleaned from examination of the package code (meaning the package.xml directory and subdirectories, including metadata and included documentation). I believe that the current rosdoc2 is pretty much there for typical packages. What remains to be done as a minimum feature-wise is allowing customization, plus getting cross-references to work on the build farm.
I don’t yet though have a good feeling that I understand an overall plan to move the documentation of the universe of packages in the direction that I would find useful as a developer. rosdoc2 seemed to initially try to simply provide a comprehensive Doxygen and Sphinx documentation of C++ and python code, but that is not particularly useful for the vast majority of packages that do not provide the appropriate Doxygen or Sphinx markup of code usage. I’ve tried to expand that with package metadata and provided textual documentation, so it is at least somewhat useful for most packages. But as rosdoc2 features expand, at some point you start to overlap what is provided by rosindex. Plus I typically find that packages are too narrow of a focus, because features typically span multiple packages.
Is there a big-picture plan from the core team for documentation of the universe of packages?
I don’t think we should exactly do this. The thing is that rosdoc2 isn’t tied to a ROS release, so there is really no reason to have ros-jazzy-rosdoc2, ros-iron-rosdoc2, etc. What I think we should probably do instead is to have a single package released called python3-rosdoc2, and include that as part of ros-dev-tools. I can bring this up with the infrastructure team next week.
Not really, no. We have https://docs.ros.org for the “core” documentation, and we have rosdoc2 for generating package-level documentation. As mentioned above, we have been leaving “higher level” documentation to the individual sub-projects; Navigation2 and Moveit, for instance, have their own documentation sites.
That said, can you maybe point to some “groups of packages” that need documentation, that wouldn’t be covered by putting documentation in the repository? Even Navigation2, for instance, is entirely contained in one repository.
1 Like
If you are asking for examples of sub-projects that are not contained in one repository, there are many. ros-controls is one. Their project is hosted in a single github account, with multiple repositories.
But I don’t think that is what you are aking. I think you are asking for a clarification of my “I typically find that packages are too narrow of a focus, because features typically span multiple packages”
I’ve mostly focused on rosdoc2, mostly because it was severely under-developed. But when I tried to organize my documentation for a demo package, I was not sure where to put stuff. rosdoc2 did not work because info needed to discuss both the nodes as well as the messages, and those are in separate packages. Documentation probably belongs in the parent repository (or a separate repository) but there really is not a ROS-wide system for displaying that. The closest that we come is the repository README in rosindex.
Looking over the repo and package list on rosindex, it looks to me like ROS is a collection of, as you describe them, sub-projects. There is no easy way to navigate a list of the major sub-projects. Documentation on how to use sub-project features should probably be in the sub-project documentation, but that is not available in a common system within ROS.