Showing 'age' of ROS wiki content

Quality Assurance
1

I’d like to post a request-for-comments for the following PR against ros-infrastructure/roswiki:

Our very active member @max-krichenbauer has submitted this PR following a suggestion I made to the SQWG some time ago to see whether we could improve the situation regarding outdated content on the ROS wiki.

This suggestion was based on my personal experience with questions on ROS Answers posted by (new) users about unclear or incomplete (or sometimes even incorrect) instructions or information on ROS wiki pages, combined with the results of one of the earlier community-oriented studies performed in the ROSIN project (which showed that newcomers are particularly frustrated by the fact that it’s almost impossible to figure out whether a source of information is still relevant (be it a wiki page, tutorial or blog post)).

As a first approach, the PR implements an extension to the wiki which shows a notice at the top of the wiki page whenever the last-edited property is older than some configurable threshold. Like so:

799×335 37.8 KB

This is of course just an example.

As this is just simple thresholding, it’s not too sophisticated, and one of the comments by @dirk-thomas was very valid (from here):

I am pretty sure there are numerous pages which haven’t been touched for years but their content is still as valid / relevant / up-to-date as it needs to be. And also some pages might have recently been touched for minor changes but still contain outdated content

The main problem is of course the fact that to do this properly (ie: determine whether content is still relevant), understanding of both the content and the state-of-the-art would be necessary (something not very easily captured in 25 lines of Python). In essence, this would require humans to be part of the process.

While it would be great if we could all keep the ROS wiki up-to-date, the past years have shown this to not happen. Including humans all the time seems infeasible.

The idea however of this simple PR would be to add at least some indication of potentially outdated information, under the assumption that with a project as active as ROS, the fact that a wiki page hasn’t been edited in (say) 2 years does indeed mean there is a relatively high chance that at least some things may not work any more as they are described or documented.

Such a notice does of course not really solve the problem (as for that a human would need to update the page and correct any issues), but instead of not showing anything (by which we implicitly state the page as-is documents the current state of things and everything should just work), we at least acknowledge the fact that things could have changed and we warn the reader that he should be aware of this.

I’d like to ask people here to comment on the PR and voice their opinions.

It would be great if we could figure out some solution, especially also because of the ROS 1 → ROS 2 migration, which will probably mean we’ll see more-and-more content becoming stale in the near future.

2 Likes
2

“Verified for melodic” badges could also be a way, I think ubuntu wiki uses those? Can and probably should be combined with the age warning. This would also allow for a very legitimate reason to “reset” the age.

Another suggestion: is it possible to search for old/outdated pages? Possibly weighted by relevance (hit counter)? This would allow highly motivated people to quickly find the worst offenders.

3 Likes
3

How about doing exactly what the title of this post says: “Showing ‘age’ of ROS wiki content”? But with no implication that a certain age automatically means “outdated”. Then the reader is aware how old the information is and can make their own decision as on any other website on the internet.

I disagree with that statement and if necessary can easily come up with numerous pages which clearly disprove this heuristic. Whatever the chances for being outdated are - marking pages which are not outdated with such a note will make readers even more confused imo.

1 Like
4

Thank you all very much for your comments!

I understand that a clear statement (“this page is outdated”) cannot be made from the last-edit date.
But if we only display the date without any context, I’m not really sure if that helps the reader in any way. It’ll just take up real estate on the page.

So my suggestion is to change the text to:

"This package documentation has not been updated since XX-XX-XXXX.
Please contact the maintainer to check if it is still up-to-date".

My reasoning is this:

(1) This PR only applies only to package pages, not to tutorials or general ROS information.

(2) If a package is still being maintained, that means that the maintainer is still looking at it at least once a year, even if nothing changes. Usually there should be at least a minor version update once a year to adjust to changing versions of tools etc. or at least answer the questions asked by users.

(3) If the package is updated or at least checked, it would be “standard operating procedure” to also check and update the wiki page. At least to increase the version number or add another user’s question to the “FAQ”.

(4) Not keeping the version in the wiki up-to-date or not sharing solutions to user’s common problems is not so good. The maintainer should receive a friendly reminder.

(5) Automated reminders (if they are even possible in the wiki system) are likely to be ignored. We love robots but in the end, we value human interactions higher. So a message from a human will have more effect. Doubly so because it is someone genuinely interested in the package. So the maintainer will be motivated as he sees that his package is still being actively used.

(6) Sorry if I offend people, but in my humble opinion a maintainer who does not at least once a year spend 5 minutes to check on his documentation, and who does not want to be contacted by users asking about his package should not be considered a “maintainer”. If he’s too busy (understandably) then he should find someone else or marked the project “abandoned”, because matter-of-fact it is.

I’m new to this community so please excuse if my views don’t reflect reality or offend anyone.
Please let me know what you think so I can learn better how to help improve the situation.

1 Like
5

I’m not convinced this is a good idea. The date of the last edit is a poor indicator of how complete and relevant a wiki page is. There are plenty of packages that are still maintained and work fine but are not under active development (i.e., the ROS API is stable), so there is no need to update the wiki page. On the other hand, there are also plenty of new packages with shoddy documentation. As a maintainer, I’d prefer to get emails (or better: GitHub issues) pointing to specific flaws in the documentation than just “hey, I got this warning message telling me to ask you if the documentation is still up-to-date. Is it?”.

Also, by my guesstimate, for every 10 users that run into a problem, there’s at most 1 who actually contacts the maintainer. A warning like this would probably just create the impression that 90% of the documentation on the wiki is outdated, and newbies would be more confused but not feel confident enough to actually contact the maintainer, so the maintainer wouldn’t even be aware of this problem.

Welcome to the community and thanks for making it better! I don’t think anyone will be offended by suggestions (and actual pull requests) for improvements.

As an aside: You are absolutely correct that there is often a mismatch between the documentation on the wiki and the actual code. In recent years, I’ve moved most documentation of my packages from the wiki to the GitHub README. This makes it easier to actually keep documentation and code in sync.

6

I really like both of these suggestions. Especially the “Verified for melodic” badges would provide a great motivation to completionists like me to actually go over every wiki page, check it and add the badge!

1 Like
7

As a maintainer, I’d prefer to get emails (or better: GitHub issues) pointing to specific flaws in the documentation than just “hey, I got this warning message telling me to ask you if the documentation is still up-to-date. Is it?”.

I understand that. Then maybe a wording like “If you run into problems following this document, please consider contacting the maintainer.”, so people understand that it may be outdated but won’t actually send an email until they actually have problems.

About the search for old/outdated pages, I don’t understand the logic.
If you say that being old doesn’t mean it’s outdated, then why are they considered “worst offenders”?
Now instead of warning the actual users of the package (who have an interest and maybe some insight), you have highly motivated zealots who contact maintainers because their pages are outdated.

8

I wasn’t aware that this discussion only applies to “package-related” wiki pages and was thinking more along the up-keeping of general information that does not have a clear maintainer.

9

I like that wording better than the previous one.

About the search for old/outdated pages , I don’t understand the logic.
If you say that being old doesn’t mean it’s outdated, then why are they considered “worst offenders”?

Ha, you got me there. :grinning: Actually this was a quote from @simonschmeisser’s post. I read it in the context of his badge proposal, to help people who want to update the wiki find the best candidates to check.

To expand a bit on the badge idea - I couldn’t find anything like this on the English Ubuntu wiki, but the German ubuntuusers wiki has a box at the very top of each page that translates to “This article has been tested on the following Ubuntu versions: * Ubuntu 19.10Eoan Ermine, …”. For example:

https://wiki.ubuntuusers.de/GIMP/

In my opinion, this conveys much more useful information to the user than a warning based purely on the last edit date.

They also have a meta page that automatically lists all untested articles: ungetestet › Wiki › Wiki › ubuntuusers.de (where “untested” means that it’s not marked as tested on any of the currently supported Ubuntu versions (16.04, 18.04, 19.04, 19.10)). This is to encourage users to find articles in need of testing and mark them as “tested” where appropriate, or update them.

2 Likes
10

@Martin_Guenther
Thank you for the input.
I’ve changed the message accordingly, but for the badges I feel this would be a different PR.
If you can gather enough support to get such a PR reviewed and accepted, I can work on implementing.
What do you think?

11

@max-krichenbauer: Since you specifically asked me for my opinion: It hasn’t changed. I still believe that this PR does more harm than good - having warnings pop up on most of the wiki pages based on an arbitrary metric will only confuse users without providing much benefit. From the comments there and on the GitHub PR, several others seem to share my concerns. Sorry for being so blunt - I’m grateful that you are working on making the wiki better!

I’m not sure how I would go about gathering support. I’ve already stated my support, and @simonschmeisser was the one that suggested the badges, so I’d count him in the “pro-badge” team as well. :slight_smile: Maybe some of the others that have commented here and on the GitHub PR could state their opinion on the badge idea? (@gavanderhoorn, @dirk-thomas, @tfoote)

12

@Martin_Guenther

No worries, thanks for the straight answer.
I don’t know if it’s a good or bad idea myself. I just implemented it because as part of my work for the ROS QA working group, where @gavanderhoorn suggested it and the other members supported it.

What I meant by support is if you can (if the implementation is deemed acceptable) merge it or can lobby for the merge.
I already have a couple of open PRs that are just being ignored (not rejected, just ignored), so this time I’d like to know before starting to work on it. :wink:

13

I can’t merge it, since I’m not a maintainer of ros-infrastructure.

That sounds like a good idea, so let’s have the discussion before the implementation. Since the proposed change affects all ROS users, it would be good to have a thorough discussion anyway. Maybe you should create an issue for this on the ros-infrastructure GitHub repo and post the link here to invite comments from the community?

14

Maybe you should create an issue for this on the ros-infrastructure GitHub repo and post the link here to invite comments from the community?

I would greatly prefer someone else can do it. It’s not my idea, I don’t understand the community around it, and it’s not up to me to decide if this is “official ROS QA workgroup” work.
So if @simonschmeisser or @gavanderhoorn or @Alami or you could start the conversation, that would be much appreciated.

1 Like