Recommendations for Survey Makers

General
1

Hi All,

There has been a proliferation of surveys in the past few weeks. I get it, COVID has made it very difficult to meet with your customers and users to discuss their needs, wants, and desires. I want to remind everyone that ROS Discourse is intended as a place for ROS discussion and not just a captive audience for data collection. Moreover, a poorly designed and executed survey can be heavily biased and deliver data that isn’t particularly useful. I would caution anyone who thinks the ROS Discourse user base is a representative sample of the ROS community. While I want to help everyone get the data they need, I am beginning to feel like surveys are being used as a replacement for having actual community discussion. As engineers, most of us thrive on data and shy away form conversation; however biased data collection is no more error prone than summarizing nuanced user stories and narratives brought about from discussion. In terms of analysis, user stories are often much more practical than trying to interpret highly biased data. From a product management perspective narrative user stories are much more useful for shaping the functionality of a piece of software.

For these reasons I want to make a few recommendations for those who wish to perform a survey:

  1. Start with a community discussion before you perform a survey. I have seen multiple surveys where the comments end up being, “this survey doesn’t apply to my situation” or, “I don’t understand this question.” You should know and talk to your intended audience before you survey them. Asking broader discussion questions on ROS discourse is perfectly fine and highly encouraged. This has the side effect of potentially bringing in more community members who may have similar, but slightly different questions. You may find that there are other groups who are interested in the same topic as you are, and could help to build a better survey.

  2. Don’t use a survey when a poll is just as good! Discourse has a poll feature! Feel free to use it. If you have just a few questions this approach is preferred. If you have insufficient karma to run a poll let me know and I can help. This feature is nice because it keeps all the data in-line with the page, and is generally pretty open.

  3. Feel free to message me with your survey prior to posting it. I am more than happy to review your work, help with analysis, and broaden your audience. I would prefer to do these things for well constructed and reviewed surveys.

10 Likes