ARIAC code release updates

ARIAC users
1

The software for competing in ARIAC 2018 has been released.

See the ARIAC wiki for competition documentation, installation instructions, and tutorials for working with the software.

2

New ARIAC software release (excerpt from https://bitbucket.org/osrf/ariac/wiki/2018/updates):

Version 2.0.3 (28 February 2018)

  1. Additional sample trial config files released. See the trial configuration tutorial for how to launch the trials. Summary of new trial config files (more details in the files themselves):
    1. sample_interruption1.yaml: high priority order interruption at a “convenient” time.
    2. sample_interruption2.yaml: high priority order interruption at an “inconvenient” time.
    3. sample_flipped.yaml: order requiring products to be flipped (see ‘Flipped products’ in the frame specifications).
    4. sample_not_enough_products.yaml: order requiring more products than available for use.
3

First qualifier has been released (excerpt from https://bitbucket.org/osrf/ariac/wiki/2018/updates):

Changes to ARIAC software

Version 2.0.4 (5 March 2018, Qualification Task 1 release)

  1. Qualification task 1 Part A trial config file released. See the first qualifier page for details.
  2. More environment collisions. The arm can no longer pass through the support frame around the storage bins. This was advised as an upcoming change on 17 February.
  3. Reduced logical camera FOV. In response to the additional workcell collisions, the logical camera range has been reduced.
  4. Segfault on competition start fixed. This would happen if the /ariac/start_competition service was called before the simulation had finished loading.

Changes to ARIAC documentation

5 March 2018

  1. First qualifier announced. See this page for details.
  2. Logging/log files documented. See the logging page for how to enable/playback simulation state logging for debug purposes.
  3. Update policy extended. The update policy now covers rules/scoring metrics in addition to software changes.

Upcoming changes

Announced 5 March 2018

  1. State logging fixes. Fixes for the simulation state logging (low RTF during recording, no arm inserted during playback) will be resolved in the next Gazebo release.
  2. Scoring of ambiguous yaw. The pulley_part product’s yaw is not distinguishable with perception sensors, but its yaw is evaluated by the scoring algorithm. This will be corrected for, either by making the yaw perceivable or by having the scoring algorithm ignore the yaw.
4

Changes to ARIAC documentation

7 March 2018

  1. Qualification task 1 Part B limitations clarified. In particular, info on the storage bins, quality control sensors, potential products, and potential order count. See this page for the added details.
  2. Static sensors only. The environment configuration tutorial/competition specifications have been updated to clarify that sensors can only be placed in static locations around the environment, they cannot be attached to the arm.

Upcoming changes

Announced 7 March 2018

  1. Docker images using gazebo8. Docker images used for the automated evaluation setup are currently using gazebo7.7 to avoid state log recording issues, but will be swapped to the next gazebo version when it is released (within the next week).
  2. State logging playback freeze fix. A fix for the simulation state log playback freezing with gazebo8.3 will be resolved in the next Gazebo release. This impacts teams looking to playback state logs from the automated evaluation setup. See these instructions for how to install gazebo8.2 as a temporary fix.
  3. /ariac/material_locations service classified as a ‘cheat’. After the first qualification task closes the /ariac/material_locations service will become classified as a cheat. Teams will need to use sensors to infer the products available in storage bins.
  4. Model naming system change. In ariac2.0.4, models in the environment of a particular type have sequential IDs. The naming system will be modified so that model IDs are randomized. As a result, logical cameras/quality control sensors will not publish TF frames, which include model IDs, correlated to the number of models in the environment.
  5. Conveyor controllable only after competition has started. In ariac2.0.4, the conveyor can be controlled before the competition has been started. This functionality will be removed and must not be exploited by teams.
5

Changes to ARIAC software

With each ARIAC software release, a new version of the automated evaluation setup will be released that uses the latest ARIAC version. The latest release will be used for evaluation of qualification task 1. See these instructions for how to update.

Version 2.0.5 (8 March 2018)

  1. Product IDs randomized. For all future trial config files released (including qual1b), products in the storage bins will no longer have sequential IDs.
  2. Updated environment collisions. Fixed misalignment of the vertical bars in the support frame around the storage bins.
  3. pulley_part model updated. It is now asymmetric on top and bottom. Pulleys (and all other products) will always start un-flipped in the storage bins.
6

Important note: Submissions for the first qualifier will be uploaded via secure online workspaces. All registered teams must contact ariac@nist.gov to have their workspace prepared in advance of when they intend to submit. If you are planning a submission for the first qualifier and do not yet have a secure workspace, you must contact ariac@nist.gov immediately or you risk missing the submission deadline.

Changes to ARIAC software

Version 2.0.6 (14 March 2018)

  1. Option to use joint-limited UR10. By default the UR10 has joint limits of [-2*PI, 2*PI]; there is now an option to use joint limits of [-PI, PI], which provides better controller performance in some situations. It can be enabled by adding joint_limited_ur10: true to your team’s config file similarly to this. Participants that do not wish to enable this option do not need to make any changes to their system. The use of this option is permitted in submissions for the first qualifier.
  2. New Gazebo release. The ariac2-server Docker image used in automated evaluation setup has been updated to use gazebo8.4 now that the logging issues have been resolved.

Changes to ARIAC documentation

14 March 2018

  1. Submission process for the first qualifier updated. Teams must have had a secure workspace created by competition controllers before they can upload their submissions for the first qualifier.
  2. Scoring metrics released. See this page. Note that these may change between rounds of the competition.
  3. Qualifying guidelines for the first qualifier updated. Clarified that only automated evaluation metrics will be used for the first qualifier and that any participants found violating submission guidelines (e.g. by starting the conveyor belt before starting the competition) will not be permitted to qualify for the Finals.
  4. ROS API clarifications. /ariac/end_competition, /ariac/competition_state, /ariac/drone/state, /ariac/current_score, /ariac/drone topics/services documentation added/updated in the competition interface documentation.
    1. Clarified that requesting that the drone deliver any unknown shipment type will cause the drone to remove the box without scoring it.

Upcoming changes

Announced 14 March 2018

  1. /ariac/current_score topic classified as a ‘cheat’.

    After the first qualification task closes the /ariac/current_score topic will become classified as a cheat.

7

Changes to ARIAC documentation

18 March 2018

  1. Re-run policy clarified. Re-runs for the first qualifier will be permitted only in the event of simulation bugs such as reported issues #115 and #116, at the discretion of the competition controllers. Any issues caused by competitors’ code does not warrant a re-run.
  2. Enabling state logging in qual1a.yaml. The qual1 scenario description page has been updated to clarify that simulation state logging will be enabled in the first qualifier trials via the gazebo_state_logging: true setting in the the trial config file. If the config file that you are testing with in the automated evaluation setup does not have logging explicitly enabled like this, please add that line to the config file to enable logging.
8

Changes to ARIAC software

Version 2.1.0 (21 March 2018)

  1. UR10 replaced by the Kuka IIWA14. The IIWA14 will be used in all future trials; the UR10 will no longer be used. The IIWA14 presents the same ROS interface but participants will need to update their system to adapt to the new arm. This was announced as an upcoming change on 17 February 2018.

  2. Updated environment. The workcell environment, particularly the shipping container, shelving and the conveyor belt, has been updated to accommodate the working area of the IIWA14. Teams will need to re-position their sensors. The dimensions of the shipping boxes, storage bins, and products have not been modified.

Changes to ARIAC documentation

21 March 2018

  1. Tutorials updated for IIWA14. Tutorials involving control of the robot arm have been updated for the switch to the IIWA14.
9

Changes to ARIAC software

Version 2.1.1 (27 March 2018)

  1. Example of the dropped product challenge released. Read/use sample_dropped_products.yaml for practice.
  2. Example of the order update challenge released. Read/use sample_order_update.yaml for practice.
  3. qual1b.yaml released. The config file that was used for evaluating Part B of the first qualifier is in config/quals/qual1b.yaml. Congratulations to the teams that qualified for the Finals.
  4. Information about product types not from sensors removed.
    1. The /ariac/material_locations service and the /ariac/current_score topic have been re-classified as cheats.
    2. The quality control sensors now publish anonymized model names instead of the types of the faulty products that are detected. As a side effect, the model numbering scheme has been updated; any randomized faulty product IDs in custom-defined trial configs will need to be updated.
  5. Devel space support. Use of the devel space is now supported for users building from source (contributed by @IanTheEngineer).

Changes to ARIAC documentation

27 March 2018

  1. Sample trial configs highlighted. The configuration file tutorial now highlights that sample config files are available for practicing with.
10

Changes to ARIAC software

Version 2.1.2 (30 March 2018)

  1. Example of the sensor blackout challenge released. Read/use sample_sensor_blackout.yaml for practice. Note that re-connecting to some sensors during development will cause them to resume publishing data, but this functionality is blocked in the automated evaluation setup.
  2. ROS interface to the congestion_sensor removed. This is the break beam positioned at the end of the conveyor belt by default.
  3. Conveyor control blocked before competition start. Service calls to set the conveyor belt power before the competition has been started will fail.

Changes to ARIAC documentation

30 March 2018

  1. Sensor placement rules clarified. The competition specifications now detail that sensors can be placed in any free space in the workcell, they do not need to be mounted such that they are touching the conveyor belt/support frame of the storage bin. Sensors must be used in a realistic manner and must not exploit any simulation technicalities such as the logical camera seeing through obstructions.
11

Changes to ARIAC software

Version 2.1.3 (2 April 2018, Qualification Task 2 release)

  1. Qualification task 2 Part A trial config file released. See the second qualifier page for details. Teams that did not participate in the first qualification task, and teams that participated but did not qualify, are still eligible to participate in the second qualification round.
12

Changes to ARIAC software

Version 2.1.4 (4 April 2018)

  1. Fixed bug with box being spawned on competition start. This issue, which would occur occasionally for some participants, should now be resolved.
  2. Updated shipping box collisions. The external walls of shipping boxes have been widened to make this issue, where boxes would fall into the conveyor belt, less likely to occur. The internal dimensions of the boxes are unchanged.

Changes to ARIAC documentation

4 April 2018

  1. Gazebo version 8.4 required. The system requirements have been updated to highlight that Gazebo 8.4 is required to avoid issues with state logging recording/playback. If you are experiencing slow simulations with state logging enabled, or freezes when you try to playback state logs, please upgrade your gazebo installation.
  2. Agility challenge page added. This page has been added, which summarizes the released agility challenges, including additional details on the “sensor blackout” challenge.
13

Upcoming changes

Proposed 10 April 2018

  1. IIWA controller updates.

    The controller has been updated to avoid an issue with joint 7 getting stuck, and to relax tolerances on the trajectory controller (see this issue for details).

    To test out the proposed changes, add the ARIAC pre-release repository with:

    sudo sh -c 'echo "deb http://packages.osrfoundation.org/gazebo/ubuntu-prerelease `lsb_release -cs` main" > /etc/apt/sources.list.d/gazebo-prerelease.list'

    and then run sudo apt-get update && sudo apt-get install ariac2. Your ARIAC version will be updated to 2.1.5, which will become the standard version if no regressions are reported. Please report regressions on the issue linked above.

    When you are finished testing we suggest you remove the /etc/apt/sources.list.d/gazebo-prerelease.list added in the above command.

14

Version 2.1.5 (11 April 2018)

  1. IIWA14 model/controller modifications. The arm simulation model/controller has been updated to avoid an issue with joint 7 getting stuck/causing instability, and to relax tolerances on the trajectory controller (see this issue for details). Participants should update their version to test the changes before submitting their qual2 submission, as the most recently released ARIAC version will be used for the qualifier evaluation.

Changes to ARIAC documentation

11 April 2018

  1. Product scoring limitation. The scoring metrics page now clarifies that products must be placed onto the base of the shipping box to be counted for scoring, not on top of other products.
15

Changes to ARIAC software

Version 2.1.6 (12 April 2018)

  1. Rollback to 2.1.5~pre1. Two “pre-release” versions of 2.1.5 were released to users for testing prior to yesterday’s ariac2.1.5 release to improve IIWA14 controller performance: 2.1.5~pre1 and 2.1.5~pre2. The version released yesterday was 2.1.5~pre2; ariac2.1.6 rolls back to 2.1.5~pre1 based on feedback from participants that it provides better arm performance.

Changes to ARIAC documentation

12 April 2018

  1. Trial time limit clarification. As mentioned in the details of qual2 scenarios, 500 sim seconds will be used for qual2b. The competition specifications now clarify that this information is not broadcast by the ARIAC server. For the Finals, as with the Qualifiers, the time limit will be set as a fixed value for all trials, which teams will know in advance. There are no time limits for individual orders.
16

ARIAC competition software updates

Version 2.1.7 (19 April 2018)

  1. qual2b.yaml released. The config file that was used for evaluating Part B of the second qualifier is in config/quals/qual2b.yaml.
  2. Conveyor belt simulation model update. The simulation model of the conveyor belt has been updated to make this issue, where boxes fall into the conveyor belt, less likely to occur. There should be no other observable impact to users.

Changes to ARIAC documentation

19 April 2018

  1. Finals schedule released. The Finals wiki page has been released, including details on what to expect in the Finals, dry-run testing on the competition machines, and the schedule for the process.
  2. Automated evaluation results structure outlined. Structure of the logs directory output by the automated evaluation setup is now described in the readme.
17

ARIAC competition software updates

Changes to ARIAC software

Version 2.1.8 (21 May 2018)

  1. Finals scenarios released. The config files used for the Finals of the competition are in the config/finals directory. This page gives an overview of each trial. Stay tuned for an announcement about the outcome of the ARIAC 2018 Finals!
  2. Simulation state logs recorded at lower frequency. (This change was included in the version of ARIAC used for evaluating the Finals).
  3. Fix state log playback of shipping boxes. Closing/collection of shipping boxes during log playback now works correctly during state log playback.
18

The software for competing in ARIAC 2019 (Version 3.0.1) has been released!

Please see the installation wiki page for instructions as they have changed, especially if you have installed a prerelease. This version is feature complete. Follow this thread to get notified of future bug fix releases.

Other links

19

The software for competing in ARIAC 2019 (Version 3.0.2) has been released!
Please see the installation page and software updates page for more information.

This release contains the trial config files for the Qualifier.
Please see the qualifier page to learn how qualification works and the qualifier scenarios page to learn how to practice on the Part A trial configs.
Qualifier submissions will be accepted until April 14th.

Other links

20

The end date of the 14th conflicts with those on the Challenge.gov site (https://challenge.gov/a/buzz/challenge/999/ideas/top):

Dates of the Competition

Qualifier Begin: April 15, 2019
Qualifier End: April 19, 2019
Competition Testing (Finals) Period Begins: May 13, 2019
Competition Testing (Finals) Period Ends: 5:00 PM, May 17, 2019
Announcement of Cash Prize Winners: May 21, 2019