Just to put one more variable in the discussion, building with Release or RelWithDebInfo mode disables the asserts in most code (due to the -DNDEBUG flag), so I am a bit against making either of those the default.
I agree that uninformed people just compile using catkin_make/ament and find that the code runs slower than expected so maybe better documentation about that would be good, maybe even show a warning at the start of build that no optimizations are enabled by default.