What to take into account
There are several things to keep in mind when watching the surge of robot demo videos and even live broadcasts. First, such robotic demonstrations are not necessarily indicative of the robots operating autonomously without human control or supervision, he said. Dipam PatelPhD candidate in computer science at Purdue University and research assistant at the US Army DevCom Army Research Laboratory. Many demonstrations still rely on human operators directly controlling the robots’ actions through teleoperation.
“Unless a research paper or a company explicitly mentions that (the robot) is completely autonomous, you have to take it with a pinch of salt,” Patel told Ars.
Another question to consider is whether the demo shows robots tackling a completely new test environment for the first time, or whether the robots are simply repeating a task they had already learned to perform in that specific training environment. The new test environment would be significantly more impressive by showcasing robots capable of autonomously performing tasks on a widespread basis, Patel said.
It’s also worth checking the video playback speed for any robot demonstration, because “normally robots are very slow” for safety and other reasons, Patel said. Companies may sometimes reveal that a robot demonstration video is running at two or four times the normal speed, meaning the robot could take two or four times as long as a human to perform the same task.
Robot demo videos can also vary greatly in their informational value and transparency. Some are clearly intended to be performative entertainment clips that can go viral on social media, or polished promotional videos from companies looking for new customers and investors. Others can offer a more behind-the-scenes look at the robot training process while acknowledging the robot’s mistakes along the way.
But even if a robot demo video looks incredibly impressive and authentic even though it comes from a more reputable company or research lab, keep in mind that it’s still a small glimpse of the bigger picture. Actual indicators of progress in robotic capabilities are not so easy to present to the Internet public.
