Last updated July 17, 2018 at 9:33 am
Technology might make driverless cars drive better.
Picture: iStock
Australian scientists hope to improve how mobile robots and driverless cars operate and interact with people by helping them to see the world from a more human perspective.
In what it believes is a world first, a team from QUT in Queensland has used visual semantics to enable high-performance place recognition from opposing viewpoints.
As PhD student Sourav Garg explains, while humans can recognise a place when re-entering it from the opposite direction, even if there are extreme variations in its appearance, robots struggle to do the same.
“For example, if a person is driving down a road and they do a U-turn and head back down that same road, in the opposite direction, they have the ability to know where they are, based on that prior experience, because they recognise key aspects of the environment,” he said.
“People can also do that if they travel down the same road at night time, and then again during the day time, or during different seasons.
“Unfortunately, it’s not so straightforward for robots. Current engineered solutions, such as those used by driverless cars, largely rely on panoramic cameras or 360-degree Light Detection and Ranging sensing. While this is effective, it is very different to how humans naturally navigate.”
The solution proposed by Garg and Dr Niko Suenderhauf and Professor Michael Milford from QUT and the Australian Centre for Robotic Vision uses a state-of-the-art semantic segmentation network called RefineNet trained on the Cityscapes Dataset, to form a Local Semantic Tensor (LoST) descriptor of images.
Credit: neyro2008; Alexander Zelnitskiy; 123rf.com; Michael Milford
This is then used to perform place recognition along with additional robotic vision techniques based on spatial layout verification checks and weighted keypoint matching.
“We wanted to replicate the process used by humans,” Milford said. “Visual semantics works by not just sensing, but understanding where key objects are in the environment, and this allows for greater predictability in the actions that follow.
“Our approach enables us to match places from opposing viewpoints with little common visual overlap and across day-night cycles.”
The team is now extending its work to handle both opposing viewpoints and lateral viewpoint change, which occurs, for example, when a vehicle changes lanes – and adds an extra degree of difficulty.
Its paper has been accepted for publication in the international conference Robotics: Science and Systems. It can be downloaded from ArXiV.
