Remote Robotic Control with 360° VR-based Telepresence

Experimentation at the 5G-VINNI UK Facility

This article describes a project to demonstrate the use of two simultaneous paths through the 5G-VINNI network to undertake remote robotic control, while at the same time providing the robot’s controller with a 360° view from the robot’s cameras, viewed through a virtual reality headset.

In February 2020, BT Applied Research and Interdigital Digital Labs collaborated on an experiment using one of Interdigital’s turtlebot robots. The robot can move around on a flat surface as determined by the robot’s controller who wears a VR headset, and who controls the robot’s movement with a game controller. The robot is able to move forwards/backwards, left/right as well as being able to spin clockwise/anti-clockwise.

Use of the 5G-VINNI network

Connections between the robot and its camera, and between the robot controller’s headset and movement controller, were made over the 5G-VINNI UK facility at Adastral Park, Suffolk, UK. The experiment makes use of the 5G 28GHz mmWave fixed-wireless access radio system deployed at the facility. This mmWave radio system provides up to 1Gbit/s (download) connectivity to a set of indoor CPEs which then present an Ethernet interface to a laptop PC (which connects to the game controller and VR headset) and to a WiFi access point (used to provide radio connectivity to the turtlebot).

The experiment, shown in Figure 1, demonstrates the use of two simultaneous paths through the end-to-end 5G network:

• The first is a low-latency path, used to control the robot’s movement. The low latency provides closed loop robotic control and ensures that the movement of the robot takes place without any lag, that is, when the controller tells the robot to move in a certain way, the robot does so immediately.
• The second is a high bandwidth path, used to stream the video content from the robot’s 360° camera to the VR headset worn by the robot’s controller. This gives the user an immersive video experience in which the high bandwidth ensures that the video is received by the headset immediately, enabling the controller to see the movement in real time.

Using this configuration, we were able to safely demonstrate full control of the robot’s movement, including successfully negotiating routes around obstacles placed in the robot’s path as shown in Figure 2.

The demonstration is the first of its kind at the UK facility and has demonstrated the low latency and high bandwidth capabilities of 5G to support a high performance use case of this type. Further experiments are planned which will build upon this use case, using the robot’s camera to perform resource recognition and an artificial intelligence system to help determine the robot’s actions.

References

A youtube video of the experiment can be seen on the 5G-VINNI channel at https://www.youtube.com/watch?v=91yIbv8aNL4

Conclusion/Outlook

The experiment described above demonstrates the way in which 5G can support the two high performance requirements of low latency and high bandwidth at the same time. The ability to undertake remote robotic control while simultaneously streaming 360° video illustrates the ability of 5G to deliver services which other cellular radio systems (for example 4G) would struggle with. Use cases such as this have a number of real-world applications, particularly for Industry 4.0, where high performance robotic control has the potential to transform the work environment.

The experiment is the first of a number of collaborations planned between BT and Interdigital, using the 5G-VINNI network and aiming to demonstrate the power of the network to support high performance 5G use cases and applications.

Article by

Paul Muschamp
BT Applied Research
paul.muschamp@bt.com