BMW Group has showcased its new revolutionary driver functionality, called BMW Natural Interaction, at the Mobile World Congress 2019 in Barcelona. The new system combines the most advanced voice command technology available with expanded gesture control and gaze recognition to enable genuine multimodal operation for the first time. The first BMW Natural Interaction functions will be available in the BMW iNEXT from 2021. 

BMW Natural Interaction represents a paradigm shift for the driving experience, allowing drivers to use their voice, gestures and even their gaze at the same time in various combinations to interact with their vehicle.

The preferred mode of operation can be selected intuitively, according to the situation and context. Voice commands, gestures and the direction of gaze can be reliably detected by the vehicle, combined and the desired operation executed.

Spoken instructions, which are registered and processed using Natural Language Understanding, and gesture controls are constantly being refined and interpreted so that a multimodal experience can be created perfectly around the driver’s wishes. The driver can also decide how they want to interact, based on their own personal preferences, habits or the current situation.  

INTERACTING WITH THE WIDER WORLD.

By combining enhanced gesture recognition with the car’s high level of connectivity, the interaction space is no longer confined to the interior. For the first time, people in the car will be able to interact with their direct surroundings, such as buildings or parking spaces. Even complex queries can be answered; like pointing a finger and saying. “What’s this building? How long is that business open? What is this restaurant called? Can I park here and what does it cost?”

“Customers should be able to communicate with their intelligent connected vehicle in a totally natural way,” explains Christoph Grote, Senior Vice President, BMW Group Electronics. “People shouldn’t have to think about which operating strategy to use to get what they want. They should always be able to decide freely – and the car should still understand them. BMW Natural Interaction is also an important step for the future of autonomous vehicles, when interior concepts will no longer be geared solely towards the driver’s position and occupants will have more freedom.”

ENHANCED SENSORS AND ARTIFICIAL INTELLIGENCE.

What makes all this possible are the improved sensor and analysis technologies. Using an infrared light signal, the gesture camera can now capture hand and finger movements in three dimensions. The high-definition camera integrated into the instrument cluster also registers head and eye direction. All this complex  information is combined and evaluated with the help of artificial intelligence. The algorithm responsible for interpreting the data in-car is continuously optimised and refined using machine-learning and evaluation of different operating scenarios.

Find out more about the BMW iNEXT here.