Google Is Now Using Gemini AI To Train Robots 'Navigate The World': Here's How

Google has joined the trend with Gemini designing robots to understand surroundings, handle complex tasks, and remember information. While these developments may not yet equate to having a personal assistant, we are getting closer to truly useful robot helpers for everyday use. Recently, Google’s DeepMind team demonstrated how Gemini1.5 enables robots to record important locations and navigate seamlessly in real-world scenarios.

In the video shared on Instagram, a team member showed how a robot took them to a whiteboard when asked to show the place where they could draw. After the command, the robot could be heard saying, “Okay, thinking with Gemini. Please give me a minute.” While the experiment looks promising, there is a noticeable delay of up to a minute between the robot receiving a request and taking action. Despite this, Google's project offers a sneak peek into how these robots might function in our homes shortly and offices in the near future.

Sharing the intriguing clip, the team explained, “With help from Gemini1.5 Pro’s long context window, we challenged our helper robots to navigate their way around our busy office.”

To train the robot, the DeepMind team took the machine through various areas and showed important locations and objects. The robot then creates a mental map to remember these places and items for later use. Although it’s still in the early stages, the Gemini robot could offer even more precise details in the future.

According to a research paper published by DeepMind, the robot showed a 90 per cent success rate on over 50 user instructions within a 9,000-square-foot area. The team also found out thatGemini1.5 Pro allows the robot to plan actions other than simple navigation.

As highlighted in the paper, if a user, who has multiple cans of Coke in their desk, asks if their favorite drink is available in the kitchen, the Gemini“knows that the robot should navigate to the fridge, inspect if there are Cokes, and then return to the user to report the result.” Meanwhile, the team plans to explore these capabilities further.