Amazon Wants Its Home Robot to Anticipate Your Every Need

Jeff Bezos has wanted a home robot for a long time. By 2017, Amazon’s founder, an avid sci-fi fan , had repeatedly asked the company’s engineers and executives about the feasibility of such a project, says Ken Kiraly, a vice president who helped create the Kindle ebook reader. That was the year Amazon’s special projects team judged that it was finally the right time to begin building a home robot, due to the maturity of artificial intelligence and robotics , falling cost of sensors and computer chips, and risk that competitors had similar plans.

They got started despite one big unknown: what Amazon’s home robot would be good for. “Robots are hard,” says Kiraly, who put together the team on the project. “We didn’t know what the value would necessarily be right then—but we knew it was going to take a while to get there, so we said we have to get started.

” The result was Astro , a cutesy, two-wheeled robot about the size of a bowling ball with a touchscreen for a face and a little periscope for peering over tabletops. It can automatically map your home using several sensors, and will then wander curiously around, avoiding obstacles, people, and pets. Users can also log in to the robot and control it remotely.

However, as the less-than-glowing early reviews pointed out , it still isn’t all that clear why anyone needs this machine underfoot. Amazon suggests that Astro can help you keep an eye on your home from afar, by recognizing different people and then sounding an alarm, and sending an alert, if it comes across a stranger. That’s neat, but not exactly a killer app.

Despite Astro’s lukewarm reception, Amazon remains committed to the project. On a recent visit to Lab126, the Amazon division responsible for developing hardware devices, in Silicon Valley, Kiraly and others said that the company firmly believes domestic robots will one day become widespread. And they described how such devices offer a powerful way for Amazon to fulfill its main objective—anticipating customers’ every want and need.

“Astro is our first robot, it’s not our last,” says Ken Washington, general manager of Amazon Consumer Robotics and head of the Astro project. At Lab126, Amazon explained a new feature publicly announced at its virtual hardware showcase today that allows the robot to recognize when a door or window inside a home has been left open, and alert its owner. That upgrade doesn’t make Astro a must-have, but it is rooted in AI technology that could allow the robot to learn all sorts of useful things about a person’s home by looking around and receiving some instruction.

Activating the new feature requires a person to take Astro on a tour of a home, pointing at doors and windows and speaking phrases such as “front door” out loud to teach the robot how to refer to them in future alerts. The robot makes use of an AI model trained on both text and images, similar to those used to generate AI art . The same approach to teaching Astro with gestures and words could in the future be extended to all kinds of furniture and objects in a home, Washington says.

The underlying AI technology could also help the robot make sense of what people are doing. “Artificial intelligence has reached this amazing inflection point,” he says. “It’s fully within reach to know ‘This is a chair,’ and ‘There is someone sitting in a chair.

’” Amazon is also planning a software update this year that will allow Astro to identify cats and dogs and automatically record videos of them, a feature users had asked for. Washington says the technology behind those new abilities is part of Amazon’s “big vision” for the smart home, which involves learning to anticipate people’s habits. Amazon executives call that “ambient intelligence.

” Getting there depends on Amazon being able to understand many of the things that a person does in their home, Washington says, yet most people would balk at a camera in every room. A cute wheeled robot provides a more acceptable way to monitor a household’s activity. “If you’ve got a mobile robot, it can be this smart glue for this future vision,” Washington says.

“When you walk into a room, the lights come on, for instance. ” When I ask Washington whether this could involve predicting what people might want or need to buy, he avoids a direct answer. He does say the robot should know whether you’ve been adding things to a grocery list, and points to how Alexa can preemptively turn the lights off if you say goodnight to it, using a feature known as Hunches .

“Today you have to ask for things,” he says. “But a lot of this asking is starting to fade into the background, because the AI is getting good enough that it’s beginning to predict what I might want. ” Amazon’s vision for a cute machine that watches your every move might feel unsettling to some, especially given the company’s already detailed view into customers’ lives .

Washington says Astro currently does almost all of its computing using its own hardware, sending little to Amazon’s servers except a map of people’s homes that needs to be relayed to the Astro smartphone app. “We took a privacy by design approach,” he says. WIRED saw Astro in action last week inside a mocked-up apartment at Lab126.

After years of writing about robots , I was impressed by its ability to navigate quickly through doorways and around obstacles, as well as its subtle interface with blinking eyes and emotive bleeps. It was clear that making even a relatively limited home robot required Amazon to cram in some impressive technology. Astro gets its bearings using cameras, motion sensors, and some clever software that turns video footage into a map , something tricky to do reliably in a small and relatively low-cost consumer device.

The overall impression is of an intelligent pet rather than a machine attempting to seem human—sensible given the robot’s limitations. But there was the occasional awkward moment when I asked Amazon executives, “Can it do anything else?” Washington and others I spoke to at Lab126 said that early Astro users typically like the robot, but want it to do more. Amazon hopes to fix that problem by keeping Astro on the market and steadily upgrading the robot until killer applications emerge.

One possibility is elder care. Washington says an early user of Astro logged in to the robot to check up on an elderly parent only to discover that they had fallen out of their wheelchair. In the future, Astro could conceivably watch for such mishaps and do many other helpful tasks automatically, Washington says.

“It could know when they took their medicine, and tell you whether they fell and needed help,” he says. Rodney Brooks , formerly an MIT roboticist and cofounder of iRobot, which made the first mass-market home robot, the Roomba vacuum, says that elderly care is the most obvious application for a home robot after cleaning. “It would be a big deal,” says Brooks, who more recently founded a company working on collaborative warehouse robots .

Amazon announced it struck a deal to acquire iRobot last month , but Brooks and Amazon both declined to speculate on how the acquisition might change the company’s robot offerings. Amazon might also try to make more of Astro’s preprogrammed personality, designing the robot to offer a kind of companionship. “I think it’s smart to make it look and act more like a pet than a human,” says Kate Darling , a roboticist at MIT’s Media Lab.

“I’m not a fan of the use case they are pushing—surveillance—but I truly believe that the real use case for social home robots will be, well, social. ” Many possible use cases for Astro’s future would depend on the robot having appendages. At one point I ask Washington whether Amazon’s vision involves it eventually having limbs so that it can climb stairs or grasp things.

“Our vision in the next five to 10 years is that robots are going to have the ability to manipulate things,” he says. He adds with a smile that Astro’s evolution could include a body plan in between the current, two-wheeled design and a robot with legs, but refuses to explain what it might be. A robot that can find and fetch the remote control or a missing set of keys might be useful.

But grasping and manipulating objects is one of the most difficult challenges in robotics. Humans can look at an object, make sense of its shape, form the correct grip, and lift it without much thought while accounting for slippage and weight. Machines struggle to do that reliably—and not because of a lack of research on the problem.

Amazon has been working on grasping for warehouse robots since long before starting work on Astro. Brooks went to Stanford University 45 years ago to work on robotic vision and grasping, and says that even today “we aren’t doing a whole lot better. ” Progress in robotics, because of the complexity of the physical world, tends to be slower than that of other technologies that have lifted Amazon, such as cloud computing or mobile devices.

“Frankly,” Washington says. “This is the beginning. ” Don’t expect Astro to evolve into something wildly more capable overnight—or Amazon to give up on the project.

.