Zoox today published a video showcasing its autonomous system driving for an hour in downtown San Francisco and performing a range of complex maneuvers in heavy, multi-lane vehicle traffic and dense pedestrian crowds.
While it’s difficult to glean much without data from additional real-world tests, the clip highlights a trend among self-driving startups: Showing rather than telling. It’s the philosophy of companies like GM’s Cruise and Zoox that metrics such as autonomous miles driven and disengagements are a poor measure of progress and won’t convince a public skeptical of driverless cars’ safety. On the other hand, videos might — perhaps to such a degree that they hasten the adoption of autonomous vehicles.
In the video, which was recorded with a safety driver behind the wheel, Zoox’s retrofitted Toyota Highlander is shown making an unprotected left turn at a six-way intersection and traversing a tunnel without the use of GPS. The Highlander also climbs up steep hills with occluded crests, makes right turns on red at four-way intersections, and at one point drives down iconic Lombard Street and overtakes a row of three double-parked cars.
A 360-degree field-of-view sensor architecture comprising 18 cameras, 10 radars, and eight lidar sensors records the car’s surroundings, explains Zoox senior director of perception and prediction James Philbin. In some parts of San Francisco, this sensor array is helping detect and track roughly a hundred people and vehicles simultaneously beyond 130 meters (426 feet) away.
According to Zoox director of computer vision Sarah Tariq, Zoox’s perception system detects, classifies, and identifies a number of behaviors and obstacles that aren’t shown in the video. For instance, it can distinguish traffic lights, trams, busses, construction vehicles, and construction obstacles like cones and lane dividers, as well as open and closed car doors and different types of animals. The perception system also tracks certain pedestrian attributes, like the direction they’re facing and whether they’re standing or walking, looking at a phone, pushing a stroller or carrying things, or even riding an escooter.
Above: One of Zoox’s retrofitted Toyota Highlander SUVs.Image Credit: Zoox
Detecting vehicles that are double-parked or actively reversing is a tougher challenge, says Philbin. To accomplish this, Zoox’s system looks for things like vehicle lights, road position, parking lane occupancy, and the behavior of other traffic around the target cars.
When it comes to decision-making, Zoox’s planner system — which takes into account both perception and prediction data, including HD maps of roads — makes decisions and feeds this data into the controls that communicate with the Highlander’s braking and steering interface. The planner employs a feature called action search, which aims to make forward progress in a “smooth and consistent” way while never violating safety limits. For instance, action search will prompt the Highlander to switch lanes in anticipation of a left-hand turn down the road.
Features like action search are partially trained in simulation, says Zoox — the company uses data collected from the real world to inform simulations that continuously improve its systems’ performance. Zoox’s cars drive city blocks to create topologies, and its engineers use these topologies to create three-dimensional representations that AI agents travel endlessly in order to self-improve.
On the subject of the above-mentioned safety limits, Zoox says it biases its perception output to be conservative in the sense that if something looks at all like a person, the system may occasionally misclassify it as a person even if it’s actually something else. “While proper classification is important for prediction purposes, the most important thing is to never miss an object that actually exists,” said Philbin.
A murky future
Foster City, California-based Zoox, which has raised over $950 million in venture capital at a multibillion-dollar valuation, appeared to be making progress toward a commercial launch before testing of its live level 3 vehicles ground to a halt as a result of the COVID-19 pandemic. (A spokesperson told VentureBeat via email that the company continues to make progress on “back-end testing and processes.”) The company recently announced it would begin deploying autonomous Highlanders in Las Vegas for blocks of time — it has a permit from the Nevada Department of Motor Vehicles to transport passengers autonomously — and it submitted documents to the California DMV showing that its 58 registered vehicles drove 67,015 autonomous miles in 2019.
Eventually, Zoox hopes to move beyond retrofitted Highlanders to fully driverless custom cars, the design of which it’s hinted at (but not yet fully revealed) in the past. The roughly BMW i3-sized electric vehicles will use cameras and four lidar sensors for perception, each of which will be able to see 270 degrees, along with four-wheel steering, active four-wheel suspension, dual power trains, and dual batteries with a combined capacity larger than that of most single-car batteries today.
The idea is to reduce congestion through fleet management and to minimize trips back to base stations for charging overnight. Zoox’s shuttle-like car — which is fully driverless — is designed to operate in a shared fleet in order to maximize efficiency and cut down on ride trip times.
But although Zoox says it’s demonstrating its fully electric vehicle for partners and insiders behind closed doors, the next phase of its deployment plans are likely to be delayed. This week, Zoox reportedly laid off 10% of its 1,000-person workforce days after letting go of 120 contract workers, moves it blamed on the economic fallout from the pandemic. And it said that it would pay Tesla an undisclosed amount of monetary damages and undergo an audit to settle a trade secret theft lawsuit filed last year in which Tesla claimed that some of its former employees brought proprietary information with them to Zoox.
The Zoox spokesperson told VentureBeat that the company’s product launch goals remain the same, though it “recognizes the rapidly evolving impact COVID-19 has” on global business. “[Zoox] remains adaptable in its path to bring a product to market,” said the spokesperson.
