Waymo’s driverless cars have driven 6.1 million autonomous miles in Phoenix, Arizona, including 65,000 miles without a human behind the wheel from 2019 through the first nine months of 2020. That’s according to data from a new internal report Waymo published today that analyzed a portion of collisions involving the robo-taxi service Waymo One, which launched in 2018. In total, Waymo’s vehicles were involved in 18 accidents with a pedestrian, cyclist, driver, or other object and experienced 29 disengagements — times human drivers were forced to take control — that likely would have otherwise resulted in an accident.
Three independent studies in 2018 — by the Brookings Institution, the think tank HNTB, and the Advocates for Highway and Auto Safety (AHAS) — found that a majority of people aren’t convinced of driverless cars’ safety. And Partners for Automated Vehicle Education (PAVE) reports a majority of Americans don’t think the technology is “ready for prime time.” These concerns are not without reason. In March 2018, Uber suspended testing of its autonomous Volvo XC90 fleet after one of its cars struck and killed a pedestrian in Tempe, Arizona. Separately, Tesla’s Autopilot driver-assistance system has been blamed for a number of fender benders, including one in which a Tesla Model S collided with a parked fire truck. Now the automaker’s Full Self Driving Beta program is raising new concerns.
Waymo has so far declined to sign onto efforts like Safety First For Automated Driving, a group of companies that includes Fiat Chrysler, Intel, and Volkswagen and is dedicated to a common framework for the development, testing, and validation of autonomous vehicles. However, Waymo is a member of the Self-Driving Coalition for Safer Streets, which launched in April 2016 with the stated goal of working “with lawmakers, regulators, and the public to realize the safety and societal benefits of self-driving vehicles.” Since October 2017, Waymo has released a self-driving report each year, ostensibly highlighting how its vehicles work and the technology it uses to ensure safety, albeit in a format some advocates say resembles marketing materials rather than regulatory filings.
Waymo says its Chrysler Pacificas and Jaguar I-Pace electric SUVs — which have driven tens of billions of miles through computer simulations and 20 million miles (74,000 driverless) on public roads in 25 cities — were providing a combined 1,000 to 2,000 rides per week in the East Valley portion of the Phoenix metropolitan region by early 2020. (Waymo One reached 100,000 rides served in December 2019.) Between 5% and 10% of these trips were driverless — without a human behind the wheel. Prior to early October, when Waymo made fully driverless rides available to the public through Waymo One, contracted safety drivers rode in most cars to note anomalies and take over in the event of an emergency.
Waymo One, which initially transitioned to driverless pickups with a group of riders from Waymo’s Early Rider program, delivers rides with a fleet of over 600 autonomous cars from Phoenix-area locations 24 hours a day, seven days a week. It prompts customers to specify pickup and drop-off points before estimating the time to arrival and cost of the ride. As with a typical ride-hailing app, users can enter payment information and rate the quality of rides using a five-star scale.
Using its cloud simulation platform, Carcraft, Waymo says it predicts what might have transpired had a driver not taken over to avert a near-accident — what the company calls a counterfactual. Waymo leverages the outcomes of these counterfactual disengagement simulations individually and in aggregate. Engineers evaluate each counterfactual to identify potential collisions, near-misses, and other metrics. If the simulation outcome reveals an opportunity to improve the system’s behavior, the engineers use it to develop and test changes to software. The counterfactual is also added to a library of scenarios used to test future software.
At an aggregate level, Waymo uses results from counterfactuals to produce metrics relevant to a vehicle’s on-road performance.
While conceding that counterfactuals can’t predict exactly what would have occurred, Waymo asserts they can be more realistic than simulations because they use the actual behavior of the vehicles and objects up to the point of disengagement. Where counterfactuals aren’t involved, Waymo synthesizes sensor data for cars and models scenes in digitized versions of real-world environments. As virtual cars drive through the scenarios, engineers modify the scenes and evaluate possible situations by adding new obstacles (such as cyclists) or by modulating the speed of oncoming traffic to gauge how the vehicle would have reacted.
As part of a collision avoidance testing program, Waymo also benchmarks the vehicles’ capabilities in thousands of scenarios where immediate braking or steering is required to avoid collisions. The company says these scenarios test competencies crucial to reducing the likelihood of collisions caused by other road users.
Waymo analyzes counterfactuals to determine their severity based on the likelihood of injury, collision object, impact velocity, and impact geometry — methods the company developed using national crash databases and periodically refines to reflect new data. Events are tallied using classes ranging from no injury expected (S0) to possible critical injuries expected (S1, S2, and S3). Waymo says it determines this rating using the change in velocity and direction of force estimated for each vehicle.
Here is a breakdown of the car data from January 1, 2019 to September 30, 2020, which covers 65,000 miles in driverless mode. The disengagement data is from January 1 to December 31, 2019, which is when Waymo’s cars drove the aforementioned 6.1 million miles.
READ full article: ‘Waymo’s’ driverless cars were involved in 18 accidents over 20 months