Menu Close

‘DARPA’ seeks to better align AI with human intentions

This week in AI, DARPA, the emerging technologies R&D agency of the U.S. Defense Department, launched a new program that aims to “align” AI systems with human decision-makers in domains where there isn’t an agreed-upon right answer. Elsewhere, two prominent cofounders from LinkedIn and DeepMind, Reid Hoffman and Mustafa Suleyman, announced a new AI startup called Inflection AI that seeks to develop software that allows humans to talk to computers using everyday language.

In a press release describing the new three-and-a-half-year program, DARPA says that the goal is to “evaluate and build trusted algorithmic decision-makers for mission-critical Department of Defense operations.” Dubbed “In the Moment,” or ITM, it focuses on the process of alignment — building AI systems that accomplish what they’re expected to accomplish.

“ITM is different from typical AI development approaches that require human agreement on the right outcomes,” ITM program manager Matt Turek said in a statement. “The lack of a right answer in difficult scenarios prevents us from using conventional AI evaluation techniques, which implicitly requires human agreement to create ground-truth data.”

For example, self-driving cars can be developed against a ground truth for right and wrong decisions based on unchanging, relatively consistent rules of the road. The designers of these cars could hard-code “risk values” into the cars that prevent them from, for example, making right turns on red in cities where they’re illegal. But Turek says that these one-size-fits-all risk values won’t work from a Department of Defense perspective. Combat situations evolve rapidly, he points out, and a commander’s intent can change from scenario to scenario.

“The [Defense Department] needs rigorous, quantifiable, and scalable approaches to evaluating and building algorithmic systems for difficult decision-making where objective ground truth is unavailable,” Turek continued. “Difficult decisions are those where trusted decision-makers disagree, no right answer exists, and uncertainty, time-pressure, and conflicting values create significant decision-making challenges.”

DARPA is only the latest organization to explore techniques that might help better align AI with a person’s intent. In January, OpenAI, the company behind the text-generating model GPT-3, detailed an alignment technique that it claims cuts down on the amount of toxic language that GPT-3 generates. Toxic text generation is a well-known problem in AI, often caused by toxic datasets. Because text-generating systems are trained on data containing problematic content, some of the content slips through.

“Although [AI systems are] quite smart today, they don’t always do what we want them to do. The goal of alignment is to produce AI systems that do [achieve] what we want them to,” OpenAI cofounder and chief scientist Ilya Sutskever told VentureBeat in a phone interview earlier this year. “[T]hat becomes more important as AI systems become more powerful.”

ITM will attempt to establish a framework to evaluate decision-making by algorithms in “very difficult domains,” including combat, through the use of “realistic, challenging” scenarios. “Trusted humans” will be asked to make decisions in these scenarios and then the results will be compared to decisions from an algorithm subjected to the same scenarios.

Full article: DARPA seeks to better align AI with human intentions

Leave a Reply

Your email address will not be published. Required fields are marked *