Menu Close

‘DARPA’ seeks to better align AI with human intentions

This week in AI, DARPA, the emerging technologies R&D agency of the U.S. Defense Department, launched a new program that aims to “align” AI systems with human decision-makers in domains where there isn’t an agreed-upon right answer. Elsewhere, two prominent cofounders from LinkedIn and DeepMind, Reid Hoffman and Mustafa Suleyman, announced a new AI startup called Inflection AI that seeks to develop software that allows humans to talk to computers using everyday language.

In a press release describing the new three-and-a-half-year program, DARPA says that the goal is to “evaluate and build trusted algorithmic decision-makers for mission-critical Department of Defense operations.” Dubbed “In the Moment,” or ITM, it focuses on the process of alignment — building AI systems that accomplish what they’re expected to accomplish.

“ITM is different from typical AI development approaches that require human agreement on the right outcomes,” ITM program manager Matt Turek said in a statement. “The lack of a right answer in difficult scenarios prevents us from using conventional AI evaluation techniques, which implicitly requires human agreement to create ground-truth data.”

For example, self-driving cars can be developed against a ground truth for right and wrong decisions based on unchanging, relatively consistent rules of the road. The designers of these cars could hard-code “risk values” into the cars that prevent them from, for example, making right turns on red in cities where they’re illegal. But Turek says that these one-size-fits-all risk values won’t work from a Department of Defense perspective. Combat situations evolve rapidly, he points out, and a commander’s intent can change from scenario to scenario.

“The [Defense Department] needs rigorous, quantifiable, and scalable approaches to evaluating and building algorithmic systems for difficult decision-making where objective ground truth is unavailable,” Turek continued. “Difficult decisions are those where trusted decision-makers disagree, no right answer exists, and uncertainty, time-pressure, and conflicting values create significant decision-making challenges.”

DARPA is only the latest organization to explore techniques that might help better align AI with a person’s intent. In January, OpenAI, the company behind the text-generating model GPT-3, detailed an alignment technique that it claims cuts down on the amount of toxic language that GPT-3 generates. Toxic text generation is a well-known problem in AI, often caused by toxic datasets. Because text-generating systems are trained on data containing problematic content, some of the content slips through.

“Although [AI systems are] quite smart today, they don’t always do what we want them to do. The goal of alignment is to produce AI systems that do [achieve] what we want them to,” OpenAI cofounder and chief scientist Ilya Sutskever told VentureBeat in a phone interview earlier this year. “[T]hat becomes more important as AI systems become more powerful.”

ITM will attempt to establish a framework to evaluate decision-making by algorithms in “very difficult domains,” including combat, through the use of “realistic, challenging” scenarios. “Trusted humans” will be asked to make decisions in these scenarios and then the results will be compared to decisions from an algorithm subjected to the same scenarios.

Full article: DARPA seeks to better align AI with human intentions

Leave a Reply

Your email address will not be published. Required fields are marked *

(UN General Assembly, 1948) The Universal Declaration of Human Rights: 1. All human beings are free and equal 2. No discrimination 3. Right to life 4. No slavery 5. No torture and inhuman treatment 6. Same right to use law 7. Equal before the law 8. Right to be treated fair by court 9. No unfair detainment 10. Right to trial 11. Innocent until proved guilty 12. Right to privacy 13. Freedom to movement and residence 14. Right to asylum 15. Right to nationality 16. Rights to marry and have family 17. Right to own things 18. Freedom of thought and religion 19. Freedom of opinion and expression 20. Right to assemble 21. Right to democracy 22. Right to social security 23. Right to work 24. Right to rest and holiday 25. Right of social service 26. Right to education 27. Right of cultural and art 28. Freedom around the world 29. Subject to law 30. Human rights can’t be taken away