Tech company, Raytheon, says it is creating a learning system that knows what it knows – an innovation which the Massachusetts-based firm hopes will improve trust between human operators and AI machinery.
Raytheon, which specialises in solutions for defence, civil government and cybersecurity, says the machine-learning development is part of a $6 million contract from the Defence Advanced Research Projects Agency for the Competency Aware Machine Learning program.
Systems will be able to communicate the abilities they have learned, the conditions under which the abilities were learned, the strategies they recommend as a result and the situations for which those strategies can be used.
“The CAML system turns tools into partners,” said Ilana Heintz, principal investigator for CAML at Raytheon BBN Technologies.
“It will understand the conditions where it makes decisions and communicate the reasons for those decisions,” Heintz added.
The system will learn from a video game like process. Instead of giving the system rules, the researchers will tell the system what choices it has in the game and what the ultimate goal is. By repeatedly playing the game, the system will learn the most effective ways to meet the goal.
The system will explain itself by recording the conditions and strategies it used to come up with successful outcomes.
“People need to understand an autonomous system’s skills and limitations to trust it with critical decisions,” she continued.
Once the system has developed these skills, the team will apply it to a simulated search and rescue mission. Users will create the conditions surrounding the mission, while the system will make recommendations and give users information about its competence in those particular conditions. For example, the system might say, “In the rain, at night, I can distinguish between a person and an inanimate object with 90 percent accuracy, and I have done this over 1,000 times.”