Researchers are developing a approach to incorporate one of the human of characteristics – uncertainty – into machine learning systems.

Human error and uncertainty are concepts that many artificial intelligence systems fail to know, particularly in systems where a human provides feedback to a machine learning model. Lots of these systems are programmed to assume that humans are at all times certain and proper, but real-world decision-making includes occasional mistakes and uncertainty.

Researchers from the University of Cambridge, together with The Alan Turing Institute, Princeton, and Google DeepMind, have been attempting to bridge the gap between human behavior and machine learning, in order that uncertainty might be more fully accounted for in AI applications where humans and machines are working together. This might help reduce risk and improve trust and reliability of those applications, especially where safety is critical, equivalent to medical diagnosis.

The team adapted a well known image classification dataset in order that humans could provide feedback and indicate their level of uncertainty when labeling a specific image. The researchers found that training with uncertain labels can improve these systems’ performance in handling uncertain feedback, although humans also cause the general performance of those hybrid systems to drop. Their results will probably be reported on the AAAI/ACM Conference on Artificial Intelligence, Ethics and Society (AIES 2023) in Montréal.

‘Human-in-the-loop’ machine learning systems – a form of AI system that allows human feedback – are sometimes framed as a promising approach to reduce risks in settings where automated models can’t be relied upon to make decisions alone. But what if the humans are unsure?

Uncertainty is central in how humans reason in regards to the world but many AI models fail to take this into consideration. A variety of developers are working to handle model uncertainty, but less work has been done on addressing uncertainty from the person’s viewpoint.”

Katherine Collins, First Writer, Cambridge’s Department of Engineering

We’re continually making decisions based on the balance of probabilities, often without really occupied with it. More often than not – for instance, if we wave at someone who looks similar to a friend but seems to be a complete stranger – there is not any harm if we get things flawed. Nevertheless, in certain applications, uncertainty comes with real safety risks.

“Many human-AI systems assume that humans are at all times certain of their decisions, which is not how humans work – all of us make mistakes,” said Collins. “We wanted to take a look at what happens when people express uncertainty, which is very essential in safety-critical settings, like a clinician working with a medical AI system.”

“We want higher tools to recalibrate these models, in order that the people working with them are empowered to say after they’re uncertain,” said co-author Matthew Barker, who recently accomplished his MEng degree at Gonville and Caius College, Cambridge. “Although machines might be trained with complete confidence, humans often cannot provide this, and machine learning models struggle with that uncertainty.”

For his or her study, the researchers used a number of the benchmark machine learning datasets: one was for digit classification, one other for classifying chest X-rays, and one for classifying images of birds. For the primary two datasets, the researchers simulated uncertainty, but for the bird dataset, that they had human participants indicate how certain they were of the pictures they were : whether a bird was red or orange, for instance. These annotated ‘soft labels’ provided by the human participants allowed the researchers to find out how the ultimate output was modified. Nevertheless, they found that performance degraded rapidly when machines were replaced with humans.

“We all know from a long time of behavioral research that humans are almost never 100% certain, however it’s a challenge to include this into machine learning,” said Barker. “We’re attempting to bridge the 2 fields, in order that machine learning can begin to cope with human uncertainty where humans are a part of the system.”

The researchers say their results have identified several open challenges when incorporating humans into machine learning models. They’re releasing their datasets in order that further research might be carried out and uncertainty could be built into machine learning systems.

“As a few of our colleagues so brilliantly put it, uncertainty is a type of transparency, and that is hugely essential,” said Collins. “We want to determine when we will trust a model and when to trust a human and why. In certain applications, we’re a probability over possibilities. Especially with the rise of chatbots for instance, we’d like models that higher incorporate the language of possibility, which can result in a more natural, secure experience.”

“In some ways, this work raised more questions than it answered,” said Barker. “But although humans could also be miscalibrated of their uncertainty, we will improve the trustworthiness and reliability of those human-in-the-loop systems by accounting for human behavior.”

The research was supported partially by the Cambridge Trust, the Marshall Commission, the Leverhulme Trust, the Gates Cambridge Trust and the Engineering and Physical Sciences Research Council (EPSRC), a part of UK Research and Innovation (UKRI).