The stochastic nature of machine learning and its implications for high-consequence AI.

The stochastic nature of machine learning and its implications for high-consequence AI: Almost all relevant contemporary AI systems are based on ML models that are high dimensional probability density functions, which output the most likely predictions given the input data, leading to likely errors that have decidedly different error patterns than human experts. While the field has developed various mitigation strategies, these approaches address symptoms rather than the underlying statistical nature of ML. This has important ramifications for any operator (and their organization) relying on respective results (predictions):

1.) AI results are not ’neutral’. They have picked up human biases via the training data and are ignorant of anything not represented in the data or not representable or inferable by the chosen model.

2.) AI results tend to regurgitate the past. Applied in the straight-forward way, they reproduce the most likely pattern found in the training data. This can lead to an impoverishment of strategic decisions and have effects worth of consideration when conflicting parties rely on basically similar AI decision support.

3.) AI results are statistically plausible predictions with a certain likelihood of failure. Being error-free is not part of the methodology; plausible might still be wrong. If a result is wrong is not known to the model and difficult to predict technically, but typically, any result will be reported with optimistic confidence by an AI system. It must hence be verified by a human capable of doing so independently.

4.) Human errors and AI errors are very different such that AI systems’ errors might seem very stupid (and hence unexpected) from a human point of view. This stems from the completely different mechanisms by which these results are achieved, even when based on the same data.

Link to the full article on linkedin.