Clever Hans does AI – FAKE performance in natural language processing

Better language models -> more and more subtle forms of overfitting! pic.twitter.com/y8e10jkPs2

— Scott Enderle (@scottenderle) January 11, 2020

Wikipedia:

Clever Hans (in German: der Kluge Hans) was an Orlov Trotter horse that was claimed to have performed arithmetic and other intellectual tasks.

After a formal investigation in 1907, psychologist Oskar Pfungst demonstrated that the horse was not actually performing these mental tasks, but was watching the reactions of his trainer. He discovered this artifact in the research methodology, wherein the horse was responding directly to involuntary cues in the body language of the human trainer, who had the faculties to solve each problem. The trainer was entirely unaware that he was providing such cues. In honour of Pfungst's study, the anomalous artifact has since been referred to as the Clever Hans effect and has continued to be important knowledge in the observer-expectancy effect and later studies in animal cognition.

BTW, I believe "SOTA" = "State of the Art."