What strange beasts are these new-fangled AIs?

Cade Metz, How Smart Are the Robots Getting? The New York Times, Jan. 20, 2023.

Well into the piece, after introducing Turing’s “imitation game,” after telling us how, “Five years ago, Google, OpenAI and other A.I. labs started designing neural networks that analyzed enormous amounts of digital text” to build large language models, such as the GPT series, and after telling us about DALL-E, and other multi-modal systems, Metz tells us:

These are not systems that anyone can properly evaluate with the Turing test — or any other simple method. Their end goal is not conversation.

Researchers at Google and DeepMind, which is owned by Google’s parent company, are developing tests meant to evaluate chatbots and systems like DALL-E, to judge what they do well, where they lack reason and common sense, and more. One test shows videos to artificial intelligence systems and asks them to explain what has happened. After watching someone tinker with an electric shaver, for instance, the A.I. must explain why the shaver did not turn on.

These tests feel like academic exercises — much like the Turing test. We need something that is more practical, that can really tell us what these systems do well and what they cannot, how they will replace human labor in the near term and how they will not.

YES!

We could also use a change in attitude. “We need a paradigm shift — where we no longer judge intelligence by comparing machines to human behavior,” said Oren Etzioni, professor emeritus at the University of Washington and founding chief executive of the Allen Institute for AI, a prominent lab in Seattle.

In some ways these systems are superior to us – who has the range of knowledge that any large-language-model has? – and in other ways they are inferior: “They cannot deal with concepts they have never seen before. And they cannot take ideas and explore them in the physical world.”

Metz concludes:

Certainly, these bots will change the world. But the onus is on you to be wary of what these systems say and do, to edit what they give you, to approach everything you see online with skepticism. Researchers know how to give these systems a wide range of skills, but they do not yet know how to give them reason or common sense or a sense of truth.

That still lies with you.