Are LLMs approaching the limiting wall beyond which performance simply wanders around in the same general region?

Hot take on Google Gemini and GPT-4:

👉Google Gemini seems to have by many measures matched (or slightly exceeded) GPT-4, but not to have blown it away.

👉From a commercial standpoint GPT-4 is no longer unique. That’s a huge problem for OpenAI, especially post drama, when many… https://t.co/0i0yyTEtRY

— Gary Marcus (@GaryMarcus) December 6, 2023

Cade Metz and Nico Grant, Google Updates Bard Chatbot With ‘Gemini’ A.I. As It Chases ChatGPT, NYTimes, 2023. From the article:

Google has built three versions of Gemini with three different sets of skills. The largest, Ultra, is designed to tackle complex tasks and will debut next year. Pro, the mid-tier offering, will be rolled out to numerous Google services, starting Wednesday, with the Bard chatbot. Nano, the smallest version, will power some features on the Pixel 8 Pro smartphone, such as summarizing audio recordings and offering suggested text responses in WhatsApp starting Wednesday. [...]

With Gemini, Google has also trained the technology on digital images and sounds. It is what researchers call a “multimodal” system, meaning it can analyze and respond to both images and sounds. If you give it a math problem that includes lines, shapes and other images, for example, it can answer in much the way a high school student would.

That portion of the technology, however, will not be available to consumers until sometime next year. Google also acknowledged that like similar systems, Gemini is prone to mistakes. It can get facts wrong or even “hallucinate” — make stuff up.