Monday, October 23, 2023

I'll say it again, "predict the next token" is a reductive and misleading way of thinking about LLMs

And shame shame shame on the experts for allowing, encouraging, instructing so many to think that this is how they work. 

This technology is too important to be left in the hands of these experts. They may be expert in programming the engines and "training" them, but that's as far as their expertise goes. They need to rethink their "understanding," if you can call it that, of how they function to produce text.

3 comments:

  1. Assuming they have to "dumb down" the information for other people?

    ReplyDelete
    Replies
    1. Alas, I fear it's more complicated than that. I think many of them half-way believe it themselves. I certainly got a lot of push-back when I posted on the subject over at LessWrong, where there are technical experts.

      Delete