Note that that graph is training cost, NOT inference cost. Training is a single fixed cost used to create ChatGPT, running it in practice is much, much cheaper
— Neel Nanda (@NeelNanda5) February 2, 2023
To which I have replied:
FWIW, to a first approximation, the training cost of The Human Mind, considered in maximum generality, is human history, from clever ape to the present. The training cost of any particular mind is considerably lower. In any case, inference cost is cheap.
— Bill Benzon, BAM! Bootstrapping Artificial Minds (@bbenzon) February 2, 2023
One might also ask, what is the training cost of clever ape minds? Considered in maximum generality, it is the cost of animal evolution – and I suppose that, if we wish, we could push that back to the cost of evolution in general. The cost of training the mind of one specific clever ape is, of course, considerably lower.
What am I trying to say there? I am certainly saying that real minds, at any rate, organic minds, are creatures from history. The start from close to nothing and bootstrap themselves into existence through some kind of procedure. An evolutionary procedure?
What is it that allowed clever apes to become human? I present one kind of answer in this post, which is based on a conceptual impulse that struct me in graduate school: UFO Events, a Thought Experiment about the Evolution of Language. I present a somewhat more fleshed out answer in Synch, Song, and Society, which is a review of Steven Mithen's book, The Singing Neanderthals (pp. 78-82), which I published in Human Nature Review. In any event, the procedure, whatever its mechanical details, involves communicating with others.
No comments:
Post a Comment