Bill, I think you will appreciate Smolensky in 2026. See [2.] below.
Has the future arrived? Moving your dialetic forward Bill... "We take ideas from both territories and create a new territory, on a “higher” level: Synthesis."" [1.] ... utilising... "Tensor Product Representations (TPRs; Smolensky, 1990)" [2.].
Smolensky et al just published (what is to me, ymmv) "a “higher” level: Synthesis.".
Smolenaky must be patient, as his TPR 1990 paper is used as a basis for synthesis in; "The Emergent Symbolic Structure of Artificial Neural Networks" [2.] ###
[1.] BB on the problem... "A dialectical view of the history of AI, Part 1: We’re only in the antithesis phase. [A synthesis is in the future.]" Wednesday, November 15, 2023 ... "No, to continue forward, we are going to need approaches that combine ideas from both families. Just what form that synthesis will take, who knows? ...We take ideas from both territories and create a new territory, on a “higher” level: Synthesis." https://new-savanna.blogspot.com/2023/11/a-dialectical-view-of-history-of-ai.html ###
This is the finding in [2.] I'd like expanded... a lobotomy - "closed-form equation instantiating a symbolic structure" - works?!... "we can replace the network's entire representation-generating process with a closed-form equation instantiating a symbolic structure, and the network's behavior remains largely unchanged."
If the "network's entire representation-generating process" is lobotomised and replaced "with a closed-form equation", is it still an 'ai', or just ML, or??? New name as is 'transformer TPR synthetic symbol ai'???
The paper Smolensky et al in 2026 [2.]. (not abstract) informs us;
"This approach requires a proposal about how vectors could capture such structure—a consideration that is far from trivial given the apparent incompatibility of vectors and symbols. For this purpose, we adopt a mathematical formalism from cognitive science called Tensor Product Representations (TPRs; Smolensky, 1990). In a TPR, a symbolic structure is framed as a collection of fillers—the elements of the structure—each of which is paired with a role that denotes its position. For instance, to encode the sentence cats chase dogs, we might use the fillers cats, chase, and dogs paired with the roles subject, verb, and object, respectively. The collection of role-filler pairs can then be translated into a vector using an approach described in Section 2.3. ... "Specifically, we can replace each neural network’s entire representation-generating process with a single, interpretable, TPR-based equation, and the network’s behavior changes minimally if at all. Our core findings are the following:"
The Conclusion says - (tl:dr) we still don't understand models... "... Such an understanding would illuminate how a system can unify systematic compositionality with gradience and idiosyncrasies—a type of unification that large language models seem to have achieved, at least in a partial sense, yet which remains poorly understood."
This 2026 paper with Smolensky, who's 1990 paper... "Tensor Product Representations and Holographic Reduced Representations", seems to have finally reached the light.
[2.] "The Emergent Symbolic Structure of Artificial Neural Networks R. Thomas McCoy, Paul Soulos, Tal Linzen, Paul Smolensky 30 Aug 2026 https://arxiv.org/abs/2608.29530 ###
And a bonus for us beginners... "A gentle introduction to tensor product representations (TPRs) ... "The key components of tensor product representations are the concepts of “roles” and “fillers.” https://csinva.io/blog/misc/24_tensor_product_repr.html
Approaching symbolic.
ReplyDeleteBill, I think you will appreciate Smolensky in 2026. See [2.] below.
Has the future arrived? Moving your dialetic forward Bill... "We take ideas from both territories and create a new territory, on a “higher” level: Synthesis."" [1.] ... utilising... "Tensor Product Representations (TPRs; Smolensky, 1990)" [2.].
Smolensky et al just published (what is to me, ymmv) "a “higher” level: Synthesis.".
Smolenaky must be patient, as his TPR 1990 paper is used as a basis for synthesis in;
"The Emergent Symbolic Structure of Artificial Neural Networks" [2.]
###
[1.] BB on the problem...
"A dialectical view of the history of AI, Part 1: We’re only in the antithesis phase. [A synthesis is in the future.]"
Wednesday, November 15, 2023
...
"No, to continue forward, we are going to need approaches that combine ideas from both families. Just what form that synthesis will take, who knows? ...We take ideas from both territories and create a new territory, on a “higher” level: Synthesis."
https://new-savanna.blogspot.com/2023/11/a-dialectical-view-of-history-of-ai.html
###
This is the finding in [2.] I'd like expanded... a lobotomy - "closed-form equation instantiating a symbolic structure" - works?!... "we can replace the network's entire representation-generating process with a closed-form equation instantiating a symbolic structure, and the network's behavior remains largely unchanged."
If the "network's entire representation-generating process" is lobotomised and replaced "with a closed-form equation", is it still an 'ai', or just ML, or??? New name as is 'transformer TPR synthetic symbol ai'???
The paper Smolensky et al in 2026 [2.]. (not abstract) informs us;
"This approach requires a proposal about how vectors could capture such structure—a consideration that is far from trivial given the apparent incompatibility of vectors and symbols. For this purpose, we adopt a
mathematical formalism from cognitive science called Tensor Product Representations (TPRs; Smolensky, 1990).
In a TPR, a symbolic structure is framed as a collection of fillers—the elements of the structure—each of
which is paired with a role that denotes its position. For instance, to encode the sentence cats chase dogs, we
might use the fillers cats, chase, and dogs paired with the roles subject, verb, and object, respectively. The
collection of role-filler pairs can then be translated into a vector using an approach described in Section 2.3.
...
"Specifically, we can replace each
neural network’s entire representation-generating process with a single, interpretable, TPR-based
equation, and the network’s behavior changes minimally if at all. Our core findings are the following:"
The Conclusion says - (tl:dr) we still don't understand models...
"... Such an understanding would illuminate
how a system can unify systematic compositionality with gradience and idiosyncrasies—a type of unification that
large language models seem to have achieved, at least in a partial sense, yet which remains poorly understood."
This 2026 paper with Smolensky, who's 1990 paper... "Tensor Product Representations and Holographic Reduced Representations", seems to have finally reached the light.
[2.] "The Emergent Symbolic Structure of Artificial Neural Networks
R. Thomas McCoy, Paul Soulos, Tal Linzen, Paul Smolensky
30 Aug 2026
https://arxiv.org/abs/2608.29530
###
And a bonus for us beginners...
"A gentle introduction to tensor product representations (TPRs)
... "The key components of tensor product representations are the concepts of “roles” and “fillers.”
https://csinva.io/blog/misc/24_tensor_product_repr.html
SD