Many important tasks solved by transformers can't be done in subquadratic time

New paper proving that many important tasks solved by transformers can't be done in subquadratic time.

Sorry Linear Transformers, Mamba and all the others.

https://t.co/OGUEASmL23
By @firebat03 and @HantaoYu_Theory. pic.twitter.com/XlEmmtDn58

— Thomas Ahle (@thomasahle) October 14, 2024