Tom Ritchford
Oct 9, 2023

--

Words have actual meanings, you know, you can't just throw a large number like "quintillion" in without some actual argument.

You're talking about a model with 20 million times as many parameters as today's largest. Note that the cost of training models grows much faster than linearly in the number of parameters.

Even if Moore's Law continued, and forgetting about the non-linearity, that's 25 doublings, or about 50 years. But we are getting close to fundamental physical limits - no one believes Moore's Law can hold for much longer.

--

--

No responses yet