Compressing Random Data - Search Videos

Revolutionary Data Compression - 90% Cost Reduction Target #innovation #shortsfeed #shorts

Revolutionary Data Compression - 90% Cost Reduction Target #inno…

1.2K views2 days ago

YouTubeStark Drones Corporation

Karpathy told Dwarkesh that a 1 billion parameter model, trained on clean data, could hit the intelligence of today's 1.8 trillion parameter frontier.That is a 1,800x compression claim. The math behind it is more defensible than it sounds.When researchers at frontier labs look at random samples from their training corpus, they see stock ticker symbols, broken HTML, forum spam, autogenerated gibberish. Not Wikipedia. Not the Wall Street Journal. The actual pretraining dataset is mostly noise, and

Karpathy told Dwarkesh that a 1 billion parameter model, trained o…

36.2K views10 hours ago

x.comAakash Gupta

See more videos