LLM training data mixture optimization breaks when training pools shift — every prior proxy experiment becomes stale.

“The rapid release cycle in the AI industry has accelerated to the point where barely a day goes past without a new LLM being announced. But the same cannot be said for the underlying data,” notes ...

Both humans and other animals are good at learning by inference, using information we do have to figure out things we cannot observe directly. New research from the Center for Mind and Brain at the ...