Data efficiency

…counterfactual reasoning, data efficiency, evaluation, deployment costs and the limits of current benchmarks. Invited talks, contributed spotlights and posters connect theory to empirical comparisons, including negative results. Confirmed speakers include Chelsea Finn, Yunzhu Li, Georgia Chalvatzaki and Vincent Sitzmann. Paper submissions closed on 12 October; camera-ready papers are due on 6 November and

…models of the world can improve data efficiency. The conversation examines belief-state compression, Transformer caches, JEPA-style objectives and his Next Latent research. It also covers the value of algorithmic research alongside scaling, agent-assisted experiments, open models, the history of CAPTCHA and the behavior of modern optimizers. Hosted by Ravid Shwartz Ziv and Allen Roush. Watch the full conversation on the publisher’s YouTube channel.

…He discusses scaling laws, data selection and the origins of diffusion models, then turns to neural experiments on perception, self-related processing and describing neuronal responses. The conversation closes with questions about human versus machine data efficiency and the different ways brains and artificial networks acquire useful algorithms. Hosted by Ravid Shwartz Ziv and Allen Roush. Watch the full conversation on the publisher’s YouTube channel.

Small language models trained on limited data can provide economical, controlled simulations for studying human language acquisition. Alex Warstadt discusses the BabyLM Challenge and its goals of improving data efficiency, cognitive plausibility and accessibility. Studies from his UC San Diego lab train models from scratch under controlled conditions to investigate critical periods in second-language learning, inductive biases behind cross-linguistic patterns, and distributional signatures of word-learning trajectories. Comparing cases where these models resemble human learners with cases wher

…evolution and development as routes to data efficiency, the coordination of multiple behavioural objectives, the possible energy benefits of spikes, and the historical and current relationship between neuroscience research and AI engineering.

Feedforward convolutional neural networks are currently the state-of-the-art approach for characterizing the transfer function of early visual cortical neurons. It is well known that the visual cortex has massive recurrent connections. Would models using recurrent circuits provide better neural response prediction than feedforward models? Here, we show that recurrent models consistently outperform feedforward ones when their parameters and hyperparameters are carefully matched. These benefits are robust against variation in the type of activation functions, loss function, number of channels, a

…Energy efficiency and data efficiency are the major features of the brain and human cognition that today’s deep learning has yet to deliver. The brain can be seen as a multi-agent system of heterogeneous learners using different representations and algorithms. The flexible use of reactive, model-free control and model-based “mental simulation” appears to be the basis for computational and d