Portfolio item number 1
Short description of portfolio item number 1
Posts by Collection
portfolio
Portfolio item number 2
Short description of portfolio item number 2
preprints
LoRA ensembles for large language model fine-tuning
Published in NeurIPS Workshop: Efficient Natural Language and Speech Processing, 2023
This paper investigates the use of LoRA ensembles for fine-tuning large language models.
Recommended citation: Wang X, Aitchison L, and Rudolph M. (2023). "LoRA ensembles for large language model fine-tuning." NeurIPS Workshop: Efficient Natural Language and Speech Processing.
Bayesian reward models for LLM alignment
Published in Workshop on Secure and Trustworthy Large Language Models, 2024
This paper explores Bayesian reward models for aligning large language models.
Recommended citation: Yang AX, Robeyns M, Coste T, Wang J, Bou-Ammar H, and Aitchison L. (2024). "Bayesian reward models for LLM alignment." Workshop on Secure and Trustworthy Large Language Models.
Sparse autoencoders can interpret randomly initialized transformers
Published in arXiv, 2024
This paper demonstrates how sparse autoencoders can be used to interpret randomly initialized transformers.
Recommended citation: Heap T, Lawson T, Farnik L, and Aitchison L. (2024). "Sparse autoencoders can interpret randomly initialized transformers." arXiv:2501.17727.
Download Paper
Function-space learning rates
Published in arXiv, 2024
This paper explores learning rates in function space.
Recommended citation: Milsom E, Anson B, and Aitchison L. (2024). "Function-space learning rates." arXiv:2502.17405.
Download Paper
Batch size invariant Adam
Published in arXiv, 2024
This paper introduces a variant of Adam that is invariant to batch size.
Recommended citation: Wang X and Aitchison L. (2024). "Batch size invariant Adam." arXiv:2402.18824.
Download Paper
Jacobian sparse autoencoders: Sparsify computations, not just activations
Published in arXiv, 2024
This paper introduces Jacobian sparse autoencoders that sparsify computations rather than just activations.
Recommended citation: Farnik L, Lawson T, Houghton C, and Aitchison L. (2024). "Jacobian sparse autoencoders: Sparsify computations, not just activations." arXiv:2502.18147.
Download Paper
Position: Don’t use the CLT in LLM evals with fewer than a few hundred datapoints
Published in arXiv, 2024
This position paper argues against using the Central Limit Theorem in LLM evaluations with small datasets.
Recommended citation: Bowyer S, Aitchison L, and Ivanova DR. (2024). "Position: Don't use the CLT in LLM evals with fewer than a few hundred datapoints." arXiv:2503.01747.
Download Paper
How to set AdamW’s weight decay as you scale model and dataset size
Published in arXiv, 2024
This paper provides guidelines for setting AdamW weight decay parameters when scaling model and dataset size.
Recommended citation: Wang X and Aitchison L. (2024). "How to set AdamW's weight decay as you scale model and dataset size." arXiv:2405.13698.
Download Paper
Questionable practices in machine learning
Published in arXiv, 2024
This paper examines questionable research practices in the field of machine learning.
Recommended citation: Leech G, Vazquez JJ, Kupper N, Yagudin M, and Aitchison L. (2024). "Questionable practices in machine learning." arXiv:2407.12220.
Download Paper
A self-improving coding agent
Published in ICLR Workshop on Scaling Self-Improving Foundation Models without Human Supervision (Oral), 2025
This paper presents a coding agent that can improve itself without human supervision.
Recommended citation: Robeyns M, Szummer M, and Aitchison L. (2025). "A self-improving coding agent." ICLR Workshop on Scaling Self-Improving Foundation Models without Human Supervision (Oral).
publications
Fast sampling-based inference in balanced neuronal networks
Published in NeurIPS, 2014
This paper presents methods for fast sampling-based inference in balanced neural networks.
Recommended citation: Hennequin G, Aitchison L, Lengyel M. (2014). "Fast sampling-based inference in balanced neuronal networks." NeurIPS.
Doubly Bayesian analysis of confidence in perceptual decision-making
Published in PLoS Computational Biology, 2015
This paper presents a Bayesian analysis of confidence in perceptual decision-making.
Recommended citation: Aitchison L, Bang D, Bahrami B, Latham PE. (2015). "Doubly Bayesian analysis of confidence in perceptual decision-making." PLoS Computational Biology.
The Hamiltonian brain: efficient probabilistic inference with excitatory-inhibitory neural circuit dynamics
Published in PLoS Computational Biology, 2016
This paper shows how excitatory-inhibitory neural circuits can implement efficient probabilistic inference.
Recommended citation: Aitchison L, Lengyel M. (2016). "The Hamiltonian brain: efficient probabilistic inference with excitatory-inhibitory neural circuit dynamics." PLoS Computational Biology.
Zipf’s law arises naturally when there are underlying, unobserved variables
Published in PLoS Computational Biology, 2016
This paper provides a theoretical explanation for the emergence of Zipf’s law in natural phenomena.
Recommended citation: Aitchison L, Corradi N, Latham PE. (2016). "Zipf's law arises naturally when there are underlying, unobserved variables." PLoS Computational Biology.
Active dendritic integration as a mechanism for robust and precise grid cell firing
Published in Nature Neuroscience, 2017
This paper shows how active dendritic integration contributes to robust and precise grid cell firing.
Recommended citation: Schmidt-Hieber C, Toleikyte G, Aitchison L, Roth A, Clark BA, Branco T, Häusser M. (2017). "Active dendritic integration as a mechanism for robust and precise grid cell firing." Nature Neuroscience.
Confidence matching in group decision-making
Published in Nature Human Behaviour, 2017
This paper examines how confidence matching affects group decision-making.
Recommended citation: Bang D, Aitchison L, Moran R, Castanon SH, Rafiee B, Mahmoodi A, Lau JYF, Latham PE, Bahrami B, Summerfield C. (2017). "Confidence matching in group decision-making." Nature Human Behaviour.
Model-based Bayesian inference of neural activity and connectivity from all-optical interrogation of a neural circuit
Published in NeurIPS, 2017
This paper presents methods for Bayesian inference of neural activity and connectivity from optical recordings.
Recommended citation: Aitchison L, Russell L, Packer A, Yan J, Castonguay P, Häusser M, Turaga SC. (2017). "Model-based Bayesian inference of neural activity and connectivity from all-optical interrogation of a neural circuit." NeurIPS.
With or without you: predictive coding and Bayesian inference in the brain
Published in Current Opinion in Neurobiology, 2017
This paper discusses the relationship between predictive coding and Bayesian inference in neural computation.
Recommended citation: Aitchison L, Lengyel M. (2017). "With or without you: predictive coding and Bayesian inference in the brain." Current Opinion in Neurobiology.
Model-based Bayesian inference of neural activity and connectivity from all-optical interrogation of a neural circuit
Published in Advances in Neural Information Processing Systems, 2017
This paper presents methods for inferring neural activity and connectivity from calcium imaging data.
Recommended citation: Aitchison L, Russell L, Packer AM, Yan J, Castonguay P, Häusser M, Turaga SC. (2017). "Model-based Bayesian inference of neural activity and connectivity from all-optical interrogation of a neural circuit." Advances in Neural Information Processing Systems.
Deep Convolutional Networks as shallow Gaussian Processes
Published in ICLR, 2019
This paper shows how deep convolutional networks can be understood as Gaussian processes.
Recommended citation: Garriga-Alonso A, Rasmussen CE, Aitchison L. (2019). "Deep Convolutional Networks as shallow Gaussian Processes." ICLR.
Tensor Monte Carlo: particle methods for the GPU era
Published in NeurIPS, 2019
This paper introduces Tensor Monte Carlo, adapting particle methods for modern GPU architectures.
Recommended citation: Aitchison L. (2019). "Tensor Monte Carlo: particle methods for the GPU era." NeurIPS.
Bayesian filtering unifies adaptive and non-adaptive neural network optimization methods
Published in NeurIPS, 2020
This paper shows how Bayesian filtering provides a unified framework for understanding neural network optimization.
Recommended citation: Aitchison L. (2020). "Bayesian filtering unifies adaptive and non-adaptive neural network optimization methods." NeurIPS.
Why bigger is not always better: on finite and infinite neural networks
Published in ICML, 2020
This paper analyzes the relationship between network size and performance in neural networks.
Recommended citation: Aitchison L. (2020). "Why bigger is not always better: on finite and infinite neural networks." ICML.
Cortical-like dynamics in recurrent circuits optimized for sampling-based probabilistic inference
Published in Nature Neuroscience, 2020
This paper shows how cortical-like dynamics emerge in circuits optimized for probabilistic inference.
Recommended citation: Echeveste R, Aitchison L, Hennequin G, Lengyel M. (2020). "Cortical-like dynamics in recurrent circuits optimized for sampling-based probabilistic inference." Nature Neuroscience.
Synaptic plasticity as Bayesian inference
Published in Nature Neuroscience, 2021
This paper shows how synaptic plasticity can be understood as a form of Bayesian inference.
Recommended citation: Aitchison L, Jegminat J, Menendez J, Pfister JP, Pouget A, Latham PE. (2021). "Synaptic plasticity as Bayesian inference." Nature Neuroscience.
A statistical theory of cold posteriors in deep neural networks
Published in ICLR, 2021
This paper develops a statistical theory explaining the cold posterior effect in deep neural networks.
Recommended citation: Aitchison L. (2021). "A statistical theory of cold posteriors in deep neural networks." ICLR.
Deep kernel processes
Published in ICML, 2021
This paper introduces deep kernel processes, a new class of probabilistic models.
Recommended citation: Aitchison L, Yang AX, Ober SW. (2021). "Deep kernel processes." ICML.
Global inducing point variational posteriors for Bayesian neural networks and deep Gaussian processes
Published in ICML, 2021
This paper presents a new variational inference method for Bayesian neural networks and deep Gaussian processes.
Recommended citation: Ober SW, Aitchison L. (2021). "Global inducing point variational posteriors for Bayesian neural networks and deep Gaussian processes." ICML.
BNNpriors: A library for Bayesian neural network inference with different prior distributions
Published in Software Impacts, 2021
This paper presents a library for Bayesian neural network inference with different prior distributions.
Recommended citation: Fortuin V, Garriga-Alonso A, van der Wilk M, Aitchison L. (2021). "BNNpriors: A library for Bayesian neural network inference with different prior distributions." Software Impacts.
Gradient regularisation as approximate variational inference
Published in Entropy, 2021
This paper shows how gradient regularization can be understood as approximate variational inference.
Recommended citation: Unlu A, Aitchison L. (2021). "Gradient regularisation as approximate variational inference." Entropy.
Tactile Image-to-Image Disentanglement of Contact Geometry from Motion-Induced Shear
Published in Conference on Robot Learning (CoRL), 2021
This paper presents methods for disentangling contact geometry from motion-induced shear in tactile images.
Recommended citation: Gupta AK, Aitchison L, Lepora NF. (2021). "Tactile Image-to-Image Disentanglement of Contact Geometry from Motion-Induced Shear." Conference on Robot Learning (CoRL).
Download Paper
Understanding the effectiveness of government interventions against the resurgence of COVID-19 in Europe
Published in Nature Communications, 2021
This paper analyzes the effectiveness of different government interventions in controlling COVID-19 resurgence in Europe.
Recommended citation: Sharma M, Mindermann S, Rogers-Smith C, Leech G, Snodin B, Ahuja J, Sandbrink JB, Monrad JT, Altman G, Dhaliwal G, Finnveden L, Norman AJ, Oehm SB, Sandkühler JF, Aitchison L, Gavenciak T, Mellan T, Kulveit J, Chindelevitch L, Flaxman S, Gal Y, Mishra S, Bhatt S, Brauner JM. (2021). "Understanding the effectiveness of government interventions against the resurgence of COVID-19 in Europe." Nature Communications.
Download Paper
A variational approximate posterior for the deep Wishart process
Published in NeurIPS, 2021
This paper presents a variational approximate posterior for the deep Wishart process.
Recommended citation: Ober SW, Aitchison L. (2021). "A variational approximate posterior for the deep Wishart process." NeurIPS.
Tactile Image-to-Image Disentanglement of Contact Geometry from Motion-Induced Shear
Published in Conference on Robot Learning (CoRL), 2021
This paper is about disentangling contact geometry from motion-induced shear in tactile images.
Recommended citation: Gupta AK, Aitchison L, Lepora NF. (2021). "Tactile Image-to-Image Disentanglement of Contact Geometry from Motion-Induced Shear." Conference on Robot Learning (CoRL).
Bayesian Neural Network Priors Revisited
Published in ICLR, 2022
This paper reexamines and analyzes priors for Bayesian neural networks.
Recommended citation: Fortuin V, Garriga-Alonso A, Wenzel F, Rätsch G, Turner R, van der Wilk M, Aitchison L. (2022). "Bayesian Neural Network Priors Revisited." ICLR.
Data augmentation in Bayesian neural networks and the cold posterior effect
Published in UAI, 2022
This paper analyzes the relationship between data augmentation in Bayesian neural networks and the cold posterior effect.
Recommended citation: Nabarro S, Ganev S, Garriga-Alonso A, Fortuin V, van der Wilk M, Aitchison L. (2022). "Data augmentation in Bayesian neural networks and the cold posterior effect." UAI.
Mass mask-wearing notably reduces COVID-19 transmission
Published in PNAS, 2022
This paper demonstrates the effectiveness of mass mask-wearing in reducing COVID-19 transmission.
Recommended citation: Leech G, Rogers-Smith C, Sandbrink JB, Snodin B, Zinkov R, Rader B, Brownstein JS, Gal Y, Bhatt S, Sharma M, Mindermann S, Brauner JM, Aitchison L. (2022). "Mass mask-wearing notably reduces COVID-19 transmission." PNAS.
A theory of representation learning gives a deep generalisation of kernel methods
Published in ICML, 2023
This paper develops a theory of representation learning that generalizes kernel methods, providing a deep generalization of traditional kernel approaches.
Recommended citation: Yang XY, Robeyns M, Milsom E, Anson B, Schoots N, Aitchison L. (2023). "A theory of representation learning gives a deep generalisation of kernel methods." ICML.
Deep Learning for Downscaling Tropical Cyclone Rainfall to Hazard-relevant Spatial Scales
Published in Journal of Geophysical Research: Atmospheres, 2023
This paper presents deep learning methods for downscaling tropical cyclone rainfall predictions.
Recommended citation: Vosper E, Watson P, Harris L, McRae A, Santos-Rodriguez R, Aitchison L, Mitchell D. (2023). "Deep Learning for Downscaling Tropical Cyclone Rainfall to Hazard-relevant Spatial Scales." Journal of Geophysical Research: Atmospheres.
An improved variational approximate posterior for the deep Wishart process
Published in UAI, 2023
This paper presents an improved variational approximate posterior for the deep Wishart process.
Recommended citation: Ober S, Anson B, Milsom E, Aitchison L. (2023). "An improved variational approximate posterior for the deep Wishart process." UAI.
Massively parallel reweighted wake-sleep
Published in UAI, 2023
This paper presents a massively parallel implementation of the reweighted wake-sleep algorithm.
Recommended citation: Heap T, Leech G, Aitchison L. (2023). "Massively parallel reweighted wake-sleep." UAI.
Robustness to corruption in pre-trained Bayesian neural networks
Published in ICLR, 2023
This paper analyzes robustness to corruption in pre-trained Bayesian neural networks.
Recommended citation: Wang X, Aitchison L. (2023). "Robustness to corruption in pre-trained Bayesian neural networks." ICLR.
Semi-supervised learning with a principled likelihood from a generative model of data curation
Published in ICLR, 2023
This paper presents a principled approach to semi-supervised learning based on a generative model of data curation.
Recommended citation: Ganev S, Aitchison L. (2023). "Semi-supervised learning with a principled likelihood from a generative model of data curation." ICLR.
Taylor TD-learning
Published in NeurIPS, 2023
This paper introduces Taylor TD-learning, a new approach to temporal difference learning.
Recommended citation: Garibbo M, Robeyns M, Aitchison L. (2023). "Taylor TD-learning." NeurIPS.
InfoNCE is variational inference in a recognition parameterised model
Published in TMLR, 2024
This paper shows how InfoNCE can be understood as variational inference in a recognition parameterised model.
Recommended citation: Aitchison L, Ganev S. (2024). "InfoNCE is variational inference in a recognition parameterised model." TMLR.
Convolutional Deep Kernel Machines
Published in ICML, 2024
This paper extends deep kernel machines to handle convolutional architectures.
Recommended citation: Milsom E, Anson B, Aitchison L. (2024). "Convolutional Deep Kernel Machines." ICML.
Bayesian low-rank adaptation for large language models
Published in ICML, 2024
This paper presents a Bayesian approach to low-rank adaptation for large language models.
Recommended citation: Yang AX, Robeyns M, Wang X, Aitchison L. (2024). "Bayesian low-rank adaptation for large language models." ICML.
Instruction Tuning With Loss Over Instructions
Published in NeurIPS, 2024
This paper presents a new approach to instruction tuning using loss over instructions.
Recommended citation: Shi Z, Yang AX, Wu B, Aitchison L, Yilmaz E, Lipani A. (2024). "Instruction Tuning With Loss Over Instructions." NeurIPS.
Position paper: Bayesian deep learning in the age of large-scale AI
Published in ICML, 2024
This position paper discusses Bayesian deep learning in the context of large-scale AI.
Recommended citation: Papamarkou T, Skoularidou M, Palla K, Aitchison L, et al. (2024). "Position paper: Bayesian deep learning in the age of large-scale AI." ICML.
Relating Human Error-Based Learning to Modern Deep RL Algorithms
Published in Neural Computation, 2024
This paper relates human error-based learning to modern deep reinforcement learning algorithms.
Recommended citation: Garibbo M, Ludwig CJH, Lepora NF, Aitchison L. (2024). "Relating Human Error-Based Learning to Modern Deep RL Algorithms." Neural Computation.
Signatures of Bayesian inference emerge from energy efficient synapses
Published in ELife, 2024
This paper demonstrates how Bayesian inference emerges naturally from energy efficient synapses.
Recommended citation: Malkin J, O'Donnell C, Houghton C, Aitchison L. (2024). "Signatures of Bayesian inference emerge from energy efficient synapses." ELife.
Snap-it, Tap-it, Splat-it: Tactile-Informed 3D Gaussian Splatting for Reconstructing Challenging Surfaces
Published in 3DV, 2024
This paper presents a method for reconstructing challenging surfaces using tactile-informed 3D Gaussian splatting.
Recommended citation: Comi M, Tonioni A, Yang M, Tremblay J, Blukis V, Lin Y, Lepora NF, Aitchison L. (2024). "Snap-it, Tap-it, Splat-it: Tactile-Informed 3D Gaussian Splatting for Reconstructing Challenging Surfaces." 3DV.
Stochastic Kernel Regularisation Improves Generalisation in Deep Kernel Machines
Published in NeurIPS, 2024
This paper shows how stochastic kernel regularization can improve generalization in deep kernel machines.
Recommended citation: Milsom E, Anson B, Aitchison L. (2024). "Stochastic Kernel Regularisation Improves Generalisation in Deep Kernel Machines." NeurIPS.
TouchSDF: A DeepSDF Approach for 3D Shape Reconstruction using Vision-Based Tactile Sensing
Published in IEEE Robotics and Automation Letters, 2024
This paper presents a DeepSDF approach for 3D shape reconstruction using vision-based tactile sensing.
Recommended citation: Mauro C, Yijiong L, Church A, Alessio T, Aitchison L, Lepora NF. (2024). "TouchSDF: A DeepSDF Approach for 3D Shape Reconstruction using Vision-Based Tactile Sensing." IEEE Robotics and Automation Letters.
Using autodiff to estimate posterior moments, marginals and samples
Published in UAI, 2024
This paper presents methods for using automatic differentiation to estimate posterior moments, marginals and samples.
Recommended citation: Bowyer S, Heap T, Aitchison L. (2024). "Using autodiff to estimate posterior moments, marginals and samples." UAI.
Residual Stream Analysis with Multi-Layer SAEs
Published in ICLR, 2025
This paper introduces a novel approach to analyzing the residual stream in transformer models using multi-layer sparse autoencoders.
Recommended citation: Lawson T, Farnik L, Houghton C, Aitchison L. (2025). "Residual Stream Analysis with Multi-Layer SAEs." ICLR.
Postprocessing East African rainfall forecasts using a generative machine learning model
Published in Journal of Advances in Modeling Earth Systems, 2025
This paper presents a generative machine learning approach for postprocessing East African rainfall forecasts.
Recommended citation: Antonio B, McRae ATT, MacLeod D, Cooper FC, Marsham J, Aitchison L, Palmer TN, Watson PAG. (2025). "Postprocessing East African rainfall forecasts using a generative machine learning model." Journal of Advances in Modeling Earth Systems.
talks
Deep kernel processes
Published:
Invited presentation at MILA on deep kernel processes and their applications in machine learning.
Deep kernel processes
Published:
Invited talk at the Gatsby Computational Neuroscience Unit on deep kernel processes.
Deep kernel processes
Published:
Invited talk at the GenU workshop focusing on deep kernel processes and their applications in uncertainty quantification.
(Convolutional) Deep Kernel Machines
Published:
Presentation on convolutional deep kernel machines at the Deep Learning Classics and Trends seminar series.
Deep kernel processes and machines
Published:
Invited seminar at the Oxford Computational Statistics and Machine Learning group discussing deep kernel processes and machines.
Massively parallel probabilistic inference
Published:
Presentation at Bristol statistics seminars on massively parallel approaches to probabilistic inference.
Deep kernel processes and machines
Published:
Invited talk at the Functional Inference and Machine Intelligence Workshop at the University of Bristol on deep kernel processes and machines.
teaching
SEMT20003: Methods of AI
Undergraduate course, University of Bristol, 1900
An introduction to modern AI for second-year designed for engineering mathematics students.