LowRAM: Low-Rank Keys and Amortised Values for Composable KV Cache Compression at Test-Time
arXiv pre-print, 2026
Zafeirios Fountas
Co-founder & CEO · Empiria One
Brain-inspired AI · London
About
I'm the co-founder and CEO of Empiria One, a London start-up building private models that learn on the job. Previously I was a Principal Research Scientist and group leader at Huawei, where I led the brain-inspired computing team. My work sits at the intersection of cognitive neuroscience and machine learning, with a focus on memory, continual learning and decision making.
Today, I'm most excited by how ideas from the brain (episodic memory, consolidation, integrated reasoning and intrinsic motivation) can make large language models think and remember more like we do. Recent examples: EM-LLM (ICLR 2025), a brain-inspired episodic memory for LLMs; Bottlenecked Transformers (ICLR 2026), which periodically consolidate a transformer's working memory to improve reasoning; and Why the Brain Consolidates (2026), which argues that predictive forgetting is what lets memory generalise.
Before Huawei, I was a founding scientist at Emotech, where I led the AI behind Olly (a personal robot that won 4 CES Innovation Awards). I was also a research fellow at the Wellcome Centre for Human Neuroimaging, UCL, working with Karl Friston on active inference. Before that, I spent eight years at Imperial College London's Department of Computing, doing my PhD with Murray Shanahan.
News
- New preprint: LowRAM, predicting values from keys for near-lossless 4× value-cache compression in LLMs. Paper · Code · Hugging Face.
- New preprint with Haoyu Zhao and Jun Wang: Can Large Language Model Agents Learn by Watching Tutorial Videos? Tutorial Agentic Learning builds a persistent, structured memory from instructional videos without updating model weights.
- The recording of my Zangwill Club talk at Cambridge, Why Intelligence Requires Forgetting, is now on YouTube.
- Presented The Global Key-Value Workspace, an explicit global workspace inside pretrained LLMs, at ASSC 29 in Santiago de Chile. Our chapter Interactive Inference (with Roel Vertegaal and colleagues) is out in the 4th edition of the Human Computer Interaction Handbook.
- New position paper with twelve co-authors: Continual Learning Requires Evaluating Trajectories.
- New preprint with Neil Burgess: Why the Brain Consolidates: Predictive Forgetting for Optimal Generalisation (website).
- Seminars on episodic memory, consolidation and synergistic processing at UCL NeuroAI and at Imperial's ICARL seminar series.
- Bottlenecked Transformers accepted at ICLR 2026. New preprint with Imperial: A Brain-like Synergistic Core in LLMs Drives Behaviour and Learning.
- Three new preprints: SuRe on continual learning, Subjective Depth and Timescale Transformers on adaptive computation, and Emergent Bayesian Behaviour in LLMs on Bayesian cue combination.
- Talk at ASSC 28 in Heraklion: A Bayesian Account of Temporal Consciousness (slides).
- Presented EM-LLM at ICLR 2025 in Singapore. Our workshop paper on hierarchical episodic memory, led by Martin Benfeghoul, was a poster at the NFAM workshop.
Selected publications
Human-inspired Episodic Memory for Infinite Context LLMs
ICLR 2025
Talks
Why Intelligence Requires Forgetting: Memory and Consolidation in Brains and Language Models
Brains turn experience into durable, general knowledge largely by discarding. In this talk I argue that the limits of large language models are a memory problem rather than a scale problem: surprise-driven episodic segmentation produces event boundaries that track human event perception, and predictive forgetting, an outcome-conditioned compression of memory, provably optimises generalisation in high-capacity systems. Such models then become instruments for neuroscience, and raise a new question for AI evaluation.
- Invited seminar: Sussex AI, University of Sussex · Why Intelligence Requires Forgetting: Memory and Consolidation in Brains and Language Models Recording
- Invited talk: Zangwill Club, Department of Psychology, University of Cambridge · Why Intelligence Requires Forgetting: Memory and Consolidation in Brains and Language Models Listing · Recording
- Poster: ASSC 29, Santiago de Chile · The Global Key-Value Workspace: A Unifying Consciousness Theory Achieves State-of-the-Art Reasoning in LLMs Poster site
- Invited seminar: ICARL Seminar Series, Department of Computing, Imperial College London · Episodic Memory, Consolidation, and Synergistic Processing: Convergent Principles in Biological and Scalable Artificial Intelligence Listing
- Invited seminar: UCL NeuroAI online seminar series · Episodic Memory, Consolidation, and Synergistic Processing: Convergent Principles in Biological and Scalable Artificial Intelligence Listing
- Talk: ASSC 28, Heraklion · A Bayesian Account of Temporal Consciousness: From Event Processing to the Experience of Flow Slides
- Invited talk: Workshop Episodic Memory: Uniquely Human?, CPNSS, London School of Economics · Sense of time, events and episodic memory in humans and AI Programme
- Panellist: LSE Philosophy Live, London School of Economics · Living in the past: exploring memory in humans, animals, and artificial agents (with N. Clayton, F. De Brigard and J. Mahr; chair A. Boyle) Video · Podcast
- Organised workshop: Temporal Representations in Reinforcement Learning, RLDM 2022, Providence RI
- Invited keynote: 2nd International Workshop on Active Inference (IWAI), ECML/PKDD 2021
- Invited talk: Meet AI @ Shanghai · Challenges of brain-inspired AI: from lab to product
- Invited talk: AI Assistant Summit London · Imitating humans with brain-inspired AI
- Talk & demo, Royal College of Art (with A. Sylaidi) Slides
- Interview: The i newspaper · Scientists train AI to 'dream' about pink mugs like humans do
Earlier work
Olly: The 1st home robot with personality
I led the AI design behind Olly, a smart-home personal robot whose architecture was based on global workspace theory. Olly could predict users' personality and affective state, convey its own mood, and behave proactively rather than reactively.
Olly won 4 CES Innovation Awards in 2017 (the largest number ever awarded to a single robot product) and was demonstrated at the Royal Institution Christmas Lectures the same year.
NeuroBot: Passing the Turing test for AI bots?
NeuroBot was a large-scale spiking neural network of around 20,000 neurons that controlled an avatar in Unreal Tournament 2004 in a human-like manner. The high-level coordination was based on global workspace theory.
NeuroBot finished second in the 2K BotPrize human-like-bot competition at CIG 2011 in Seoul (the second-best score in the competition's history at the time), and was covered by New Scientist, EDGE and others. Early human-like AI work, well before LLMs made this fashionable.
PANDORA Robotics
Co-founded PANDORA, a student-led robotics research team at Aristotle University focused on rescue robots: vehicles that navigate disaster areas and locate human victims. The team went on to compete at RoboCup Rescue in 2011, 2013 and 2015, finishing second in the autonomy category in 2013 and 2015.
I started by designing and building a custom robotic arm, then led the arm and AI teams as we competed at the RoboCup Rescue Robot League world championships in Suzhou (2008) and Graz (2009, best European participation).
Biography
Experience
-
Co-founder & CEO
Empiria One, London · private models that learn on the job -
Principal Research Scientist, Group Leader
Huawei Technologies, UK · brain-inspired computing -
Honorary Research Fellow
Wellcome Centre for Human Neuroimaging, UCL · active inference, deep learning -
Visiting Lecturer
School of Design, Royal College of Art -
Research Associate & PI
Imperial College London · EU TimeStorm project -
Research Scientist, Founding Member
Emotech Ltd · personality modelling, AI lead for Olly -
Co-founder & Team Lead
PANDORA Robotics, Aristotle University of Thessaloniki
Education
-
PhD, Computational Neuroscience
Imperial College London, Department of Computing
Action Selection in the Rhythmic Brain. Supervisor: Murray Shanahan -
MSc, Computing Science (Distinction)
Imperial College London · Distinguished thesis -
Erasmus Exchange
City University London, Department of Mathematical Science -
BSc, Mathematics
Aristotle University of Thessaloniki
Selected awards & recognition
- 4 × CES 2017 Innovation Awards · lead AI designer for Olly (smart home, drones, home appliances, audio-video accessories), Las Vegas, January 2017
- Guest speaker, Royal Institution Christmas Lectures 2017 · presentation and demonstration of Olly
- Top Reviewer, NeurIPS 2024
- Google EMEA Scholarship, 2014–15
- EPSRC Doctoral Training Account Studentship, Imperial College London, 2012–2015
- 2nd best score in 2K BotPrize history (until 2011) · CIG 2011, Seoul
- Distinguished MSc thesis, Imperial College London, 2011
Professional service
- Area Chair, ICLR 2026
- Reviewer for NeurIPS, ICLR and ICML, and for Nature Human Behaviour, Trends in Cognitive Sciences, eLife and IEEE Access
Get in touch
Happy to hear from researchers and engineers working on memory, active inference, or anything brain-inspired. The fastest way to reach me is email.
zaf@empiria.one
Off-screen interests: chess, sailing, FPV, philosophy of mind, music.