How many cosmic rays can GPT-2 survive?
About a dozen random bit flips break it, almost all through a single exponent bit.
I'm a Turing Scholar at UT Austin studying computer science and mathematics. I'm currently Co-founder of Tacit Research. Before that I worked on automated theorem proving in the UToPiA Lab, where I worked on making LLM-based provers better at Lean 4 by using RL to learn when to retrieve custom tactics.
How many cosmic rays can GPT-2 survive?
About a dozen random bit flips break it, almost all through a single exponent bit.
Tacit Research — Co-founder, YC F26
Working on continual learning through compressed KV caches.
UToPiA Lab, UT Austin — Undergraduate Researcher
Led a project with Prof. Isil Dillig on LLM-based theorem proving in Lean 4: a dynamic retrieval system that injects relevant custom proof tactics into the prover at inference time, with RL used to learn when to retrieve them.
Lean 4 Directed Reading Program
Functional programming and formal verification in Lean 4. Proved properties of quicksort by hand, which is what got me interested in formal reasoning.
University of Texas at Austin — CS (Turing) + Mathematics