Progress on provable inference for large language models