flâneur

Speculative Programmatic Tool Calling | Alex L. Zhang

alexzhang13.github.io · 2,576 words · saved by 2 readers

Speculative programmatic tool calling is a class of techniques for overlapping tool call computation with the code being generated by a harness.

Fig 1. Speculative Programmatic Tool Calling (sPTC) Example the model's turn — streaming WITH SPECULATION DECODE EXECUTE prep sub-calls ×6 judge execute baseline answer · 7/7 claims WITHOUT SPECULATION · SAME PROGRAM answer c₁c₂ c₃c₄ c₅c₆ judge execute collapses — 2.4×, no async Time t The associated code is here: https://github.com/alexzhang13/spec-ptc It is no surprise from my previous writing and work on Recursive Language Models (RLMs) that I believe (1) code in a REPL is the only “tool” a system needs; (2) all other tools should be functions in that code tool. When code becomes…

saved by

related reading