In short. Toolformer teaches a language model to call tools by generating candidate calls, filtering them by prediction loss, and training on the retained examples.
Definition: Toolformer: Language Models Can Teach Themselves to Use Tools is a research paper presenting a method for training a language model to select and call external tools. The model generates candidate calls, retains those whose responses help predict subsequent text, and learns from the resulting examples.
Key facts
- The arXiv record dates the initial submission by Timo Schick and colleagues to 9 February 2023.
- The NeurIPS publication lists a calculator, a question answering system, a search engine, a translation system, and a calendar among the tools studied.
- The paper's tool specification requires text inputs and outputs, plus a few demonstrations of intended use for each tool.
How does the model learn to call tools?
An application programming interface (API) lets the model request a tool operation and receive its result. In Toolformer's training procedure, human-written demonstrations show how to insert calls into text. The model then proposes candidate calls at selected positions in a language modeling dataset, and the training pipeline executes them.
A loss measures the model's error when predicting subsequent tokens, the units of text it generates. The pipeline keeps calls whose returned results reduce that loss by a chosen threshold. It compares the call plus its result against both an absent call and a call with no returned result. This makes the usefulness of the response part of the selection criterion.
The retained calls and responses are inserted into the original texts. The model is then fine-tuned on this augmented dataset, meaning its parameters are updated through further training. The experiments use GPT-J as the language model and a subset of CCNet as the text dataset.
What does a tool call look like?
The paper's calculator example, written with the practical token sequences described in its implementation note, has this form:
[Calculator(400 / 1400) -> 0.29]
The tool name identifies the operation, the parentheses hold its input, and the arrow introduces the returned result. During text generation, the model produces the call up to the arrow. The surrounding execution process pauses generation, executes the tool, inserts the response and closing marker, then resumes generation.
For search, the experiment uses a system that returns short Wikipedia snippets. Its retrieval method, BM25, matches a query against an index built from the Wikipedia collection supplied with KILT. The model extracts relevant information from the returned text.
What limitations does the paper document?
The limitations section identifies an inability to chain tools, meaning that one tool's output becomes another tool's input. Training calls are generated independently for each tool, leaving the training dataset without examples of chained use. The method also lacks interactive search actions such as browsing successive results or refining a query.
Tool selection is sensitive to the wording of the input. Producing useful training examples can also require substantial processing: the authors report that more than a million documents yield only a few thousand useful calculator calls. The call decision omits the computational cost of the selected tool.
The experimental setup permits at most one API call per input to prevent repeated calls from forming a loop.
Related terms
FAQ
Does self-supervised training remove human demonstrations?
The paper's abstract specifies a handful of demonstrations for each API. The model generates additional calls, and the loss-based filter selects useful examples for further training.
Does Toolformer search the live web?
The documented search tool searches a Wikipedia collection and returns snippets. Its index comes from KILT, and its retrieval method is BM25.
Can I use the paper to choose website changes for AI search?
The same paper's experiments evaluate tasks such as factual completion, mathematics, and question answering. The paper supplies no test of website changes or citation rates in current answer engines.
Sources
- arXiv, "Toolformer: Language Models Can Teach Themselves to Use Tools", 9 February 2023
- NeurIPS, "Toolformer: Language Models Can Teach Themselves to Use Tools", 2023, publication record
- NeurIPS, "Toolformer: Language Models Can Teach Themselves to Use Tools", 2023, full paper
Check how a page reads to answer engines. Open the GEO score analyzer