ToolFormerMicro: Composable Tool Schema Compression via Gated Cross-Attention
ToolFormerMicro: Composable Tool Schema Compression via Gated Cross-Attention
ToolFormerMicro is a ~428M parameter encoder-decoder model that compresses verbose tool schemas into compact 8-token gist vectors via gated cross-attention. Initialized from Qwen2.5-0.5B and trained with a three-stage curriculum (schema auto-encoding, contrastive discrimination, end-to-end tool calling), the model achieves 0.818 Tool Selection Accuracy with zero false positives across seen, held-out, and unseen tool splits. Three composability experiments verify order independence (delta=0.000), sub-linear scaling to 200 tools, and bit-identical cache invalidation on hot-swap — properties that make gist representations a drop-in composable primitive for tool-augmented LLM serving.
encoder-decoder, tool-augmented language models, composable caching, gist tokens, context compression, cross-attention
encoder-decoder, tool-augmented language models, composable caching, gist tokens, context compression, cross-attention
selected citations These citations are derived from selected sources.This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).0 popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.Average influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).Average impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.Average
Citations provided by BIP!Popularity provided by BIP!