Tools and Introspection

The tool model is the center of the whole system. Fisk AI never lets a model run arbitrary commands. It introspects a fisk application once, turns its runnable commands into named tools, and then decides which of those tools a model is even allowed to see.

From a command tree to a tool list

ToolsForApp execs the target binary with --fisk-introspect and unmarshals the emitted fisk.ApplicationModel (fisk.go:696, fisk.go:714). ApplicationTools then walks the command tree with commandTools (fisk.go:558, fisk.go:584). The walk keeps only runnable leaf commands: a hidden command is skipped, and a grouping node with subcommands is recursed into but is not itself a tool. Every surviving leaf becomes one Tool bound to its fisk.CmdModel, the source of truth for the command’s schema and tags.

A tool’s name is its command path joined with underscores. The command auth user add becomes the tool auth_user_add (Tool.Name, fisk.go:103). That name is what the model addresses, what include and exclude rules match against, and what a remote import prefixes with its alias.

Reserved tags

Three tags carry meaning to Fisk AI itself. The rest are free-form and can only be referenced by include and exclude rules. Tags are set in the fisk command definition, or in YAML for an App Builder application.

Selecting tools

LoadTools is the pipeline (util.go:221). It always runs one filter pass first with a nil filter, which strips ai:deny and nothing else, so deny is enforced even when no rules are configured. Then, if include names any tools or tags, only matching tools are kept; if exclude names any, matching tools are dropped.

A rule’s tools entries are regular expressions matched against the tool name. Its tags entries are exact tags, where an empty string matches a tool with no tags at all. A tool matches a rule if any name regex matches or any tag is present (matchesFilter, fisk.go:666). Include and exclude can be combined: include ^cow but exclude ^cow_think.

Tool search deferral

Recent Anthropic models can search a large tool set on demand rather than receive every definition up front. Fisk AI uses this above a threshold. BuildToolParams counts local, remote, and built-in tools together; at ten or more (toolSearchThreshold, anthropic.go:21) it marks tools for deferred loading and appends the BM25 tool-search server tool. Below the threshold every tool is sent directly and no search tool is added.

A tool tagged ai:no_defer is always sent directly even in a deferred set, which keeps the handful of commands a model needs on most requests immediately available. Because the model discovers deferred tools by their description, ModelDescription appends a trailing Tags: ... line so a tag search can find them (fisk.go:126).

Running a tool

Tool.Execute turns the model’s JSON arguments into a command line with t.Model.ArgsFromJSON, the trust boundary that bounds input to the command’s schema, then runs exec.CommandContext with the argument vector (fisk.go:449). No shell is involved, so model input can never be interpreted as a shell command. A non-zero exit is a normal result carrying the exit code, not an error, so the model can react to it. Output handling and credential stripping are detailed in Safety and Human in the Loop.