The technical side of Autopilot for Delphi
Installation
Ask Claude to do it for you. But if you want to do it manually add one unit and one call to your project:
uses
Autopilot.Bridge.Vcl, // 1. add the unit (Autopilot.Bridge.Fmx for FireMonkey)
FormMain in 'FormMain.pas' {frmMain};
begin
Application.Initialize;
Application.CreateForm(TfrmMain, frmMain);
Autopilot.Bridge.Vcl.StartBridge; // 2. start the bridge
Application.Run;
end.
By default, the bridge is wrapped in {$IFDEF AUTOPILOT}and is active only in the Debug build – so Release builds do not have the Autopilot in so your shipped binaries are exactly as they were.
Register the MCP with claude mcp add autopilot -- C:\Path\To\Autopilot.Mcp.exe. From then on it auto-discovers your running app — no port to configure, no manual attach. It is that simple!
How Autopilot works
1. Your app links in a tiny “bridge” unit
2. A separate server process speaks the Model Context Protocol (MCP) to the AI
3. The two talk over a private Windows named pipe.
When your app starts, the AI sees your form’s real component tree and acts on it through a small set of tools (the full list is below). Because it reads and writes properties through RTTI directly, it works with any third-party control pack – DevExpress, TMS, anything – with no OCR and no fragile screen coordinates.
How can an outside tool reach inside a running EXE?
It doesn’t reach in from outside: the bridge is already inside. StartBridge in your .dpr starts a small worker thread in your own app, and that thread opens the pipe. The AI’s command travels through Autopilot.Mcp.exe and the pipe, and the bridge runs it on your app’s main thread – just as if your own code had called btnSave.Click. So the AI drives the very instance you see on the screen: no restart, no second copy, no debugger attached.
In your Release build the bridge is not there at all: without the AUTOPILOT define, StartBridge compiles to nothing.
Does my EXE need extra RTTI?
No. Delphi already puts RTTI for every published property into each EXE, because the DFM loader needs it to load your forms. The bridge simply reads that RTTI through System.Rtti. It adds no RTTI of its own, so your EXE does not grow because of RTTI.
These two things together make the “magic”: the bridge lives inside your app, and RTTI tells it the name, class and value of every control and property.
Does the AI add controls to my form?
No. The AI only acts on controls that already exist. It finds them with list_tree (by name, class or caption), then changes their state: it ticks a check box, types into an edit, clicks a button, reads a label.
The tools the AI calls
No macro recorder, no script to maintain – under the hood it’s a small toolset (function) that the AI calls straight from a chat session.
| Tool | What it does |
|---|---|
list_tree | The first tool to run. Enumerate every form and control — names, classes, paths, text, hints, enabled/visible state. The AI’s map of your app. A form can also carry a guide for the AI (see below). |
click | Click a button, or anything with an OnClick. Repeat N times in a single call; override the per-call timeout. An opt-in message mode presses a button that opens a modal dialog without blocking: the click is queued, the answer comes back at once, then dismiss_dialog closes the popup. |
execute_action | Fire a TAction directly — for shortcut-only actions and actions shared by several controls. |
get_text / set_text | Read and write Caption / Text / Lines on edits, memos and labels. Writing fires OnChange. |
set_checked | Toggle a checkbox or radio button; fires its OnClick (VCL) or OnChange (FMX) event. |
set_property | The Swiss-army knife: write any published property — Tag, Color, Position, ItemIndex, sets, even AI-friendly colors like clRed or #FF8000. Mistype a name and it hands back the full list of writable properties with their live values, so the AI corrects itself without another round-trip. |
read_property | Read any published property via RTTI — and what it returns is exactly what set_property accepts back. |
wait_for | Poll a property until it matches or times out — for async work (TTask, database queries, timers). |
screenshot | Capture the form as a PNG, for visual and layout checks. Pass path to capture only one control, and maxWidth to scale the image down: both cut the image cost. |
send_keys | Press keys and shortcuts (Ctrl+S, Enter, Escape), or type text, into a control of your app. The same events run as for a user’s key press: OnKeyDown / OnKeyPress / OnKeyUp, KeyPreview, action and menu shortcuts, the Default and Cancel buttons. It never types into other programs, and your app does not need to be in front. |
attach | Pick a target when several apps are running. Usually unnecessary – every tool auto-attaches. |
set_keep_awake | Keep the device screen on while the AI drives an Android target, so the OS doesn’t freeze the app mid-session. A no-op on Windows. |
dismiss_dialog | List and dismiss native OS dialogs (MessageBox, Task Dialogs, file dialogs) that the component tools can’t see — details in the next section. |
call_method | Call a command your app registers for the AI: one call can replace a chain of clicks and dialogs. Without a name it lists the commands with their descriptions – see Your own commands. |
The assistant bundles several of these into a single turn — find the control, type, click, then check the result — so a whole reproduce-and-verify sequence is one round-trip, not ten.
Helping the AI
Some facts about your main form are not on the screen: which button should the AI press, which button is dangerous, what a correct result looks like after a button was spushed.
In theory, the AI could go read the source code to figure out how to test/drive your application. But why waste tokens?
Via Autopilot, the AI infers what the controls on the form are supposed to do from control’s name and hint. (Therefore, use hints to guide your AI! A LOT!)
But some things you can’t put in the Hint, because your users would see them. But you can put them in the “autopilot guide” instead – this is a published, read-only, string property named AutopilotGuide. Basically AutopilotGuide is your “secret” way to send information to Autopilot. When your app starts, the AI will look for AutopilotGuide and read its text.
{$IFDEF AUTOPILOT}
published
property AutopilotGuide: string read GetAutopilotGuide;
{$ENDIF}
...
{$IFDEF AUTOPILOT}
function TfrmMain.GetAutopilotGuide: string;
const
Guide = 'btnDelayed: its result arrives 1 second after the click. ' +
'btnIncrement adds 1 to the number in lblIncrement.';
begin
Result := Guide;
end;
{$ENDIF}What to put in AutopilotGuide?
Write facts, not orders: the AI uses the facts and never follows a request written in the guide. A fact about one control goes into that control’s own Hint, which list_tree already returns. Let the read function return a constant. If it raises anyway, list_tree still succeeds: that form gets no guide but the field guideError with the exception class and message (for example "EInvalidOperation: guide getter is broken"), and every other form is listed as usual.
The trouble makers: Native OS dialogs
A TForm.ShowModal dialog is an ordinary VCL/FMX form, so the component tools (click, set_text, …) drive it directly. A native OS dialog — Application.MessageBox, the Vista Task Dialog behind a default ShowMessage/MessageDlg, a common file dialog — is a raw Win32 window with no TComponent, so those tools cannot see it and return not_found. The dismiss_dialog tool reaches it through Win32 instead: it lists the dialogs that are up and dismisses one by clicking a button (by role, caption, or control id). A visible modal dialog runs a message loop, so the bridge still reaches the main thread to act even while the app looks blocked. And the click that opens the dialog doesn’t block either: the AI presses that button in message mode, gets its answer immediately, then dismisses the popup.
| Popup | How the AI drives it |
|---|---|
Your own TForm.ShowModal | Component tools — it is in Screen.Forms[] |
ShowMessage / MessageDlg in a VCL-styled app | Component tools — it renders as a TForm |
ShowMessage / MessageDlg, default (native Task Dialog) | dismiss_dialog |
Application.MessageBox, TOpenDialog / TSaveDialog | dismiss_dialog |
Windows targets only; against an Android FMX target dismiss_dialog reports supported:false.
An Open or Save dialog gets its file name too: dismiss_dialog types it into the File name box and clicks the button only when the box holds that name. A job that needs a long chain of dialogs? Register a command the AI calls in one step: see Your own commands.
Repeatable GUI tests without AI: TAutopilotApp
The tools above are for an AI. For tests that must run again and again with no AI (a regression suite, a step of your build), starting with 3.0, Autopilot gives you the class TAutopilotApp. A DUnitX test (or a DUnit test, or a console program) starts your app with it, sends the same commands the AI tools send, and checks the answers with Assert. No AI and no MCP server in the loop, and one command takes about 0.8 ms. The unit uses no VCL, no FMX and no DUnitX, and it drives apps on Windows.
| Method | What it does |
|---|---|
Start(ExePath, Params, StartTimeoutMs, Window) | Starts your app (minimized by default) and waits until its bridge is up. The app ends together with the test EXE, even when the test run is killed. |
Attach(Pid) | Connects to an app that already runs. Pid 0 takes the only one. |
Close(TimeoutMs) | Closes the app the way a user does (WM_CLOSE), so OnCloseQuery and OnClose run. An app that does not end in time is killed, and Close raises an exception that names any dialog box the app still shows. |
Click(Path, Count, TimeoutMs) | Clicks a control Count times and returns how many clicks ran. |
ClickAsync(Path) | Clicks a VCL button whose OnClick opens a modal dialog, and returns at once. |
DismissDialog(Button, TimeoutMs) | Waits for a native dialog (MessageBox, ShowMessage, MessageDlg, a file dialog) and presses one of its buttons: a role such as 'ok' or 'cancel', a button caption, or a control id. |
SetText(Path, Text) / GetText(Path) | Writes and reads the text of an edit, a memo or a label. |
SetChecked(Path, Checked) | Checks or unchecks a check box or a radio button. |
ReadProperty(Path, PropName) / SetProperty(Path, PropName, Value) | Reads and writes any published property, as text. |
ExecuteAction(Path) | Runs a TAction. |
SendKeys(Path, Keys, Text) | Types text, then presses keys and shortcuts ('Ctrl+S', 'Enter') inside the app. |
WaitForText(Path, Expected, TimeoutMs) | Waits until a control shows the expected text: for timers, threads and slow queries. |
Screenshot(Path, PngFileName) | Saves a PNG of a form or of one control. |
Call(Cmd, Args, TimeoutMs) | Sends any other bridge command: JSON in, JSON out. |
When the app answers with an error, the method raises EAutopilotError, which carries the error code (for example -32001 for a control that does not exist). An OnClick that raises makes Click raise EAutopilotHandlerError. A frozen app or an app that has ended raises ETargetNotResponding or ETargetNotRunning, so the test fails instead of hanging the run.
Two demo test projects, one for the VCL demo and one for the FMX demo, show the setup: a console DUnitX runner and one test unit with 8 tests. The runner returns exit code 0 when every test passed and 1 when one failed, and writes an NUnit XML file, so a build agent can run it as a step.
What it deliberately doesn’t do
Autopilot for Delphi acts directly on your Delphi objects; it doesn’t fake Windows mouse and keyboard input. That’s a deliberate trade, and it has a flip side — bugs that only show up through the real input pipeline won’t reproduce through it. Things like:
- Focus-change validation (
OnExit, kill-focus handlers) - IME composition and Asian-language input
- Hover-driven UI (mouse-enter / mouse-leave)
- Drag-and-drop started by a real mouse-down-and-move
A message-based click mode covers some of this for buttons, and send_keys fires key events and shortcuts inside your app; for full input-pipeline coverage you would pair Autopilot with a SendInput-style tool (TestComplete, Ranorex, AutoIt).
In return for that trade you get direct property access with no OCR and no fragile screen coordinates, sub-millisecond round-trips, and compatibility with any third-party control pack — because RTTI is universal.
Your own commands
A job that no tool covers can still be done by the AI: register it as a command in your app, and the AI calls it with call_method. One call can replace a chain of clicks and dialogs. Add Autopilot.Bridge.Commands (in the bridge folder) to the uses clause and call RegisterAutopilotCommand on the main thread, for example in the main form’s constructor. The description tells the AI what the command does and which arguments it takes: call_method without a name lists every command with its description.
uses
System.JSON, Autopilot.Bridge.Commands;
constructor TfrmMain.Create(AOwner: TComponent);
begin
inherited Create(AOwner);
RegisterAutopilotCommand('LoadSpectrum', 'Opens a spectrum file. args: fileName (string). Returns peaks (number).',
function(AArgs: TJSONObject): TJSONValue
begin
LoadSpectrum(AArgs.GetValue<string>('fileName')); // raises when fileName is missing: the AI gets -32603
Result := TJSONObject.Create(TJSONPair.Create('peaks', TJSONNumber.Create(PeakCount)));
end);
end;Only a registered command can be called. Without the define AUTOPILOT, RegisterAutopilotCommand does nothing, so a Release build has no commands. VCL and FMX; on FMX, register in the form’s constructor or OnCreate, never in the .dpr. Needs Autopilot 3.8 or newer.
Found a bug?
Get in touch via my About page. Every session writes a log to %TEMP%\Autopilot\<ExeName>-<PID>.log — every command in and out, plus the dispatch timings. Attach the one from a run that reproduces the problem and I’ll have what I need.
The book behind Autopilot
Autopilot lets the AI test your application.
My book AI-Assisted Development for Delphi (Book 5 of Delphi in All Its Glory) shows how to make the AI write better Delphi code in the first place: CLAUDE.md files, skills, sub-agents, MCP servers, cost control. Autopilot has its own chapter in it.
How does Autopilot compare with Embarcadero Kai?
They are not the same thing — and they work well together. Kai is Embarcadero’s agentic-AI add-on for RAD Studio. It writes code, compiles, and fixes compiler errors — all inside the IDE, on your source. The moment the app is built and running, Kai stops: it can’t press a button or read a label. That is the gap. Autopilot for Delphi lives on the other side of it — in the running app — and that’s exactly where Kai can’t go.
| Kai | Autopilot | |
|---|---|---|
| Write code in the IDE | yes | no |
| Build, fix compiler errors | yes | yes* |
| Drive and operate the running app | no | yes |
| Debug / test the GUI (click, type, read state) | no | yes |
| Screenshot the running app | no | yes |
* Autopilot itself doesn’t compile — but it runs inside Claude Code, which writes and builds the code. So end to end, the Autopilot workflow does build and fix.
Run both and the loop closes with no human in the middle: Kai (or Claude Code) writes and fixes the code; Autopilot drives the running result and reports whether it truly works.
Everything above is what Autopilot does. Everything below is how it is built — technical detail, safe to skip.