Autopilot for Delphi – What’s New

The changes in Autopilot for Delphi, newest first.

Version 3.8

Autopilot now has 15 tools: call_method is new.

  • New tool: call_method. It calls commands that the application registers for the AI with RegisterAutopilotCommand (unit Autopilot.Bridge.Commands, VCL and FMX). One call can replace a chain of clicks and dialogs, and only a registered command can be called. Without a name it lists the commands with their descriptions. A handler that puts a part of its own arguments into its result (for example Result.AddPair('x', AArgs.GetValue('y'))) does not crash the app: the bridge sends a copy and writes a warning to its log.
  • New: the AI can drive the Delphi IDE itself. The design-time package Source\IdeBridge\Autopilot.IdeBridge.dpk loads the bridge into the Delphi 13 IDE, so the AI can read and drive the IDE’s own windows (tested: 41 IDE forms, 3,503 components listed in 58 ms).
  • New: “which control is at this point?” list_tree takes x and y (with form): it returns only the control at that pixel of the form’s screenshot, VCL and FMX. A point on a hidden tab page never matches; a control that list_tree does not list gives its nearest listed parent.
  • New: list_tree takes two optional filters, form and formsOnly. form lists only the nodes of one form, formsOnly only the form nodes, so a large application such as the Delphi IDE can be listed without filling the AI’s context window. The bridge filters, so the rest of the tree never leaves the application.
  • New: list_tree takes an optional path, to list one panel, frame or menu and everything shown inside it; useful when one form is too big.
  • New: dismiss_dialog can fill the File name box of an Open or Save dialog. The new argument fileName types the name, and the button is clicked only when the box holds exactly that name. Autopilot.Client has the same: DismissDialog takes a FileName.
  • New: dismiss_dialog has waitMs: right after the click that opens a dialog, one call waits for the dialog instead of answering “no dialog”.
  • New: set_checked and read_property take an optional index, to check, uncheck or read one item of a VCL TCheckListBox; OnClickCheck fires as for a mouse click.
  • Improved: click mode='message' now works on every control that can be clicked (menu items, toolbar buttons, speed buttons, labels) and on FMX too: the click runs after the answer, so a control that opens a modal dialog no longer blocks the AI.
  • Improved: the six tools that change nothing in your app (attach, list_tree, get_text, read_property, wait_for, screenshot) now tell the AI host they are read-only, so Claude Code can send their calls at the same time.
  • Improved: the -32004 answer says whether the command already started (it waits in a dialog it opened) or never started, and that it must not be sent again; error.data.started carries the same.
  • Fixed: dismiss_dialog could click the wrong button of a Task Dialog (a default MessageDlg or ShowMessage): button: no on a Yes/No/Abort box made the program take its default button, Yes. The bridge now sends a button’s click to the window that really holds the button, and refuses a button it cannot name for sure (new reason button_ambiguous), instead of guessing. Rebuild your app to get this fix: the bridge is compiled into it.
  • Fixed: dismiss_dialog now reads the text of a Task Dialog, the dialog Delphi’s ShowMessage and MessageDlg show (it was empty).
  • Fixed: set_text, set_checked and set_property no longer touch a control that the app’s own code freed during the command (for example an OnChange that closes its frame); the answer says freed: true.
  • Fixed: StopBridge could wait for ever when the app closed while the bridge was between two client connections. The bridge now keeps waking its worker thread until that thread has ended. Thanks to waldirpaim, who found the cause and sent the fix in GitHub pull request 4.
  • The VCL demo has a MessageDlg button and an Open and a Save file dialog button, so dismiss_dialog is tested on a Task Dialog shown by the app’s own main thread.
  • Docs: screenshot captures the client area of a form only.

Version 3.7

  • New: a form can tell the AI what the screen does not show. Give the form a published read-only string property AutopilotGuide (under {$IFDEF AUTOPILOT}, so a Release build carries none), and list_tree sends its text in the field guide of that form. Write facts there: the order of the steps, what a correct result looks like, which actions are dangerous, results that arrive later. The AI uses these facts and never follows a request written in the guide. If the property’s read function raises an exception, list_tree still lists every form and reports the exception in the field guideError of that form.
  • New: read_property names the event handler. Read an event such as OnClick or an action’s OnExecute and the answer gives the routine that runs, for example TfrmMain.btnSaveClick, and the unit that declares it, so the AI goes straight from a button on screen to its code. A handler declared in a base form names the base form. VCL and FMX. Events stay read-only: set_property never assigns a handler.
  • New: a wrong enumeration value lists the allowed ones. When set_property gets an identifier that the enumeration or the set does not have, the error message names the identifiers it accepts, for example allowed: alNone, alTop, alBottom, alLeft, alRight, alClient, alCustom, so the AI gets the next call right, also for the app’s own enumerations. VCL and FMX.
  • Fixed: set_property wrote an enumeration ordinal outside the enumeration’s range (for example '99') into the property. It now answers an error that gives the valid range and the allowed identifiers.
  • New: set_property returns the old value. The answer carries oldValue, the value the property had before the write, so the AI can put it back with one more call. VCL and FMX.
  • New: list_tree gives the AI the hint of each control. The hint you wrote for a control, a menu item or an action (its tooltip text) now comes in the field hint, so the AI learns what a control does before it clicks it. A control bound to an action shows the action’s hint. To keep the answer short, the field is left out when the hint is empty or the same as the control’s text.
  • Fix: dismiss_dialog could report a click on a message box that stayed open. Windows may ignore a button click when the dialog is not the active window, for example a message box shown while the application starts. The bridge now sends the dialog the same notification the button sends on a real click, so the box closes without being activated. The answer says via:"BN_CLICKED" (it said "BM_CLICK" before). A disabled button now answers clicked:false with reason:"button_disabled".
  • Fix: an error inside a control’s property no longer vanishes without a trace. When reading or writing a property raised an exception (for example a getter that fails), the bridge ignored it silently. The answer stays the same, but the bridge now writes a warning line to its log %TEMP%\Autopilot\<ExeName>-<PID>.log, on VCL and FMX.
  • Fix: the MCP server no longer leaks memory when it builds an error answer.

Version 3.5

  • Crash reports are now opt-in. By default the MCP server never sends a crash report: it only saves it on your PC, to %TEMP%\Autopilot\bugreport Autopilot.Mcp.txt. To send it to us, add the parameter --crash-report when you register the server: claude mcp add autopilot -- <path>\Autopilot.Mcp.exe --crash-report. The parameter --no-crash-report still works and wins over --crash-report.
  • The VCL and FMX demos now look the same. Both have the same size and layout, the Autopilot logo at the top, a Help menu with “Check for updates” (this page) and “About”, and a link to the home page at the bottom.
  • The demos link to the book behind Autopilot. A line above the home-page link opens the page of AI-Assisted Development for Delphi, Book 5 of “Delphi in All Its Glory”.
  • The native dialog of the demos explains itself. Its text now says that the main thread waits inside the message box until you close it, that this is how a Windows message box works (Autopilot does not cause it), and that an AI closes it with dismiss_dialog.
  • New: the VCL screenshot now draws TLinkLabel and TComboBoxEx controls. These two controls came out blank, so the bridge now paints them into the picture itself. The links of the VCL demo now show in its screenshots too.
  • No more hint H1054 in your builds. The bridge no longer prints a hint each time your project compiles it.

Version 3.1.3

  • The MCP server is now built from its source. The repository holds the source of Autopilot.Mcp.exe, not the EXE. Give your AI the install prompt: it builds the EXE and registers it. With Delphi Community Edition, which cannot compile from the command line, the AI asks you to build it once in Delphi (Source\McpServer\Autopilot.Mcp.dproj, Release, Win32). The steps: How to install it.
  • Fixed: the MCP server finds the bridge folder when it runs from its build folder. An EXE in Source\McpServer\Win32_<config>\ now names Source\Bridge in its setup steps, the same as an EXE in Source\McpServer\.
  • Fixed: the description of the list_tree tool. It said that each part of a path like Form.A.B.C is a direct child of the part before it. It now says what the bridge does: each part is owned by the part before it, and a path the owner tree cannot resolve is tried again through the visual parents. It also warns that the number in a name like @TButton#3 changes when an earlier unnamed component is freed.
  • Fixed: the Claude Code agent autopilot-script-runner can close dialogs. It now has all 14 tools. dismiss_dialog and set_keep_awake were missing, so a script that opened a message box could not close it.
  • The AI briefing AI-INSTRUCTIONS.md is shorter and corrected. Each rule is stated once, and stale statements were fixed: for example, it said that keyboard shortcuts cannot be tested, but send_keys presses them.

Version 3.1.2

  • The source of the MCP server is in the repository (Source\McpServer\), and it builds with or without madExcept.

Version 3.1.1

Version 3.1

  • New: the parameter --no-crash-report for Autopilot.Mcp.exe. With it, the MCP server never sends a crash report: it only saves it on your PC, to %TEMP%\Autopilot\bugreport Autopilot.Mcp.txt. Add it when you register the server: claude mcp add autopilot -- <path>\Autopilot.Mcp.exe --no-crash-report.

Version 3.0

  • New page: Security and privacy. The short answers for a Compliance or Risk team: what Autopilot can read, where the data goes, and how to keep it to test builds, test data and one project.
  • New: repeatable GUI tests in DUnitX, with no AI. The new unit Autopilot.Client lets a DUnitX test (or any Delphi program) start your app, click, type, read labels and properties, close dialogs and take screenshots, then check the result with Assert. No AI and no MCP server in the loop, and about 0.8 ms per command. Your app ends together with the test EXE, even when the test run is killed. Two demo test projects, one for the VCL demo and one for the FMX demo, show the setup. Their runner returns exit code 0 when every test passed and 1 when one failed, and writes an NUnit XML file, so a build agent can run it as a step. For Windows targets. How to use it: Automated GUI testing in Delphi.
  • Fixed: a screenshot of a minimized VCL form now shows its labels and speed buttons. A minimized form paints only its icon, so the controls without a window of their own were missing from the image. Now they are there, the image has the form’s normal (restored) size, and the form stays minimized.

Version 2.1

  • New tool send_keys. The AI can now press keys and shortcuts, or type text, into a control of your app. The same events and shortcuts run as for a user’s key press. It never types into other programs, and your app does not need to be in front.
  • Cheaper screenshots. screenshot takes two new options: path captures only one control, and maxWidth scales the image down. Both cut the image cost.

Version 2.0 and earlier

  • A new home for the documentation. Everything about Autopilot now lives on this website: the main page, the technical details, and this list of changes.
  • Screenshots arrive as a picture. screenshot now returns the PNG as an image your assistant can look at, plus a short text part with the form name, width and height. Before, the picture came as a long base64 text, and the AI never saw it as an image.
  • A frozen or busy app gives a clear answer. When the app is stopped on a breakpoint or hung, so that its connection stays busy, the tools now answer error -32098 target_not_responding instead of a raw “could not open pipe” text. wait_for stops at the first -32098 instead of polling on until its deadline.
  • A control moved onto another form is found through the form that shows it. A panel created on one form and placed on a tab sheet of another can now be reached as MainForm.tabFixEnters.Container. If two controls fit such a path, the answer is -32002 ambiguous_path with the paths to use instead, and list_tree marks a moved control with a parent field.
  • Fixed trap: A click on a control bound to an action runs the action, the same as a mouse click: OnExecute gets the action as Sender, AutoCheck works, and a standard action such as TFileExit no longer fails.
  • No more debugger stops on third-party components with unusual property types, such as DevExpress bar items, whose Visible is an enumeration. The bridge now checks a property’s type before it reads or writes it, so it raises no hidden exception, and a debugger that stops on exceptions no longer halts your app during list_tree. The same holds when your app answers after the assistant has stopped waiting.
  • set_property can enable a disabled control again. On a disabled control it now accepts the property Enabled (true or false); every other property still answers -32003 control_disabled.
  • Named colors read back as claRed, as documented: a TAlphaColor (the FMX color type) used to read back as Red. A color with no name reads as #AARRGGBB.
  • etc
  • Lighter on every session. The text the server gives your AI at the start of every session shrank from 1,397 to 857 characters (39% less). The bridge setup steps used to ride along in every session; now the AI gets them only when it needs them. Want the rest too? Register the server only where you use it – see step 1 of the Quick start in the README. On my machine that took 3,604 characters out of every session that does not drive a Delphi app.
  • A clear answer when your app is not running. Every tool now answers with error -32099 target_not_running plus the steps to start the app or link the bridge in (for an Android target, also the adb forward step). attach always carries the setup steps too.
  • wait_for stops at once when no app is running, instead of waiting out its whole timeout.
  • Plain-ASCII tool descriptions, so no AI host shows stray characters where a dash or an arrow was meant.
  • dismiss_dialog tells you when it cannot confirm a click. When it had to send the click by command id instead of pressing the button itself, the answer says via:"WM_COMMAND", so the AI knows to list the dialogs again and check.

 

Fixes (older versions)

  • Fixed: click in message mode on a minimized form. On a minimized VCL form the click never ran; now it does.
  • Fixed: Enter on an inactive form no longer clicks a button that had the focus earlier.
  • Fixed trap: A click that frees its own control stops cleanly. When click runs with a count above 1 and one click frees the control (for example, a button that closes and frees its form), the remaining clicks stop and the answer says stoppedReason:"freed". Before, the bridge clicked the freed control again.
  • Fixed: a call to a tool that does not exist answers the JSON-RPC error -32602, as the MCP specification asks. A call with no tool name, or with arguments that are not a JSON object, answers -32602 too. Before, these came back as a normal tool result with an error text, and arguments that were not an object ran the tool with every argument at its default.
  • Fixed: an argument of the wrong type, or a missing required argument, is refused. The answer is an error result that names the argument, for example Parameter "pid": expected a whole number in the Integer range, got "abc". Before, the server could run the tool with a wrong or empty value and say nothing.
  • Fixed: null for an optional argument now means “not given”. Before, a text argument received the word null.
  • Fixed: the tool list publishes whole-number arguments as integer, not number. This covers pid, count, timeoutMs, maxWidth, pollIntervalMs and hwnd, so an AI knows not to send a value like 1.5.
  • Fixed: a request line that is not valid UTF-8 answers the JSON-RPC parse error -32700. Before, the server replaced the bad bytes and ran the request with the changed text.
  • Fixed, in an MCP server built with Delphi 11 or 12: a number where the protocol expects a text (the method name or the tool name) is refused, not read as text.
  • Fixed: enumeration and record arguments, which no tool uses yet. The schema now lists the allowed names of an enumeration, and an unknown name is refused instead of becoming the first value. An argument of a kind the server does not handle, such as a record, now fails in the server’s own tests instead of being published as a string and then ignored.
Scroll to Top