Hoplite: Give a Coding Agent a Task You Can Verify

Write a reproduction someone else can run

Describe the starting state, action, actual result and expected result. For example: “On the saved-items page, choose a category, then go to page two. The category resets. Pagination should retain the selected category.” Include a sample URL or harmless fixture when it removes ambiguity.

Point to the relevant route and existing test command if you know them. Let the agent trace the implementation; avoid prescribing a patch before the cause is understood.

1. Define one user-visible outcome and the smallest reproduction.
2. List the existing behavior that must keep working, such as keyboard navigation or saved filters.
3. Ask for the changed files, verification result and any unresolved failure.

Make the environment reproducible

Hoplite’s site describes connecting GitHub, importing local setup and using cloud agents. Confirm the repository can install dependencies and run its checks in that environment before starting a longer task.

Use scoped development credentials when a service is needed. A missing secret should produce a clear setup failure, not a workaround that removes authentication or silently disables the feature under test.

Review the result against the brief

Read the diff before merging. Check whether the patch fixes the shared cause, whether it touches unrelated files and whether a test would have failed before the fix. A successful build alone does not prove that the original interaction works.

Run the reproduction again, then one nearby case: another filter, an empty result or the last page. If the task expanded into a redesign, split that work out and finish the original correction first. Agent execution can save time; acceptance still needs an observable result.