
A pull request says the tests passed. The attached transcript shows a successful command. Before treating that result as security evidence, an AppSec reviewer needs to establish which code was tested, what the command actually checked, and whether anything changed afterwards.
This is a practical problem for teams using coding agents. A confident summary can compress several different events into one claim: a test was proposed, a process started, a command exited successfully, or the intended security requirement was verified. A useful review keeps those events distinguishable and gives missing evidence an explicit outcome.
The following checklist can be added to an existing pull request template. It complements independent CI and security review; the quality of the underlying tests still determines what a pass establishes.
Identify the code and configuration under test
Record the commit or immutable source snapshot used by each required check. If tests ran before a commit, identify the actual workspace contents, including relevant untracked files. A branch name or timestamp alone cannot establish that relationship.
Also capture the test configuration, dependency lockfile and environment identity. A security test against a local database with permissive settings may say little about the deployment configuration being reviewed. Describe the relevant difference instead of labelling both runs simply as integration tests.
GitHub requires checks on the applicable latest commit, with separate considerations for the head and test merge commits. Apply comparable discipline to evidence produced inside an agent session. If the agent edits an authorization rule after the recorded pass, obtain fresh evidence for the new code or document why the earlier result remains applicable.
Verify the check that actually ran
Retain the executed command, working directory, start and completion records, exit status, and a link to the relevant output. Record the test selection as well. A command that discovers zero tests should not satisfy a requirement to exercise an access-control boundary.
For static analysis, identify the ruleset and the files examined. For dependency analysis, identify the dependency manifest, resolved versions and advisory data used. These checks address different risks; one successful scan should not silently stand in for the other.
Read enough output to confirm that the reported result belongs to the intended task. A wrapper script can exit successfully after swallowing a failing subprocess. A shell transcript proves less when the collector cannot establish how exit codes were propagated.
Connect evidence to a security requirement
A passing command is useful when a reviewer knows the claim it supports. For an authorization change, that might be that a user from tenant A cannot retrieve tenant B’s object, including through a direct identifier lookup.
Write that expectation before selecting the test. Then link the requirement to the relevant negative test and its observed result. The OWASP Application Security Verification Standard provides security requirements that teams can use to structure this work; collecting a log does not establish conformance to the whole standard.
Consider an illustrative change to an invoice endpoint. Unit tests pass, but they mock the authorization service. The evidence supports the tested handler behaviour under those mocks. It does not establish that the real service rejects a cross-tenant request. That gap should remain visible until an appropriate test or review addresses it.
Preserve failures and unknown outcomes
Use distinct states for completed success, completed failure, incomplete execution and missing evidence. Treat relevance separately: a completed successful check can still be stale or belong to the wrong environment.
For example, an interrupted dependency scan is incomplete. An absent scan record is missing. A failing scan followed by a successful run after a fix should retain both attempts, with the later run tied to the revised code. Deleting the failure destroys useful context about what changed.
Avoid translating every nonblocking platform status into proof that a test executed. GitHub’s documented status behaviour includes successful, skipped and neutral conclusions for required checks. Inspect the underlying execution when your policy specifically requires a completed test.
Protect evidence and define exceptions
Evidence generated by the same process that edits the code has a limited trust boundary. An agent that can modify its own test wrapper or overwrite its logs may also influence what a reviewer sees. Use independent CI where practical, restrict who can publish authoritative check results, and retain the source artifacts needed to investigate a discrepancy.
Do not put secrets, customer data or unnecessary prompts into the review record. Preserve the information needed to reproduce the check and apply appropriate access and retention controls to detailed logs.
When an urgent release proceeds without a required check, record a human exception with an owner, reason, scope and expiry. Keep the absent evidence visible. An approved exception describes an accepted risk; it does not change an unexecuted test into a pass.
Make the review decision explicit
Start with one security-sensitive workflow and a small set of required checks. Ask reviewers whether they can identify the tested revision, explain the security claim, find the result and see unresolved gaps without reconstructing an entire chat session.
The resulting record should make the next action clear: rerun a stale check, investigate a failure, supply missing evidence, or approve a documented exception. That is a more useful review outcome than a reassuring summary whose supporting facts cannot be inspected.
Join our LinkedIn group Information Security Community!











