Read 2026-10-23
humanlayer/12-factor-agents: technical due diligence, read from its repository
What Borehole read in humanlayer/12-factor-agents's code and history at d20c728: its licence, security advisories, releases, review practice, tests, dependencies and operations. Every finding cites the file and line it rests on. The full report also covers the team and asks the questions the code cannot answer.
| Dimension | Rating |
|---|---|
| Architecture & Codebase — including AI provenance | strong |
| Engineering Process & SDLC Maturity | adequate |
| Product & Engineering Maturity | not assessable |
| Security & Reliability Posture | adequate |
| Observability & Operations | adequate |
| Fit with the Acquirer | not assessable |
Architecture & Codebase — including AI provenance
strongCommit messages are templated rather than written
adequate40% of commit subjects are exact repeats of another subject, and subject length varies by only 24.0 characters. A log written by someone explaining their reasoning to a future reader does not look like this. The log is therefore weak evidence of intent, whatever the code is like.
| 38× 'Update README.md' |
| 27× 'wip' |
| 7× 'Update factor-3-own-your-context-window.md' |
Engineering Process & SDLC Maturity
adequateTests cover a thin slice of the codebase
adequate5 test files against 279 source files (1.8%). This does not measure whether the tests are good, only whether enough of them exist for the codebase to be changed safely by someone who did not write it — which is exactly the situation after an acquisition.
| 5 test files / 279 source files |
40% of commit subjects say nothing about the change
adequate91 of 229 subjects are either under twelve characters or one of the standard placeholders — wip, fix, update. The history is therefore not usable for reconstructing why a decision was taken, which is the main thing an inheriting engineer needs it for. It says nothing about code quality and a great deal about how much context transfers with the asset.
| 91/229 low-content subjects (39.7%) |
| 'wip' |
| 'wip' |
| 'wip' |
| 'wip' |
Almost no process scaffolding exists in the repository
adequateOf five ordinary markers — CI workflows, CODEOWNERS, a pull-request template, pre-commit hooks, contribution guidance — none are present. Branch protection itself is a GitHub setting rather than a file, so this is a reading of proxies and not of the rules themselves. What it does establish is that the process is not written down anywhere a new engineer would find it.
| 0 of 5 markers present across 499 tracked files |
12% of recent commits change more than 2,000 lines
adequate26 of 224 commits in the last 24 months changed over 2,000 lines of code, not counting reformatting, renames, vendored or generated code. A change that size is rarely reviewed line by line. Ask how they were reviewed, and what made them that large.
| 26 commits over 2,000 lines |
Product & Engineering Maturity
not assessableNot rated: only 9 of 19 checks could reach a conclusion here; the rest do not apply to this repository or could not read it.
Security & Reliability Posture
adequateNone of the standard security guard rails are configured
adequateThere is no disclosure policy, no automated dependency updates, no named code owners and no pre-commit configuration. None of these is load-bearing on its own. Their collective absence says that security posture here is whatever the current team happens to remember to do, which is not a posture that survives the team changing.
| checked 5 kinds of guard rail across 499 tracked files; 0 present |
25 locked dependencies have published vulnerabilities, 47 high or critical in what ships
adequate25 of 270 locked dependencies match 121 published advisories in OSV, 47 of them rated high or critical. A lockfile says which version is installed, not whether the flawed code is reachable from this product, so each needs a look. Most are fixed by an upgrade, and the advisory names the version.
workshops/2025-07-16/uv.lock — anyio 4.9.0: GHSA-82r6-8w77-94w6 (critical), fixed in 4.14.2 |
workshops/2025-07-16/uv.lock — certifi 2025.7.14: GHSA-6q48-6c7w-9gw2 (critical), fixed in 2026.10.19 |
workshops/2025-07-16/uv.lock — jupyter-server 2.16.0: GHSA-fcw5-x6j4-ccmp (critical), fixed in 2.20.0 |
Observability & Operations
adequateNo structured logging library is in use
adequateNothing references a structured logging library. Logs that are written to be read by a person are not searchable by a machine, which means incident response here is proportional to how well someone remembers the codebase rather than to what the tooling can find.
| searched 420 files for 10 structured logging libraries; 0 matched |
The service logs with print: 411 calls against 0 logger calls
adequate411 `print` or `console.log` calls in the service's code, and 0 calls through a logger. Printed lines carry no level and no context, cannot be switched off, and are hard to find again in production.
packages/create-12-factor-agent/template/src/agent.ts:56 — print used for logging |
packages/create-12-factor-agent/template/src/agent.ts:64 — print used for logging |
packages/create-12-factor-agent/template/src/agent.ts:72 — print used for logging |
One request cannot be followed through the logs
adequateNothing in the service carries a request or correlation id, and there is no tracing. When a customer reports a failure, the log lines that belong to their request cannot be picked out from everyone else's.
| no request id, correlation id or tracing found |
No error reporter is wired in
adequateNothing in the service initialises an error reporter (Sentry, Rollbar, Bugsnag, Honeybadger or an APM agent). An exception in production reaches a person only if someone is reading the logs, which usually means a customer reports it first.
| no error reporter initialised |
229 commits in a year, and no release marked
adequateThe service took 229 commits in the last year with no tag, no release commit and no changelog entry. What is running in production cannot be read from the repository, and a bug report cannot be mapped to a version. Ask how the team knows what is deployed.
| no tag, release commit or changelog change in 12 months |
Fit with the Acquirer
not assessableNot rated: structurally not assessable from a repository.
Read more, or read your own
The full report on humanlayer/12-factor-agents: every dimension, the questions for the room, and the audit log.
Survey a repository: free on public repositories. Private code is read on your own machine by a collector whose source you can read first.