Read 2026-11-03
traceloop/openllmetry: technical due diligence, read from its repository
What Borehole read in traceloop/openllmetry's code and history at 4a294cd: its licence, security advisories, releases, review practice, tests, dependencies and operations. Every finding cites the file and line it rests on. The full report also covers the team and asks the questions the code cannot answer.
| Dimension | Rating |
|---|---|
| Architecture & Codebase — including AI provenance | strong |
| Engineering Process & SDLC Maturity | adequate |
| Product & Engineering Maturity | strong |
| Security & Reliability Posture | adequate |
| Observability & Operations | not assessable |
| Fit with the Acquirer | not assessable |
Architecture & Codebase — including AI provenance
strongpackages/opentelemetry-instrumentation-bedrock/opentelemetry/instrumentation/bedrock/__init__.py is 1,299 lines, 14× this codebase's median file
adequateThe median source file here is 92 lines. 7 files exceed 1,000 lines, the largest being 1,299. Size is not a defect on its own — some problems genuinely live in one place — but these are where merge conflicts, review fatigue and single-owner knowledge concentrate, and they are the first thing that slows an inheriting engineer down.
packages/opentelemetry-instrumentation-bedrock/opentelemetry/instrumentation/bedrock/__init__.py:1 — 1,299 lines (14× median) |
packages/opentelemetry-instrumentation-langchain/opentelemetry/instrumentation/langchain/callback_handler.py:1 — 1,261 lines (13× median) |
packages/opentelemetry-instrumentation-openai/opentelemetry/instrumentation/openai/shared/chat_wrappers.py:1 — 1,254 lines (13× median) |
packages/traceloop-sdk/traceloop/sdk/tracing/tracing.py:1 — 1,229 lines (13× median) |
packages/opentelemetry-instrumentation-openai/opentelemetry/instrumentation/openai/v1/responses_wrappers.py:1 — 1,192 lines (12× median) |
Engineering Process & SDLC Maturity
adequateCI runs tests that cannot fail the build
adequate1 test command in CI ends in `|| true` or an equivalent, so the pipeline passes whatever the tests say. A green build then means the tests ran, not that they passed.
.github/workflows/ci.yml:195 — a test command that cannot fail |
10% of recent commits change more than 2,000 lines
adequate36 of 365 commits in the last 24 months changed over 2,000 lines of code, not counting reformatting, renames, vendored or generated code. A change that size is rarely reviewed line by line. Ask how they were reviewed, and what made them that large.
| 36 commits over 2,000 lines |
Product & Engineering Maturity
strongNo finding.
Security & Reliability Posture
adequate15 credential-shaped strings in the tree sit in tests, examples, documentation or local development settings
adequateStrings shaped like credentials appear only where fakes are usual: test fixtures, examples, documentation, placeholders and local stacks. They are almost always fakes. Ask someone to confirm that none of them works anywhere, because a real key in a development file is still a real key.
| 5 cited: AWS access key, where fakes are usual |
The build runs third-party code it does not pin
adequate6 third-party CI actions are referenced by a tag or branch rather than a commit, so they run whatever that tag points to on the day. Tags have been moved to malicious commits in real supply-chain attacks, and a piped script is whatever the server returns that minute. Pinning to a commit is the fix, and it is small.
.github/workflows/ci.yml:23 — action pinned by tag |
.github/workflows/ci.yml:58 — action pinned by tag |
67 locked dependencies have published vulnerabilities, 56 high or critical in what ships
adequate67 of 627 locked dependencies match 226 published advisories in OSV, 110 of them rated high or critical. A lockfile says which version is installed, not whether the flawed code is reachable from this product, so each needs a look. Most are fixed by an upgrade, and the advisory names the version.
packages/sample-app/uv.lock — authlib 1.6.6: GHSA-6v38-4h7w-rj9h (critical), fixed in 1.9.0 |
packages/sample-app/uv.lock — certifi 2026.1.4: GHSA-6q48-6c7w-9gw2 (critical), fixed in 2026.10.18 |
packages/sample-app/uv.lock — chromadb 1.1.1: GHSA-36p7-vc44-83pf (critical) |
Manifest says MIT, licence file says Apache-2.0
adequatepackage.json declares MIT. LICENSE contains the text of Apache-2.0. Package managers and downstream users read the first; auditors and acquirers read the second. One of them is wrong, and until it is settled the terms this code is offered under are genuinely ambiguous.
package.json:4 — declares MIT |
LICENSE — text of Apache-2.0 |
Observability & Operations
not assessableNot rated: only 5 of 16 checks could reach a conclusion here; the rest do not apply to this repository or could not read it.
Fit with the Acquirer
not assessableNot rated: structurally not assessable from a repository.
Read more, or read your own
The full report on traceloop/openllmetry: every dimension, the questions for the room, and the audit log.
Survey a repository: free on public repositories. Private code is read on your own machine by a collector whose source you can read first.