Read 2026-10-22
huggingface/transformers: technical due diligence, read from its repository
What Borehole read in huggingface/transformers's code and history at 6b6792f: its licence, security advisories, releases, review practice, tests, dependencies and operations. Every finding cites the file and line it rests on. The full report also covers the team and asks the questions the code cannot answer.
| Dimension | Rating |
|---|---|
| Architecture & Codebase — including AI provenance | adequate |
| Engineering Process & SDLC Maturity | adequate |
| Product & Engineering Maturity | strong |
| Security & Reliability Posture | adequate |
| Observability & Operations | adequate |
| Fit with the Acquirer | not assessable |
What the repository says
- Releases are cut from another branch than the one surveyed
CONTRIBUTING.md:220: states a policy on AI-generated contributions
Architecture & Codebase — including AI provenance
adequatesrc/transformers/modeling_utils.py is 5,215 lines, 27× this codebase's median file
adequateThe median source file here is 193 lines. 78 files exceed 1,930 lines, the largest being 5,215. Size is not a defect on its own — some problems genuinely live in one place — but these are where merge conflicts, review fatigue and single-owner knowledge concentrate, and they are the first thing that slows an inheriting engineer down.
src/transformers/modeling_utils.py:1 — 5,215 lines (27× median) |
src/transformers/testing_utils.py:1 — 4,956 lines (25× median) |
src/transformers/utils/auto_docstring.py:1 — 4,862 lines (25× median) |
src/transformers/trainer.py:1 — 4,727 lines (24× median) |
src/transformers/generation/utils.py:1 — 4,444 lines (23× median) |
6 base images float: no version, or `latest`
adequate6 of 18 `FROM` lines in product Dockerfiles name an image without a version, or with `latest`. Each build pulls whatever the registry serves that day, so the image that ships can change without a commit, and last month's build cannot be reproduced. Pin a version, or a digest.
docker/consistency.dockerfile:1 — base image without a version |
docker/custom-tokenizers.dockerfile:1 — base image without a version |
docker/exotic-models.dockerfile:1 — base image without a version |
Engineering Process & SDLC Maturity
adequate2641 tests are switched off
adequate2641 skip markers across 578 of 1825 test files. Each is a failure somebody chose to stop seeing; the question is whether anyone is tracking them.
tests/generation/test_utils.py:4666 — skipped test |
tests/models/afmoe/test_modeling_afmoe.py:112 — skipped test |
tests/models/afmoe/test_modeling_afmoe.py:116 — skipped test |
5% of recent commits change more than 2,000 lines
adequate357 of 6821 commits in the last 24 months changed over 2,000 lines of code, not counting reformatting, renames, vendored or generated code. A change that size is rarely reviewed line by line. Ask how they were reviewed, and what made them that large.
| 357 commits over 2,000 lines |
Product & Engineering Maturity
strongReleases are cut from another branch than the one surveyed
adequateThe newest release, v5.20.2, is not on the branch this survey read; the newest release on it is v4.33.1. The shipped product may differ from what this report describes. Survey the release branch too.
| tag v5.20.2 |
Security & Reliability Posture
adequate9 dependencies float, and no lockfile is committed
adequateDependency versions are declared as ranges and no lockfile is committed, so a build today does not necessarily produce what a build last month produced. That matters twice over in a transaction: the thing being bought cannot be reproduced exactly, and a compromised upstream release arrives without anyone choosing to take it.
benchmark/requirements.txt:4 — >=1.5.0 |
benchmark_v2/requirements.txt:1 — >=1.21.0 |
benchmark_v2/requirements.txt:2 — >=5.8.0 |
benchmark_v2/requirements.txt:3 — >=12.0.0 |
benchmark_v2/requirements.txt:4 — >=2.0.0 |
11 Dockerfiles run the application as root
adequateNo USER instruction switches away from root, so the process runs with full privileges inside the container. It is a one-line fix and an ordinary expectation, which is why its absence is worth asking about: it usually says more about whether anyone has reviewed the deployment than about this container specifically.
docker/transformers-all-latest-gpu/Dockerfile — runs as root |
docker/transformers-doc-builder/Dockerfile — runs as root |
docker/transformers-gpu/Dockerfile — runs as root |
docker/transformers-intel-cpu/Dockerfile — runs as root |
The build runs third-party code it does not pin
adequate8 third-party CI actions are referenced by a tag or branch rather than a commit, so they run whatever that tag points to on the day; 4 steps pipe a download straight into a shell. Tags have been moved to malicious commits in real supply-chain attacks, and a piped script is whatever the server returns that minute. Pinning to a commit is the fix, and it is small.
.github/workflows/assign-reviewers.yml:26 — action pinned by tag |
.github/workflows/nightly-integration-failure-triage-caller.yml:39 — action pinned by tag |
.github/workflows/build_pr_documentation.yml:41 — download piped into a shell |
18 outbound calls can wait forever
adequate18 HTTP calls use a client with no timeout (Python `requests` without `timeout=`, or Go's default client). When a dependency slows down, each one holds a worker until the connection gives up, and enough of them take the service down with the dependency.
src/transformers/models/beit/modeling_beit.py:628 — outbound call without a timeout |
src/transformers/models/beit/modeling_beit.py:1019 — outbound call without a timeout |
src/transformers/models/beit/modeling_beit.py:1107 — outbound call without a timeout |
17 deserialisers that can run code
adequate17 places read data with a deserialiser that executes what it reads (`yaml.load` without a safe loader, pickle). That is safe on data the system wrote itself and remote code execution on anything else. Ask where each one's input comes from.
src/transformers/models/canary/convert_canary_nemo_to_hf.py:198 — code-executing deserialiser |
src/transformers/models/marian/convert_marian_to_pytorch.py:107 — code-executing deserialiser |
src/transformers/models/marian/convert_marian_to_pytorch.py:684 — code-executing deserialiser |
19 advisories against this project in the last 24 months, 11 high or critical
adequateThe newest was published on 2026-09-01. A project that publishes advisories is telling its users about its fixes, and the count and pace say how much security work it carries. CVE-2026-1839 (high), CVE-2025-3933 (moderate), CVE-2025-2099 (high) and 2 more are fixed in the surveyed commit; which release carries them cannot be told from this history, which does not place the release users run. Answer lives in: the version you run. The record is incomplete: NVD: 12 records name transformers with nothing tying them to this repository (CVE-2024-21799, CVE-2025-14920, CVE-2025-14921, …).
| 2026-09-01: CVE-2026-80047 (high): A vulnerability in Hugging Face Transformers (versions 4.57.0 to 5.16.1) allows remote Python files to be written to local disk without user consent when using |
| 2026-08-17: CVE-2026-75104 (moderate): Hugging Face Transformers fails to validate shard filenames in checkpoint index files, allowing attackers to read arbitrary files outside the model directory. A |
| 2026-05-24: CVE-2026-4372 (high): A critical remote code execution vulnerability exists in all versions of the HuggingFace transformers library prior to version 5.3.0. The vulnerability allows a |
| 2026-04-07: CVE-2026-1839 (high): A vulnerability in the HuggingFace Transformers library, specifically in the `Trainer` class, allows for arbitrary code execution. The `_load_rng_state()` metho |
| 2025-09-23: CVE-2025-6921 (high): The huggingface/transformers library, versions prior to 4.53.0, is vulnerable to Regular Expression Denial of Service (ReDoS) in the AdamWeightDecay optimizer. |
Observability & Operations
adequateNo structured logging library is in use
adequateNothing references a structured logging library. Logs that are written to be read by a person are not searchable by a machine, which means incident response here is proportional to how well someone remembers the codebase rather than to what the tooling can find.
| searched 5,161 files for 10 structured logging libraries; 0 matched |
No error reporter is wired in
adequateNothing in the service initialises an error reporter (Sentry, Rollbar, Bugsnag, Honeybadger or an APM agent). An exception in production reaches a person only if someone is reading the logs, which usually means a customer reports it first.
| no error reporter initialised |
Fit with the Acquirer
not assessableNot rated: structurally not assessable from a repository.
Read more, or read your own
The full report on huggingface/transformers: every dimension, the questions for the room, and the audit log.
Survey a repository: free on public repositories. Private code is read on your own machine by a collector whose source you can read first.