A model name is convenient configuration. It is not a complete record of the bytes that produced an embedding. If files change while the name stays the same, a system needs a way to notice—or an explicit procedure for rebuilding what depends on them.
METABOLIQ Brain’s accepted foundation includes a model lockfile, checksummed artifact loading and a local embedding provider. For Article 009, we examined those controls and tested their boundaries against an isolated export of the accepted implementation. We also ran the real default embedding model with container networking disabled.
The result is useful, but narrower than “the model is always verified.” Verification checks particular files against a trusted lockfile at particular points. It does not continuously monitor the running engine, establish model quality or automatically give every changed artifact a new vector identity.
Start with a revision and a list of files
The lockfile associates a supported model name with a model-hub repository, a pinned revision and SHA-256 hashes for listed files. The standard API image invokes the fetch helper during its build, then includes the downloaded model directory in the image.
We reused the default BAAI/bge-base-en-v1.5 artifacts from an existing local image. All nine listed files matched the accepted lockfile before the real-model checks ran. The recorded hub revision was 199291fdd1aa89faf9c20b722dc72ad5e17aa0d0. No fresh model download was needed for this verification.
The fetch helper passes the pinned revision and any configured file patterns to the downloader, then verifies the resulting files. We checked that contract with a controlled downloader stand-in: the expected revision and patterns were passed, an unknown model was refused before a download call, and incorrect downloaded bytes caused verification to fail. This was not a new end-to-end image build or a fresh hub-download test.
A checksum answers a specific question
The verifier walks the files named in the lock entry. A missing listed file or a different hash is an integrity error. In tests using real model copies, appending a byte to the ONNX weights or removing the tokenizer file prevented a new pinned provider from completing its first inference.
An additional file outside the lock entry was different: the verifier accepted the directory. A harmless extra text file also did not prevent the real model from loading. These checks establish coverage of the listed files, not an exact inventory rule that rejects every unlisted file. They do not establish that arbitrary extra files are harmless.
The expected hashes must themselves be trusted. In a small fictional-artifact test, changing both a file and its expected hash made verification pass. That is the intended comparison mechanism, not an independent approval of the replacement. Reviewing the lockfile and the code that uses it remains part of the trust boundary.
Verification happens before the engine first loads
The pinned provider verifies its directory before constructing the embedding engine on first use. Merely constructing the provider object does not load and hash the model. Once the engine is successfully loaded, the provider retains it and subsequent calls reuse it.
We loaded a model from a temporary copy, ran a query and then changed its listed README file. Calling the verifier directly now failed. The already-loaded provider still returned the same query vector; a fresh provider pointing at that changed copy refused to load.
This is evidence that the loading check is not continuous file verification. We deliberately changed documentation rather than live, potentially memory-mapped model weights. The test makes no claim about the safety or behavior of mutating weights after loading. It also does not prove protection against a file change between verification and loading.
Configuration matters too. The standard image sets a pinned model directory. Outside that configuration, the provider supports a development path without a pinned directory. The presence of integrity-checking code does not prove that every possible deployment uses it.
Offline inference needs an execution test
The standard image sets the model directory and an offline hub setting. To test more than configuration, we ran the accepted provider code in a separate container with its network mode set to none. The container had only its loopback interface, a read-only root filesystem and temporary writable storage. No existing Brain database was mounted or contacted by these inference checks.
The real default model loaded and produced both query and document embeddings from fictional text. The vectors had the expected 768 dimensions, contained finite values and were normalized. Repeating the same query in that provider returned the same vector in this run. A deliberately incorrect declared dimension was rejected.
The Python socket-connection hook recorded no connection attempt during the dedicated offline inference check. The container’s network isolation also constrained native dependencies. These observations establish that this model could load and infer without an external network connection in the tested environment. They do not establish bit-for-bit reproducibility across hardware, library versions or other environments.
There was a diagnostic worth preserving: ONNX Runtime warned that it could not persist a telemetry device identifier and was using an in-memory identifier. The warning did not stop inference. We did not use this test to certify telemetry settings, enumerate native network attempts or establish that an otherwise network-enabled runtime never attempts communication. Offline hub configuration alone is not a complete network privacy policy.
Artifact identity and stored model identity are different
The accepted FastEmbed provider constructs its model identifier from the provider prefix and model name. The identifier does not include the lockfile revision, artifact hashes or model-directory path. Different configured roots for the same model name therefore produce the same identifier.
We tested the consequence with an instrumented fake provider and a real isolated PostgreSQL database. After indexing a fictional memory, we created another application instance whose provider returned different document vectors but kept the same model identifier and dimensions. Configuration synchronization left the existing, current-version embedding ready and unchanged. An explicit rebuild replaced the vector while preserving the source memory.
That test used a controlled provider stand-in; it did not replace the real model’s weights. It shows that a same-name implementation change is not automatically a new model to the indexing lifecycle. Artifact verification and derived-vector migration need to be considered separately.
The query cache has a related boundary. Its key includes the provider’s model identifier, the caller’s scope and a digest of the query. In a focused in-process provider-replacement test, the same-name replacement left the warmed query using its earlier cached vector, while a new query used the replacement provider. A newly created cache used the replacement immediately. This does not mean query caches survive an application restart, and it does not describe a supported hot-swap deployment procedure.
Diagnostics should be precise without overstating privacy
The tested integrity errors identified the missing or mismatched model file; mismatch errors can include shortened hashes. In the selected provider errors and a service-level health probe with a stubbed database check, the fictional query and document markers did not appear. Health correctly reported the provider as unavailable for the missing-file case.
Those are bounded diagnostic checks. They do not audit every API response, dependency diagnostic, host log or external telemetry system. Model artifact metadata and memory text also have different disclosure implications; calling an error “content-free” should not be read as promising that it contains no technical detail.
What the evidence supports
All 19 focused artifact, loading, identity and offline checks passed. A separate run passed 49 database-backed checks: the accepted retrieval suite, two existing lockfile/unpinned-model checks and the new same-name provider identity case. The real-model container used Python 3.13.15, FastEmbed 0.9.1 and ONNX Runtime 1.32.0; the evidence package records the remaining environment details.
The source snapshot was accepted local Brain commit 489f4a870d398220b1ec500f956498f20543ec75. The available GitHub connection could not confirm the remote head. Active MB-003 ingestion work was excluded. The operating scope remains one owner on a trusted local machine, and reranking remains disabled by default.
The checks do not establish retrieval quality, model safety, whole-system supply-chain security, production readiness or cross-platform numerical reproducibility. Original model artifacts, active Brain source and existing memory databases were not changed by the test harness. Artifact mutations affected only disposable test copies.
A useful model record includes more than a name: which revision and files were intended, which runtime loaded them, and what procedure keeps derived vectors aligned when those inputs change. Brain’s accepted implementation provides a concrete artifact-verification boundary. Understanding where that boundary ends is part of using it accurately.
← All engineering articles