Start with a question.
Our first objective is to understand where inference is constrained in a specific deployment. A useful experiment starts with a narrow question: can a change reduce resource use while preserving an agreed level of output quality and service responsiveness?
This note describes a proposed evaluation process. METABOLIQ is not presenting a new model, performance improvement or commercial serving product here.
Define the baseline.
Before comparing systems, fix the conditions that make the comparison meaningful.
- Model and task: model revision, tokenizer, precision, task definition and quality checks.
- Workload: prompt and output lengths, concurrency, request distribution and representative inputs.
- Environment: hardware, memory capacity, software versions, runtime settings and warmup procedure.
- Constraints: acceptable latency, quality floor, operating budget and deployment requirements.
Use a well-configured baseline. Record tuning choices for both the baseline and the proposed change.
Measure the tradeoffs.
Quality
Evaluate outputs on the chosen task. An efficiency gain should be reported alongside any change in quality, rather than hidden behind an aggregate speed figure.
Responsiveness
Record time to first token, time between generated tokens and end-to-end request latency. Report distributions and tail behavior at stated concurrency, including failed or timed-out requests.
Capacity and resources
Report useful throughput, peak memory use and the resource configuration. Specify whether throughput counts input tokens, output tokens or completed requests.
Cost
State the price assumptions, utilization and billing period used in any cost estimate. Keep measured resource consumption separate from a cost model.
Make it repeatable.
- Save the configuration, versions and evaluation inputs or a shareable description of them.
- Warm up each system consistently, repeat runs and record variability.
- Test more than one workload shape and identify where an approach stops helping.
- Keep raw measurements, scripts and known limitations with the report.
- Have a second reviewer reproduce the central comparison before publication.
Use permitted, non-sensitive evaluation data. Publish only material we have the right to share.
Decide what follows.
A promising result is a reason to investigate further. The next stage should test whether the effect survives a broader workload, stricter quality checks and realistic operating conditions.
When results are ready, the report should include the baseline, method, measurements, limitations and reproducibility materials together.
Discuss the approach