RFQFuzz: Regression Tests for Drawing Reviewers
Compare reviewer behavior on controlled CAD and drawing packages, with evidence for each changed finding.
RFQFuzz is a local regression toolkit for engineers developing or adopting reviewers of CNC request-for-quotation packages. It checks whether a change to a review prompt, rule engine, or capability profile changes the findings on controlled STEP models and drawings.
I initially considered a drawing-review tool. The research into existing products led to a narrower question: how can someone test the reviewer itself? RFQFuzz supplies known defects, corrected counterparts, and valid alternatives so that flagging every difference is not a successful strategy.
How it is used
The workflow is to validate a challenge package, pass its public files to a reviewer, import the observations, and compare the results across versions. Reports connect changed findings to the relevant drawing and geometry. Package consistency and conditional CNC advisories are scored separately.
The installation and demo guide provides a 15-case offline replay. After setup, run this from the checkout with the environment active:
python -m rfqfuzz.v1 demo examples/v1-demo work/my-demo
Open work/my-demo/report/index.html. The demo compares retained responses with two deliberately injected changes; it requires no live model or API key.
A small example
One challenge pairs a STEP model containing 6.00 mm bores with a drawing specifying 6.80 ±0.05 mm at the same manufacturing stage. A corrected drawing resolves the contradiction. Another package legitimately differs because it describes a later reaming operation. That context matters as much as the numbers.
Validation and limits
The publication audit passed 278 tests, up from 270 after repairs to local-unit interpretation and process output limits. The audit records preserve that distinction.
Its 145 validated core packets are development cases, with no holdout. The tool supports three finite geometry families, and its geometry checks share the OCCT kernel. It does not establish general drawing-review accuracy or approve manufacture. I see its potential in making regressions explainable; independent engineering use is still needed to assess its value.