Methodology

Where specifications come from

Manufacturer documentation is the primary source for specifications. A vendor is authoritative about what it built: socket, core count, memory type, dimensions, connector layout.

A vendor is not a credible source about how its product performs against a competitor. Manufacturer benchmark claims are marketing, they are run on hardware and settings the manufacturer chose, and we do not publish them as results.

Retailer listings are used for prices and availability, never for specifications. Retailer spec fields are copied, stale and frequently wrong.

Normalising

Before two products can be compared, the same fact has to be stored under the same name, in the same unit, meaning the same thing.

We keep one canonical label per fact per category. Values are stored as a number followed by its unit, so the table can subtract them. Where a source gives a range or a qualifier that the number alone would lose, such as a configurable 10 to 25 watt power envelope, we keep the text and show no percentage.

Which benchmarks we publish

We publish Blender Open Data results. The dataset is released under CC0, it is reproducible, and the scores are submitted by users running a published, versioned benchmark rather than by the hardware vendor.

We do not republish benchmark databases whose licences forbid commercial reuse. That rules out Geekbench, UL's 3DMark and PCMark, and PassMark. Cinebench has no official public database to cite. Where we cannot publish a number, we say so rather than reproducing someone else's table.

Sources we do not use

We do not use UserBenchmark. Its weightings have been changed repeatedly in ways that depress multi core results, its published methodology does not account for the spread of its own submissions, and its rankings disagree with every reproducible workload benchmark on the same parts. A source that has to be corrected before it can be read is not a source.

When we show a percentage

A percentage is only meaningful on a ratio scale, where zero means none of the thing and twice the number means twice the quantity.

Temperature in Celsius is an interval scale. 80 degrees is not "33 percent hotter" than 60. Noise in dBA is logarithmic. We show the difference in those units and no percentage.

We also withhold percentages on component counts and clock speeds across different generations, even though the arithmetic is valid, because the conclusion a reader would draw is not. Performance is roughly instructions per clock times clock speed times cores. A percentage on any one of those three assumes the other two are equal, and across generations they never are. Two cores of a tenth generation part can beat four cores of a seventh generation one. Where we want to say something about performance we show a benchmark, which measures all three at once.

Scores

Scores are ours, not user submitted, and they are category specific. A score is withheld when there is not enough verified data behind it. A missing score means we do not know, not that the product is bad.

Corrections

If a number here is wrong, tell us at hello@consumercarts.com or tools@incrementors.com and we will fix it and say what changed.