By John Siau
Do Specifications Lie? Do Our Ears Lie? Where does the Truth Lie?
Recording Engineers and Audiophiles often distrust audio measurements and specifications. It is not uncommon to hear claims that a product measures poorly but sounds good. Occasionally we also hear claims that a product measures well but sounds bad.
This whitepaper documents significant differences between three headphone amplifiers that have nearly identical published specifications.
All three products have a high-quality headphone output with very low output impedance, high drive capability, low distortion, and a flat frequency response. The published specifications strongly suggest that it should be impossible to hear any difference between these products. Nevertheless, in our informal listening tests, we had the distinct impression that there were significant sonic differences between these products when listening to music.
We replicated the manufacturers' tests and found that the published specifications were accurate and correct. All three manufacturers followed standard measurement procedures and provided accurate specifications.
This paper shows some additional measurements which expose significant differences that do not appear in traditional published specifications:
At Benchmark, listening is the final exam that determines if a design passes from engineering to production. When all of the measurements show that a product is working flawlessly, we spend time listening for issues that may not have shown up on the test station. If we hear something, we go back and figure out how to measure what we heard. We then add this test to our arsenal of measurements.
Benchmark's listening room is equipped with a variety of signal sources, amplifiers and loudspeakers, including the selection of nearfield monitors shown in the photo. It is also equipped with ABX switch boxes that can be used to switch sources while the music is playing.
Benchmark's lab is equipped with Audio Precision test stations that include the top-of-the-line APx555 and the older AP2722 and AP2522. We don't just use these test stations for R&D - every product must pass a full set of tests on one of our Audio Precision test stations before it ships from our factory in Syracuse, NY.
Paul Seydor of The Absolute Sound interviews John Siau, VP and chief designer at Benchmark Media Systems. The interview accompanies Paul's review of the LA4 in the December, 2020 issue of TAS.
"At Benchmark, listening is the final exam that determines if a design passes from engineering to production. But since listening tests are never perfect, it’s essential we develop measurements for each artifact we identify in a listening test. An APx555 test set has far more resolution than human hearing, but it has no intelligence. We have to tell it exactly what to measure and how to measure it. When we hear something we cannot measure, we are not doing the right measurements. If we just listen, redesign, then repeat, we may arrive at a solution that just masks the artifact with another less-objectionable artifact. But if we focus on eliminating every artifact that we can measure, we can quickly converge on a solution that approaches sonic transparency. If we can measure an artifact, we don't try to determine if it’s low enough to be inaudible, we simply try to eliminate it."
- John Siau