By John Siau
March 14, 2014
A couple years ago, some of us at Benchmark noticed a weird discrepancy between our HPA2™ headphone amp (built into our DAC1 and DAC2) and some comparably-priced headphone amps. The advertised specifications of all the amps were basically the same, but they sounded noticeably different. Benchmark launched a detailed investigation to identify the differences. The results were surprising and are detailed in "An Examination of Headphone Amplifier Performance Specifications", a white paper by John Siau.
If you care to read the whole whitepaper, you can follow the link above. Otherwise, please continue reading the summary presented in this blog post. Either way, our findings were the same:
We tested three high-quality pro-audio headphone amplifiers with built-in D/A converters. All three had similar published specifications. All units are priced between $1000 and $2000. We verified that the manufacturer’s specifications were accurate, but we have shown that these published specifications are not sufficient to tell the whole story. In this case, the published specifications were not a good representation of typical operating conditions.
The key to the riddle is that all of the published measurements were made with an ideal resistive load. Performance changed dramatically when headphones were connected in place of the resistor loads. With actual headphones loading the amplifiers, the specifications mirrored what we had experienced in listening tests. There were significant differences in the measured performance of the three units when driving headphones, but not when driving resistive loads. We found that our subjective listening tests were validated by the measurements when the tests accurately reflected real-world conditions.
It would be nice if we could build headphones that would behave like an ideal resistor. This would make a headphone amplifier's job easy. Unfortunately, electro-mechanical transducers are far from ideal. The burden falls squarely on the headphone amplifier. As it turns out, headphone amplifiers are not created equal:
In order to make a fair test, we compared the HPA2™ (again, included with our DAC1 and DAC2) to comparably spec’d headphone amps costing between $1,000 to $2,000 - not inexpensive models you'd expect to be deficient. All these amplifiers had similar “ideal” specs, but how would they hold up in real-world tests? We wanted to find out.
You can see in the graph above, the ideal test, everything is basically even. None of the three amps show much distortion. Without actually running the signal into a load, the Total Harmonic Distortion + Noise (THD+N) is comparable. While the HPA2™ measure slightly better than the others, you probably wouldn't hear a difference between these amplifiers in this ideal world. These are the measurements/specifications that manufacturers show consumers.
In our second test we added a 60-Ohm resistive load to simulate headphone loading. Notice that distortion begins to increase in the non-HPA2 amplifiers (green and magenta curves). In contrast, the performance of the HPA2™ is nearly unchanged when driving a 60-Ohm resistor (blue curve).
In our third test, we replaced the 60-Ohm resistor (ideal load) with a pair of 60-Ohm headphones. Distortion rose significantly in the non-Benchmark headphone amplifiers. The 60-Ohm headphones do not behave like 60-Ohm resistors. Clearly the other two amplifiers were having difficulty controlling the headphone transducers. In contrast, the HPA2™ measured almost the same as it did in the unloaded conditions (1st test).
This third graph shows the THD+N for the three amplifiers while driving a pair of Sony MDR-V6. The Sony headphones aren’t extremely hard to drive, but they do tax the performance of the non-Benchmark amplifiers.
All three headphone amplifiers have enough power to drive the MDR-V6 headphones at their 500 mW rated power. Nevertheless, units 2 and 3 were unable to fully damp the mechanical resonances of these popular headphones. Consequently, units 2 and 3 show significant distortion at low frequencies.
Units 2 and 3 produce audible levels of distortion when driving the popular Sony MDR-V6 headphones. Clearly these two amps are not able to maintain control over the headphone drivers. Distortion can color the voicing of the headphones and cause listener fatigue.This correlated with the original listening tests that prompted this investigation.
In conclusion, all three of these headphone amps have the same specifications under ideal loads, but they don’t perform the same when driving headphones.
If you’d like to learn more, get in touch with someone at Benchmark, and we’d be happy to answer your questions. We work hard to develop products that perform and measure well under all real-world conditions - not just selected ideal conditions.
Secrets contributor Sumit Chawla recently caught up with Benchmark’s VP and Chief Designer, John Siau to get a little more in-depth on several subjects.
Q: "Benchmark is one of the few companies that publishes an extensive set of measurements, but you also balance that with subjective testing. Can you talk about the equipment, the listening room, and the process for subjective testing?"
Q: "Was there ever a time where you learned something from a subjective test that was not captured by measurements?"
Q: "You conducted some listening tests to determine whether distortion in the “First Watt” was audible. What test material did you use for this, and what did you find?"
Q: "The AHB2 amplifier incorporates THX Audio Achromatic Amplifier technology. When and how did the partnership with THX come about?"
Q: "Linear power supplies have been and remain quite popular in high-end devices. You favor switch-mode power supplies. When and why did you make this switch?"
... and more!
At Benchmark, listening is the final exam that determines if a design passes from engineering to production. When all of the measurements show that a product is working flawlessly, we spend time listening for issues that may not have shown up on the test station. If we hear something, we go back and figure out how to measure what we heard. We then add this test to our arsenal of measurements.
Benchmark's listening room is equipped with a variety of signal sources, amplifiers and loudspeakers, including the selection of nearfield monitors shown in the photo. It is also equipped with ABX switch boxes that can be used to switch sources while the music is playing.
Benchmark's lab is equipped with Audio Precision test stations that include the top-of-the-line APx555 and the older AP2722 and AP2522. We don't just use these test stations for R&D - every product must pass a full set of tests on one of our Audio Precision test stations before it ships from our factory in Syracuse, NY.
Paul Seydor of The Absolute Sound interviews John Siau, VP and chief designer at Benchmark Media Systems. The interview accompanies Paul's review of the LA4 in the December, 2020 issue of TAS.
"At Benchmark, listening is the final exam that determines if a design passes from engineering to production. But since listening tests are never perfect, it’s essential we develop measurements for each artifact we identify in a listening test. An APx555 test set has far more resolution than human hearing, but it has no intelligence. We have to tell it exactly what to measure and how to measure it. When we hear something we cannot measure, we are not doing the right measurements. If we just listen, redesign, then repeat, we may arrive at a solution that just masks the artifact with another less-objectionable artifact. But if we focus on eliminating every artifact that we can measure, we can quickly converge on a solution that approaches sonic transparency. If we can measure an artifact, we don't try to determine if it’s low enough to be inaudible, we simply try to eliminate it."
- John Siau