A/B and A/B/X listening tests are important methods of comparing two audio sources or two audio components. In the studio an engineer may want to switch an effect or EQ setting on and off to decide if it contributes positively to a mix. Hi-Fi enthusiasts may wish to compare audio components, signal sources and interconnects. The DAC1 and DAC2 converters have input selector switches that allow fast and easy switching between signal sources. Before attempting to conduct these tests, it is important to understand how these converters and their switches work. A/B tests using the DAC1 input selector can be very misleading. In contrast, A/B testing using the input selector on the DAC2 will produce reliable results. This application note provides guidance for conducting reliable A/B or A/B/X listening tests with your D/A converter.
The DAC1 converters allow instantaneous switching between digital sources but the audio unmutes before the low-jitter UltraLock PLL (phase locked loop) has fully settled. When we created the DAC1, some of our professional customers needed a D/A converter with a very short lock time. To this day, the DAC1 has one of the fastest lock times in the industry. Audio begins playing only a few milliseconds after a digital signal is applied, and this happens before the clock system has fully settled. This can cause a very slight pitch shift in the first 250 ms of playback. This pitch shift is imperceptible when switching between different musical selections, but it can become perceptible when switching between two identical (or nearly identical) signal sources. While it is convenient to run A/B tests with the input selectors on the DAC1, we do not recommend this! We strongly recommend switching the source within a computer, an upstream digital console, or DAW. The upstream switching will produce a continuous data stream to the DAC1 and the PLL will not be transitioning between unlocked and locked states.
There have been a number of anecdotal accounts of optical and coaxial cables producing differing results when a DAC1 input selector switch is used to switch between two cables that are receiving identical data. Usually this is done by simultaneously feeding the DAC1 from two outputs on a disk player. Conducting an A/B test in this manner will give erroneous results with the DAC1 but not with the DAC2. The DAC1 unmutes just before the end of the pitch shift. The DAC2 unmutes just after the end of the pitch shift.
If you use the DAC1 input selector to conduct an A/B comparison, the very first thing you hear after the switch is a pitch shift that persists for about 250 milliseconds. It is a very short event, but your ears will detect the momentary change in pitch and this will color your opinion of what you hear after the pitch shift. It is a really interesting psychoacoustical effect and I don't know that anyone has really done any research on this. Nevertheless, the effect is real and it is easily reproducible. With two identical inputs to the DAC1, one will be perceived as being brighter than the other. For this reason, we do not recommend using the DAC1 input selector for A/B comparisons (use the console or DAW instead). Other brands of D/A converters have similar pitch-shift artifacts, but the magnitude and duration of these artifacts will vary. It is probably safe to say that most D/A converters will have pitch-shift artifacts that can produce false results in A/B tests (if D/A is switched between two different digital inputs).
A slightly longer mute delay on the DAC2 prevents the erroneous results of A/B tests that are tainted by the pitch-shift transient of the PLL. The trade-off is that there is a momentary silence when switching. This muting can be avoided by switching within the DAW or digital console, because these devices normally provide a continuous digital data stream while switching between signal sources.
In an A/B test, signal source A is simply compared to signal source B. In an A/B/X test, the listener has access to A and B and an unknown X which is either A or B. On each trial, the listener must determine if X is A or B. The listener is free to switch between A, B, and X as many times as necessary before making a decision about X. X is randomly selected for each trial and the system keeps score to determine if the listener actually heard a difference. Rigorous A/B/X tests can scientifically confirm that a listener can hear a difference between two sources. We use these types of tests in our listening room here at Benchmark. Our listening room is equipped with a QSC ABX test set, DAW ABX testing, and software-based ABX testing. We also have an A/B system that uses multiple DAC2 converters. The photo shows the remote control for the QSC ABX system in Benchmark's listening room.
Rigorous A/B/X tests can also be conducted with specialized software running on a Windows, Mac, or Linux computer. Lactinato ABX/Shootout-er is a free software-based ABX system that operates across each of these three major operating systems. Other similar software solutions can be found with this search.
The PLL lock times on the DAC1 and DAC2 are among the fastest in the industry. Unfortunately, low-jitter phase locked loops cannot lock instantaneously. A/B tests should be conducted in a way that prevents the locking and unlocking of a PLL. In general, D/A converters will produce some pitch-shift anomalies when switching between two different digital sources. The DAC2 mutes for about 500 milliseconds to eliminate the pitch-shift transient. All of these effects can be eliminated by switching the audio source within a computer program or a digital mixer (instead of using the input switch on the D/A converter).
Secrets contributor Sumit Chawla recently caught up with Benchmark’s VP and Chief Designer, John Siau to get a little more in-depth on several subjects.
Q: "Benchmark is one of the few companies that publishes an extensive set of measurements, but you also balance that with subjective testing. Can you talk about the equipment, the listening room, and the process for subjective testing?"
Q: "Was there ever a time where you learned something from a subjective test that was not captured by measurements?"
Q: "You conducted some listening tests to determine whether distortion in the “First Watt” was audible. What test material did you use for this, and what did you find?"
Q: "The AHB2 amplifier incorporates THX Audio Achromatic Amplifier technology. When and how did the partnership with THX come about?"
Q: "Linear power supplies have been and remain quite popular in high-end devices. You favor switch-mode power supplies. When and why did you make this switch?"
... and more!
At Benchmark, listening is the final exam that determines if a design passes from engineering to production. When all of the measurements show that a product is working flawlessly, we spend time listening for issues that may not have shown up on the test station. If we hear something, we go back and figure out how to measure what we heard. We then add this test to our arsenal of measurements.
Benchmark's listening room is equipped with a variety of signal sources, amplifiers and loudspeakers, including the selection of nearfield monitors shown in the photo. It is also equipped with ABX switch boxes that can be used to switch sources while the music is playing.
Benchmark's lab is equipped with Audio Precision test stations that include the top-of-the-line APx555 and the older AP2722 and AP2522. We don't just use these test stations for R&D - every product must pass a full set of tests on one of our Audio Precision test stations before it ships from our factory in Syracuse, NY.
Paul Seydor of The Absolute Sound interviews John Siau, VP and chief designer at Benchmark Media Systems. The interview accompanies Paul's review of the LA4 in the December, 2020 issue of TAS.
"At Benchmark, listening is the final exam that determines if a design passes from engineering to production. But since listening tests are never perfect, it’s essential we develop measurements for each artifact we identify in a listening test. An APx555 test set has far more resolution than human hearing, but it has no intelligence. We have to tell it exactly what to measure and how to measure it. When we hear something we cannot measure, we are not doing the right measurements. If we just listen, redesign, then repeat, we may arrive at a solution that just masks the artifact with another less-objectionable artifact. But if we focus on eliminating every artifact that we can measure, we can quickly converge on a solution that approaches sonic transparency. If we can measure an artifact, we don't try to determine if it’s low enough to be inaudible, we simply try to eliminate it."
- John Siau