Everything is subjective
Contents
Everything you have read so far is a long introduction to the next sentence: double-blind research in audio at most tells you what the largest group of people prefer, not whether it is then objectively better. Which is fine, we listen to music for the sake of music.
But what are we actually testing?
If you would like to investigate whether an interlink makes an audible difference, you have to consider beforehand in your test methodology that you are not testing efficacy, but a difference. So what is your neutral measurement, equal to all test subjects, the so-called baseline measurement? That is quite complex to answer.
This is because the baseline itself is not the same to everyone; there is no placebo. I have never heard even the beginnings of an answer to that question from anyone shouting very loudly ‘double-blind or it is not true’.
Complex real world challenges: the testing conditions
But the biggest problem is that in audio it is impossible to keep the conditions, the test protocol, strictly regulated. Imagine testing an interlink. Something as simple as temperature can already affect the outcome of the test. So you will have to have strictly controlled conditions that are always the same when you test. So that also means that you need to have a sufficiently large randomly selected group of listeners who will therefore travel to your test setup to listen. If your name is Sonos you could do something like that, but for Alpha Audio it is impossible.
The human factor
Not to mention research questions as how big your listening group should be and what the test protocol should look like. Looking at my own experience, I know that if I listen to something on day 1 and listen to the same thing on day 2, I will write down two different findings. It depends on so many things: How tired am I? What is my mood? What kind of music am I listening to? Do I have a bit of a cold?
It is hard to always, consistently, listen in the same way. To always listen to exactly the same aspects of reproduction in high, mid or low, to how transients are rendered, timing differences, timbre, and so on. I trained myself on it, but I know I am not a robot.
To check for this variable outcome, I will have to listen more than once on different days and some kind of average should be taken from my findings for analysis. How often would I have to listen? I have no idea.
… even worse:
If you manage to successfully differentiate between let’s say some cables or amplifiers in your controlled environment, this may not at all reflect how these samples behave in a different environment. Talking different floors (cable risers?), different qualities of AC from the wall, and so on.
Maybe two power cables with a specific amp sound the very same in one lab, but manage to sound different from each other in another environment.
In the end to me it seems that the “double-blind-A/B/X police” constantly fails in explaining why two different cables/amplifiers/whatever should sound indistinguishable from each other. Their standard phrase then is “if it sounds different, it’s broken”. 😉
Thank you! Now, we could use a similar discussion of why statements about cables like, “It has no coloration of its own and just lets the sound of the components through,” or “It recreates just what it was like in the studio,.” reflect fundamental misunderstanding of how cables and audio recordings are developed and created.
To add to my previous comment, I would say that in audio a double-blind study method is also unnecessary, since if one has a helper and they follow a method blind to the listener that’s sufficient, i.e., single blind. But if one doesn’t, or it takes some time to switch, then the effect of the physical — time and/or energy expenditure — come into play.
For audio, however, your point about the absence of a placebo is something of a straw man, since the question is difference not so much effect (measurable or scalable symptoms). Affect is a different matter and that’s really a Yes or No query — I like it or don’t, or some gradient of that. Then the question becomes which do I like more (and then other questions, such as function needed and cost).
Finally, and I think this is the “bottom line,” the person who feels they need a blinded method has two big problems on the face of it: hearing/listening ability, and honesty with themselves. For that person, the only answer is to judge on affect, which one likes better. If it’s a toss up, then go with the cheaper or most functional for one’s purpose.
To quote from the article: “ A/B is testing two variants of a test subject against each other. […] After all, the results then do not tell you whether A or B is better, but whether there is a clear preference in the listening group between A or B.“