Scientific evidence
Contents
Let us for a moment assume we can address all of the challenges. The next hard question is: when is it proven that an interlink – I keep using the interlink as an example – has an audible effect on sound quality? This paragraph is a bit more complicated, but there are different sorts of scientific evidence.
Under (the almost impossible to organise) controlled conditions, the evidence of a double-blind study is at most that a certain percentage of people hear a difference, and is thus only indirectly proven that an interlink apparently makes a difference because enough people hear differences. You could qualify that as ‘sufficient evidence’, but there is still a scientific discussion to start on when ‘enough’ is statistically significant to be seen as evidence.
What is the research worth in terms of practical experience?
Suppose you can devise a method by which you can provide scientifically watertight evidence that an interlink makes an audible difference, or disprove it once and for all. In that case, if 1 (test) person in the world hears a difference, it is proven that there is a difference. There is no threshold value, as with a blind test, because you have made the study scientifically watertight in advance.
What does that mean for everyone else on the planet? If they cannot hear the difference, what good is the evidence except scientific satisfaction?
The reverse is also true: the lack of scientifically watertight proof that an interlink makes an audible difference does not mean that interlinks therefore do not make an audible difference. All you can conclude is that it has never been proven that interlinks make audible difference. But you cannot conclude that pending that evidence there is no audible difference. Scientifically, you cannot make any statement.
… even worse:
If you manage to successfully differentiate between let’s say some cables or amplifiers in your controlled environment, this may not at all reflect how these samples behave in a different environment. Talking different floors (cable risers?), different qualities of AC from the wall, and so on.
Maybe two power cables with a specific amp sound the very same in one lab, but manage to sound different from each other in another environment.
In the end to me it seems that the “double-blind-A/B/X police” constantly fails in explaining why two different cables/amplifiers/whatever should sound indistinguishable from each other. Their standard phrase then is “if it sounds different, it’s broken”. 😉
Thank you! Now, we could use a similar discussion of why statements about cables like, “It has no coloration of its own and just lets the sound of the components through,” or “It recreates just what it was like in the studio,.” reflect fundamental misunderstanding of how cables and audio recordings are developed and created.
To add to my previous comment, I would say that in audio a double-blind study method is also unnecessary, since if one has a helper and they follow a method blind to the listener that’s sufficient, i.e., single blind. But if one doesn’t, or it takes some time to switch, then the effect of the physical — time and/or energy expenditure — come into play.
For audio, however, your point about the absence of a placebo is something of a straw man, since the question is difference not so much effect (measurable or scalable symptoms). Affect is a different matter and that’s really a Yes or No query — I like it or don’t, or some gradient of that. Then the question becomes which do I like more (and then other questions, such as function needed and cost).
Finally, and I think this is the “bottom line,” the person who feels they need a blinded method has two big problems on the face of it: hearing/listening ability, and honesty with themselves. For that person, the only answer is to judge on affect, which one likes better. If it’s a toss up, then go with the cheaper or most functional for one’s purpose.
To quote from the article: “ A/B is testing two variants of a test subject against each other. […] After all, the results then do not tell you whether A or B is better, but whether there is a clear preference in the listening group between A or B.“