Double-blind
Contents
Double-blind research is a method where both the person conducting the test and the test subject do not know whether they are working with the subject under scrutiny, or whether they are dealing with a neutral placebo. A placebo is an ineffective alternative to the test subject, which is indistinguishable from the real test subject. I think that with this wording, a little light bulb may already switch on for some readers, because what does a placebo look like with an audio object? I am getting a bit ahead of myself, but it is good to keep this question in mind.
The pharmaceutical industry
The well-known example of double-blind research, which almost everyone has heard of, is the pharmaceutical industry. If a new pill needs to be shown to be effective, a test is set up where, in a large study group, patients are divided randomly into two groups. One half gets a pill with the active ingredient, the other half an identical-looking pill without the active ingredient, the placebo. The doctor treating the patient does not know which of the two pills is administered and assesses the outcome of each individual treatment.
All the results of the two groups are combined and statistical analysis – and there is a lot to say about this too, but for this article I will leave that aside – is used to determine whether the pill is effective or not.
Conditions under which a double-blind study is conducted
There are a few conditions that need to be met for this research method to work:
- The study group must be large enough to allow all kinds of other variables to be averaged out. These include gender, age and all sorts of other variables that may affect the effect of the substance being studied.
- The assignment of test persons to a group should be random, but each group must continue to meet the first condition.
- The conditions under which the test is performed must be controlled or identical. In the case of a drug test, it sounds simple, but the prescription or administration protocol of a pharmaceutical test is carefully set up.
- The method of measuring the results is also strictly regulated. Especially if you want to include in the results findings like patient experience or quality of life during treatment. These are subjective, but often very important aspects of a study. How do you make these outcomes objectifiable in your statistical analysis? You have to have thought about that very carefully beforehand and therefore the protocol and outcome measurement of the test has to be guaranteed and auditable. If you read something in the news about questioning the efficacy of a new drug, or questioning that afterwards, it is always about the validity of the test or outcome measurement method.
The placebo effect
I would like to add a comment about analysis, especially since the word ‘placebo effect’ is often dropped in the comments. Many people are not aware that double-blind studies were not devised to avoid the placebo effect, but that placebo effect is one of the variables to be taken into account in the analysis.
How should we translate this to what we do at Alpha Audio, when comparing, say, 32 interlinks?
… even worse:
If you manage to successfully differentiate between let’s say some cables or amplifiers in your controlled environment, this may not at all reflect how these samples behave in a different environment. Talking different floors (cable risers?), different qualities of AC from the wall, and so on.
Maybe two power cables with a specific amp sound the very same in one lab, but manage to sound different from each other in another environment.
In the end to me it seems that the “double-blind-A/B/X police” constantly fails in explaining why two different cables/amplifiers/whatever should sound indistinguishable from each other. Their standard phrase then is “if it sounds different, it’s broken”. 😉
Thank you! Now, we could use a similar discussion of why statements about cables like, “It has no coloration of its own and just lets the sound of the components through,” or “It recreates just what it was like in the studio,.” reflect fundamental misunderstanding of how cables and audio recordings are developed and created.
To add to my previous comment, I would say that in audio a double-blind study method is also unnecessary, since if one has a helper and they follow a method blind to the listener that’s sufficient, i.e., single blind. But if one doesn’t, or it takes some time to switch, then the effect of the physical — time and/or energy expenditure — come into play.
For audio, however, your point about the absence of a placebo is something of a straw man, since the question is difference not so much effect (measurable or scalable symptoms). Affect is a different matter and that’s really a Yes or No query — I like it or don’t, or some gradient of that. Then the question becomes which do I like more (and then other questions, such as function needed and cost).
Finally, and I think this is the “bottom line,” the person who feels they need a blinded method has two big problems on the face of it: hearing/listening ability, and honesty with themselves. For that person, the only answer is to judge on affect, which one likes better. If it’s a toss up, then go with the cheaper or most functional for one’s purpose.
To quote from the article: “ A/B is testing two variants of a test subject against each other. […] After all, the results then do not tell you whether A or B is better, but whether there is a clear preference in the listening group between A or B.“