False-positives in the Radin Double-slit Experiment on the Psycho-physical Effects of Observer Consciousness Jan Walleczek , Von Stillfried, Nikolaus (Phenoscience Laboratories, Berlin, Berlin, Germany) C14
In a series of publications, Radin and coworkers claim to provide replicable evidence for an effect of certain states of human consciousness on quantum-interference in a double-slit (DS) apparatus (for references see [1,2]). This research was partially funded by the Fetzer Franklin Fund (FFF) of which one of the authors (JW) is the scientific director. Given the radical implications of this claim, two further studies were performed to evaluate whether the reported effects could be false-positives caused by experimental and/or statistical error: Firstly, a conceptual replication was commissioned by the FFF to be performed by Radin and coworkers. This study implemented the advanced meta-experimental protocol (AMP), a confirmatory (pre-specified and predictive) methodology designed by JW specifically for revealing false-positives. Only null results were observed in the intervention groups, but the sham-experiments (i.e. data collected in the absence of any test subjects) produced a statistically significant (p = 0.021) 'false-positive effect which would be indistinguishable from the predicted true-positive effect' . Secondly, the statistical methods used by Radin et al. in three of their DS-studies were empirically analyzed : Radin kindly provided a DS-apparatus and a Matlab-script for data-analysis using three analysis-parameters closely resembling the ones used in the respective studies. We found that (i) an erroneous data-processing procedure ('trimming') created false-positives and (ii) the different analysis-parameters differ so strongly that they cannot be considered replications of each other and were likely not pre-specified by Radin et al. but developed post-hoc (based on the data to be analyzed). Our findings partially match with two other investigations: Firstly, Tremblay  reanalyzed the data from one DS-experiment by Radin et al. and, as the commissioned study , reported false-positive effects under control conditions (without test subjects). Furthermore, without the 'trimming' procedure he detected no statistically significant effects in the data at all (pers. comm.). Secondly, Guerrer  attempted a conceptual replication with advice from Radin. Like in , no observer-dependent effect was detected using a pre-registered analysis; only exploratory post-hoc analyses produced statistically significant effects. In summary, our presentation will argue that research standards in the Radin double-slit experiment need to improve. At least some of the respective results reported by Radin et al. are likely false positives and some of these were probably caused by post hoc adjustment of analysis methods. The latter is adequate in exploratory research only and must always be explicitly stated. Undisclosed hypothesizing after the results are known (HARKing) is known to be a source of false positive conclusions. We will also clarify why the criticism voiced by Radin et al. in  against our conclusions in  is not applicable (see ). References:  Walleczek, J. and N. von Stillfried, 2019: https://doi.org/10.3389/fpsyg.2019.01891;  von Stillfried, N. and J. Walleczek, 2019: https://osf.io/nyq5x;  Tremblay, N., 2019: https://doi.org/10.1371/journal.pone.0211511;  Guerrer, G., 2019: https://osf.io/qdkvx;  Radin, D., et al., 2019: https://osf.io/9csgu;  Walleczek, J. and N. von Stillfried, 2020: False-Positive Effect in the Radin Double-slit Experiment: Radin et al. (2019) Misinterpret the Confirmatory Design in Walleczek and von Stillfried (2019), forthcoming.