From 57ff1912e49eb4cc9ec8694785adcb7aed113dd6 Mon Sep 17 00:00:00 2001 From: Lukas Wallrich Date: Fri, 13 Nov 2026 09:11:19 +0000 Subject: [PATCH 1/9] Reversals: correct verified reporting errors and wrong-work links Fixes found in the FLoRA cross-check and verified against primary sources: bottomless soup bowl now cites the preregistered replication (Lopez et al. 2024); hurricane, depressive realism, AoA recognition, object distance, object colour, facial feedback, loss aversion, slow blinking, colour approach/avoidance and peptic ulcer entries correct study descriptions, samples or labels; 25 links that pointed to unrelated works now point to the cited work or are unlinked where the intended DOI is unverified. Co-Authored-By: Claude Opus 5.5 --- content/reversals/reversals.md | 94 +++++++++++++++++----------------- 1 file changed, 47 insertions(+), 47 deletions(-) diff --git a/content/reversals/reversals.md b/content/reversals/reversals.md index e302c753b10..5d73430391c 100644 --- a/content/reversals/reversals.md +++ b/content/reversals/reversals.md @@ -457,9 +457,9 @@ You can find a list of all effects we are working on [here](https://docs.google. {{< spoiler text="Statistics" >}} * Status: reversed * Original paper: '[Female hurricanes are deadlier than male hurricanes](https://www.pnas.org/content/111/24/8782)', Jung 2014; observational study, n=92 hurricanes discarding two important outliers. [citations = 113(GS, Mar 2022)]. -* Critiques: [Christensen 2014](https://www.pnas.org/content/111/34/E3497) [same data, citations = 114(GS, March 2022)]. [Smith 2016 ](https://doi.org/10.1016/j.wace.2015.11.006)[same data, citations = 8(GS, March 2022)]. +* Critiques: [Christensen 2014](https://www.pnas.org/content/111/34/E3497) [same data, citations = 114(GS, March 2022)]. [Smith 2016 ](https://doi.org/10.1016/j.wace.2015.11.006)[expanded Atlantic data, 1979–2014, including tropical storms and non-U.S. deaths, n = 420 storms, and new Pacific data, n = 579 storms; citations = 8(GS, March 2022)]. * Original effect size: _d_=0.65: [176%](https://www.google.co.uk/search?source=hp&ei=HC1yXtKbBqeRlwTgzplw&q=41.84+%2F+15.15+&oq=41.84+%2F+15.15+&gs_l=psy-ab.3...606.10628..10753...0.0..0.91.412.5......3....1..gws-wiz.......0i7i30j0.raVAib7H4Jk&ved=0ahUKEwjSmciGm6ToAhWnyIUKHWBnBg4Q4dUDCAg&uact=5) increase in deaths from flipping names from relatively masculine to relatively feminine. -* Replication effect size: Smith: 264% decrease in deaths (Atlantic); 103% decrease (Pacific). +* Replication effect size: Smith: female-named storms caused fewer deaths on average than male-named storms, 35.98 vs. 131.01 (Atlantic, about 73% fewer, _p_ = .31) and 4.03 vs. 8.21 (Pacific, about 51% fewer, _p_ = .50); none of the differences was statistically significant. {{< /spoiler >}} * **Implicit bias testing for racism**. Implicit bias scores poorly predict actual bias, _r_ = 0.15. The operationalisations used to measure that predictive power are often unrelated to actual discrimination (e.g. ambiguous brain activations). Test-retest reliability of 0.44 for race, which is usually classed as “unacceptable”. This isn’t news; the original study also found very low test-criterion correlations. @@ -691,7 +691,7 @@ You can find a list of all effects we are working on [here](https://docs.google. {{< spoiler text="Statistics" >}} * Status: not replicated * Original paper: ‘[Blue or Red? Exploring the Effect of Color on Cognitive Task Performances](https://www.science.org/doi/abs/10.1126/science.1169144)’, Mehta and Zhu 2009; six studies, studies 1-5 between-subject experiments, study 6 correlational, Study 1: n = 69, Study 2: n = 208, Study 3: n = 118, Study 4: n = 42, Study 5: n = 161, Study 6: n = 68. [citations=1003 (GS, November 2022)]​. -* Critiques:[ Steele et al. 2010](http://www.appstate.edu/~steelekm/documents/Psychonomic2010Poster_Steele.pdf) direct replication of Mehta an d Zhu Study 1 [n=172, citations=2(GS, November 2022)].[ Steele et al. 2013](http://www.appstate.edu/~steelekm/documents/APS_Poster_2013_Steele.pdf) direct replication of Mehta andZhu Study 1 [n=263, citations=2(GS, November 2022)].[ Steele 2014](https://doi.org/10.3758/s13423-013-0548-3) direct replication of Mehta and Zhu Study 1 [n=263, citations=45(GS, November 2022)].​ +* Critiques:[ Steele et al. 2010](http://www.appstate.edu/~steelekm/documents/Psychonomic2010Poster_Steele.pdf) direct replication of Mehta an d Zhu Study 1 [n=172, citations=2(GS, November 2022)].[ Steele et al. 2013](http://www.appstate.edu/~steelekm/documents/APS_Poster_2013_Steele.pdf) direct replication of Mehta andZhu Study 1 [n=263; poster of the same sample as Steele 2014, not an independent replication; citations=2(GS, November 2022)].[ Steele 2014](https://doi.org/10.3758/s13423-013-0548-3) direct replication of Mehta and Zhu Study 1 [n=263, citations=45(GS, November 2022)].​ * Original effect size: Study 1 – blue versus red condition comparison for approach-related anagrams, _d_ = 0.81, and for avoidance-related anagrams _d_ = 0.96; Study 2 – blue versus red condition on detailed-oriented task, _d_ = 0.64, and on creative task, _d_ = 0.6; Study 3 - blue versus red condition on detailed-oriented task, _d_ = 1.05, and on creative task, _d_ = 0.69; Study 4 - blue versus red condition on the practicality of the designed toy, _d_ = 0.64, and on the originality/novelty of the designed toy, _d_ = 0.67; Study 5​ - blue versus red condition on detailed-oriented processing style, _d_ = 0.42, and creative thinking, _d_ = 0.56; * Replication effect size:​ Steele et al.: colour by word-type interaction _ηp2_ = 0.038 / _d_ = 0.20 [_ηp2 _calculated from the reported F statistics and converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)] (not replicated).​ Steele et al.: colour by word-type interaction _ηp2_ = 0.014 / _d_ = 0.12 [_ηp2 _calculated from the reported F statistics and converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)] (not replicated).​ Steele: The colour X word type interaction _ηp2_ = 0.007 [reported] / _d_ = 0.083 [converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)] (not replicated). {{< /spoiler >}} @@ -781,7 +781,7 @@ You can find a list of all effects we are working on [here](https://docs.google. {{< spoiler text="Statistics" >}} * Status: mixed. * Original paper: ‘[The Physical Burdens of Secrecy](https://doi.org/10.1037/a0027598)’, Slepian et al. 2012; studies 1, 2 and 4 experimental mixed model design, study 3 correlational, study 1 n = 40, study 2 n = 36, study 3 n = 40, study 4 n = 30. [citations=113 (GS, November 2022)]​. -* Critiques: [LeBel and Wilbur 2014](https://doi.org/10.3758/s13423-013-0549-2), direct Slepian et al. 2012 Study 1 replication [Study 1 n=240, Study 2 n = 90, citations=24(GS, November 2022)]. [Pecher et al. 2015](https://doi.org/10.1177/1948550613498516), direct Slepian et al.2012 Study 1 and Study 2 replication [Study 1 n=100, Study 2 n = 100, Study 3 n = 118, citations=11(GS, November 2022)]. [Slepian et al. 2014](https://doi.org/10.1177/1948550613498516) [Study 1 n=83, Study 2 n = 174, citations=51(GS, November 2022)]. ​[Slepian et al. 2015](https://doi.org/10.1037/xge0000052), [Study 1 n = 100, Study 2 n = 100, Study 3 n = 100, Study 4 n = 352, citations=42(GS, November 2022)]. +* Critiques: [LeBel and Wilbur 2014](https://doi.org/10.3758/s13423-013-0549-2), direct Slepian et al. 2012 Study 1 replication [Study 1 n=240, Study 2 n = 90, citations=24(GS, November 2022)]. [Pecher et al. 2015](https://doi.org/10.1037/xge0000090), direct Slepian et al.2012 Study 1 and Study 2 replication [Study 1 n=100, Study 2 n = 100, Study 3 n = 118, citations=11(GS, November 2022)]. [Slepian et al. 2014](https://doi.org/10.1177/1948550613498516) [Study 1 n=83, Study 2 n = 174, citations=51(GS, November 2022)]. ​[Slepian et al. 2015](https://doi.org/10.1037/xge0000052), [Study 1 n = 100, Study 2 n = 100, Study 3 n = 100, Study 4 n = 352, citations=42(GS, November 2022)]. * Original effect size: Study 1 – Big/meaningful vs. small/trivial secret hill steepness comparisons _d_ = 0.78 (calculated from M and SD data in the paper, also reported in[ LeBel and Wilbur 2014](https://doi.org/10.3758/s13423-013-0549-2)); Study 2 - Big/meaningful vs. small/trivial distant perception comparisons _d_ = 0.67 (calculated from M and SD data in the paper, also reported in[ Pecher et al. 2015](https://doi.org/10.1037/xge0000090)); Study 3 – effects of the frequencies of thought of infidelity on estimated effort required by physical task _R2 _= .21 / _d_ = 1.03 [converted using this[ conversion](https://www.escal.site/)]; Study 4 – more burdensome vs. less burdensome secret concealment effects on willingness to help others with physical task _r_ = .44 / _d_= 0.98 [converted using this[ conversion](https://www.escal.site/)]. * Replication effect size: LeBel and Wilbur: Study 1 - ​ Big/meaningful vs. small/trivial secret hill steepness comparisons _d_ = 0.176 [-.08, .43] [reported] (not replicated); Study 2 - Big/meaningful vs. small/trivial secret hill steepness comparisons _d_ = -0.319 [-.73, .10] [reported] (not replicated). Pecher et al.: Study 1 - Big/meaningful vs. small/trivial secret hill steepness comparisons _d_ = 0.08 [-0.31, 0.47] [reported] (not replicated); Study 2 - Big/meaningful vs. small/trivial secret hill steepness comparisons _d_ = 0.21 [-0.18, 0.60] [reported] (not replicated); Study 3 - Big/meaningful vs. small/trivial secret perceived distance comparisons _d_ = 0.21 [-0.15, 0.57] [reported] (not replicated). Slepian et al. 2014: Study 1 - ​ Big/meaningful secret recollection condition effects on hill slant estimation in comparison to reveаling a secret, _r_ = .29 [reported] / _d_= 0.61, and control condition _r_ = .34 [reported] / _d_= 0.72 [_d_'s converted using this[ conversion](https://www.escal.site/)] (replicated); Study 2 - Big/meaningful secret recollection condition effects on distance estimation in comparison to revealing a secret, _r_ = .24 [reported] / _d_= 0.49, and control condition _r_ = .30 [reported] / _d_= 0.62 [_d_s converted using this[ conversion](https://www.escal.site/)] (replicated). Slepian et al. 2015: Study 1 - Big/meaningful vs. small/trivial secret hill steepness comparisons _d_ = 0.31 (calculated from M and SD data in the paper, non-significant) (not replicated); Study 2 - Big/meaningful vs. small/trivial secret hill steepness comparisons _r_ = .28 [reported] / _d_= 0.58 [converted using this[ conversion](https://www.escal.site/)] (replicated); Study 3 – Recalling preoccupying vs. non-preoccupying secret effects on hill slant judgements _r_ = .23 [reported] / _d_= 0.47 [converted using this[ conversion](https://www.escal.site/)] (replicated); Study 4 - Recalling preoccupying vs. non-preoccupying secret effects on hill slant judgements _r_ = .11 [reported] /_d_= 0.22 [converted using this[ conversion](https://www.escal.site/)] (replicated). {{< /spoiler >}} @@ -808,7 +808,7 @@ You can find a list of all effects we are working on [here](https://docs.google. {{< spoiler text="Statistics" >}} * Status: mixed. * Original paper: ‘[Learning About What Others Were Doing: Verb Aspect and Attributions of Mundane and Criminal Intent for Past Actions](https://doi.org/10.1177/0956797610395393)’, Hart and Albarracín 2011; three between-subject experiments, Experiment 1 n = 54, Experiment 2 n = 37, Experiment 3 n = 48. [citations=40(GS, October 2022)]​. -* Critiques:[ Eerland et al. 2016](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0141181) Multilab direct replication of Study 3 [N=685 across 12 studies, citations=82(gs, October 2022)].​ [Sherrill et al. 2015](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0141181) [N=699 across 4 Experiments, citations=14(GS, October 2022)].[ ](https://doi.org/10.1177/1745691615605826) +* Critiques:[ Eerland et al. 2016](https://doi.org/10.1177/1745691615605826) Multilab direct replication of Study 3 [N=685 across 12 studies, citations=82(gs, October 2022)].​ [Sherrill et al. 2015](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0141181) [N=699 across 4 Experiments, citations=14(GS, October 2022)]. * Original effect size: Experiment 1 – accessibility to intention-relevant concepts _d_= 1.00 [reported]; Experiment 2 – attribution of intentionality _d_= 1.00 [reported], detailed segmentation of behaviour descriptions _d_= 1.23 [reported]; Experiment 3 – criminal intentionality _d_= 0.76 [reported], intention attributions _d_= 0.66 [reported], imagery _d_= 0.73 [reported]. * Replication effect size: Eerland et al.: intentionality _d_= -0.98 to _d_= 0.65 [reported], Meta-analytic effect for laboratory replications _d_= -0.24 [-0.50, 0.02] [non-significant, reported]; imagery _d_= -0.45 to _d_= 0.33, Meta-analytic effect for laboratory replications _d_= -0.08 [-0.23, 0.07] [non-significant, reported]; intention attribution _d_= -0.29 to _d_= 0.19, Meta-analytic effect for laboratory replications _d_= 0.00 [-0.07, 0.08] [non-significant, reported]. Sherrill et al.: Experiment 2 – murder intentionality judgement in imperfective vs. perfective condition _ηp2_ = 0.036 [reported] / _d_ = 0.19 [converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)] (replicated); Experiment 3 – murder intentionality judgement in imperfective vs. perfective condition _ηp2_ = 0.040 [reported] / _d_ = 0.20 [converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)] (replicated); Experiment 4 – imperfective murder vs. perfective murder condition _d_= 0.15 [non-significant, reported]. {{< /spoiler >}} @@ -1162,9 +1162,9 @@ You can find a list of all effects we are working on [here](https://docs.google. {{< spoiler text="Statistics" >}} * Status: not replicated * Original paper: '[Inhibiting and Facilitating Conditions of the Human Smile: A Nonobtrusive Test of the Facial Feedback Hypothesis](https://jhrcook.github.io/wagenmaker-data-analysis/papers/Strack_1988_Inhibiting-and-facilitating-conditions-of-the-human-smile-a-nonobtrusive-test-of-the-facial-feedback-hypothesis.pdf)', Strack et al. 1988; 2 experimental studies, n1= 92, n2=83. [citation= 2577(GS, February, 2022)]. -* Critiques: [Coles et al. 2019](https://psyarxiv.com/svjru/) [meta-analysis _k_ = 98, n=3878, citation= 115 (GS, February 2022)]. [Wagenmakers et al. 2016](https://www.ejwagenmakers.com/2016/WagenmakersEtAl2016Strack.pdf) [meta-analysis n=1894, citation=349 (GS, February 2022)]. [Schimmack 2017](https://replicationindex.com/2017/09/04/the-power-of-the-pen-paradigm-a-replicability-analysis/) [replicability analysis _k_=19, citation=0 (GS, March 2023)]. +* Critiques: [Coles et al. 2019](https://psyarxiv.com/svjru/) [meta-analysis of 286 effect sizes from 138 studies, citation= 115 (GS, February 2022)]. [Coles et al. 2022](https://doi.org/10.1038/s41562-022-01458-9) [Many Smiles Collaboration, preregistered multi-lab study, n = 3,878]. [Wagenmakers et al. 2016](https://www.ejwagenmakers.com/2016/WagenmakersEtAl2016Strack.pdf) [meta-analysis n=1894, citation=349 (GS, February 2022)]. [Schimmack 2017](https://replicationindex.com/2017/09/04/the-power-of-the-pen-paradigm-a-replicability-analysis/) [replicability analysis _k_=19, citation=0 (GS, March 2023)]. * Original effect size: Study 1: _d_ = 0.82, _d_ = 0.43 (0.82 out of 9). -* Replication effect size: Wagenmakers et al.: 0.03 out of 9, CI overlapping 0. Coles et al.: [A meta-analysis of 98 studies](https://psyarxiv.com/svjru/) finds _d_= 0.2 [0.14, 0.26] with an absurdly low p value, and doesn't find publication bias. But this latter point simply can't be right. Given _d_ = 0.2 and the convention of targeting 80% power to detect a real phenomenon, you would need very high sample sizes, n > 500. And almost all of the included studies are N < 100. All reported in Coles et al.: Andréasson & Dimberg: _d_=-0.22. Andréasson: _d_=-0.05 to _d_ =0.49. Baumeister et al.: _d_=0.63 to _d_ =1.26. Bodenhausen et al.: _d_ =0.55. Bush et al.: _d_ =0.16. Butler et al.: _d_ =-0.1 to _d_ =-0.83. Butler et al.: _d_ =-0.03. Cai et al.: _d_ =-0.08. Ceschi & Scherer: _d_ =0.74. Clapp: _d_ =0.08 to _d_ =0.69. Laird & Crosby: _d_ =-0.13 to _d_ =0.35. Davey et al.: _d_ =-0.25 to _d_ =0.73. Davis: _d_ =-0.19 to _d_ =0.99. Davis: _d_ =-0.19 to _d_ =0.87. Davis et al.: _d_ =0.07 to _d_ =0.51. Davis et al.: _d_ =-0.15 to _d_ =0.1. Davis et al.: _d_ =-0.16. Demaree et al.: _d_ =0.16 to _d_ =0.62. Demaree et al.: _d_ =-0.64 to _d_ =0.06. Dillon et al.: _d_ =0.11. Dimberg & Söderkvist: _d_ =0.1 to 0.51. Duncan & Laird:_ d_ =0.38 to _d_ =0.51. Duncan & Laird: _d_ =0.44 to _d_ =0.59. Dzokoto et al.: _d_ =0.2 to _d_ =1.07. Flack, Laird & Cavallaro: _d_ =-0.49 to _d_ =1.31. Flack, Laird & Cavallaro: _d_ =0.29 to _d_ =1.41. Flack: _d_ =0.35 to _d_ =0.72. Gan et al.: _d_ =-0.11. Goldin et al.: _d_ =0.8. Gross & Levenson: _d_ =0.04 to _d_ =0.37. Gross: _d_ =-0.23 to _d_ =0.37. Gross: _d_ =0.18. Harris: _d_ =0.07. Hawk et al.: _d_ =0.85. Helt & Fein: _d_ =0.42. Hendricks & Buchanan: _d_ =-0.08. Hendricks: _d_ =0.02. Henry et al.: _d_ =-0.49 to _d_ =0.25. Henry et al.: _d_ =-0.05 to d_ d_ 0.53. Hess et al.: _d_ =-0.28 to _d_ =0.14. Hofmann et al.: _d_ =-0.03. Ito et al.: _d_ =-0.39 to _d_ =-0.25. Kalokerinos et al.: _d_ =-0.06 to _d_ =1.32. Kao et al.: _d_ =-0.67 to _d_ =0.98. Kircher et al.: _d_ =1.14 to _d_ =1.89. Korb et al.: _d_ =0.21. Labott & Teleha: _d_ =0.04 to _d_ =0.91. Laird: _d_ =0.13 to _d_ =0.55. Lalot et al.: _d_ =-0.17. Larsen et al.: _d_ =0.43. Lee: _d_ =-0.27 to _d_ =0.48. Lewis & Bowler: _d_ =1.35. Lewis: _d_ =0.56 to _d_ =0.71. Ma: _d_ =-0.21. Maldonado et al.: _d_ =0.12. Marmolejo-Ramos & Dunn: _d_ =-0.07 to _d_ =0.38. Martijn et al.: _d_ =-0.24. McCanne & Anderson: _d_ =-2.16 to _d_ =4.73. McCaul et al.: _d_ =0.25. McIntosh et al.: _d_ =0.54. Meeten et al.: _d_ =0.49. Miyamoto: _d_ =0.49 to _d_ =0.17. Moore & Zoellner: _d_ =-0.87. Kappas: _d_ =0.08 to _d_ =0.74. Ohira & Kurono: _d_ =-1.38 to _d_ =1.23. Paredes et al.: _d_ =0.85. Paul et al.: _d_ =0.91. Pedder et al.: _d_ =0.07 to _d_ =0.22. Phillips et al.: _d_ =0.08 to _d_ =0.18. Reisenzein & Studtmann: _d_ =-0.08 to _d_ =0.034. Richards, Butler & Gross: _d_ =-0.12 to _d_ = 0.19. Richards & Gross: _d_ =-0.1 to d=0.36. Richards & Gross: _d_ =-0.12 to _d_ =0.39. Richards & Gross: _d_ =0.34. Roberts et al.: _d_ =0.07. Robinson & Demaree: _d_ =-0.04 to _d_ =0. Roemer: _d_ =0.29 to _d_ =0.58. Rohrmann et al.: _d_ =0.13 to _d_ =0.16. Rummer et al.: _d_ =0.46 to _d_ =0.57. Schmeichel , Vohs, & Baumeister: _d_ =-0.23. Schmeichel et al.: _d_ =0.1. Söderkvist & Dimberg (unpublished): _d_ =0.36. Söderkvist et al.: _d_ =0.17 to _d_ =0.34. Soussignan: _d_ =-0.17 to _d_ =1.11. Stel et al.: _d_ =1 to _d_ =1.11. Strack et al.: _d_ =-0.51 to _d_ =0.55. Tamir et al.: _d_ =-0.16. Tourangeau & Ellsworth: _d_ =0.3. Trent: _d_ =-0.22 to _d_ =-0.06. Vieillard et al.: _d_ =-0.12 to _d_ =0.66. Wagenmakers et al.: _d_ =-0.17 to _d_ =0.25. Wittmer: _d_ =-0.36 to _d_ =-0.21. Yartz: _d_ =-0.18 to _d_ =0.5. Zajonc et al.: _d_ =0.31 to _d_ =1.27. Zariffa et al.: _d_ =-0.57 to _d_ =-0.14. Zhu et al.: _d_ =1.74. [Schimmack](https://replicationindex.com/2017/09/04/the-power-of-the-pen-paradigm-a-replicability-analysis/): strong evidence of publication bias on a subset of these papers, using a proper power analysis. Direct replications of the original pen-in-mouth protocol fail; but new conceptual replications appear to work. [Coles et al.:](https://www.nature.com/articles/s41562-022-01458-9) effect is an absolute 5% increase in a happiness metric. +* Replication effect size: Wagenmakers et al.: 0.03 out of 9, CI overlapping 0. Coles et al.: [A meta-analysis of 138 studies](https://psyarxiv.com/svjru/) finds _d_= 0.2 [0.14, 0.26] with an absurdly low p value, and doesn't find publication bias in studies of emotional experience (it does find it in studies of affective judgements). But this latter point simply can't be right. Given _d_ = 0.2 and the convention of targeting 80% power to detect a real phenomenon, you would need very high sample sizes, n > 500. And almost all of the included studies are N < 100. All reported in Coles et al.: Andréasson & Dimberg: _d_=-0.22. Andréasson: _d_=-0.05 to _d_ =0.49. Baumeister et al.: _d_=0.63 to _d_ =1.26. Bodenhausen et al.: _d_ =0.55. Bush et al.: _d_ =0.16. Butler et al.: _d_ =-0.1 to _d_ =-0.83. Butler et al.: _d_ =-0.03. Cai et al.: _d_ =-0.08. Ceschi & Scherer: _d_ =0.74. Clapp: _d_ =0.08 to _d_ =0.69. Laird & Crosby: _d_ =-0.13 to _d_ =0.35. Davey et al.: _d_ =-0.25 to _d_ =0.73. Davis: _d_ =-0.19 to _d_ =0.99. Davis: _d_ =-0.19 to _d_ =0.87. Davis et al.: _d_ =0.07 to _d_ =0.51. Davis et al.: _d_ =-0.15 to _d_ =0.1. Davis et al.: _d_ =-0.16. Demaree et al.: _d_ =0.16 to _d_ =0.62. Demaree et al.: _d_ =-0.64 to _d_ =0.06. Dillon et al.: _d_ =0.11. Dimberg & Söderkvist: _d_ =0.1 to 0.51. Duncan & Laird:_ d_ =0.38 to _d_ =0.51. Duncan & Laird: _d_ =0.44 to _d_ =0.59. Dzokoto et al.: _d_ =0.2 to _d_ =1.07. Flack, Laird & Cavallaro: _d_ =-0.49 to _d_ =1.31. Flack, Laird & Cavallaro: _d_ =0.29 to _d_ =1.41. Flack: _d_ =0.35 to _d_ =0.72. Gan et al.: _d_ =-0.11. Goldin et al.: _d_ =0.8. Gross & Levenson: _d_ =0.04 to _d_ =0.37. Gross: _d_ =-0.23 to _d_ =0.37. Gross: _d_ =0.18. Harris: _d_ =0.07. Hawk et al.: _d_ =0.85. Helt & Fein: _d_ =0.42. Hendricks & Buchanan: _d_ =-0.08. Hendricks: _d_ =0.02. Henry et al.: _d_ =-0.49 to _d_ =0.25. Henry et al.: _d_ =-0.05 to d_ d_ 0.53. Hess et al.: _d_ =-0.28 to _d_ =0.14. Hofmann et al.: _d_ =-0.03. Ito et al.: _d_ =-0.39 to _d_ =-0.25. Kalokerinos et al.: _d_ =-0.06 to _d_ =1.32. Kao et al.: _d_ =-0.67 to _d_ =0.98. Kircher et al.: _d_ =1.14 to _d_ =1.89. Korb et al.: _d_ =0.21. Labott & Teleha: _d_ =0.04 to _d_ =0.91. Laird: _d_ =0.13 to _d_ =0.55. Lalot et al.: _d_ =-0.17. Larsen et al.: _d_ =0.43. Lee: _d_ =-0.27 to _d_ =0.48. Lewis & Bowler: _d_ =1.35. Lewis: _d_ =0.56 to _d_ =0.71. Ma: _d_ =-0.21. Maldonado et al.: _d_ =0.12. Marmolejo-Ramos & Dunn: _d_ =-0.07 to _d_ =0.38. Martijn et al.: _d_ =-0.24. McCanne & Anderson: _d_ =-2.16 to _d_ =4.73. McCaul et al.: _d_ =0.25. McIntosh et al.: _d_ =0.54. Meeten et al.: _d_ =0.49. Miyamoto: _d_ =0.49 to _d_ =0.17. Moore & Zoellner: _d_ =-0.87. Kappas: _d_ =0.08 to _d_ =0.74. Ohira & Kurono: _d_ =-1.38 to _d_ =1.23. Paredes et al.: _d_ =0.85. Paul et al.: _d_ =0.91. Pedder et al.: _d_ =0.07 to _d_ =0.22. Phillips et al.: _d_ =0.08 to _d_ =0.18. Reisenzein & Studtmann: _d_ =-0.08 to _d_ =0.034. Richards, Butler & Gross: _d_ =-0.12 to _d_ = 0.19. Richards & Gross: _d_ =-0.1 to d=0.36. Richards & Gross: _d_ =-0.12 to _d_ =0.39. Richards & Gross: _d_ =0.34. Roberts et al.: _d_ =0.07. Robinson & Demaree: _d_ =-0.04 to _d_ =0. Roemer: _d_ =0.29 to _d_ =0.58. Rohrmann et al.: _d_ =0.13 to _d_ =0.16. Rummer et al.: _d_ =0.46 to _d_ =0.57. Schmeichel , Vohs, & Baumeister: _d_ =-0.23. Schmeichel et al.: _d_ =0.1. Söderkvist & Dimberg (unpublished): _d_ =0.36. Söderkvist et al.: _d_ =0.17 to _d_ =0.34. Soussignan: _d_ =-0.17 to _d_ =1.11. Stel et al.: _d_ =1 to _d_ =1.11. Strack et al.: _d_ =-0.51 to _d_ =0.55. Tamir et al.: _d_ =-0.16. Tourangeau & Ellsworth: _d_ =0.3. Trent: _d_ =-0.22 to _d_ =-0.06. Vieillard et al.: _d_ =-0.12 to _d_ =0.66. Wagenmakers et al.: _d_ =-0.17 to _d_ =0.25. Wittmer: _d_ =-0.36 to _d_ =-0.21. Yartz: _d_ =-0.18 to _d_ =0.5. Zajonc et al.: _d_ =0.31 to _d_ =1.27. Zariffa et al.: _d_ =-0.57 to _d_ =-0.14. Zhu et al.: _d_ =1.74. [Schimmack](https://replicationindex.com/2017/09/04/the-power-of-the-pen-paradigm-a-replicability-analysis/): strong evidence of publication bias on a subset of these papers, using a proper power analysis. Direct replications of the original pen-in-mouth protocol fail; but new conceptual replications appear to work. [Coles et al.:](https://www.nature.com/articles/s41562-022-01458-9) effect is an absolute 5% increase in a happiness metric. {{< /spoiler >}} * **Positive affirmation on mood**. Positive self-statements boost mood for people with high self-esteem and reduce mood for people with low self-esteem. @@ -1180,7 +1180,7 @@ You can find a list of all effects we are working on [here](https://docs.google. {{< spoiler text="Statistics" >}} * Status: mixed * Original paper: ‘[An Outpatient Program in Behavioral Medicine for Chronic Pain Patients Based on the Practice of Mindfulness Meditation](https://v-assets.cdnsw.com/fs/Articles_scientifiques_Mindfulness/6lwfg-Mindfulness_chronic_pains_kabatzinn_mbsr_1982.pdf)’, Kabat-Zinn, 1982; longitudinal study, n=51. [citations = 5726 (PubMed, January 2023)]. -* Critiques: Hoffmann & Witt 2010 [meta-analysis, k=39 studies, n=1,140, citations= 5337 (GS, May 2023)]. Khoury et al. 2013 [meta-analysis, k= 209 studies, n=12,145, citations= 2484 (GS, May 2023)]. Strauss et al., 2014 [meta-analysis, k= 19 studies, n=578, citations= 659 (GS, May 2023)]. [Fumero et al. 2020](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8314302/) [_n_= 91, citations= 30 (PubMed, March 2023)]. Tao et al. 2022 [meta-analysis, k=7 trials, n=502, citations= 2 (GS, May 2023)]. Chayadi et al., 2022 [meta-analysis, k=36, n=1677, citations= 8 (GS, May 2023)]. [Coronado-Montoya 2016](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0153220) [n=357, citations= 21 (GS, March 2023)]. [Britton 2019](https://doi.org/10.1016/j.copsyc.2018.12.011) [_n_= NA, citations= 169 (GS, March 2023)]. [Hsiao et al. 2020](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6435335/) [_n_ = NA, citations= 17(GS, May 2023)]. [Kuyken et al. 2022](https://pubmed.ncbi.nlm.nih.gov/35820992/) [n=8376, citations= 21 (PubMed, March, 2023)]. [Sephton et al. 2007](https://doi.org/10.1007/s10880-009-9153-z) [_n_= 91, citations= 495 (PubMed, March, 2023)]. +* Critiques: Hoffmann & Witt 2010 [meta-analysis, k=39 studies, n=1,140, citations= 5337 (GS, May 2023)]. Khoury et al. 2013 [meta-analysis, k= 209 studies, n=12,145, citations= 2484 (GS, May 2023)]. Strauss et al., 2014 [meta-analysis, k= 19 studies, n=578, citations= 659 (GS, May 2023)]. [Fumero et al. 2020](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8314302/) [_n_= 91, citations= 30 (PubMed, March 2023)]. Tao et al. 2022 [meta-analysis, k=7 trials, n=502, citations= 2 (GS, May 2023)]. Chayadi et al., 2022 [meta-analysis, k=36, n=1677, citations= 8 (GS, May 2023)]. [Coronado-Montoya 2016](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0153220) [n=357, citations= 21 (GS, March 2023)]. [Britton 2019](https://doi.org/10.1016/j.copsyc.2018.12.011) [_n_= NA, citations= 169 (GS, March 2023)]. [Hsiao et al. 2020](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6435335/) [_n_ = NA, citations= 17(GS, May 2023)]. [Kuyken et al. 2022](https://pubmed.ncbi.nlm.nih.gov/35820992/) [n=8376, citations= 21 (PubMed, March, 2023)]. [Sephton et al. 2007](https://doi.org/10.1002/art.22478) [_n_= 91, citations= 495 (PubMed, March, 2023)]. * Original effect size: prima facie, [d = 0.3](https://jamanetwork.com/journals/jamainternalmedicine/article-abstract/1809754) for anxiety or depression. * Replication effect size: [Fumero et al:](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8314302/) 75% (9/12) of reviews revealed a positive effect of MBIs, comparing pre-post intervention anxiety scores and compared with a control group (SMD) = 0.57 [0.22, 0.89]; for those with negative results, SMD = -0.27 [-0.52, 0.02]. The range of effect size for studies that yielded positive results for mindfulness in comparison to control or other intervention groups was diverse. 20% of studies exhibited a large range of effect size, 50% displayed a moderate range, and 30% demonstrated a small range. Khoury et al.:meta-analysis on pre-post-comparisons revealed a mean effect size on anxiety for ten pre-post studies, Hedge's g = .89 [.71, 1.08], p <.001. [Hoffmann & Witt](http://europepmc.org/article/MED/20350028): average pre-post effect size estimate (Hedges’ g) based on 10 studies was 0.67 [0.47, 0.87], p < .01. Tao et al.: the effects of MBIs (MBSR: Mindfulness-based Stress Reduction, MBCT: Mindfulness-based Cognitive Therapy) on depressive poststroke patients has revealed a positive effect on depression in poststroke depression patients compared with the control group (MBSR: n= 196, Hedges' g = 0.49 [0.42, 0.56], p < 0.01; MBCT: n= 301, Hedges' g = 0.85 [0.71, 1.00], p < 0.01.). [Chayadi et al.:](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0269519) the medium effect sizes of MBIs on reducing anxiety (Hedges’ g = 0.56, S.E = 0.107 [0.35, 0.77], p < 0.01) and depression (Hedges’ g = 0.43, S.E = 0.059 [0.32, 0.55], p < 0.01) among cancer patients with fatigue. [Strauss et al.:](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0096110)the medium effect sizes of MBIs for depressive symptom severity (Hedges g = −0.73  [−0.09, −1.36]) but not for anxiety symptom severity (Hedges g = −0.55 n[0.09 to −1.18]) among cancer patients. Coronado-Montoya: indications of possible reporting bias; 108 (87%) of 124 published trials reported ≥1 positive outcome in the abstract, and 109 (88%) concluded that mindfulness-based therapy was effective, 1.6 times greater than the expected number of positive trials based on effect size d = 0.55 (expected number positive trials = 65.7); of 21 trial registrations, 13 (62%) remained unpublished 30 months post-trial completion. Britton: ES=NA; a number of mindfulness-related processes—including, mindful attention (observing awareness, interoception), mindfulness qualities, mindful emotion regulation (prefrontal control, decentering, exposure, acceptance), and meditation practice—show signs of non-monotonicity, boundary conditions, or negative effects under certain conditions. Hsiao et al.: the effects of Mindfulness-Based Relapse Prevention for Substance Use Disorders were small-to-medium in Study 1 (d = .08 to .48) and were much smaller in Study 2 (d =.03 to .21). Kuyken et al.: no evidence of school-based mindfulness training being superior to teaching-as-usual at 1 year; standardised mean differences (intervention minus control) were: 0.005 [-0.05 to 0.06] for risk for depression; 0.02 [-0.02 to 0.07] for social-emotional-behavioural functioning; and 0.02 [-0.03 to 0.07] for well-being. Sephton et al.: The MBSR treatment significantly reduced basal electrodermal (skin conductance level; SCL) activity (t = 3.298, p = .005) and SCL activity during meditation (t = 4.389, p = .001), consistent with reduced basal sympathetic (SNS) activation among women with fibromyalgia. {{< /spoiler >}} @@ -1202,7 +1202,7 @@ You can find a list of all effects we are working on [here](https://docs.google. {{< spoiler text="Statistics" >}} * Status: replicated * Original paper: [‘Word imagery but not age of acquisition affects episodic memory](https://doi.org/10.3758/BF03198377)’, Coltheart and Winograd 1986; experiment, Experiment 1a: n = 42. [citations=44(GS, November 2022)]​. -* Critiques: [Almond and Morrison 2014](https://doi.org/10.1080/13825585.2013.849653) [n =80, citations = 8(GS, November 2022)]. [Dewhurst et al. 1998](https://www.researchgate.net/profile/Christopher-Barry-6/publication/232448882_Separate_effects_of_word_frequency_and_age_of_acquisition_in_recognition_and_recall/links/5735b45708ae298602e08719/Separate-effects-of-word-frequency-and-age-of-acquisition-in-recognition-and-recall.pdf) [n=128, citations=117(GS, November 2022)]. [Macmillan et al. 2022](https://doi.org/10.3758/s13421-021-01137-6) [n = 44, citations = 9 (GS, November 2022)]. +* Critiques: [Almond and Morrison 2014](https://doi.org/10.1080/13825585.2013.849653) [n =80, citations = 8(GS, November 2022)]. [Dewhurst et al. 1998](https://www.researchgate.net/profile/Christopher-Barry-6/publication/232448882_Separate_effects_of_word_frequency_and_age_of_acquisition_in_recognition_and_recall/links/5735b45708ae298602e08719/Separate-effects-of-word-frequency-and-age-of-acquisition-in-recognition-and-recall.pdf) [n=128, citations=117(GS, November 2022)]. [Macmillan et al. 2021](https://doi.org/10.3758/s13421-021-01137-6) [n = 44, citations = 9 (GS, November 2022)]. * Original effect size: ηp² = .05 [ηp2 calculated from reported F statistic and converted using this [ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)]. * Replication effect size: Almond and Morrison: η² = .111. Dewhurst et al.: _F_ < 1. Macmillan et al.: ηp² = 0.008. {{< /spoiler >}} @@ -1211,16 +1211,16 @@ You can find a list of all effects we are working on [here](https://docs.google. {{< spoiler text="Statistics" >}} * Status: mixed * Original paper: ‘[Separate Effects of Word Frequency and Age of Acquisition in Recognition and Recall](https://www.researchgate.net/profile/Christopher-Barry-6/publication/232448882_Separate_effects_of_word_frequency_and_age_of_acquisition_in_recognition_and_recall/links/5735b45708ae298602e08719/Separate-effects-of-word-frequency-and-age-of-acquisition-in-recognition-and-recall.pdf)’, Dewhurst et al. 1998; experiment, n=80. [citations=117(GS, November 2022)]. -* Critiques: [Macmillan et al. 2022](https://doi.org/10.3758/s13421-021-01137-6) [n = 44, citations = 9 (GS, November 2022)]. +* Critiques: [Macmillan et al. 2021](https://doi.org/10.3758/s13421-021-01137-6) [n = 44, citations = 9 (GS, November 2022)]. * Original effect size: ηp² = 0.15 [np2 calculated from reported F statistic and converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)]. * Replication effect size: Macmillan et al.: ηp² = 0.003. {{< /spoiler >}} -* **Age of acquisition effects - influence on recognition (mixed block)**. Early-acquired items are recalled more accurately than late-acquired items. +* **Age of acquisition effects - influence on recognition (mixed block)**. The age at which a word is acquired does not affect recognition memory (Coltheart and Winograd found no age-of-acquisition effect on recognition or free recall). Later studies found that late-acquired words are recognised more accurately than early-acquired words. {{< spoiler text="Statistics" >}} -* Status: reversed +* Status: not replicated (the original found no age-of-acquisition effect on recognition; later studies found better recognition of late-acquired words) * Original paper: [‘Word imagery but not age of acquisition affects episodic memory](https://doi.org/10.3758/BF03198377)’, Coltheart and Winograd 1986; experiment, Experiment 2: n = 102. [citations=44(GS, November 2022)]​. -* Critiques: [Dewhurst et al. 1998](https://www.researchgate.net/profile/Christopher-Barry-6/publication/232448882_Separate_effects_of_word_frequency_and_age_of_acquisition_in_recognition_and_recall/links/5735b45708ae298602e08719/Separate-effects-of-word-frequency-and-age-of-acquisition-in-recognition-and-recall.pdf) [Experiment 1: n=30, Experiment 2: n = 30; citations=117(GS, November 2022)]. [Macmillan et al. 2022](https://doi.org/10.3758/s13421-021-01137-6) [n = 44, citations = 9 (GS, November 2022)]. +* Critiques: [Dewhurst et al. 1998](https://www.researchgate.net/profile/Christopher-Barry-6/publication/232448882_Separate_effects_of_word_frequency_and_age_of_acquisition_in_recognition_and_recall/links/5735b45708ae298602e08719/Separate-effects-of-word-frequency-and-age-of-acquisition-in-recognition-and-recall.pdf) [Experiment 1: n=30, Experiment 2: n = 30; citations=117(GS, November 2022)]. [Macmillan et al. 2021](https://doi.org/10.3758/s13421-021-01137-6) [n = 44, citations = 9 (GS, November 2022)]. * Original effect size: ηp² = .03 [ηp2 calculated from reported F statistic and converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)]. * Replication effect size: Dewhurst et al.: Experiment 1: Hits: ηp² = 0.42 [ηp2 calculated from reported F statistic and converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)], False alarms: _F_ < 1, d’: ηp² = 0.31 [ηp2 calculated from reported F statistic and converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)]; Experiment 2: Hits: _d_ = 0.74 [_d_ calculated from reported t statistic and converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)], False alarms: _d_= 0.57 [d_ _calculated from reported t statistic and converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)], d’: _d_ = 0.09 [_d_ calculated from reported t statistic and converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)]. Macmillan et al.: Hits: _d_ = 0.023, False alarms: _d_= 0.56, d’: _d_ = 0.44, C = 0.35, da = 0.65, slope = 0.25. {{< /spoiler >}} @@ -1545,11 +1545,11 @@ You can find a list of all effects we are working on [here](https://docs.google. * **Depressive realism effect**. Increased predictive accuracy or decreased cognitive bias among the clinically depressed. {{< spoiler text="Statistics" >}} -* Status: reversed +* Status: mixed (much smaller effect in the original direction) * Original paper: '[Judgment of contingency in depressed and nondepressed students: sadder but wiser?](https://www.ncbi.nlm.nih.gov/pubmed/528910)', Alloy and Abramson (1979): 4 experiments with Study 1: n1 = 48, n2 = 48, Study 2: n1 = 32, n2 = 32; Study 3: n1 = 32, n2 = 32; Study 4: n1 = 32, n2 = 32 [citations = 2855 (GS, June 2022)]. * Critiques: [Moore and Fresco 2012 ](https://www.ncbi.nlm.nih.gov/pubmed/22717337)[meta analysis, n = 7305, citations = 311 (GS, June 2022)] * Original effect size: not reported. [d= -0.32 calculated for bias about 'contingency', how much the outcome actually depends on what you do] -* Replication effect size: Moore and Fresco 2012: d = -0.07. +* Replication effect size: Moore and Fresco 2012: d = -0.07, a small overall depressive realism effect (negative values indicate greater accuracy among depressed participants); both groups showed a positive bias, which was larger among nondepressed (d = 0.29) than depressed participants (d = 0.14). {{< /spoiler >}} * **Hungry judge effect**, of massively reduced acquittals just before lunch. Case order isn’t independent of acquittal probability (“unrepresented prisoners usually go last and are less likely to be granted parole”); favourable cases may take predictably longer and so are pushed until after recess; effect size is implausible on priors; explanation involved ego depletion. @@ -1565,7 +1565,7 @@ You can find a list of all effects we are working on [here](https://docs.google. {{< spoiler text="Statistics" >}} * Status: NA * Original paper: ‘[Frames of Mind: The Theory of Multiple Intelligences](https://www.goodreads.com/book/show/294035.Frames_of_Mind)’, Gardener 1983; book/theoretical work, n=NA. [citation=45591(GS, March 2022)]​. -* Critiques: [Shearer and Karanian 2017](https://doi.org/10.1016/j.tine.2017.02.002) [n = 172 neuroscience reports, citations = 92 (GS, February 2023)]. [Sternberg 1994](https://doi.org/10.1177/016146819409500411) [n = NA, citations = 103 (GS, February 2023)].[ Tirri and Nokelainen 2008](https://doi.org/10.1207/s15326985ep4104_1) [n = 410, citations = 92 (GS, February 2023)]. [Visser et al. 2006](https://doi.org/10.1016/j.intell.2006.02.004) [n = 200, citations = 379 (GS, February 2023)]. [Waterhouse 2006](https://doi.org/10.1207/s15326985ep4104_1) [n = NA, citations = 103 (GS, February 2023)]. +* Critiques: [Shearer and Karanian 2017](https://doi.org/10.1016/j.tine.2017.02.002) [n = 172 neuroscience reports, citations = 92 (GS, February 2023)]. [Sternberg 1994](https://doi.org/10.1177/016146819409500411) [n = NA, citations = 103 (GS, February 2023)]. Tirri and Nokelainen 2008 [n = 410, citations = 92 (GS, February 2023)]. [Visser et al. 2006](https://doi.org/10.1016/j.intell.2006.02.004) [n = 200, citations = 379 (GS, February 2023)]. [Waterhouse 2006](https://doi.org/10.1207/s15326985ep4104_1) [n = NA, citations = 103 (GS, February 2023)]. * Original effect size: No empirical data collected [Allix, [2000](https://doi.org/10.1177/000494410004400306); Lubinski & Benbow, [1995](https://doi.org/10.1037/004016); Sternberg 1994; Waterhouse, [2006](https://doi.org/10.1207/s15326985ep4104_1); Gardner acknowledged lack of empirical data, [2004](https://doi.org/10.1111/j.1467-9620.2004.00329.x) (p. 214)]. * Replication effect size: Shearer and Karanian: NA; descriptive statistics, neural patterns consistent with Gardner’s hypothesis. Tirri and Nokelainen: NA; Confirmatory Factor Analysis; supports existence of logical-mathematical and spatial intelligences. Visser et al.: NA; Factor Analysis, modest support for Gardner. Strong loadings on g factor. {{< /spoiler >}} @@ -1610,7 +1610,7 @@ You can find a list of all effects we are working on [here](https://docs.google. {{< spoiler text="Statistics" >}} * Status: mixed * Original paper: ‘[The effects of bilingualism on theory of mind development](https://www.cambridge.org/core/journals/bilingualism-language-and-cognition/article/abs/effects-of-bilingualism-on-theory-of-mind-development/5D9E2C458663A60A255268A839CA8D31)’, Goetz 2003; experiment, English monolinguals: n = 32, Mandarin monolinguals: n = 32, English-Mandarin bilinguals: n = 40. [citations = 486 (GS, January 2023)]. -* Critiques: [Dahlgren et al. 2017](https://doi.org/10.1080/00221325.2017.1361376) [Monolinguals: n = 14, bilinguals: n = 14, citations = 23 (GS, January 2023)]. [Diaz and Farrar 2018](https://doi.org/10.1177/0142723717752741) [Monolinguals: n = 33, Bilinguals: n = 32, citations = 23(GS, January 2023)]. [Farhadian et al. 2010](https://doi.org/10.1080/00221325.2017.1361376) [Monolinguals: n = 65, bilinguals: n = 98, citations = 61 (GS, January 2023)]. [Gordon 2016](https://www.cambridge.org/core/journals/journal-of-child-language/article/high-proficiency-across-two-languages-is-related-to-better-mental-state-reasoning-for-bilingual-children/F618A28BA95AB3FAAF200BE9CDB505B1) [Monolinguals: n = 26, bilinguals n = 26, citations = 24(GS, January, 2023)]. +* Critiques: [Dahlgren et al. 2017](https://doi.org/10.1080/00221325.2017.1361376) [Monolinguals: n = 14, bilinguals: n = 14, citations = 23 (GS, January 2023)]. [Diaz and Farrar 2018](https://doi.org/10.1177/0142723717752741) [Monolinguals: n = 33, Bilinguals: n = 32, citations = 23(GS, January 2023)]. Farhadian et al. 2010 [Monolinguals: n = 65, bilinguals: n = 98, citations = 61 (GS, January 2023)]. [Gordon 2016](https://www.cambridge.org/core/journals/journal-of-child-language/article/high-proficiency-across-two-languages-is-related-to-better-mental-state-reasoning-for-bilingual-children/F618A28BA95AB3FAAF200BE9CDB505B1) [Monolinguals: n = 26, bilinguals n = 26, citations = 24(GS, January, 2023)]. * Original effect size: _ηp2 _= 0.06 [_ηp2 _calculated from reported F statistic and converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)]. * Replication effect size: Dahlgren et al.: not reported. Diaz and Farrar: _ηp2 _= .063. Farhadian et al.: _d_ = 0.40 [_d _calculated from mean differences and standard deviation and converted using this [conversion](https://lbecker.uccs.edu/)]. Gordon: _d_ = 0.123. {{< /spoiler >}} @@ -1698,7 +1698,7 @@ You can find a list of all effects we are working on [here](https://docs.google. {{< spoiler text="Statistics" >}} * Status: mixed * Original paper: ‘[Semantic priming and retrieval from lexical memory: Roles of inhibitionless spreading activation and limited-capacity attention](https://doi.org/10.1037/0096-3445.106.3.226)’, Neely 1977; speeded word–nonword classification task, n = 120. [citation = 3963 (PSYCNET.APA, January 2023)]. -* Critiques: [Aczel et al. 2021](https://doi.org/10.1098/rsos.191353) [Kan et al. 2013 replication, n=103, 70 and 38 participants for Experiments 1, 2 and 3, citations=4(GS, Feb 2022)]. [Gratton et al. 1992](https://doi.org/10.1037/0096-3445.121.4.480) [n1=6, n2=5, n3a=6, n3b= 8, citation = 2004 (GS, April 2023)].[ Gyurkovics et al. 2020 ](https://doi.org/10.1016/j.cognition.2013.09.001)[n=489 over four tasks, citations=3(GS, April 2023)]. [Kan et al. 2013](https://doi.org/10.1016/j.cognition.2013.09.001) [n = 41 in Experiment 1; n = 28 in Experiment 2; n = 15 in Experiment 3, citation=81(GS, February 2022)] +* Critiques: [Aczel et al. 2021](https://doi.org/10.1098/rsos.191353) [Kan et al. 2013 replication, n=103, 70 and 38 participants for Experiments 1, 2 and 3, citations=4(GS, Feb 2022)]. [Gratton et al. 1992](https://doi.org/10.1037/0096-3445.121.4.480) [n1=6, n2=5, n3a=6, n3b= 8, citation = 2004 (GS, April 2023)].[ Gyurkovics et al. 2020 ](https://doi.org/10.3758/s13414-020-02021-2)[n=489 over four tasks, citations=3(GS, April 2023)]. [Kan et al. 2013](https://doi.org/10.1016/j.cognition.2013.09.001) [n = 41 in Experiment 1; n = 28 in Experiment 2; n = 15 in Experiment 3, citation=81(GS, February 2022)] * Original effect size: Greatest facilitation in Non-shift-Expected-Related word X target condition and greatest inhibition effects in Shift-Unexpected-Unrelated and Nonshift-Unexpected-Unrelated conditions; _η2 _= 0.689 (calculated from the reported _F_(4, 84) = 46.85, using this conversion). * Replication effect size: Aczel et al.: The congruency sequence effect for the RT analysis was inconclusive in all three experiments, _ηp2 _=0.00 to 0.02 (calculated from the reported _F _statistic), and for the accuracy in two out of three experiments, _ηp2 _=0.00 to 0.04 (calculated from the reported _F _statistic). Gratton et al.: compatible vs. incompatible trials, Reaction time _ηp2 _=0.88 to 0.94 (calculated from the reported _F _statistic), Error rate - _ηp2 _=0.59 to 0.98 (calculated from the reported _F _statistic). Gyurkovics et al.: _ηp2_=.40-.96. Kan et al.: congruent vs. incongruent trials, Stroop accuracy _ηp2 _=0.14 to 0.57 (calculated from the reported _F_ statistic), Stroop reaction time _ηp2 _=0.19 to 0.46 (calculated from the reported _F_ statistic). {{< /spoiler >}} @@ -1813,11 +1813,11 @@ You can find a list of all effects we are working on [here](https://docs.google. * **Bottomless soup bowl**. Visual cues related to portion size increase intake volume of soup. {{< spoiler text="Statistics" >}} -* Status: not replicated -* Original paper: [‘Bottomless bowls: why visual cues of portion size may influence intake](https://pubmed.ncbi.nlm.nih.gov/15761167/)’, Warsink et al. 2005; experiment, n = 54. [citation=767(GS, December 2021)]​. -* Critiques: [Andrew 2019](https://statmodeling.stat.columbia.edu/2019/08/20/did-that-bottomless-soup-bowl-experiment-ever-happen/) [blog post]. [Heathers 2018](https://jamesheathers.medium.com/sprite-case-study-5-sunset-for-souper-man-ee898b6af9f5) [blog post]. +* Status: replicated (about half the original effect size) +* Original paper: [‘Bottomless bowls: why visual cues of portion size may influence intake](https://pubmed.ncbi.nlm.nih.gov/15761167/)’, Wansink et al. 2005; experiment, n = 54. [citation=767(GS, December 2021)]​. +* Critiques: [Lopez et al. 2024](https://doi.org/10.1037/xge0001503) [preregistered direct replication, n = 464]. [Andrew 2019](https://statmodeling.stat.columbia.edu/2019/08/20/did-that-bottomless-soup-bowl-experiment-ever-happen/) [blog post]. [Heathers 2018](https://jamesheathers.medium.com/sprite-case-study-5-sunset-for-souper-man-ee898b6af9f5) [blog post]. * Original effect size: _ηp2_ = 0.147 [not reported but [calculated](https://effect-size-calculator.herokuapp.com/) using this conversion]. -* Replication effect size: not reported. +* Replication effect size: Lopez et al.: _d_ = 0.45; participants eating from self-refilling bowls ate 32% more soup (original: _d_ = 0.84, 73% more). As in the original, the groups did not differ in satiation, but some secondary findings did not replicate; for example, the control group estimated that it had eaten more soup than the self-refilling group. {{< /spoiler >}} * **Simon effect**. Faster responses are observed when the stimulus and response are on the same side than when the stimulus and response are on opposite sides. @@ -1869,7 +1869,7 @@ You can find a list of all effects we are working on [here](https://docs.google. {{< spoiler text="Statistics" >}} * Status: replicated * Original paper: ‘[Memory: A contribution to experimental psychology](https://archive.org/details/memorycontributi00ebbiuoft/page/80/mode/2up?view=theater)’, Ebbinghaus 1964; series of single-case studies, n=1. [citations=6103 (GS, September, 2022)]. -* Critiques: [Cepeda et al. 2006](https://doi.org/10.1037/0033-2909.132.3.354), meta-analysis [n= 184 articles, citations=1894 (GS, September 2022)]. [Janiszewski et al. 2003](https://doi.org/10.58052/neon0dkjs), meta-analysis [n= 97 verbal learning studies, citations= 373 (GS, September 2022)]. +* Critiques: [Cepeda et al. 2006](https://doi.org/10.1037/0033-2909.132.3.354), meta-analysis [n= 184 articles, citations=1894 (GS, September 2022)]. [Janiszewski et al. 2003](https://doi.org/10.1086/374692), meta-analysis [n= 97 verbal learning studies, citations= 373 (GS, September 2022)]. * Original effect size: N/A. * Replication effect size: Cepeda et al.: Cohen’s _d_ for the difference in the accuracy between massed and spaced learning trials in verbal recall tasks= 0.567 (calculated). Janiszewski et al.: _ηp2_= 0.093 (calculated from the reported _F_(1, 478)=49.23,_p_<.01 using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)) for a linear relationship between the number of lags between learning events and the accuracy of recall; _ηp2_= 0.051 for the log relationship (calculated from the reported _F_(1, 478)=25.69, _p_<.01 using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)). {{< /spoiler >}} @@ -1928,7 +1928,7 @@ You can find a list of all effects we are working on [here](https://docs.google. * Replication effect size: Conway et al.: Detection rate = 43%. Röer and Cowan: Detection rate = 29%. Wood and Cowan: Detection rate = 35%. {{< /spoiler >}} -* **Mental simulation - mismatch advantage: object colour**. Readers verify pictures more quickly when they match rather than mismatch the object colour from the preceding sentence. +* **Mental simulation - mismatch advantage: object colour**. Connell (2007) found that readers verified pictures more quickly when they mismatched rather than matched the object colour implied by the preceding sentence. Later replications found the opposite, a match advantage. {{< spoiler text="Statistics" >}} * Status: reversed * Original paper: ‘[Representing object colour in language comprehension](https://www.lancaster.ac.uk/staff/connelll/papers/Connell-2007-Cognition.pdf)’, Connell (2007); experimental design, n = 44. [citation=155(GS, November 2022)]​. @@ -1948,11 +1948,11 @@ You can find a list of all effects we are working on [here](https://docs.google. * **Mental simulation - match advantage object distance**. Readers verify small pictures more quickly when they are far from the protagonist, in contrast to big pictures, while big pictures are verified more quickly when closer to the protagonist, as opposed to smaller pictures. {{< spoiler text="Statistics" >}} -* Status: reversed +* Status: NA (the only cited follow-up used a different task and does not test the original effect directly) * Original paper: ‘[Language comprehenders represent object distance both visually and auditorily](https://www.cambridge.org/core/services/aop-cambridge-core/content/view/9320F4D38E43C3EABFAFAAC50259FF40/S1866980800000880a.pdf/div-class-title-language-comprehenders-represent-object-distance-both-visually-and-auditorily-div.pdf)’, Winter and Bergen 2012; experimental design, Experiment 1: n =22. [citation=54(GS, November 2022)]​. -* Critiques: [Vukovic and Williams 2014](https://doi.org/10.1016/j.actpsy.2013.11.002) [n=20, citations=36(GS, November 2022)]. ​ +* Critiques: [Vukovic and Williams 2014](https://doi.org/10.1016/j.actpsy.2013.11.002) [conceptual test with Dutch–English bilinguals hearing interlingual homophones, n=20, citations=36(GS, November 2022)]. ​ * Original effect size: Experiment 1: _ηp2_= 0.230. -* Replication effect size: Vukovic and Williams: _ηp2_=0.44. +* Replication effect size: Vukovic and Williams: _ηp2_=0.44. Participants were slower to reject pictures whose size matched the distance implied by a task-irrelevant meaning of the homophone. The authors read this interference as evidence of automatic perceptual simulation; it is not a reversal of the original match advantage. {{< /spoiler >}} * **Mental simulation - match advantage object number**. Verification response was faster for concept-object match when there was numerical congruence (compared with incongruence) between the number word and quantity. @@ -2185,8 +2185,8 @@ You can find a list of all effects we are working on [here](https://docs.google. {{< spoiler text="Statistics" >}} * Status: replicated * Original paper: ‘[Organizational deficits in dyslexia: Possible frontal lobe dysfunction](https://doi.org/10.1080/87565649009540453)’, Levin 1990; experiment, children with dyslexia: n = 20, dyslexic children: n = 20. [citation = 97(GS, November 2022)]. -* Critiques: [Frith et al. 1994](https://www.researchgate.net/file.PostFileLoader.html?id=52dc2ab7d5a3f20b1c8b4657&assetKey=AS:272424631767040@1441962501465) [NT: n = 19, LD: n = 19, citations = 80(GS, November 2022)]. [Hatcher et al. 2002](https://doi.org/10.1348/000709902158801) [NT: n = 50, AWD: n = 23, citations = 426(GS, November 2022)]. [Marzocchi et al. 2008](https://doi.org/10.1111/j.1469-7610.2007.01859.x) [neurotypical children: n = 30, dyslexic children: n = 22, ADHD children: n = 35, citations = 202(GS, November 2022)]. [Menghini et al. 2010](https://doi.org/10.1016/j.neuropsychologia.2009.11.003) [neurotypical children and adolescents: n = 65, dyslexic children and adolescent: n = 60, citations = 330(GS, November 2022)]. [Moura et al. 2014 ](https://doi.org/10.1080/13854046.2014.964326)[neurotypical children: n = 50, dyslexic children: n = 50, citations = 104(GS, November 2022)]. [Plaza et al. 2002 ](https://europepmc.org/article/med/12030497)[neurotypical age-matched children: n = 26, neurotypical reading-age matched children: n = 26, dyslexic children: n = 26, citations = 81 (GS, November 2022)]. [Reiter et al. 2005 ](https://doi.org/10.1002/dys.289)[neurotypical children: n = 42, dyslexic children: n = 42, citations = 485 (GS, November 2022)]. [Shareef et al. 2019](https://www.cambridge.org/core/journals/applied-psycholinguistics/article/abs/verbal-fluency-in-relation-to-reading-ability-in-students-with-and-without-dyslexia/E50F5FA55028070CF24F4812DDB15114) AWD: n = 16, NT: n = 26, citations = 6 (GS, November [2022)]. Smith-Spark et al. 2017 [AWD: n = 28, NT: n = 28, citations = 36 (GS, November 2022)]. Snowling et al. 1997(https://doi.org/10.1080/02699931.2018.1468732) NT: n = 19, AWD: n = 14, citations = 262 (GS, November [2022)]. [Varvava et al. 2014](https://www.frontiersin.org/articles/10.3389/fnhum.2014.00120/full) AWD: n = 60, NT: n = 65, citations = 177 (GS, November 2022). [Wilson and Lesaux 2001](https://doi.org/10.1080/02699931.2018.1468732), NT: n = 31, AWD: n = 28, citations = 265 [GS, November (2022)]. -* Critiques: [Frith et al. 1994](https://www.researchgate.net/file.PostFileLoader.html?id=52dc2ab7d5a3f20b1c8b4657&assetKey=AS:272424631767040@1441962501465) [NT: n = 19, LD: n = 19, citations = 80(GS, November 2022)] [Hatcher et al. 2002](https://doi.org/10.1348/000709902158801) [NT: n = 50, AWD: n = 23, citations = 426(GS, November 2022)]. [Marzocchi et al. 2008](https://doi.org/10.1111/j.1469-7610.2007.01859.x) [neurotypical children: n = 30, dyslexic children: n = 22, ADHD children: n = 35, citations = 202(GS, November 2022)]. [Menghini et al. 2010](https://doi.org/10.1016/j.neuropsychologia.2009.11.003) [neurotypical children and adolescents: n = 65, dyslexic children and adolescent: n = 60, citations = 330(GS, November 2022)]. [Moura et al. 2014 ](https://doi.org/10.1080/13854046.2014.964326)[neurotypical children: n = 50, dyslexic children: n = 50, citations = 104(GS, November 2022)]. [Plaza et al. 2002 ](https://europepmc.org/article/med/12030497) [neurotypical age-matched children: n = 26, neurotypical reading-age matched children: n = 26, dyslexic children: n = 26, citations = 81 (GS, November 2022)]. [Reiter et al. 2005 ](https://doi.org/10.1002/dys.289)[neurotypical children: n = 42, dyslexic children: n = 42, citations = 485 (GS, November 2022)]. [Shareef et al. 2019](https://www.cambridge.org/core/journals/applied-psycholinguistics/article/abs/verbal-fluency-in-relation-to-reading-ability-in-students-with-and-without-dyslexia/E50F5FA55028070CF24F4812DDB15114) AWD: n = 16, NT: n = 26, citations = 6 [GS, November 2022]. Smith-Spark et al. 2017 [AWD: n = 28, NT: n = 28, citations = 36 (GS, November 2022)]. [Snowling et al. 1997](https://doi.org/10.1080/02699931.2018.1468732) NT: n = 19, AWD: n = 14, citations = 262 (GS, November [2022)]. [Varvava et al. 2014](https://www.frontiersin.org/articles/10.3389/fnhum.2014.00120/full) AWD: n = 60, NT: n = 65, citations = 177 (GS, November 2022). [Wilson and Lesaux 2001](https://doi.org/10.1080/02699931.2018.1468732), NT: n = 31, AWD: n = 28, citations = 265 [GS, November 2022]. +* Critiques: [Frith et al. 1994](https://www.researchgate.net/file.PostFileLoader.html?id=52dc2ab7d5a3f20b1c8b4657&assetKey=AS:272424631767040@1441962501465) [NT: n = 19, LD: n = 19, citations = 80(GS, November 2022)]. [Hatcher et al. 2002](https://doi.org/10.1348/000709902158801) [NT: n = 50, AWD: n = 23, citations = 426(GS, November 2022)]. [Marzocchi et al. 2008](https://doi.org/10.1111/j.1469-7610.2007.01859.x) [neurotypical children: n = 30, dyslexic children: n = 22, ADHD children: n = 35, citations = 202(GS, November 2022)]. [Menghini et al. 2010](https://doi.org/10.1016/j.neuropsychologia.2009.11.003) [neurotypical children and adolescents: n = 65, dyslexic children and adolescent: n = 60, citations = 330(GS, November 2022)]. [Moura et al. 2014 ](https://doi.org/10.1080/13854046.2014.964326)[neurotypical children: n = 50, dyslexic children: n = 50, citations = 104(GS, November 2022)]. [Plaza et al. 2002 ](https://europepmc.org/article/med/12030497)[neurotypical age-matched children: n = 26, neurotypical reading-age matched children: n = 26, dyslexic children: n = 26, citations = 81 (GS, November 2022)]. [Reiter et al. 2005 ](https://doi.org/10.1002/dys.289)[neurotypical children: n = 42, dyslexic children: n = 42, citations = 485 (GS, November 2022)]. [Shareef et al. 2019](https://www.cambridge.org/core/journals/applied-psycholinguistics/article/abs/verbal-fluency-in-relation-to-reading-ability-in-students-with-and-without-dyslexia/E50F5FA55028070CF24F4812DDB15114) AWD: n = 16, NT: n = 26, citations = 6 (GS, November [2022)]. Smith-Spark et al. 2017 [AWD: n = 28, NT: n = 28, citations = 36 (GS, November 2022)]. Snowling et al. 1997(https://doi.org/10.1111/1467-9817.00018) NT: n = 19, AWD: n = 14, citations = 262 (GS, November [2022)]. [Varvava et al. 2014](https://www.frontiersin.org/articles/10.3389/fnhum.2014.00120/full) AWD: n = 60, NT: n = 65, citations = 177 (GS, November 2022). [Wilson and Lesaux 2001](https://doi.org/10.1177/002221940103400501), NT: n = 31, AWD: n = 28, citations = 265 [GS, November (2022)]. +* Critiques: [Frith et al. 1994](https://www.researchgate.net/file.PostFileLoader.html?id=52dc2ab7d5a3f20b1c8b4657&assetKey=AS:272424631767040@1441962501465) [NT: n = 19, LD: n = 19, citations = 80(GS, November 2022)] [Hatcher et al. 2002](https://doi.org/10.1348/000709902158801) [NT: n = 50, AWD: n = 23, citations = 426(GS, November 2022)]. [Marzocchi et al. 2008](https://doi.org/10.1111/j.1469-7610.2007.01859.x) [neurotypical children: n = 30, dyslexic children: n = 22, ADHD children: n = 35, citations = 202(GS, November 2022)]. [Menghini et al. 2010](https://doi.org/10.1016/j.neuropsychologia.2009.11.003) [neurotypical children and adolescents: n = 65, dyslexic children and adolescent: n = 60, citations = 330(GS, November 2022)]. [Moura et al. 2014 ](https://doi.org/10.1080/13854046.2014.964326)[neurotypical children: n = 50, dyslexic children: n = 50, citations = 104(GS, November 2022)]. [Plaza et al. 2002 ](https://europepmc.org/article/med/12030497) [neurotypical age-matched children: n = 26, neurotypical reading-age matched children: n = 26, dyslexic children: n = 26, citations = 81 (GS, November 2022)]. [Reiter et al. 2005 ](https://doi.org/10.1002/dys.289)[neurotypical children: n = 42, dyslexic children: n = 42, citations = 485 (GS, November 2022)]. [Shareef et al. 2019](https://www.cambridge.org/core/journals/applied-psycholinguistics/article/abs/verbal-fluency-in-relation-to-reading-ability-in-students-with-and-without-dyslexia/E50F5FA55028070CF24F4812DDB15114) AWD: n = 16, NT: n = 26, citations = 6 [GS, November 2022]. Smith-Spark et al. 2017 [AWD: n = 28, NT: n = 28, citations = 36 (GS, November 2022)]. [Snowling et al. 1997](https://doi.org/10.1111/1467-9817.00018) NT: n = 19, AWD: n = 14, citations = 262 (GS, November [2022)]. [Varvava et al. 2014](https://www.frontiersin.org/articles/10.3389/fnhum.2014.00120/full) AWD: n = 60, NT: n = 65, citations = 177 (GS, November 2022). [Wilson and Lesaux 2001](https://doi.org/10.1177/002221940103400501), NT: n = 31, AWD: n = 28, citations = 265 [GS, November 2022]. * Original effect size: _d_ =-0.32 [[calculated using this conversion from t to Cohen’s d](https://haiyangjin.github.io/2020/05/eta2d/#conversion-between-cohens-d-and-partial-eta-squared)]. * Replication effect size: Frith et al.: _r_ = 0.50 [[calculated using the conversion from Mann Whitney U test to r](https://datatab.net/tutorial/mann-whitney-u-test)]. Hatcher et al.: Effect size = 0.82; Marzocchi et al. (2008): _ηp2_= .20/ d = 0.25[[calculated using this conversion](https://haiyangjin.github.io/2020/05/eta2d/#conversion-between-cohens-d-and-partial-eta-squared)]. Menghini et al.: Effect size (%)= 7.3. Moura et al.: _ηp2_ = .134/ _d_ = 0.15 [[calculated using this conversion](https://haiyangjin.github.io/2020/05/eta2d/#conversion-between-cohens-d-and-partial-eta-squared)]. Plaza et al.: age-matched neurotypical children vs dyslexic children: _ηp2_= 0.33 [[calculated using this conversion](https://haiyangjin.github.io/2020/05/eta2d/#conversion-between-cohens-d-and-partial-eta-squared)]/ _d_ = 0.48 [[calculated using this conversion](https://haiyangjin.github.io/2020/05/eta2d/#conversion-between-cohens-d-and-partial-eta-squared)], reading-aged matched neurotypical children vs. dyslexic children: _ηp2_= 0.08 [[calculated using this conversion](https://haiyangjin.github.io/2020/05/eta2d/#conversion-between-cohens-d-and-partial-eta-squared) _ηp2_]/ _d_ = 0.09 [[calculated using this conversion](https://haiyangjin.github.io/2020/05/eta2d/#conversion-between-cohens-d-and-partial-eta-squared)]. Reiter et al.: NA = 0.489. Shareef et al.: _d_ = 1.02. Smith-Spark et al.: _ß_ = .388. Snowling et al.: Effect size (% of variance explained) = 9.56. Varvava et al.: φ = 0.26 [[calculated using the conversion from Chi square to Phi coefficient]](http://www.people.vcu.edu/~pdattalo/702SuppRead/MeasAssoc/NominalAssoc.html#:~:text=Computationally%2C%20phi%20is%20the%20square,(X2%2Fn).). Wilson and Lesaux: _d_ = 0.57. {{< /spoiler >}} @@ -2195,7 +2195,7 @@ You can find a list of all effects we are working on [here](https://docs.google. {{< spoiler text="Statistics" >}} * Status: mixed * Original paper: ‘[Organizational deficits in dyslexia: Possible frontal lobe dysfunction](https://doi.org/10.1080/87565649009540453)’, Levin 1990; experiment, children with dyslexia: n = 20, dyslexic children: n = 20. [citation = 97(GS, November 2022)]. -* Critiques: [Frith et al. 1994](https://www.researchgate.net/file.PostFileLoader.html?id=52dc2ab7d5a3f20b1c8b4657&assetKey=AS:272424631767040@1441962501465) [NT: n = 19, LD: n = 19, citations = 80(GS, November 2022)]. [Hall and McGregor 2017](https://pubs.asha.org/doi/10.1044/2016_JSLHR-L-15-0440?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub%20%200pubmed) [NT: n = 132, LD: n = 53, citations = 25(GS, November 2022)]. [Hatcher et al. 2002](https://doi.org/10.1348/000709902158801) [NT: n = 50, AWD: n = 23, citations = 426(GS, November 2022)]. [Kinsbourne et al. 1991 ](https://doi.org/10.1111/j.1469-8749.1991.tb14960.x)[[AWD: ](https://d1wqtxts1xzle7.cloudfront.net/35428191/Kinsbourne__Rufo__Gamzu__Palmer___Roland__1991._Neuropsychological_deficits_in_adults_with_dyslexia-with-cover-page-v2.pdf?Expires=1668215635&Signature=GVGd-9RsJxo5kfqpoKpS20D719tmqqS~nxqNw07S83farr9J0Gc8tf9xkTbzAOf8~RKnVI8dXwKth-cQVX9bEKlcEmXWHIwpYUQ9rKXJ5pbtohC9I5GqZrdM25SvraXIer29rY5Vh8bnByCMiY~UUc0L6bHuBJ6UZoa8WyRklEmnkEOD-FyShfvwbQeHrEBeSA-x~nz1Kec75MRZahayDA2uQRKujcNk~0rBuzaELeW9LZ76pyvOW8FiUWyW7FrFDYN0o~fD3zPFiGHyBarmvL1Le4SMLQ6PF5eYiMLTD0iw~d07GUflQL~KNm~91~UzRO~YuVKufX42khZrXGKI4A__&Key-Pair-Id=APKAJLOHF5GGSLRBV4ZA)n= 23, NT: n =21; Recovered AWD: n = 11, citation=144(GS, November 2022)]. [Marzocchi et al. 2008](https://doi.org/10.1111/j.1469-7610.2007.01859.x) [neurotypical children: n = 30, dyslexic children: n = 22, ADHD children: n = 35, citations = 202(GS, November 2022)]. [Menghini et al. 2010](https://doi.org/10.1016/j.neuropsychologia.2009.11.003) [neurotypical children and adolescents: n = 65, dyslexic children and adolescent: n = 60, citations = 330(GS, November 2022)]. [Moura et al. 2014 ](https://doi.org/10.1080/13854046.2014.964326)[neurotypical children: n = 50, dyslexic children: n = 50, citations = 104(GS, November 2022)]. [Plaza et al. 2002 ](https://europepmc.org/article/med/12030497)[neurotypical age-matched children: n = 26, neurotypical reading-age matched children: n = 26, dyslexic children: n = 26, citations = 81 (GS, November 2022)]. [Reid et al. 2007](https://doi.org/10.1002/dys.321) [neurotypical students: n = 15, AWD: n = 15, citations = 134 (GS, November 2022)]. [Reiter et al. 2005 ](https://doi.org/10.1002/dys.289)[neurotypical children: n = 42, dyslexic children: n = 42, citations = 485 (GS, November 2022)]. [Shareef et al. 2019](https://www.cambridge.org/core/journals/applied-psycholinguistics/article/abs/verbal-fluency-in-relation-to-reading-ability-in-students-with-and-without-dyslexia/E50F5FA55028070CF24F4812DDB15114) [AWD: n = 16, NT: n = 26, citations = 6 (GS, November [2022)]. Smith-Spark et al. 2017 [AWD: n = 28, NT: n = 28, citations = 36 (GS, November 2022)]. Snowling et al. 1997](https://doi.org/10.1080/02699931.2018.1468732) [NT: n = 19, AWD: n = 14, citations = 262 (GS, November 2022)]. [Varvava et al. 2014](https://www.frontiersin.org/articles/10.3389/fnhum.2014.00120/full) [AWD: n = 60, NT: n = 65, citations = 177 (GS, November 2022)]. +* Critiques: [Frith et al. 1994](https://www.researchgate.net/file.PostFileLoader.html?id=52dc2ab7d5a3f20b1c8b4657&assetKey=AS:272424631767040@1441962501465) [NT: n = 19, LD: n = 19, citations = 80(GS, November 2022)]. [Hall and McGregor 2017](https://pubs.asha.org/doi/10.1044/2016_JSLHR-L-15-0440?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub%20%200pubmed) [NT: n = 132, LD: n = 53, citations = 25(GS, November 2022)]. [Hatcher et al. 2002](https://doi.org/10.1348/000709902158801) [NT: n = 50, AWD: n = 23, citations = 426(GS, November 2022)]. [Kinsbourne et al. 1991 ](https://doi.org/10.1111/j.1469-8749.1991.tb14960.x)[[AWD: ](https://d1wqtxts1xzle7.cloudfront.net/35428191/Kinsbourne__Rufo__Gamzu__Palmer___Roland__1991._Neuropsychological_deficits_in_adults_with_dyslexia-with-cover-page-v2.pdf?Expires=1668215635&Signature=GVGd-9RsJxo5kfqpoKpS20D719tmqqS~nxqNw07S83farr9J0Gc8tf9xkTbzAOf8~RKnVI8dXwKth-cQVX9bEKlcEmXWHIwpYUQ9rKXJ5pbtohC9I5GqZrdM25SvraXIer29rY5Vh8bnByCMiY~UUc0L6bHuBJ6UZoa8WyRklEmnkEOD-FyShfvwbQeHrEBeSA-x~nz1Kec75MRZahayDA2uQRKujcNk~0rBuzaELeW9LZ76pyvOW8FiUWyW7FrFDYN0o~fD3zPFiGHyBarmvL1Le4SMLQ6PF5eYiMLTD0iw~d07GUflQL~KNm~91~UzRO~YuVKufX42khZrXGKI4A__&Key-Pair-Id=APKAJLOHF5GGSLRBV4ZA)n= 23, NT: n =21; Recovered AWD: n = 11, citation=144(GS, November 2022)]. [Marzocchi et al. 2008](https://doi.org/10.1111/j.1469-7610.2007.01859.x) [neurotypical children: n = 30, dyslexic children: n = 22, ADHD children: n = 35, citations = 202(GS, November 2022)]. [Menghini et al. 2010](https://doi.org/10.1016/j.neuropsychologia.2009.11.003) [neurotypical children and adolescents: n = 65, dyslexic children and adolescent: n = 60, citations = 330(GS, November 2022)]. [Moura et al. 2014 ](https://doi.org/10.1080/13854046.2014.964326)[neurotypical children: n = 50, dyslexic children: n = 50, citations = 104(GS, November 2022)]. [Plaza et al. 2002 ](https://europepmc.org/article/med/12030497)[neurotypical age-matched children: n = 26, neurotypical reading-age matched children: n = 26, dyslexic children: n = 26, citations = 81 (GS, November 2022)]. [Reid et al. 2007](https://doi.org/10.1002/dys.321) [neurotypical students: n = 15, AWD: n = 15, citations = 134 (GS, November 2022)]. [Reiter et al. 2005 ](https://doi.org/10.1002/dys.289)[neurotypical children: n = 42, dyslexic children: n = 42, citations = 485 (GS, November 2022)]. [Shareef et al. 2019](https://www.cambridge.org/core/journals/applied-psycholinguistics/article/abs/verbal-fluency-in-relation-to-reading-ability-in-students-with-and-without-dyslexia/E50F5FA55028070CF24F4812DDB15114) [AWD: n = 16, NT: n = 26, citations = 6 (GS, November [2022)]. Smith-Spark et al. 2017 [AWD: n = 28, NT: n = 28, citations = 36 (GS, November 2022)]. Snowling et al. 1997](https://doi.org/10.1111/1467-9817.00018) [NT: n = 19, AWD: n = 14, citations = 262 (GS, November 2022)]. [Varvava et al. 2014](https://www.frontiersin.org/articles/10.3389/fnhum.2014.00120/full) [AWD: n = 60, NT: n = 65, citations = 177 (GS, November 2022)]. * Original effect size: _d_ =0.37 [[calculated using this conversion from t to Cohen’s d](https://haiyangjin.github.io/2020/05/eta2d/#conversion-between-cohens-d-and-partial-eta-squared)]. * Replication effect size: Frith et al.: not reported. Hall and McGregor: _ηp2_= .04/ _d_ = 0.04 [[calculated using this conversion from to Cohen’s d](https://haiyangjin.github.io/2020/05/eta2d/#conversion-between-cohens-d-and-partial-eta-squared)] Hatcher et al.: Effect size = 0.46. Kinsbourne et al.: recovered dyslexics vs. controls: _d_ = 0.49; severe dyslexics vs. control = not reported. Marzocchi et al.: _ηp2_= .01/ _d_ = 0.01 [[calculated using this conversion from to Cohen’s d](https://haiyangjin.github.io/2020/05/eta2d/#conversion-between-cohens-d-and-partial-eta-squared)]. Menghini et al.: NA (%)= 17.2. Moura et al.: _ηp2_ = .115 / _d_ = 0.129 [[calculated using this conversion from to Cohen’s d](https://haiyangjin.github.io/2020/05/eta2d/#conversion-between-cohens-d-and-partial-eta-squared)]. Plaza et al.: age-matched neurotypical children vs dyslexic children: _ηp2_= 0.26 [calculated using this conversion](https://haiyangjin.github.io/2020/05/eta2d/#conversion-between-cohens-d-and-partial-eta-squared)/ _d_ = 0.34 [[calculated using this conversion to Cohen’s d](https://haiyangjin.github.io/2020/05/eta2d/#conversion-between-cohens-d-and-partial-eta-squared)], reading-aged matched neurotypical children vs. dyslexic children: not reported. Reid et al.: _d_ = -0.2. Reiter et al.: NA = 0.814. Shareef et al.: _d_ = 0.80. Smith-Spark et al.: _ß_ = -.216. Snowling et al.: Effect size (% of variance explained) = 17.2. Varvava et al.: φ = 0.40 [[calculated using the conversion from Chi square to Phi coefficient]](http://www.people.vcu.edu/~pdattalo/702SuppRead/MeasAssoc/NominalAssoc.html#:~:text=Computationally%2C%20phi%20is%20the%20square,(X2%2Fn).). {{< /spoiler >}} @@ -2204,7 +2204,7 @@ You can find a list of all effects we are working on [here](https://docs.google. {{< spoiler text="Statistics" >}} * Status: mixed * Original paper: ‘[Lexical Precision in Skilled Readers: Individual Differences in Masked Neighbor Priming](https://www.researchgate.net/profile/Jolyn-Hersch/publication/44569234_Lexical_Precision_in_Skilled_Readers_Individual_Differences_in_Masked_Neighbor_Priming/links/09e41501b5628cc7bb000000/Lexical-Precision-in-Skilled-Readers-Individual-Differences-in-Masked-Neighbor-Priming.pdf)’, Andrews and Hersch 2010; experiment, experiment 1: n= 97, Experiment 2: n = 123. [citation=207(GS, November 2022)]​. -* Critiques: [Elsherif et al. 2022a](https://doi.org/10.1177/17470218211046350) [n=84, citations=[ 8(GS, November 2022)]. Elsherif et al. 2022b](https://doi.org/10.1080/02699931.2018.1468732) n=84, citations=[28(GS, November 2022)]. +* Critiques: [Elsherif et al. 2022a](https://doi.org/10.1177/17470218211046350) [n=84, citations=[ 8(GS, November 2022)]. Elsherif et al. 2022b](https://doi.org/10.1080/23273798.2022.2108091) n=84, citations=[28(GS, November 2022)]. * Original effect size: experiment 1: _ηp2_= 0.06/ d= 0.062[calculated, using this [conversion](https://haiyangjin.github.io/2020/05/eta2d/#conversion-between-cohens-d-and-partial-eta-squared)], Experiment 2: _ηp2_= 0.04/ _d_ = 0.043 [calculated, using this [conversion](https://haiyangjin.github.io/2020/05/eta2d/#conversion-between-cohens-d-and-partial-eta-squared)]. * Replication effect size: Elsherif et al.: _d_ = 0.28 [[calculated using this conversion from t to Cohen’s d](https://haiyangjin.github.io/2020/05/eta2d/#conversion-between-cohens-d-and-partial-eta-squared)]; Elsherif et al.: _d_ = 0.34 [[calculated using this conversion from t to Cohen’s d](https://haiyangjin.github.io/2020/05/eta2d/#conversion-between-cohens-d-and-partial-eta-squared)]. {{< /spoiler >}} @@ -2771,9 +2771,9 @@ You can find a list of all effects we are working on [here](https://docs.google. {{< spoiler text="Statistics" >}} * Status: mixed * Original paper: ‘ [Prospect theory: An analysis of decision under risk](https://www.worldscientific.com/doi/abs/10.1142/9789814417358_0006)’ , Kahneman and Tversky, 1979; multiple experiments with n between 66 and 95 [citations=72861(GS, June 2022)] -* Critiques: [Brown et al. 2021](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3772089) [n=607 estimates from 150 articles, citations=10 (GS, December 2021)]. Meta-analyses: [Nieuwenstein et al. 2020](http://journal.sjdm.org/14/14321/jdm14321.html) [total n = 399, citations=109(GS, April 2023)]. [Walasek et al. 2018](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3189088) [_k_=19 studies, citations=11 (GS, December 2021)]. +* Critiques: [Brown et al. 2021](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3772089) [n=607 estimates from 150 articles, citations=10 (GS, December 2021)]. Meta-analysis: [Walasek et al. 2018](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3189088) [_k_=19 studies, citations=11 (GS, December 2021)]. * Original effect size: λ = 2.25 (reported in Walasek et al. 2018). -* Replication effect size: Brown et al.: λ = 1.955 [1.824, 2.104]. Walasek et al.: λ = 1.31 [1.10, 1.53]. All reported in Nieuwenstein et al.: Abadie et al.: _g_=-0.62 to _g_=0.22. Acker: _g_ =-0.47. Aczel et al.: _g_ =-0.35. Ashby et al.: _g_ =-0.21 to _g_ =1.48. Bos et al.: _g_ =-0.10. Bos et al.: _g_ =1.48. Calvillo & Penaloza: _g_ =-0.28 to _g_ =-0.09. Dijksterhuis: _g_ =0.24 to _g_ =0.46. Dijksterhuis et al.: _g_ =0.70 to _g_ =0.86. González Vallejo et al.: _g_ =0.00. Hasford: _g_ =0.43. Hess et al.: _g_ =-0.14. Huizenga et al.: _g_ =-0.26 to _g_ =-0.50. Lassiter et al.: _g_ =0.27 to _g_ =0.51. Lerouge: _g_ =0.38 to _g_ =0.47. McMahon et al.: _g_ =0.62 to _g_ =0.67. Messner et al.: _g_ =0.63. Newell et al.: _g_ =-0.50 to _g_ =0.17. Newell and Rakow: _g_ =-0.37 to _g_ =0.31. Nieuwenstein and Van Rijn: _g_ =-0.74 to _g_ =0.87. Nieuwenstein et al.: _g_ =-0.01. Nordgren et al.: _g_ =0.27 to _g_ =0.36. Payne et al.: _g_ =-0.10. Queen&Hess: _g_ =-0.21. Rey et al.: _g_ =0.27. Smith et al.: _g_ =0.25 to _g_ =0.32. Strick et al.: _g_ =0.58 to _g_ =1.21. Thorsteinson & Withrow: _g_ =0.18 to _g_ =0.34. Usher et al.: _g_ =0.78 to _g_ =1.04. Waroquier et al.: _g_ =-0.56 to _g_ =0.35. +* Replication effect size: Brown et al.: λ = 1.955 [1.824, 2.104]. Walasek et al.: λ = 1.31 [1.10, 1.53]. {{< /spoiler >}} * **Effort heuristic**. People judge products that took longer time to complete as higher in quality and monetary value. @@ -3025,7 +3025,7 @@ You can find a list of all effects we are working on [here](https://docs.google. {{< spoiler text="Statistics" >}} * Status: replicated * Original paper: ‘[The Framing of Decisions and the Psychology of Choice](https://www.science.org/doi/abs/10.1126/science.7455683)’, Tversky & Kahneman 1981; correlational, two “Asian disease problem” situations N1=152, N2=155. [citations=25,330(GS, February 2023)]. -* Critiques: [Diederich et al. 2018](https://www.cambridge.org/core/journals/judgment-and-decision-making/article/moderators-of-framing-effects-in-variations-of-the-asian-disease-problem-time-constraint-need-and-disease-type/7FE127D11A024B3EA8D1A8D18A244F7F) [N = 43, citations=31(GS, February 2023)]. Meta-analysis:[ Kühberger 1998](https://doi.org/10.1006/obhd.1998.2781) [n≈30,000 respondents over 136 empirical papers, citations=1607(GS, February 2023)].[ Otterbring et al. 2021](https://www.cambridge.org/core/journals/judgment-and-decision-making/article/moderators-of-framing-effects-in-variations-of-the-asian-disease-problem-time-constraint-need-and-disease-type/7FE127D11A024B3EA8D1A8D18A244F7F) [Study 1 N = 200, Study 2 N=800, citations=27(GS, February 2023)].[ Peterson and Tollefson 2023](https://www.mdpi.com/2673-7116/3/1/12) [N = 1,209, citations=0(GS, February 2023)].​ +* Critiques: [Diederich et al. 2018](https://www.cambridge.org/core/journals/judgment-and-decision-making/article/moderators-of-framing-effects-in-variations-of-the-asian-disease-problem-time-constraint-need-and-disease-type/7FE127D11A024B3EA8D1A8D18A244F7F) [N = 43, citations=31(GS, February 2023)]. Meta-analysis:[ Kühberger 1998](https://doi.org/10.1006/obhd.1998.2781) [n≈30,000 respondents over 136 empirical papers, citations=1607(GS, February 2023)].[ Otterbring et al. 2021](https://doi.org/10.1016/j.ssci.2020.105065) [Study 1 N = 200, Study 2 N=800, citations=27(GS, February 2023)].[ Peterson and Tollefson 2023](https://www.mdpi.com/2673-7116/3/1/12) [N = 1,209, citations=0(GS, February 2023)].​ * Original effect size: _d_=1. 16 [reported in[ Kühberger 1998](https://doi.org/10.1006/obhd.1998.2781)]. * Replication effect size: Diederich et al.: Gain versus Loss effect on Risky option preference significant in two regression models, β=-0.206 and β=-0.303, respectively. (replicated). Kühberger: mean effect size for the 80 studies with Asian _d_=0.57 [0.53, 0.61] (replicated). Otterbring et al.: Study 1 - statistically significant effect of framing on participants’ choice of program (_b_ = 1.52, _Z_ = 4.80, _p_ < 0.001), such that a larger proportion of participants chose the risky program under conditions of negative (78.0%) compared to positive framing (44.0%); Study 2 - statistically significant effect of framing on participants’ choice of program (_b_ = 1.82, _Z_ = 11.13, _p_ < 0.001), such that a larger proportion of participants chose the risky program under conditions of negative (68.8%) compared to positive framing (26.6%) (replicated). Peterson and Tollefson: _d_ =0.26 [calculated from the reported chi-square and sample size, χ2 =17.41, N = 1021] (replicated). {{< /spoiler >}} @@ -3052,7 +3052,7 @@ You can find a list of all effects we are working on [here](https://docs.google. {{< spoiler text="Statistics" >}} * Status: replicated * Original paper: ‘[Experimental Tests of the Endowment Effect and the Coase Theorem](https://www.journals.uchicago.edu/doi/10.1086/261737)’, Kahneman et al. 1990; experimental design, Experiment 1: n=42, Experiment 2: n=38, Experiment 3: n=26, Experiment 4: n=74 [citations=6392 (GS, March 2023)]. -* Critiques: [Carmon and Ariely 2000](https://doi.org/10.17780/ksujes.1224587) [study 1: n=91, study 2: n=472, study 3: n=75, study 4: n=250, citations=776 (GS, March 2023)]. [Shogren et al. 1994](https://www.jstor.org/stable/pdf/2117981.pdf) [n=142, citations=776 (GS, March 2023)]. +* Critiques: [Carmon and Ariely 2000](https://doi.org/10.1086/317590) [study 1: n=91, study 2: n=472, study 3: n=75, study 4: n=250, citations=776 (GS, March 2023)]. [Shogren et al. 1994](https://www.jstor.org/stable/pdf/2117981.pdf) [n=142, citations=776 (GS, March 2023)]. * Original effect size: 5 (selling price divided by buying price). * Replication effect size: Carmon and Ariely: study 1: _d_= 0.03 (calculated from converting Pearson’s _r_ to Cohen’s _d_ through[ this calculator](https://www.escal.site/)); study 2: NA; study 3: NA, study 4: NA. Shogren et al.: 1.05 over trial 5 (selling price divided by buying price); _d_=-0.069 (calculated from M and SD reported in Table 2) . {{< /spoiler >}} @@ -3065,7 +3065,7 @@ You can find a list of all effects we are working on [here](https://docs.google. {{< spoiler text="Statistics" >}} * Status: mixed (duelling meta-analyses, mix of successful and failed replications). * Original study: ‘[When choice is demotivating](https://doi.org/10.1037/0022-3514.79.6.995)’, Iyengar and Lepper, 2000; field experiment, 3 experiments, Study 1: n=502, Study 2: n=197, Study 3: 134. [citations = 2460(GS, April 2023)]. -* Critiques: [Chernev et al. 2010](https://doi.org/10.58052/neon0dz5s) [commentary, n=NA, citations=98 (GS, April 2023)]. [Chernev et al. 2015](https://doi.org/10.1016/j.jcps.2014.08.002) [meta-analysis of 99 observations, N = 7202, citations=717 (GS, April 2023)]. [Scheibehenne 2008](https://edoc.hu-berlin.de/handle/18452/16392) [three replications in the field and in the lab with a total of n= 850 participants and six laboratory experiments with n=595, citations=50 (GS, April 2023)]. Greifeneder 2008 [unpublished manuscript (link not available), n=NA, citations=4 (GS, April 2023)]. [Scheibehenne et al. 2010 ](https://doi.org/10.58052/neon0e00a)[meta-analysis of 63 conditions from 50 published and unpublished experiments, N = 5,036, citations=1241 (GS, April 2023)]. [Simonsohn et al. 2014](https://doi.org/10.1177/1745691614553988) [n=NA, citations=681 (GS, April 2023)]. +* Critiques: [Chernev et al. 2010](https://doi.org/10.1086/655200) [commentary, n=NA, citations=98 (GS, April 2023)]. [Chernev et al. 2015](https://doi.org/10.1016/j.jcps.2014.08.002) [meta-analysis of 99 observations, N = 7202, citations=717 (GS, April 2023)]. [Scheibehenne 2008](https://edoc.hu-berlin.de/handle/18452/16392) [three replications in the field and in the lab with a total of n= 850 participants and six laboratory experiments with n=595, citations=50 (GS, April 2023)]. Greifeneder 2008 [unpublished manuscript (link not available), n=NA, citations=4 (GS, April 2023)]. [Scheibehenne et al. 2010 ](https://doi.org/10.1086/651235)[meta-analysis of 63 conditions from 50 published and unpublished experiments, N = 5,036, citations=1241 (GS, April 2023)]. [Simonsohn et al. 2014](https://doi.org/10.1177/1745691614553988) [n=NA, citations=681 (GS, April 2023)]. * Original effect size: _d_=0.77 (study1) and _d_=0.29 (study2), and _d_=0.88 (study3) (as calculated from the χ2 values in the text with [this](https://www.campbellcollaboration.org/calculator/) online calculator). * Replication effect sizes: Scheibehenne: failed to directly replicate Iyengar and Lepper (2000) jam study. Greifeneder: a lab experiment with chocolates and also failed to conceptually replicate. Scheibehenne et al. 2010: “We found a mean effect size of virtually zero” (_d_=.02). Chernev et al. 2010: That’s because many of the studies were designed to show instances when there is no effect. You need to split the data into “choice is good” vs. “choice is bad.” Simonsohn et al.: We agree with Chernev et al.: When we split it up, we found that the choice is bad studies (choice overload) lack collective evidential value (uniform _p_-curve). Chernev et al.: <ignoring Simonsohn et al. 2014> Choice overload is a reliable effect under certain conditions (moderators). {{< /spoiler >}} @@ -3191,7 +3191,7 @@ You can find a list of all effects we are working on [here](https://docs.google. {{< spoiler text="Statistics" >}} * Status: replicated. * Original paper: ‘[Penny Wise and Pound Foolish: The Left-Digit Effect in Price Cognition](https://doi.org/10.3390/s23094221)’, Thomas and Morwitz 2005; five experiments with various designs, Study 1a: n= 52, Study 1b: n=63, Study 2: n = 154, Study 3: n = 53, Study 4: n= 27. [citations=474(GS, February 2023)]​. -* Critiques: [Bhattacharya et al. 2012](https://pubsonline.informs.org/doi/abs/10.1287/mnsc.1110.1364) [n≈100 million stock transactions, citations=109(GS, February 2023)]. [Lacetera et al. 2012](https://www.aeaweb.org/articles?id=10.1257/aer.102.5.2206) [n=22 million wholesale used-car transactions, citations=386(GS, February 2023)].​[ ](https://pubsonline.informs.org/doi/abs/10.1287/mnsc.1110.1364) [Manning and Sprott 2009](https://doi.org/10.1016/j.ebiom.2021.103613) [Study 1: n=442, Study 2a: n=409, Study 2b: n = 329, citations=149(GS, February 2023)].[ Sokolova et al. 2020](https://www.aeaweb.org/articles?id=10.1257/aer.102.5.2206) [Study 1: n=145, Study 2: n=120, Study 3: n=99, Study 4: n=150, Study 5: n=201, Study 6: n=15,236 choices across 3 product categories, citations=30(GS, February 2023)].​ +* Critiques: [Bhattacharya et al. 2012](https://pubsonline.informs.org/doi/abs/10.1287/mnsc.1110.1364) [n≈100 million stock transactions, citations=109(GS, February 2023)]. [Lacetera et al. 2012](https://www.aeaweb.org/articles?id=10.1257/aer.102.5.2206) [n=22 million wholesale used-car transactions, citations=386(GS, February 2023)].​[ ](https://pubsonline.informs.org/doi/abs/10.1287/mnsc.1110.1364) [Manning and Sprott 2009](https://doi.org/10.1086/597215) [Study 1: n=442, Study 2a: n=409, Study 2b: n = 329, citations=149(GS, February 2023)].[ Sokolova et al. 2020](https://doi.org/10.1177/0022243720932532) [Study 1: n=145, Study 2: n=120, Study 3: n=99, Study 4: n=150, Study 5: n=201, Study 6: n=15,236 choices across 3 product categories, citations=30(GS, February 2023)].​ * Original effect size: Study 1a – Significant effect of price ending (nine vs zero ending) on price magnitude perception when left digits differed, _η_ ²=.16; Study 1b – Significant left-digit change caused by a nine-ending price effects on the price’s magnitude perception, when the comparison standard is perceived to be close, _η_ ²=.15; Study 2 – For all levels of comparison standard, nine-ending target prices were perceived to have lower magnitude than zero-ending ones, _η_ ²=.15; Study 3 – the significant effects of nine-ending numbers on response times when the distance between the target number and the comparison standard is small, _η_ ²=.08; Study 4 – Significant effect of nine-ending in the target on the perceived quality ratings differences, when the psychological distance was low, _η²_=.41. * Replication effect size: Bhattacharya et al.: Buys outnumber sells at trade prices immediately below a round number, regression coefficients from _b_=0,270 to _b_=1.493, and sells outnumber buys at trade prices immediately above a round number, regression coefficients from _b_=-0,177 to _b_=-0.367; the highest ratio of buys to sells by liquidity demanders occurs at trade prices ending in .99, and the lowest ratio of buys to sells by liquidity demanders occurs at trade prices ending in .01 (replicated). Lacetera et al.: significant discontinuous drops in car value/auction price for car sale at mileage thresholds where left digits change (e.g., 10,000-mile marks) in two regression models (average -$157 and -$173, respectively) (replicated). Manning and Sprott: Study 1 – ES not reported but the significant effects of price endings on the choice of products (replicated); Study 2a - perceived price difference between the two products was affected by the price endings manipulation, β= -.32 (replicated); Study 2b - perceived price differences were affected by the price endings manipulation, β = -.32 (replicated). Sokolova et al.: Study 1 – Left-digit effect size in private label price evaluation is stronger in stimulus-based, _ηp2_ =0.054, than in memory-based evaluations, _ηp2 _=0.001; Study 2 – Left-digit effect size in discount evaluation is stronger in stimulus-based, _ηp2 _ =0.209, than in memory-based evaluations, _ηp2_ =0.057; Study 3 – Left-digit effect size in discount evaluation is stronger in stimulus-based, _ηp2 _ = 0.455, than in memory-based evaluations,_ ηp2 _ =0.296; Study 4 – Left-digit affect response times more in stimulus-based, _ηp2 _ =0.001, than in memory-based evaluations,_ηp2_ =0.00004; left-digit effect size in precise memory-based evaluations similar to stimulus-based, _ηp2_ =0.004; Study 5 – Left-digit effect size in price evaluations is stronger in stimulus-based, _ηp2 _ =0.033, than in memory-based evaluations, _ηp2 _ =0.001; Study 6 – Left-digit bias is stronger among light category users, _b_ =-0.15 to _b_ =-0.55, than in heavy category users, _b_=0.00 to _b_ =-0.40 (replicated). {{< /spoiler >}} @@ -3376,7 +3376,7 @@ You can find a list of all effects we are working on [here](https://docs.google. {{< spoiler text="Statistics" >}} * Status: mixed * Original paper:[ ‘Disease state prediction from resting state functional connectivity’, ](https://doi.org/10.1002/mrm.22159)Craddock et al. 2009; quasi-experimental design, ncontrols = 20, nclinical = 20. [citations=464(GS, April 2023)]​. -* Critiques: [Bhaumik et al. 2017](https://doi.org/10.1016/j.nicl.2016.02.018) [ncontrols = 29, nclinical = 38, Citations=58 (GS, April 2023)]. [Cao et al. 2014](https://doi.org/10.1111/pcn.12106) [ncontrols = 37, nclinical = 39, Citations= 51 (GS, April 2023)].[Guo et al. 2014](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4146162/) [ncontrols = 27, nclinical = 36, Citations= 73 (GS, April 2023)]. [Lord et al. 2012](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0041282) [ncontrols = 22, nclinical = 21, Citations=177 (GS, April 2023)].[Ma et al. 2013](https://doi.org/10.1016/j.brainres.2012.12.002)[ncontrols = 29, nclinical = 24, Citations=89 (GS, April 2023)]. [Qin et al. 2015](https://journals.lww.com/neuroreport/Abstract/2015/08020/Predicting_clinical_responses_in_major_depression.3.aspx) [ncontrols = 29, nclinical = 24, Citations=38 (GS, April 2023)]. [Ramasubbu et al. 2016](https://doi.org/10.1016/j.nicl.2016.07.012) [ncontrols = 19, nclinical = 45, Citations= 51 (GS, April 2023)]. [Sundermann et al. 2017](https://doi.org/10.1007/s00702-016-1673-8) [ncontrols = 180/60 (whole sample/severe symptoms only), nclinical = 180/60, Citations= 17 (GS, April 2023)]. [Yu et al. 2013](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0068250) [ncontrols = 38, nclinical = 19, Citations= 69 (GS, April 2023)]. [Zeng et al. 2012](https://doi.org/10.1023/a:1016071027177) [ncontrols = 29, nclinical = 24, Citations=753 (GS, April 2023)]. [Zeng et al. 2014](https://doi.org/10.1002/hbm.22278) [ncontrols = 29, nclinical = 24, Citations=169 (GS, April 2023)] +* Critiques: [Bhaumik et al. 2017](https://doi.org/10.1016/j.nicl.2016.02.018) [ncontrols = 29, nclinical = 38, Citations=58 (GS, April 2023)]. [Cao et al. 2014](https://doi.org/10.1111/pcn.12106) [ncontrols = 37, nclinical = 39, Citations= 51 (GS, April 2023)].[Guo et al. 2014](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4146162/) [ncontrols = 27, nclinical = 36, Citations= 73 (GS, April 2023)]. [Lord et al. 2012](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0041282) [ncontrols = 22, nclinical = 21, Citations=177 (GS, April 2023)].[Ma et al. 2013](https://doi.org/10.1016/j.brainres.2012.12.002)[ncontrols = 29, nclinical = 24, Citations=89 (GS, April 2023)]. [Qin et al. 2015](https://journals.lww.com/neuroreport/Abstract/2015/08020/Predicting_clinical_responses_in_major_depression.3.aspx) [ncontrols = 29, nclinical = 24, Citations=38 (GS, April 2023)]. [Ramasubbu et al. 2016](https://doi.org/10.1016/j.nicl.2016.07.012) [ncontrols = 19, nclinical = 45, Citations= 51 (GS, April 2023)]. [Sundermann et al. 2017](https://doi.org/10.1007/s00702-016-1673-8) [ncontrols = 180/60 (whole sample/severe symptoms only), nclinical = 180/60, Citations= 17 (GS, April 2023)]. [Yu et al. 2013](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0068250) [ncontrols = 38, nclinical = 19, Citations= 69 (GS, April 2023)]. [Zeng et al. 2012](https://doi.org/10.1093/brain/aws059) [ncontrols = 29, nclinical = 24, Citations=753 (GS, April 2023)]. [Zeng et al. 2014](https://doi.org/10.1002/hbm.22278) [ncontrols = 29, nclinical = 24, Citations=169 (GS, April 2023)] * Original effect size: 62.5% - 95% Classification Accuracy (cross-validation; CV); 16.7–83.3% (Hold-out validation). * Replication effect size: Bhaumik et al.: 76.1% (CV); 77.8%. (Hold-out validation). Lord et al.: 99.3% (CV). Zeng et al. /[ Zeng et al.](https://doi.org/10.1002/hbm.22278)/ [Ma et al.](https://doi.org/10.1016/j.brainres.2012.12.002)/[ Qin et al.](https://journals.lww.com/neuroreport/Abstract/2015/08020/Predicting_clinical_responses_in_major_depression.3.aspx): 69.8–96.2% (CV). Yu et al.: 80.9% (CV). Guo et al.: 90.5% (CV). Cao et al.: 84.2% (CV). Ramasubbu et al.: 49–66% (CV) – mixed, only significant in group with most severe symptoms. Sundermann et al.: no significant results in main analysis on whole sample (ES not reported); only significant in group with most severe symptoms 40.8 to 65.0% (CV), 54.2-61.7 (hold-out validation). {{< /spoiler >}} @@ -3541,7 +3541,7 @@ Critiques: [Gelman and Loken 2013](http://www.stat.columbia.edu/~gelman/research {{< spoiler text="Statistics" >}} * Status: replicated. * Original paper: ‘[The Incidental Orgasm: The Presence of Clitoral Knowledge and the Absence of Orgasm for Women](https://doi.org/10.1300/J013v42n01_07)’, Wade et al. 2005; correlational study, n=833 [citations=135(GS, May 2023)]​. -* Critiques:[ Garcia et al. 2014](https://doi.org/10.35919/rbsh.v36.1286) [n=2,850 single individuals, citations=134(GS, May 2023)]. ​[Mahar et al. 2020](https://doi.org/10.1007/s11930-020-00237-9) [systematic review, n=NA, citations=74(GS, May 2023)]. +* Critiques:[ Garcia et al. 2014](https://doi.org/10.1111/jsm.12669) [n=2,850 single individuals, citations=134(GS, May 2023)]. ​[Mahar et al. 2020](https://doi.org/10.1007/s11930-020-00237-9) [systematic review, n=NA, citations=74(GS, May 2023)]. * Original effect size: The orgasm gap was 52 percent: 39 percent of women, compared to 91 percent of men, usually or always experienced orgasm in partnered sex; _d_= -1.26 [-1.44, -1.07] (estimated from the data in Table 6 and using this[ conversion](https://www.campbellcollaboration.org/calculator/)). * Replication effect size: Garcia et al.: Compared with women, men reported a significantly higher mean occurrence rate of orgasm frequency, _η2_ = 0.12. Mahar et al.: ES not reported; six covered studies all showed that males report more frequent orgasm than females. {{< /spoiler >}} @@ -3630,7 +3630,7 @@ Critiques: [Gelman and Loken 2013](http://www.stat.columbia.edu/~gelman/research {{< spoiler text="Statistics" >}} * Status: replicated * Original paper: ‘[Word-blindness’ in school children](https://jamanetwork.com/journals/archneurpsyc/article-abstract/643441)’, Orton 1928; experimental clinic observational, n=15. [citation=535(GS, March 2023)]​. -* Critiques: [Lavers 1981](https://doi.org/10.1023/a:1015492015263)[anecdotal account ~1-2/year, citations=3(GS, March 2023)]. [Rosen 1955](https://doi.org/10.1080/00797308.1955.11822551) [n=1, citations=61(GS, March 2023)]. Original effect size: NA. +* Critiques: [Lavers 1981](https://doi.org/10.58680/ce198113769)[anecdotal account ~1-2/year, citations=3(GS, March 2023)]. [Rosen 1955](https://doi.org/10.1080/00797308.1955.11822551) [n=1, citations=61(GS, March 2023)]. Original effect size: NA. * Replication effect size: NA. {{< /spoiler >}} @@ -3647,7 +3647,7 @@ Critiques: [Gelman and Loken 2013](http://www.stat.columbia.edu/~gelman/research {{< spoiler text="Statistics" >}} * Status: mixed * Original paper: Various sources but the one of the earliest published studies ‘[The breaking of a habit by suggestion during sleep](https://psycnet.apa.org/record/1943-00131-001)’, Leshan 1942; between-subject experimental design, n=40 boys. [Citations=36(GS, March 2023)]​. -* Critiques: [Aarons 1976](https://psycnet.apa.org/record/1976-08859-001) review paper [n = 11 studies, citations=120(GS, March 2023)]. [Arzi et al. 2012](https://www.nature.com/articles/nn.3193) [n = 69 - 14 exclusions = 55 total, citations=261(GS, March 2023)]. [Simon and Emmons 1955](https://doi.org/10.1037/h0043733) review paper [n=10 studies, citations=80(GS, March 2023)].[ Wood et al. 1992](https://psycnet.apa.org/record/1976-08859-001) [n = 31, citations=261(GS, March 2023)]. +* Critiques: [Aarons 1976](https://psycnet.apa.org/record/1976-08859-001) review paper [n = 11 studies, citations=120(GS, March 2023)]. [Arzi et al. 2012](https://www.nature.com/articles/nn.3193) [n = 69 - 14 exclusions = 55 total, citations=261(GS, March 2023)]. [Simon and Emmons 1955](https://doi.org/10.1037/h0043733) review paper [n=10 studies, citations=80(GS, March 2023)]. Wood et al. 1992 [n = 31, citations=261(GS, March 2023)]. * Original effect size: ES not reported, but 40% of boys broke their fingernail-biting habit after listening to "My fingernails taste terribly bitter” message during 54 nights, as compared to 0% in two control groups. * Replication effect size: Aarons: _ES_ not reported; unclear if sleep-assisted learning works, preliminary results indicate that age, sex, health, wake learning capacity, suggestibility, and motivation to learn are important factors. ​[Arzi et al.](https://www.nature.com/articles/nn.3193): sleeping subjects learned novel associations between tones and odours: Odour pleasantness is processed during sleep in a pattern resembling that during wake – Sniff volume during sleep was greater following pleasant odorants than unpleasant odorants (_d=_ 1.39, estimated from the reported _t(27)_ = 3.7, _p_< 0.001 using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#conversion-between-cohens-d-and-partial-eta-squared)); Participants learned a novel association and acted on this learning, all during sleep – the sniff volume during sleep was larger after a tone that was previously paired during sleep with a pleasant odour than after a tone that was previously paired during sleep with an unpleasant odour (_d=_ 1.29 , estimated from the _t_(19) = 2.9, _p_< 0.01 using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#conversion-between-cohens-d-and-partial-eta-squared)). Simon and Emmons: ES not reported, but the critical analysis of ten sleep-learning studies concludes that it is highly speculative whether or not the studies reviewed in this paper have presented any acceptable evidence that learning during _sleep _is possible; even when the results is found they cannot be construed as _sleep-learning. _Wood et al.: Homophone test - presentation of items affected the implicit memory scores of waking controls but not sleeping subjects, indicating the absence of implicit learning during sleep (_ηp_ ²= 0.85, estimated from the reported _F_(l, 20) = 118.41. _p_ < .001); Category-instance task - waking controls but not sleeping subjects reporting the paired instance more frequently for presented than non-presented items (_ηp_ ²= 0.77, estimated from the reported _F_(l, 20)= 67.8, _p_ <.001 using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#conversion-between-cohens-d-and-partial-eta-squared)); Recognition tests – Waking subjects were significantly more likely to recognize the presented homophones (_ηp_ ²= 0.84, estimated from the reported _F_(1, 22) = 106.46, _p_ < .001 using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#conversion-between-cohens-d-and-partial-eta-squared)), and the presented categories of the category-instance pairs (_ηp_ ²= 0.77, estimated from the reported _F_(l, 23) = 70.1, _p_ < .001 using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#conversion-between-cohens-d-and-partial-eta-squared)). {{< /spoiler >}} @@ -3670,9 +3670,9 @@ Critiques: [Gelman and Loken 2013](http://www.stat.columbia.edu/~gelman/research {{< spoiler text="Statistics" >}} * Status: mixed. * Original paper: Multiple sources, but one of the earliest published ‘[Chronic stress and peptic ulcer](https://jamanetwork.com/journals/jama/article-abstract/313068)’ Gray et al. 1951; experimental and observational study, n= 7. [citations=279(GS, March 2023)]. -* Critiques: [Skillman et al. 1969](https://doi.org/10.1016/0002-9610(69)90011-7) [n=150, citations=406(GS, March 2023)].[ Marshall and Warren 1983](https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(84)91816-6/fulltext) [n=100, citations=7515(GS, March 2023)].[ Gough et al. 1984](https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(84)91224-8/fulltext) [n=484, citations=145(GS, March 2023)].[ Levenstein et al. 1997](https://journals.lww.com/jcge/Abstract/1997/04000/Psychological_Predictors_of_Peptic_Ulcer_Incidence.4.aspx) [n=484, citations=113(GS, March 2023)].[ Levenstein et al. 1999](https://jamanetwork.com/journals/jama/article-abstract/188314) [n=4,595, citations=84(GS, March 2023)].[ Levenstein et al. 2015](https://doi.org/10.1016/j.cgh.2014.07.052) [n=3,379, citations=188(GS, March 2023)]. +* Critiques: [Skillman et al. 1969](https://doi.org/10.1016/0002-9610(69)90011-7) [n=150, citations=406(GS, March 2023)].[ Marshall and Warren 1983](https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(84)91816-6/fulltext) [n=100, citations=7515(GS, March 2023)].[ Gough et al. 1984](https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(84)91224-8/fulltext) [n=484, citations=145(GS, March 2023)].[ Levenstein et al. 1997](https://journals.lww.com/jcge/Abstract/1997/04000/Psychological_Predictors_of_Peptic_Ulcer_Incidence.4.aspx) [n=484, citations=113(GS, March 2023)].[ Levenstein et al. 1999](https://jamanetwork.com/journals/jama/article-abstract/188314) [review, no new sample, citations=84(GS, March 2023)].[ Levenstein et al. 2015](https://doi.org/10.1016/j.cgh.2014.07.052) [n=3,379, citations=188(GS, March 2023)]. * Original effect size: ES not reported but the studies suggested that chronic emotional and physical stress is transmitted to the stomach by a hormonal mechanism mediated through the adrenal gland and may induce gastrointestinal ulceration through the hypothalamic-pituitary-gastric pathway. -* Replication effect size: Skillman et al.: Stress ulceration described; increased secretion of acid may be an important cause of this disease (replicated). Marshall and Warren: ES not reported, but the bacterium _Helicobacter pylori_ identified as correlated with ulcers and were present in almost all patients with active chronic gastritis, duodenal ulcer, or gastric ulcer and thus may be an important factor in the aetiology of these diseases (not replicated). Gough et al.: treating ulcers with antibiotics reduced recurrence by approximately 90-95% (not replicated). Leventstein et al.: five baseline psychological measures (depression, hostility, ego resiliency, social alienation or anomy, and personal uncertainty) had significant age-adjusted associations with incident ulcer [OR= 1.8 to 2.6]. Levenstein et al.: ES not reported, but conclusion that in most ulcer cases where stress is involved, _H. pylori_ is likely to be present as well. The impact of the two factors may be additive (replicated). Levenstein et al.: ulcer incidence was significantly higher among subjects in the highest tertile of stress scores (3.5%) than the lowest tertile (1.6%) (adjusted odds ratio, 2.2; 95% confidence interval [CI], 1.2–3.9; _P_ < .01); Stress had similar effects on ulcers associated with _H. pylori_ infection and those unrelated to either _H. pylori_ or use of nonsteroidal anti-inflammatory drugs (replicated). +* Replication effect size: Skillman et al.: Stress ulceration described; increased secretion of acid may be an important cause of this disease (replicated). Marshall and Warren: ES not reported, but the bacterium _Helicobacter pylori_ identified as correlated with ulcers and were present in almost all patients with active chronic gastritis, duodenal ulcer, or gastric ulcer and thus may be an important factor in the aetiology of these diseases (not replicated). Gough et al.: treating ulcers with antibiotics reduced recurrence by approximately 90-95% (not replicated). Leventstein et al.: five baseline psychological measures (depression, hostility, ego resiliency, social alienation or anomy, and personal uncertainty) had significant age-adjusted associations with incident ulcer [OR= 1.8 to 2.6]. Levenstein et al. (1999 review): ES not reported; the review concludes from earlier studies that in most ulcer cases where stress is involved, _H. pylori_ is likely to be present as well, and that the impact of the two factors may be additive (a summary of earlier evidence, not a new replication). Levenstein et al.: ulcer incidence was significantly higher among subjects in the highest tertile of stress scores (3.5%) than the lowest tertile (1.6%) (adjusted odds ratio, 2.2; 95% confidence interval [CI], 1.2–3.9; _P_ < .01); Stress had similar effects on ulcers associated with _H. pylori_ infection and those unrelated to either _H. pylori_ or use of nonsteroidal anti-inflammatory drugs (replicated). {{< /spoiler >}} * **Graphic images and cigarette use** (Graphic warning labels). Introducing graphic warning labels (GWL) on cigarette packages reduces smoking prevalence.​ @@ -3710,7 +3710,7 @@ Critiques: [Gelman and Loken 2013](http://www.stat.columbia.edu/~gelman/research {{< spoiler text="Statistics" >}} * Status: replicated. * Original paper: ‘[Sex, age, and equity behavior](https://psycnet.apa.org/record/1970-20861-001)’, Leventhal & Lane 1970; between-subjects experiment, N = 61. [citation=269(GS, October 2022)]​. -* Critiques:[ Callahan-Levy and Messe 1979](https://doi.org/10.1037/h0031684) [Study 1 n=126, Study 2 n = 80, citation=222(GS, October 2022)].​.​[ Jost 1997](https://doi.org/10.1111/j.1471-6402.1997.tb00120.x) [N=132, citation=192(GS, October 2022)]​. [ Hogue and Yoder 2003](https://doi.org/10.1111/1471-6402.00113) [N=180, citation=78(GS, October 2022)] [Lane and Messe 1971](https://doi.org/10.1037/h0031684) [N=128, citation=174(GS, October 2022)]..[ Major et al. 1984](https://doi.org/10.1037/0022-3514.47.6.1399) [Experiment 1 n=76, Experiment 2 n=80, citation=408(GS, October 2022)]​ +* Critiques:[ Callahan-Levy and Messe 1979](https://doi.org/10.1037/0022-3514.37.3.433) [Study 1 n=126, Study 2 n = 80, citation=222(GS, October 2022)].​.​[ Jost 1997](https://doi.org/10.1111/j.1471-6402.1997.tb00120.x) [N=132, citation=192(GS, October 2022)]​. [ Hogue and Yoder 2003](https://doi.org/10.1111/1471-6402.00113) [N=180, citation=78(GS, October 2022)] [Lane and Messe 1971](https://doi.org/10.1037/h0031684) [N=128, citation=174(GS, October 2022)]..[ Major et al. 1984](https://doi.org/10.1037/0022-3514.47.6.1399) [Experiment 1 n=76, Experiment 2 n=80, citation=408(GS, October 2022)]​ * Original effect size: main effects of sex on rewards taking_ ηp2 _= 0.22 / _d_ = 0.53 [_ηp2_ calculated from F statistic values and Table 1 data and converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)]. * Replication effect size: Callahan-Levy and Messe: Study 1 – Target X Sex of Target interaction effects on actual pay _ηp2 _= 0.16 / _d_ = 0.43 [_ηp2_ calculated from F statistic values and converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)], Target X Sex of Target interaction effects on fair pay estimates_ ηp2 _ = 0.16 / _d_ = 0.41 [_ηp2_ calculated from F statistic values and converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)] (replicated); Study 2 – main effect of sex on pay allocation _ηp2 _= 0.18 / _d_ = 0.47 [_ηp2_ calculated from F statistic values and converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)], main effect of sex on fair pay estimates _ηp2 _= 0.11 / _d_ = 0.35 [_ηp2_ calculated from F statistic values and converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)] (replicated). Jost: _ηp2 _= 0.044 / _d_ = 0.21 [_ηp2_ calculated from F statistic values in Table 1 and converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)] (replicated). Hogue and Yoder: _ηp2_ = 0.165 [reported] / _d_ = 0.87 [calculated from M, SD and (sub)sample size data for two independent samples in control condition] (replicated). Lane and Messe: main effects of sex on self-interest responses _ηp2 _= 0.099 / _d_ = 0.33 [_ηp2_ calculated from F statistic values and converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)] (replicated). Major et al.: ​ Study 1 –_ηp2 _= 0.13 / _d_ = 0.38 [Sex X Social Comparison Condition interaction effects, _ηp2_ calculated from F statistic values and converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)]; Study 2 –_ηp2 _= 0.18 / _d_ = 0.47 [main effects of Sex amount of work for the same/fixed amount of money, _ηp2_ calculated from F statistic values and converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)] (replicated). {{< /spoiler >}} @@ -3737,7 +3737,7 @@ Critiques: [Gelman and Loken 2013](http://www.stat.columbia.edu/~gelman/research {{< spoiler text="Statistics" >}} * Status: replicated * Original paper: ‘[Why Don´t Democracies Fight Each Other? An Experimental Assessment of the ”Political Incentive" Explanation.](https://doi.org/10.5114/aoms/174265)‘, Mintz and Geva 1993; experiment, n = 44/34/39. [citations = 268 (GS, February 2023)] -* Critiques: [Tomz and Weeks 2013](https://doi.org/10.21900/j.jams.v5.1566) [n = 762/1273/1944, citations = 426 (GS, February 2023)]. +* Critiques: [Tomz and Weeks 2013](https://doi.org/10.1017/s0003055413000488) [n = 762/1273/1944, citations = 426 (GS, February 2023)]. * Original effect size: First experiment: -1.73 approval (on a scale from 0 to 10) to use force against democracies (_p_<0.05), compared to autocracies; Second experiment: -3.06 approval (_p_<0.01); Third experiment: -2.68 approval (_p_<0.01). * Replication effect size: Tomz and Weeks: First experiment: -13.3% approval to use force against democracies [-19.6, -6.9], compared to autocracies; Second experiment: -11.4% approval [-17, -5.9]; Third experiment: -11.5% approval [14.7, 5.3]. {{< /spoiler >}} @@ -3947,7 +3947,7 @@ Critiques: [Gelman and Loken 2013](http://www.stat.columbia.edu/~gelman/research {{< spoiler text="Statistics" >}} * Status: replicated * Original paper: ‘[The role of cat eye narrowing movements in cat–human communication](https://www.nature.com/articles/s41598-020-73426-0#Abs1); Humphreys et al. 2020a; experiments, experiment 1: n = 21, experiment 2: n = 24. [citation=21(GS, January 2023)]​. -* Critiques: [Humphreys et al. 2020b](https://www.mdpi.com/2076-2615/10/12/2256#:~:text=We%20demonstrated%20that%20cats%20that%20responded%20to%20human,blink%20movements%20in%20response%20to%20human%20slow%20blinking.) [[n = 24](https://www.biorxiv.org/content/biorxiv/early/2022/10/08/2022.03.12.484069.full.pdf), citations = 7(GS, January 2023)]. +* Critiques: [Humphreys et al. 2020b](https://www.mdpi.com/2076-2615/10/12/2256#:~:text=We%20demonstrated%20that%20cats%20that%20responded%20to%20human,blink%20movements%20in%20response%20to%20human%20slow%20blinking.) [n = 18, citations = 7(GS, January 2023)]. * Original effect size: Experiment 1: _d_ = 0.56 Experiment 2: Half-blinks vs. neutral: _d_ =0.75 [_d _calculated from mean differences and standard deviation using this [conversion](https://lbecker.uccs.edu/)], eye narrowing vs neutral: _d_ = 0.90 [_d _calculated from mean differences and standard deviation and converted using this [conversion](https://lbecker.uccs.edu/)]. * Replication effect size: Humphreys et al.: slow blinking versus control trials: _d_ = 0.71, slow blinking versus neutral trials: _d_ = 0.52. {{< /spoiler >}} @@ -4019,7 +4019,7 @@ Critiques: [Gelman and Loken 2013](http://www.stat.columbia.edu/~gelman/research {{< spoiler text="Statistics" >}} * Status: reversed * Original paper: ‘[Relative quantity judgment by Asian elephants (Elephas maximus)](https://doi.org/10.1007/s10071-008-0185-9)’, Irie-Sugimoto et al. 2009; experiment, n = 3. [citations = 107 (GS, January 2023)]​. -* Critiques: [Perdue et al. 2012](https://doi.org/10.1007/s10071-012-0521-y) [n = 2, citations = 114 (GS, January 2023)].​ +* Critiques: [Perdue et al. 2012](https://doi.org/10.1007/s10071-012-0521-y) [n = 2 African elephants; the original tested Asian elephants, citations = 114 (GS, January 2023)].​ * Original effect size: individual correlations for each subject: _r_ = 0.23, _r_ = 0.01, _r_ = 0.02. * Replication effect size: Perdue et al.: individual correlations for each subject: _r_ = −0.64, −0.96. {{< /spoiler >}} From cfbd31cc93fac4b1d57ecec9c55b986c81eaad95 Mon Sep 17 00:00:00 2001 From: Lukas Wallrich Date: Fri, 13 Nov 2026 09:14:16 +0000 Subject: [PATCH 2/9] Reversals: AoA verification entry reports a null original, not a reversal Chalard and Bonin (2006) found no AoA effect on name-object verification; later verification studies mostly found an early-acquired advantage, and Holmes and Ellis's Experiment 8 (category verification) found none. Fixes the experiment numbers and task labels of the cited studies. Co-Authored-By: Claude Opus 5.5 --- content/reversals/reversals.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/content/reversals/reversals.md b/content/reversals/reversals.md index 5d73430391c..10320a86f13 100644 --- a/content/reversals/reversals.md +++ b/content/reversals/reversals.md @@ -1261,13 +1261,13 @@ You can find a list of all effects we are working on [here](https://docs.google. * Replication effect size: Chen et al.: ηp² = 0.46 [ηp2 calculated from reported F statistic and converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)]. {{< /spoiler >}} -* **Age of acquisition influence on the pre-conceptual stages of lexical retrieval (category verification)**. The age at which one acquires an object does not contribute to the speed and accuracy of category verification during a semantic categorisation task (where objects have to be decided whether they represent one group or another, e.g. tools vs. furniture). +* **Age of acquisition influence on the pre-conceptual stages of lexical retrieval (verification tasks)**. The age at which one acquires an object's name does not affect how quickly people verify that a picture matches a name (Chalard and Bonin found age-of-acquisition effects in picture naming but not in name–object verification). Later studies used related verification tasks: word–picture and picture–picture verification, and category verification, where people decide whether an object belongs to a named category (e.g. tools vs. furniture). {{< spoiler text="Statistics" >}} -* Status: reversed +* Status: not replicated (the original reported no reliable age-of-acquisition effect on name–object verification times; other word–picture verification studies reported an early-acquired advantage. Holmes and Ellis's Experiment 8 found no reliable age-of-acquisition effect in category verification with typicality controlled) * Original paper: ‘[Age-of-acquisition effects in picture naming: Are they structural and/or semantic in nature?](https://doi.org/10.1080/13506280544000084), Chalard and Bonin 2006; experiment, n = 27. [citations=36(GS, December 2022)]​. -* Critiques: [Catling and Elsherif 2020](https://doi.org/10.1016/j.actpsy.2020.103138) [Experiment 1a: n = 48, Experiment 2a: n = 48, citations = 12(GS, December 2022)]. [Catling and Johnston 2006](https://doi.org/10.1348/000712605X53515) [Experiment 1: n = 15, citations = 17(GS, December 2022)]. [Catling and Johnston 2009 ](https://doi.org/10.1080/17470210701814352)[Experiment 1: n = 24, citations = 54 (GS, December 2022)]. [Holmes and Ellis 2006 ](https://doi.org/10.1080/13506280544000093)[Experiment 4: n = 20, Experiment 7: n = 30, citations = 87 (GS, December 2022)]. [Räling et al. 2015](https://doi.org/10.1016/j.neuropsychologia.2015.05.031) [n = 36, citations = 24(GS, December 2022)]. [Stadthagen-Gonzalez et al. 2009 ](https://doi.org/10.1080/17470210802511139)[n = 100, citations = 51 (GS, December 2022)]. +* Critiques: [Catling and Elsherif 2020](https://doi.org/10.1016/j.actpsy.2020.103138) [Experiment 1a: n = 48, Experiment 2a: n = 48, citations = 12(GS, December 2022)]. [Catling and Johnston 2006](https://doi.org/10.1348/000712605X53515) [Experiment 1: n = 15, citations = 17(GS, December 2022)]. [Catling and Johnston 2009 ](https://doi.org/10.1080/17470210701814352)[Experiment 1, picture–picture verification: n = 24, citations = 54 (GS, December 2022)]. [Holmes and Ellis 2006 ](https://doi.org/10.1080/13506280544000093)[Experiment 4, category verification: n = 20 in the positive-decision condition, total N = 40; Experiment 8, category verification: n = 30, citations = 87 (GS, December 2022)]. [Räling et al. 2015](https://doi.org/10.1016/j.neuropsychologia.2015.05.031) [n = 36, citations = 24(GS, December 2022)]. [Stadthagen-Gonzalez et al. 2009 ](https://doi.org/10.1080/17470210802511139)[n = 100, citations = 51 (GS, December 2022)]. * Original effect size: NA. -* Replication effect size: Catling and Elsherif: Experiment 1a: _d_ = 0.23, Experiment 2a: _d_ = 0.25. Catling and Johnston: _ηp2_ = 0.46. Catling and Johnston: _ηp2_ = 0.19[_ηp2_ calculated from reported F statistic and converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)]. Holmes and Ellis: Experiment 4: _d_ = 1.62 [_d_ calculated from reported t statistic in category verification and converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)]; Experiment 8: _t_ < 1. Räling et al.: _ηp2_ = 0.45[_ηp2_ calculated from reported F statistic and converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)]. Stadthagen-Gonzalez et al.: Beta = 2.43. +* Replication effect size: Catling and Elsherif: Experiment 1a: _d_ = 0.23, Experiment 2a: _d_ = 0.25. Catling and Johnston: _ηp2_ = 0.46. Catling and Johnston: _ηp2_ = 0.19[_ηp2_ calculated from reported F statistic and converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)]. Holmes and Ellis: Experiment 4: _d_ = 1.62 [_d_ calculated from reported t statistic in category verification and converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)]; Experiment 8: _F_ < 1. Räling et al.: _ηp2_ = 0.45[_ηp2_ calculated from reported F statistic and converted using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)]. Stadthagen-Gonzalez et al.: Beta = 2.43. {{< /spoiler >}} * **Age of acquisition influence on the pre-conceptual stages of lexical retrieval (category falsification)**. The age at which one acquires the name of an object object does not contribute to the speed and accuracy of category falsification (i.e. deciding that a different word and the picture of the acquired concept do not match; e.g. the picture of the acquired concept of a rabbit, paired with the non-matching word “mouse”). From 2b378f666e74c55fcfb642e5445ab28e160c9681 Mon Sep 17 00:00:00 2001 From: Lukas Wallrich Date: Fri, 13 Nov 2026 11:14:57 +0000 Subject: [PATCH 3/9] Reversals: non-significant opposite estimates are not reversals Hurricane names (Smith 2016), elderly priming (Doyen et al. 2012, automated timing) and scarcity resource allocation (O'Donnell et al. 2021) relabelled "not replicated" with short qualifiers. Co-Authored-By: Claude Opus 5.5 --- content/reversals/reversals.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/content/reversals/reversals.md b/content/reversals/reversals.md index 10320a86f13..6351029c25a 100644 --- a/content/reversals/reversals.md +++ b/content/reversals/reversals.md @@ -176,7 +176,7 @@ You can find a list of all effects we are working on [here](https://docs.google. * **Elderly priming**. that hearing about old age makes people walk slower. {{< spoiler text="Statistics" >}} -* Status: reversed +* Status: not replicated (no effect with automated timing; slower walking appeared only when experimenters expected it) * Original paper: '[Automaticity of social behaviour](https://scholar.google.com/scholar?cluster=3335859380278379099&hl=en&as_sdt=0,5&sciodt=0,5)', Bargh 1996; 2 experiments with Study 2a: n = 30, Study 2b: n = 30. [citations = 5938(GS, October 2021)]​. * Critiques:[ Doyen 2012](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0029081#s2) [n=120, citations=757(GS, October 2021)]. [Lakens 2017](https://psyarxiv.com/3m5y9) [meta analysis: citations = 21(GS, October 2021)]. [Pashler et al. 2011](https://web.archive.org/web/20170923093613/http://www.psychfiledrawer.org/replication.php?attempt=MTU%3D) [n=66, citations=21(GS, October 2021)]. * Original effect size: not reported. ​ @@ -455,7 +455,7 @@ You can find a list of all effects we are working on [here](https://docs.google. * **Hurricane names**. Female-named hurricanes are more deadly than male-named ones. Original effect size was a 176% increase in deaths, driven entirely by four outliers; reanalysis using a greatly expanded historical dataset found a nonsignificant decrease in deaths from female named storms. {{< spoiler text="Statistics" >}} -* Status: reversed +* Status: not replicated (no significant difference in a larger dataset) * Original paper: '[Female hurricanes are deadlier than male hurricanes](https://www.pnas.org/content/111/24/8782)', Jung 2014; observational study, n=92 hurricanes discarding two important outliers. [citations = 113(GS, Mar 2022)]. * Critiques: [Christensen 2014](https://www.pnas.org/content/111/34/E3497) [same data, citations = 114(GS, March 2022)]. [Smith 2016 ](https://doi.org/10.1016/j.wace.2015.11.006)[expanded Atlantic data, 1979–2014, including tropical storms and non-U.S. deaths, n = 420 storms, and new Pacific data, n = 579 storms; citations = 8(GS, March 2022)]. * Original effect size: _d_=0.65: [176%](https://www.google.co.uk/search?source=hp&ei=HC1yXtKbBqeRlwTgzplw&q=41.84+%2F+15.15+&oq=41.84+%2F+15.15+&gs_l=psy-ab.3...606.10628..10753...0.0..0.91.412.5......3....1..gws-wiz.......0i7i30j0.raVAib7H4Jk&ved=0ahUKEwjSmciGm6ToAhWnyIUKHWBnBg4Q4dUDCAg&uact=5) increase in deaths from flipping names from relatively masculine to relatively feminine. @@ -3090,11 +3090,11 @@ You can find a list of all effects we are working on [here](https://docs.google. * **Scarcity effect - _Resource allocation_**. Poor economic conditions favour resource allocations to daughters over sons. {{< spoiler text="Statistics" >}} -* Status: reversed +* Status: not replicated (the replication estimate was in the opposite direction but not significant) * Original paper: ‘[Spending on daughters versus sons in economic recessions](https://doi.org/10.1093/jcr/ucv023)’, Durante et al. 2015; within-subjects design, n=198. [citation=57(GS, November 2021)]​. * Critiques: [O’Donnell et al. 2021](https://doi.org/10.1073/pnas.2103313118) [n=573, citations=0(GS, November 2021)]. * Original effect size: _r_= .179. -* Replication effect size: O’Donnell et al.: _r_= -.082. +* Replication effect size: O’Donnell et al.: _r_= -.082 (not significant). {{< /spoiler >}} * **Scarcity effect - _Planning_**. Consumers who feel resource constrained shift to engage in relatively more priority planning, rather than efficiency planning. From cd1225dfe399326475640ff92442ae2f384e40e2 Mon Sep 17 00:00:00 2001 From: Lukas Wallrich Date: Fri, 13 Nov 2026 11:17:22 +0000 Subject: [PATCH 4/9] Reversals: elephant ratio entry reports a null original, not a reversal Irie-Sugimoto et al. 2009 found no ratio effect (Experiment 1, n = 5; Experiment 2, n = 4); Perdue et al. 2012 found ratio effects in two African elephants. Status, description, sample and effect lines updated. Co-Authored-By: Claude Opus 5.5 --- content/reversals/reversals.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/content/reversals/reversals.md b/content/reversals/reversals.md index 6351029c25a..e39b59dffe3 100644 --- a/content/reversals/reversals.md +++ b/content/reversals/reversals.md @@ -4015,13 +4015,13 @@ Critiques: [Gelman and Loken 2013](http://www.stat.columbia.edu/~gelman/research * Replication effect size: Mongillo et al.: _r_ = 0.01. Stevens et al.: _r_ = −0.10 [−0.34, 0.15] (Study 1), _r_ = 0.04 [−0.26, 0.34] (Study 2), _r_ = −0.11 [−0.27, 0.04] (meta-analysis). {{< /spoiler >}} -* **Numerical ratio effects on quantity discrimination in elephants** (Number discrimination, quantity/number judgments). The numerical ratio between quantities has been shown to predict the ability to discriminate quantities across a wide range of species. However, Irie-Sugimoto et al., (2009) found that elephants did not follow this pattern because numerical ratio did not predict performance. Perdue et al. (2012) replicated the study in elephants with a similar design and found that ratio predicted performance, reversing the findings of the original study.​ +* **Numerical ratio effects on quantity discrimination in elephants** (Number discrimination, quantity/number judgments). The numerical ratio between quantities has been shown to predict the ability to discriminate quantities across a wide range of species. However, Irie-Sugimoto et al., (2009) found that elephants did not follow this pattern because numerical ratio did not predict performance. Perdue et al. (2012) used a similar paradigm with additional methodological controls in two African elephants and found that accuracy declined significantly as the two quantities became more similar, unlike the original null findings in Asian elephants.​ {{< spoiler text="Statistics" >}} -* Status: reversed -* Original paper: ‘[Relative quantity judgment by Asian elephants (Elephas maximus)](https://doi.org/10.1007/s10071-008-0185-9)’, Irie-Sugimoto et al. 2009; experiment, n = 3. [citations = 107 (GS, January 2023)]​. +* Status: not replicated (the original found no ratio effect in Asian elephants; a later study found ratio effects in African elephants) +* Original paper: ‘[Relative quantity judgment by Asian elephants (Elephas maximus)](https://doi.org/10.1007/s10071-008-0185-9)’, Irie-Sugimoto et al. 2009; two experiments, Experiment 1: n = 5, Experiment 2: n = 4. [citations = 107 (GS, January 2023)]​. * Critiques: [Perdue et al. 2012](https://doi.org/10.1007/s10071-012-0521-y) [n = 2 African elephants; the original tested Asian elephants, citations = 114 (GS, January 2023)].​ -* Original effect size: individual correlations for each subject: _r_ = 0.23, _r_ = 0.01, _r_ = 0.02. -* Replication effect size: Perdue et al.: individual correlations for each subject: _r_ = −0.64, −0.96. +* Original effect size: Experiment 1, individual correlations between accuracy and ratio for the three elephants with enough comparisons: _r_ = 0.23, _r_ = 0.01, _r_ = 0.02 (none significant). +* Replication effect size: Perdue et al.: individual correlations between accuracy and ratio, visible condition: _r_ = −0.64, −0.96; nonvisible condition: _r_ = −0.79, −0.67 (all significant). {{< /spoiler >}} * **Contagious yawning in dogs**. Dogs have been shown to yawn more after observing a person yawn than after a control condition with other mouth movements. The original study (Joly-Mascheroni et al., 2008) found 72% of dogs yawned after watching a human yawn but none yawned after watching other mouth movements. Two subsequent studies did not replicate a difference in yawning between conditions (Harr et al., 2009; O’Hara and Reeve, 2010), but a third did find a difference (Madsen and Persson, 2012). From 032a4e8eec9fd94913773f08dc0c820879f603c1 Mon Sep 17 00:00:00 2001 From: Lukas Wallrich Date: Fri, 13 Nov 2026 11:41:16 +0000 Subject: [PATCH 5/9] Reversals: relabel entries where FLoRA pointed to missing replications Checked against the papers after FLoRA outcome codes disagreed with the page labels. Abstract rule learning, induced-compliance dissonance, choosing vs rejecting and facial feedback become mixed; prosocial spending gets a qualifier; brand attitudes and neonate imitation become not replicated. Adds the omitted replications (Geambasu et al. 2022, Vaidis et al. 2024, Many Labs 2, Shafir & Cheek 2024, Koepke et al. 1983), fixes the Last-place aversion DOI and Buell citation, the Chandrashekar sign and the less-is-better effect sizes. Co-Authored-By: Claude Opus 5.5 --- content/reversals/reversals.md | 44 +++++++++++++++++----------------- 1 file changed, 22 insertions(+), 22 deletions(-) diff --git a/content/reversals/reversals.md b/content/reversals/reversals.md index e39b59dffe3..e95bc1227d5 100644 --- a/content/reversals/reversals.md +++ b/content/reversals/reversals.md @@ -365,9 +365,9 @@ You can find a list of all effects we are working on [here](https://docs.google. * **Prosocial spending**. Spending money on other people leads to greater happiness than spending money on oneself. {{< spoiler text="Statistics" >}} -* Status: replicated +* Status: replicated (smaller effects; one of three experiments was null) * Original paper: ‘[Spending Money on Others Promotes Happiness](https://www.science.org/doi/epdf/10.1126/science.1150952)’, Dunn et al. , 2008; cross-sectional survey, n=632. [citations = 2008 (GS, March 2022)]. -* Critiques: [Akinn et al., 2020](https://doi.apa.org/fulltext/2020-23349-001.pdf?auth_token=24a0cdbea49548a6e0de651a3de753b3d84652eb&returnUrl=https%3A%2F%2Fdoi.apa.org%2FdoiLanding%3Fdoi%3D10.1037%252Fpspa0000191) [3 Experiments, Experiment 1: n=712, Experiment 2: n =1,950, Experiment 3: n =5,199, citations = 51 (GS, March 2022)]. +* Critiques: [Aknin et al., 2020](https://doi.org/10.1037/pspa0000191) [3 Experiments, Experiment 1: n=712, Experiment 2: n =1,950, Experiment 3: n =5,199, citations = 51 (GS, March 2022)]. * Original effect size: _b _= 0.11. * Replication effect size: Experiment 1: positive affect: _d_ = .36, positive emotion: _d_ = .32; Experiment 2: positive affect: _d_ = .03, positive emotion: _d_ = .02; Experiment 3: positive affect: _d_ = .06, positive emotion: _d_ = .06, positive meotion after spending one’s own money: _d_ = .17. {{< /spoiler >}} @@ -986,10 +986,10 @@ You can find a list of all effects we are working on [here](https://docs.google. * **Personal cognitive dissonance - induced-compliance paradigm**. In this paradigm, subjects are led to perform, in a context of free choice, an inconsistent act with their own norms or social norms (e.g., agree to perform a counter-attitudinal act). Dissonance can be resolved through multiple modes of reduction (e.g., social support, trivialization, etv.), but attitude change remains the most studied mode of reduction.​ {{< spoiler text="Statistics" >}} -* Status: replicated +* Status: mixed (meta-analyses positive; the core choice effect was not replicated in a 39-lab study) * Original paper: ‘[Dissonance arousal: Physiological evidence](https://doi.org/10.1037/0022-3514.45.4.782)’, Croyle and Cooper 1983; between-subjects design, n1 = 30, n2=30. [citations= 447 (GS, February 2023)]​. -* Critiques: [Kenworthy et al. 2011 ](https://doi.org/10.1080/10463283.2011.580155)[meta-analyse; _n_= 31 studies, citations = 71 (GS, February 2023)]. [Kim et al. 2014](https://doi.org/10.1080/01463373.2014.949385) [meta-analyse; _k_= 230 effects, citations = 11 (GS, February 2023)]. [Vaidis et al. 2018](https://osf.io/9xsmj/) [multi-Lab Replication; _in preparation_].​ Original effect size: Effect of induced-compliance on attitude change: _d_= 2.40 [1.40, 3.37]​. -* Replication effect size: Kenworthy et al.: Effect of induced-compliance on dissonance effects: _d_= 0.81 [0.70, 0.91]​.​ Kim et al.: Effect of induced-compliance on attitude change: _r_= .22 [.21, .24]​.​ +* Critiques: [Kenworthy et al. 2011 ](https://doi.org/10.1080/10463283.2011.580155)[meta-analyse; _n_= 31 studies, citations = 71 (GS, February 2023)]. [Kim et al. 2014](https://doi.org/10.1080/01463373.2014.949385) [meta-analyse; _k_= 230 effects, citations = 11 (GS, February 2023)]. [Vaidis et al. 2024](https://doi.org/10.1177/25152459231213375) [multi-lab constructive replication of Experiment 1, 39 labs, n = 4,898].​ Original effect size: Effect of induced-compliance on attitude change: _d_= 2.40 [1.40, 3.37]​. +* Replication effect size: Kenworthy et al.: Effect of induced-compliance on dissonance effects: _d_= 0.81 [0.70, 0.91]​.​ Kim et al.: Effect of induced-compliance on attitude change: _r_= .22 [.21, .24]​.​ Vaidis et al.: high- vs low-choice counterattitudinal essay, attitude change _d_ = 0.017 [−0.071, 0.11] (not replicated); counterattitudinal vs neutral essay, _d_ = 0.31 [0.22, 0.41], regardless of choice. {{< /spoiler >}} * **Personal cognitive dissonance - Induced-hypocrisy paradigm**. In this paradigm, dissonance is aroused by making individuals aware of the discrepancy between a socially desirable behaviour (e.g., not wasting water; stage 1: normative commitment phase) and their own past transgressive behaviours (e.g., remembering one's past water waste; stage 2: transgression salience phase). Most of the dissonance reduction work is done through behavioural means, and leads subjects to express behavioural intentions, and/or perform behaviours in the direction of the socially desirable behaviours expressed in step 1 (i.e., allowing for the reduction of the inconsistency between the norm, step 1, and the recall of transgressions, step 2).​ @@ -1160,11 +1160,11 @@ You can find a list of all effects we are working on [here](https://docs.google. * **Facial Feedback.** Smiling causes a good mood, while pouting produces a bad mood. {{< spoiler text="Statistics" >}} -* Status: not replicated +* Status: mixed (posed and mimicked expressions affected happiness in a large multi-lab study; pen-in-mouth results were inconsistent) * Original paper: '[Inhibiting and Facilitating Conditions of the Human Smile: A Nonobtrusive Test of the Facial Feedback Hypothesis](https://jhrcook.github.io/wagenmaker-data-analysis/papers/Strack_1988_Inhibiting-and-facilitating-conditions-of-the-human-smile-a-nonobtrusive-test-of-the-facial-feedback-hypothesis.pdf)', Strack et al. 1988; 2 experimental studies, n1= 92, n2=83. [citation= 2577(GS, February, 2022)]. * Critiques: [Coles et al. 2019](https://psyarxiv.com/svjru/) [meta-analysis of 286 effect sizes from 138 studies, citation= 115 (GS, February 2022)]. [Coles et al. 2022](https://doi.org/10.1038/s41562-022-01458-9) [Many Smiles Collaboration, preregistered multi-lab study, n = 3,878]. [Wagenmakers et al. 2016](https://www.ejwagenmakers.com/2016/WagenmakersEtAl2016Strack.pdf) [meta-analysis n=1894, citation=349 (GS, February 2022)]. [Schimmack 2017](https://replicationindex.com/2017/09/04/the-power-of-the-pen-paradigm-a-replicability-analysis/) [replicability analysis _k_=19, citation=0 (GS, March 2023)]. * Original effect size: Study 1: _d_ = 0.82, _d_ = 0.43 (0.82 out of 9). -* Replication effect size: Wagenmakers et al.: 0.03 out of 9, CI overlapping 0. Coles et al.: [A meta-analysis of 138 studies](https://psyarxiv.com/svjru/) finds _d_= 0.2 [0.14, 0.26] with an absurdly low p value, and doesn't find publication bias in studies of emotional experience (it does find it in studies of affective judgements). But this latter point simply can't be right. Given _d_ = 0.2 and the convention of targeting 80% power to detect a real phenomenon, you would need very high sample sizes, n > 500. And almost all of the included studies are N < 100. All reported in Coles et al.: Andréasson & Dimberg: _d_=-0.22. Andréasson: _d_=-0.05 to _d_ =0.49. Baumeister et al.: _d_=0.63 to _d_ =1.26. Bodenhausen et al.: _d_ =0.55. Bush et al.: _d_ =0.16. Butler et al.: _d_ =-0.1 to _d_ =-0.83. Butler et al.: _d_ =-0.03. Cai et al.: _d_ =-0.08. Ceschi & Scherer: _d_ =0.74. Clapp: _d_ =0.08 to _d_ =0.69. Laird & Crosby: _d_ =-0.13 to _d_ =0.35. Davey et al.: _d_ =-0.25 to _d_ =0.73. Davis: _d_ =-0.19 to _d_ =0.99. Davis: _d_ =-0.19 to _d_ =0.87. Davis et al.: _d_ =0.07 to _d_ =0.51. Davis et al.: _d_ =-0.15 to _d_ =0.1. Davis et al.: _d_ =-0.16. Demaree et al.: _d_ =0.16 to _d_ =0.62. Demaree et al.: _d_ =-0.64 to _d_ =0.06. Dillon et al.: _d_ =0.11. Dimberg & Söderkvist: _d_ =0.1 to 0.51. Duncan & Laird:_ d_ =0.38 to _d_ =0.51. Duncan & Laird: _d_ =0.44 to _d_ =0.59. Dzokoto et al.: _d_ =0.2 to _d_ =1.07. Flack, Laird & Cavallaro: _d_ =-0.49 to _d_ =1.31. Flack, Laird & Cavallaro: _d_ =0.29 to _d_ =1.41. Flack: _d_ =0.35 to _d_ =0.72. Gan et al.: _d_ =-0.11. Goldin et al.: _d_ =0.8. Gross & Levenson: _d_ =0.04 to _d_ =0.37. Gross: _d_ =-0.23 to _d_ =0.37. Gross: _d_ =0.18. Harris: _d_ =0.07. Hawk et al.: _d_ =0.85. Helt & Fein: _d_ =0.42. Hendricks & Buchanan: _d_ =-0.08. Hendricks: _d_ =0.02. Henry et al.: _d_ =-0.49 to _d_ =0.25. Henry et al.: _d_ =-0.05 to d_ d_ 0.53. Hess et al.: _d_ =-0.28 to _d_ =0.14. Hofmann et al.: _d_ =-0.03. Ito et al.: _d_ =-0.39 to _d_ =-0.25. Kalokerinos et al.: _d_ =-0.06 to _d_ =1.32. Kao et al.: _d_ =-0.67 to _d_ =0.98. Kircher et al.: _d_ =1.14 to _d_ =1.89. Korb et al.: _d_ =0.21. Labott & Teleha: _d_ =0.04 to _d_ =0.91. Laird: _d_ =0.13 to _d_ =0.55. Lalot et al.: _d_ =-0.17. Larsen et al.: _d_ =0.43. Lee: _d_ =-0.27 to _d_ =0.48. Lewis & Bowler: _d_ =1.35. Lewis: _d_ =0.56 to _d_ =0.71. Ma: _d_ =-0.21. Maldonado et al.: _d_ =0.12. Marmolejo-Ramos & Dunn: _d_ =-0.07 to _d_ =0.38. Martijn et al.: _d_ =-0.24. McCanne & Anderson: _d_ =-2.16 to _d_ =4.73. McCaul et al.: _d_ =0.25. McIntosh et al.: _d_ =0.54. Meeten et al.: _d_ =0.49. Miyamoto: _d_ =0.49 to _d_ =0.17. Moore & Zoellner: _d_ =-0.87. Kappas: _d_ =0.08 to _d_ =0.74. Ohira & Kurono: _d_ =-1.38 to _d_ =1.23. Paredes et al.: _d_ =0.85. Paul et al.: _d_ =0.91. Pedder et al.: _d_ =0.07 to _d_ =0.22. Phillips et al.: _d_ =0.08 to _d_ =0.18. Reisenzein & Studtmann: _d_ =-0.08 to _d_ =0.034. Richards, Butler & Gross: _d_ =-0.12 to _d_ = 0.19. Richards & Gross: _d_ =-0.1 to d=0.36. Richards & Gross: _d_ =-0.12 to _d_ =0.39. Richards & Gross: _d_ =0.34. Roberts et al.: _d_ =0.07. Robinson & Demaree: _d_ =-0.04 to _d_ =0. Roemer: _d_ =0.29 to _d_ =0.58. Rohrmann et al.: _d_ =0.13 to _d_ =0.16. Rummer et al.: _d_ =0.46 to _d_ =0.57. Schmeichel , Vohs, & Baumeister: _d_ =-0.23. Schmeichel et al.: _d_ =0.1. Söderkvist & Dimberg (unpublished): _d_ =0.36. Söderkvist et al.: _d_ =0.17 to _d_ =0.34. Soussignan: _d_ =-0.17 to _d_ =1.11. Stel et al.: _d_ =1 to _d_ =1.11. Strack et al.: _d_ =-0.51 to _d_ =0.55. Tamir et al.: _d_ =-0.16. Tourangeau & Ellsworth: _d_ =0.3. Trent: _d_ =-0.22 to _d_ =-0.06. Vieillard et al.: _d_ =-0.12 to _d_ =0.66. Wagenmakers et al.: _d_ =-0.17 to _d_ =0.25. Wittmer: _d_ =-0.36 to _d_ =-0.21. Yartz: _d_ =-0.18 to _d_ =0.5. Zajonc et al.: _d_ =0.31 to _d_ =1.27. Zariffa et al.: _d_ =-0.57 to _d_ =-0.14. Zhu et al.: _d_ =1.74. [Schimmack](https://replicationindex.com/2017/09/04/the-power-of-the-pen-paradigm-a-replicability-analysis/): strong evidence of publication bias on a subset of these papers, using a proper power analysis. Direct replications of the original pen-in-mouth protocol fail; but new conceptual replications appear to work. [Coles et al.:](https://www.nature.com/articles/s41562-022-01458-9) effect is an absolute 5% increase in a happiness metric. +* Replication effect size: Wagenmakers et al.: 0.03 out of 9, CI overlapping 0. Many Smiles (Coles et al. 2022): facial mimicry and voluntary facial action both increased happiness; evidence was "less conclusive" for the pen-in-mouth task. Coles et al.: [A meta-analysis of 138 studies](https://psyarxiv.com/svjru/) finds _d_= 0.2 [0.14, 0.26] with an absurdly low p value, and doesn't find publication bias in studies of emotional experience (it does find it in studies of affective judgements). But this latter point simply can't be right. Given _d_ = 0.2 and the convention of targeting 80% power to detect a real phenomenon, you would need very high sample sizes, n > 500. And almost all of the included studies are N < 100. All reported in Coles et al.: Andréasson & Dimberg: _d_=-0.22. Andréasson: _d_=-0.05 to _d_ =0.49. Baumeister et al.: _d_=0.63 to _d_ =1.26. Bodenhausen et al.: _d_ =0.55. Bush et al.: _d_ =0.16. Butler et al.: _d_ =-0.1 to _d_ =-0.83. Butler et al.: _d_ =-0.03. Cai et al.: _d_ =-0.08. Ceschi & Scherer: _d_ =0.74. Clapp: _d_ =0.08 to _d_ =0.69. Laird & Crosby: _d_ =-0.13 to _d_ =0.35. Davey et al.: _d_ =-0.25 to _d_ =0.73. Davis: _d_ =-0.19 to _d_ =0.99. Davis: _d_ =-0.19 to _d_ =0.87. Davis et al.: _d_ =0.07 to _d_ =0.51. Davis et al.: _d_ =-0.15 to _d_ =0.1. Davis et al.: _d_ =-0.16. Demaree et al.: _d_ =0.16 to _d_ =0.62. Demaree et al.: _d_ =-0.64 to _d_ =0.06. Dillon et al.: _d_ =0.11. Dimberg & Söderkvist: _d_ =0.1 to 0.51. Duncan & Laird:_ d_ =0.38 to _d_ =0.51. Duncan & Laird: _d_ =0.44 to _d_ =0.59. Dzokoto et al.: _d_ =0.2 to _d_ =1.07. Flack, Laird & Cavallaro: _d_ =-0.49 to _d_ =1.31. Flack, Laird & Cavallaro: _d_ =0.29 to _d_ =1.41. Flack: _d_ =0.35 to _d_ =0.72. Gan et al.: _d_ =-0.11. Goldin et al.: _d_ =0.8. Gross & Levenson: _d_ =0.04 to _d_ =0.37. Gross: _d_ =-0.23 to _d_ =0.37. Gross: _d_ =0.18. Harris: _d_ =0.07. Hawk et al.: _d_ =0.85. Helt & Fein: _d_ =0.42. Hendricks & Buchanan: _d_ =-0.08. Hendricks: _d_ =0.02. Henry et al.: _d_ =-0.49 to _d_ =0.25. Henry et al.: _d_ =-0.05 to d_ d_ 0.53. Hess et al.: _d_ =-0.28 to _d_ =0.14. Hofmann et al.: _d_ =-0.03. Ito et al.: _d_ =-0.39 to _d_ =-0.25. Kalokerinos et al.: _d_ =-0.06 to _d_ =1.32. Kao et al.: _d_ =-0.67 to _d_ =0.98. Kircher et al.: _d_ =1.14 to _d_ =1.89. Korb et al.: _d_ =0.21. Labott & Teleha: _d_ =0.04 to _d_ =0.91. Laird: _d_ =0.13 to _d_ =0.55. Lalot et al.: _d_ =-0.17. Larsen et al.: _d_ =0.43. Lee: _d_ =-0.27 to _d_ =0.48. Lewis & Bowler: _d_ =1.35. Lewis: _d_ =0.56 to _d_ =0.71. Ma: _d_ =-0.21. Maldonado et al.: _d_ =0.12. Marmolejo-Ramos & Dunn: _d_ =-0.07 to _d_ =0.38. Martijn et al.: _d_ =-0.24. McCanne & Anderson: _d_ =-2.16 to _d_ =4.73. McCaul et al.: _d_ =0.25. McIntosh et al.: _d_ =0.54. Meeten et al.: _d_ =0.49. Miyamoto: _d_ =0.49 to _d_ =0.17. Moore & Zoellner: _d_ =-0.87. Kappas: _d_ =0.08 to _d_ =0.74. Ohira & Kurono: _d_ =-1.38 to _d_ =1.23. Paredes et al.: _d_ =0.85. Paul et al.: _d_ =0.91. Pedder et al.: _d_ =0.07 to _d_ =0.22. Phillips et al.: _d_ =0.08 to _d_ =0.18. Reisenzein & Studtmann: _d_ =-0.08 to _d_ =0.034. Richards, Butler & Gross: _d_ =-0.12 to _d_ = 0.19. Richards & Gross: _d_ =-0.1 to d=0.36. Richards & Gross: _d_ =-0.12 to _d_ =0.39. Richards & Gross: _d_ =0.34. Roberts et al.: _d_ =0.07. Robinson & Demaree: _d_ =-0.04 to _d_ =0. Roemer: _d_ =0.29 to _d_ =0.58. Rohrmann et al.: _d_ =0.13 to _d_ =0.16. Rummer et al.: _d_ =0.46 to _d_ =0.57. Schmeichel , Vohs, & Baumeister: _d_ =-0.23. Schmeichel et al.: _d_ =0.1. Söderkvist & Dimberg (unpublished): _d_ =0.36. Söderkvist et al.: _d_ =0.17 to _d_ =0.34. Soussignan: _d_ =-0.17 to _d_ =1.11. Stel et al.: _d_ =1 to _d_ =1.11. Strack et al.: _d_ =-0.51 to _d_ =0.55. Tamir et al.: _d_ =-0.16. Tourangeau & Ellsworth: _d_ =0.3. Trent: _d_ =-0.22 to _d_ =-0.06. Vieillard et al.: _d_ =-0.12 to _d_ =0.66. Wagenmakers et al.: _d_ =-0.17 to _d_ =0.25. Wittmer: _d_ =-0.36 to _d_ =-0.21. Yartz: _d_ =-0.18 to _d_ =0.5. Zajonc et al.: _d_ =0.31 to _d_ =1.27. Zariffa et al.: _d_ =-0.57 to _d_ =-0.14. Zhu et al.: _d_ =1.74. [Schimmack](https://replicationindex.com/2017/09/04/the-power-of-the-pen-paradigm-a-replicability-analysis/): strong evidence of publication bias on a subset of these papers, using a proper power analysis. Direct replications of the original pen-in-mouth protocol fail; but new conceptual replications appear to work. [Coles et al.:](https://www.nature.com/articles/s41562-022-01458-9) effect is an absolute 5% increase in a happiness metric. {{< /spoiler >}} * **Positive affirmation on mood**. Positive self-statements boost mood for people with high self-esteem and reduce mood for people with low self-esteem. @@ -2410,11 +2410,11 @@ You can find a list of all effects we are working on [here](https://docs.google. * **Neonate imitation**. Babies are born with the ability to imitate. {{< spoiler text="Statistics" >}} -* Status: NA +* Status: not replicated (no imitation in a large longitudinal study or in two direct replications) * Original paper: ‘Imitation of facial and manual gestures by human neonates’, [Meltzoff and Moore 1977](https://www.science.org/doi/pdf/10.1126/science.198.4312.75); observational and experimental design, 2 studies with: Study 1: n=6, Study 2: n=12. [citation=5311 (GS, December 2021)]​. -* Critiques: [Oostenbroek et al. 2016](https://doi.org/10.1016/j.cub.2016.03.047) [n=106, citations=259 (GS, December 2021)]. +* Critiques: [Oostenbroek et al. 2016](https://doi.org/10.1016/j.cub.2016.03.047) [n=106, citations=259 (GS, December 2021)]. [Koepke et al. 1983](https://doi.org/10.1016/S0163-6383(83)80012-5) [two direct replications]. * Original effect size: Not reported​. -* Replication effect size: Not reported. +* Replication effect size: Oostenbroek et al.: infants were as likely to produce the gestures after control models as after matching models (not replicated). Koepke et al.: both replications failed. {{< /spoiler >}} * **Violent media content on aggression**. Violence content in media can affect people to be more aggressive. Notably, the studies of this effect differ by media (TV, games, etc.) and whether long, medium, or short-term effects have been investigated. The variety of methods/tests further complicates the literature. Distinct media types are marked for each reference below. @@ -2446,11 +2446,11 @@ You can find a list of all effects we are working on [here](https://docs.google. * **Abstract rule learning**. After training on strings on stimuli generated from a particular repetition pattern (e.g., AAB), young infants attend longer to novel stimuli generated from a novel pattern (e.g., ABA) than novel stimuli generated from the original pattern. {{< spoiler text="Statistics" >}} -* Status: replicated +* Status: mixed (positive meta-analysis; a four-lab close replication found no effect) * Original paper: [‘Rule learning by seven-month-old infants’](https://www.science.org/doi/10.1126/science.283.5398.77), Marcus et al. 1999; preferential looking time, Experiment 1: n=16. [Citations=1707 (GS, February 2023)]. -* Critiques: [Rabagliati et al. 2018](https://doi.org/10.1111/desc.12704) [meta-analysis, n=1318, citations=53 (GS, February 2023)]. +* Critiques: [Rabagliati et al. 2018](https://doi.org/10.1111/desc.12704) [meta-analysis, n=1318, citations=53 (GS, February 2023)]. [Geambașu et al. 2022](https://doi.org/10.1111/desc.13244) [close replication across four labs with the original stimuli, n = 96]. * Original effect size: _η2p_=0.647 [0.252, 0.789]. -* Replication effect size: Rabagliati et al.: _g_=0.25 [0.09, 0.40]. +* Replication effect size: Rabagliati et al.: _g_=0.25 [0.09, 0.40]. Geambașu et al.: no difference in looking times between consistent and inconsistent test patterns, _F_(1, 94) = 0.155, _p_ = .695 (not replicated). {{< /spoiler >}} * **Theory of mind in below-4 year olds** (false belief). Children under four years of age fail at theory of mind tasks. @@ -2678,11 +2678,11 @@ You can find a list of all effects we are working on [here](https://docs.google. * **Choosing versus rejecting** (Framing effects, compatibility principle). People are inconsistent in their preferences when faced with choosing versus rejecting decision-making scenarios. {{< spoiler text="Statistics" >}} -* Status: not replicated +* Status: mixed (with the original materials, Many Labs 2 found a significant opposite effect and a close replication found none; updated materials replicated the original) * Original paper: ‘[Choosing versus rejecting: Why some options are both better and worse than others](https://doi.org/10.3758/BF03197186)’, Shafir 1993; 8 experiments with between-subjects design (i.e., two between-subjects conditions), across 8 studies sample size ranged from 170 to 398. [citations=779 (GS, July 2022)]​. -* Critiques: [Chandrashekar et al. 2021](https://journal.sjdm.org/20/200131/jdm200131.html) [n = 1026, citations = 1 (GS, July 2022)]. Proposed and tested alternative theoretical predictions to that of Shafir 1993.[Ganzach 1995](https://doi.org/10.1006/obhd.1995.1036) [n = 41 & 96, citations = 94 (GS, July 2022)]. [Wedell 1997 ](https://doi.org/10.3758/BF03211332)[n1=225, n2= 125, citations = 79(GS, July 2022)]. +* Critiques: [Chandrashekar et al. 2021](https://journal.sjdm.org/20/200131/jdm200131.html) [n = 1026, citations = 1 (GS, July 2022)]. [Klein et al. 2018](https://doi.org/10.1177/2515245918810225) [Many Labs 2, custody problem, n = 7,901]. [Shafir and Cheek 2024](https://doi.org/10.1017/jdm.2023.48) [two preregistered experiments with updated custody problems]. Proposed and tested alternative theoretical predictions to that of Shafir 1993.[Ganzach 1995](https://doi.org/10.1006/obhd.1995.1036) [n = 41 & 96, citations = 94 (GS, July 2022)]. [Wedell 1997 ](https://doi.org/10.3758/BF03211332)[n1=225, n2= 125, citations = 79(GS, July 2022)]. * Original effect size: _d_= 0.22 to 0.51​. -* Replication effect size: Chandrashekar et al.: _d_ = 0.01. Ganzach: Experiment 1 - _ηp_ ² = 0.065 (calculated from the reported _F_(1,39)=6.6, _p_<.01 using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)), Experiment 2 - _ηp_ ²= 0.105 (calculated from the reported _F_(1,94)=11,_p_<.001 using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)). +* Replication effect size: Chandrashekar et al.: _d_ = −0.01 [−0.06, 0.03]. Many Labs 2: _d_ = −0.13 [−0.18, −0.09], opposite direction. Shafir and Cheek: choose-plus-reject totals of 128% and 119%, both _p_ < .001 (replicated). Ganzach: Experiment 1 - _ηp_ ² = 0.065 (calculated from the reported _F_(1,39)=6.6, _p_<.01 using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)), Experiment 2 - _ηp_ ²= 0.105 (calculated from the reported _F_(1,94)=11,_p_<.001 using this[ conversion](https://haiyangjin.github.io/2020/05/eta2d/#from-cohens-d-to-partial-eta-squared)). {{< /spoiler >}} * **Conjunction bias** (conjunction fallacy). The fallacy consists of judging the conjunction of two events as more likely than any of the two specific events, violating one of the most fundamental tenets of probability theory. @@ -3033,10 +3033,10 @@ You can find a list of all effects we are working on [here](https://docs.google. * **Last place aversion**. A phenomenon where individuals are averse to being in last place and choose gambles with the potential to move them out of last place that they reject when randomly placed in other parts of the distribution. {{< spoiler text="Statistics" >}} * Status: replicated -* Original paper: ‘[Last-Place Aversion”: Evidence and Redistributive Implications](https://doi.org/10.1002/adhm.201970054)’, Kuziemko et. al. 2014; laboratory experiment, N=84. [citations=358(GS, March 2023)]​. -* Critiques: [Bull 2020](https://pubsonline.informs.org/doi/abs/10.1287/mnsc.2020.3619) [Study 1: n=1144, Study 2-4: n=1203, citations= 30 (GS, March 2023)] +* Original paper: ‘[Last-Place Aversion”: Evidence and Redistributive Implications](https://doi.org/10.1093/qje/qjt035)’, Kuziemko et. al. 2014; laboratory experiment, N=84. [citations=358(GS, March 2023)]​. +* Critiques: [Buell 2021](https://doi.org/10.1287/mnsc.2020.3619) [Study 1: n=1144, Study 2-4: n=1203, citations= 30 (GS, March 2023)] * Original effect size: Paper does not provide enough information to convert to effect size. Lottery experiment (Appendix Table 2): Relevant coefficient: "Last or fifth place", Coefficient value: 0.448, P-value: < 0.01. -* Replication effect size: Bull: Study 1: Observational analysis of customers queuing at a grocery store, where the author recorded the queue positions, wait times, and switching and abandonment behaviours of 1,144 customers. The results showed that being last in line doubled the probability of switching queues and quadrupled the chances of leaving the line altogether. The last-place indicator has a coefficient of 1.255 with a p-value < 0.05. This suggests that, holding all else constant, customers were 3.5 times more likely to switch queues when they were in the last place compared to having a single person waiting in line behind them. Studies 2-4: All were online experiments participants waited in a virtual queue for a chance to win a gift card. Show that being in last place increased 1)reduced wait satisfaction, increased abandonment rates 2) increased perceived value of the service, reduced 3) queue transparency moderated effects. +* Replication effect size: Buell: Study 1: Observational analysis of customers queuing at a grocery store, where the author recorded the queue positions, wait times, and switching and abandonment behaviours of 1,144 customers. The results showed that being last in line doubled the probability of switching queues and quadrupled the chances of leaving the line altogether. The last-place indicator has a coefficient of 1.255 with a p-value < 0.05. This suggests that, holding all else constant, customers were 3.5 times more likely to switch queues when they were in the last place compared to having a single person waiting in line behind them. Studies 2-4: All were online experiments participants waited in a virtual queue for a chance to win a gift card. Show that being in last place increased 1)reduced wait satisfaction, increased abandonment rates 2) increased perceived value of the service, reduced 3) queue transparency moderated effects. {{< /spoiler >}} * **Ikea effect**. When compared to objectively similar goods not produced by themselves, consumers place a higher value on goods they have assembled. Consumers show a higher willingness-to-pay when they assemble products themselves. @@ -3117,7 +3117,7 @@ You can find a list of all effects we are working on [here](https://docs.google. * **Scarcity effect - _Brand attitudes_**. Observing luxury brand consumers whose consumption arises from unearned financial resources reduces observers’ brand attitudes when observers place a high value on fairness.​ {{< spoiler text="Statistics" >}} -* Status: replicated (NS); reported effects were not replicated, while a non-reported effect remained NS: +* Status: not replicated (the key resource × fairness interaction and the mediation were not supported) * Original paper: ‘[Did they earn it? Observing unearned luxury consumption decreases brand attitude when observers value fairness](https://doi.org/10.1002/jcpy.1028)’, Lee et al. 2018; experimental design, n=164. [citation=23(GS, November 2021)]​. * Critiques: [O’Donnell et al. 2021](https://doi.org/10.1073/pnas.2103313118) [n=401, citations=0(GS, November 2021)]. * Original effect size: _r_ = .128. @@ -3182,9 +3182,9 @@ You can find a list of all effects we are working on [here](https://docs.google. {{< spoiler text="Statistics" >}} * Status: replicated * Original paper:[ ‘Less is better: When low-value options are valued more highly than high-value options](https://doi.org/10.1002/(SICI)1099-0771(199806)11:2%3C107::AID-BDM292%3E3.0.CO;2-Y)’, Hsee 1998; between-subjects manipulation of lower amount with more complete/full vs. higher amount with less complete/broken/full, Study 1 n=83, Study 2 n= 69, Study 3 n=98, Study 4 n=104. [citations=491(GS, January 2023)]. -* Critique: Study 1 was also replicated in [Klein et al. 2018](https://doi.org/10.1177/2515245918810225) [aggregate replication sample N=7,646, citations=779(GS, February 2023)]. [Vonasch et al 2023](https://osf.io/ubn2f) [preprint, Study 1 n=132, Study 2 n=133, Study 4 n = 131, citations=0(GS, January 2023)]. +* Critique: Study 1 was also replicated in [Klein et al. 2018](https://doi.org/10.1177/2515245918810225) [aggregate replication sample N=7,646, citations=779(GS, February 2023)]. [Vonasch et al 2023](https://doi.org/10.1525/collabra.77859) [preregistered replication of Studies 1, 2 and 4, N = 403, citations=0(GS, January 2023)]. * Original effect sizes: Study 1: _d_ = 0.70 [0.24, 1.15]; Study 2: _d_ = 0.74 [0.12, 1.35]; Study 4: _d_ = 0.97 [0.43, 1.50]. -* Replication effect sizes: Klein et al.: Study 1 also replicated with _d_=.78 [.74, .83]. Vonasch et al.: Study 1: _d_ = 0.99 [0.72, 1.25]; Study 2: _d_ = 0.32 [0.05, 0.56]; Study 4: _d_ = 76 [.50, 1.02]. +* Replication effect sizes: Klein et al.: Study 1 also replicated with _d_=.78 [.74, .83]. Vonasch et al.: Study 1: _d_ = 0.99 [0.72, 1.25]; Study 2: _d_ = 0.32 [0.07, 0.56]; Study 4: _d_ = 0.76 [0.50, 1.02]. {{< /spoiler >}} * **Left digit bias**. The leftmost digit of a number disproportionately influences decision making. Consumers judge the difference between $4.00 and $2.99 to be larger than that between $4.01 and $3.00, even though the numeric differences are identical; it is this change in the left digit, rather than the one cent drop, that affects the magnitude perception. From ef024f04df9d3fbecca7feeddf8165214cb440e5 Mon Sep 17 00:00:00 2001 From: Lukas Wallrich Date: Fri, 13 Nov 2026 11:50:37 +0000 Subject: [PATCH 6/9] Reversals: apply independently re-checked labels After a second Sol review: moral priming becomes mixed (adds Huang 2014), men's preference for competition becomes mixed (adds Carlsson et al. 2020), and last place aversion becomes a scoped NA (adds Camerer et al. 2016's failed redistribution replication). Neonate imitation wording calibrated to absence of evidence. Co-Authored-By: Claude Opus 5.5 --- content/reversals/reversals.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/content/reversals/reversals.md b/content/reversals/reversals.md index e95bc1227d5..bbadc352630 100644 --- a/content/reversals/reversals.md +++ b/content/reversals/reversals.md @@ -203,11 +203,11 @@ You can find a list of all effects we are working on [here](https://docs.google. * **Moral priming (cleanliness)**. Participants exposed to physical cleanliness were shown to reduce the severity of their moral judgments. Direct, well-powered replications did not find evidence for the phenomenon. {{< spoiler text="Statistics" >}} -* Status: not replicated +* Status: mixed (large direct replications failed; one study found support only when participants paid little attention to the primes) * Original paper: [With a Clean Conscience: Cleanliness Reduces the Severity of Moral Judgments](https://doi.org/10.1111/j.1467-9280.2008.02227.x), Schnall, Benton, and Harvey, 2008; 2 experiments with Study 1: n = 40, Study 2: n = 44. [citations=645 (GS November 2021)]. -* Critiques: [Johnson et al.](https://econtent.hogrefe.com/doi/full/10.1027/1864-9335/a000186) 2014, [Study 1: n = 208, Study 2: n = 126. citations=128(GS November 2021)]. +* Critiques: [Johnson et al.](https://econtent.hogrefe.com/doi/full/10.1027/1864-9335/a000186) 2014, [Study 1: n = 208, Study 2: n = 126. citations=128(GS November 2021)]. [Huang 2014](https://doi.org/10.3389/fpsyg.2014.01276) [Study 2: n = 440; Study 2a, a direct replication of Study 2: n = 436]. * Original effect size: Study 1: d = -0.60, 95% CI [-1.23, 0.04]; Study 2: d = -0.85, 95% CI [-1.47, -0.22] -* Replication effect size: Study 1: d = -0.01, 95% CI [-0.28, 0.26]; Study 2: d = 0.01, 95% CI [-0.34, 0.36] +* Replication effect size: Johnson et al.: Study 1: d = -0.01, 95% CI [-0.28, 0.26]; Study 2: d = 0.01, 95% CI [-0.34, 0.36]. Huang, Study 2: low response effort, d = -0.29, p = .04; high response effort, d = 0.19, p = .16. {{< /spoiler >}} * **Moral priming (contemplation)**. Participants exposed to a moral-reminder prime would demonstrate reduced cheating. @@ -2410,7 +2410,7 @@ You can find a list of all effects we are working on [here](https://docs.google. * **Neonate imitation**. Babies are born with the ability to imitate. {{< spoiler text="Statistics" >}} -* Status: not replicated (no imitation in a large longitudinal study or in two direct replications) +* Status: not replicated (two direct replications and a large longitudinal study found no evidence of imitation) * Original paper: ‘Imitation of facial and manual gestures by human neonates’, [Meltzoff and Moore 1977](https://www.science.org/doi/pdf/10.1126/science.198.4312.75); observational and experimental design, 2 studies with: Study 1: n=6, Study 2: n=12. [citation=5311 (GS, December 2021)]​. * Critiques: [Oostenbroek et al. 2016](https://doi.org/10.1016/j.cub.2016.03.047) [n=106, citations=259 (GS, December 2021)]. [Koepke et al. 1983](https://doi.org/10.1016/S0163-6383(83)80012-5) [two direct replications]. * Original effect size: Not reported​. @@ -3032,11 +3032,11 @@ You can find a list of all effects we are working on [here](https://docs.google. * **Last place aversion**. A phenomenon where individuals are averse to being in last place and choose gambles with the potential to move them out of last place that they reject when randomly placed in other parts of the distribution. {{< spoiler text="Statistics" >}} -* Status: replicated +* Status: NA (no follow-up tests the gambling result; a direct replication of the paper's redistribution game failed, and the queueing study is a conceptual extension) * Original paper: ‘[Last-Place Aversion”: Evidence and Redistributive Implications](https://doi.org/10.1093/qje/qjt035)’, Kuziemko et. al. 2014; laboratory experiment, N=84. [citations=358(GS, March 2023)]​. -* Critiques: [Buell 2021](https://doi.org/10.1287/mnsc.2020.3619) [Study 1: n=1144, Study 2-4: n=1203, citations= 30 (GS, March 2023)] +* Critiques: [Camerer et al. 2016](https://doi.org/10.1126/science.aaf0918) [replication of Experiment 2, redistribution game with 6-person groups, n = 144]. [Buell 2021](https://doi.org/10.1287/mnsc.2020.3619) [Study 1: n=1144, Study 2-4: n=1203, citations= 30 (GS, March 2023)] * Original effect size: Paper does not provide enough information to convert to effect size. Lottery experiment (Appendix Table 2): Relevant coefficient: "Last or fifth place", Coefficient value: 0.448, P-value: < 0.01. -* Replication effect size: Buell: Study 1: Observational analysis of customers queuing at a grocery store, where the author recorded the queue positions, wait times, and switching and abandonment behaviours of 1,144 customers. The results showed that being last in line doubled the probability of switching queues and quadrupled the chances of leaving the line altogether. The last-place indicator has a coefficient of 1.255 with a p-value < 0.05. This suggests that, holding all else constant, customers were 3.5 times more likely to switch queues when they were in the last place compared to having a single person waiting in line behind them. Studies 2-4: All were online experiments participants waited in a virtual queue for a chance to win a gift card. Show that being in last place increased 1)reduced wait satisfaction, increased abandonment rates 2) increased perceived value of the service, reduced 3) queue transparency moderated effects. +* Replication effect size: Camerer et al.: marginal effect of second-to-last place on giving 0.045 (_p_ = .154) vs −0.116 (_p_ = .070) in the original (not replicated). Buell: Study 1: Observational analysis of customers queuing at a grocery store, where the author recorded the queue positions, wait times, and switching and abandonment behaviours of 1,144 customers. The results showed that being last in line doubled the probability of switching queues and quadrupled the chances of leaving the line altogether. The last-place indicator has a coefficient of 1.255 with a p-value < 0.05. This suggests that, holding all else constant, customers were 3.5 times more likely to switch queues when they were in the last place compared to having a single person waiting in line behind them. Studies 2-4: All were online experiments participants waited in a virtual queue for a chance to win a gift card. Show that being in last place increased 1)reduced wait satisfaction, increased abandonment rates 2) increased perceived value of the service, reduced 3) queue transparency moderated effects. {{< /spoiler >}} * **Ikea effect**. When compared to objectively similar goods not produced by themselves, consumers place a higher value on goods they have assembled. Consumers show a higher willingness-to-pay when they assemble products themselves. @@ -3530,11 +3530,11 @@ Critiques: [Gelman and Loken 2013](http://www.stat.columbia.edu/~gelman/research * **Men’s preference for competition**. Men are more likely to select tournaments than women, because women tend to avoid competition and men look for competition. {{< spoiler text="Statistics" >}} -* Status: not replicated +* Status: mixed (Price's direct replication found no gender gap; a larger replication with Chinese adults found one) * Original paper: ‘[Do women shy away from competition? Do men compete too much?’, ](https://web.stanford.edu/~niederle/Niederle.Vesterlund.QJE.2007.pdf)Niederle and Vesterlund 2007; within-subject experiment, n=80. [citations=4001(GS, February 2023)]​. -* Critiques: [Price 2020](http://www.accessecon.com/Pubs/EB/2020/Volume40/EB-20-V40-I2-P132.pdf) direct replication of Niederle & Vesterlund study [n=60, citations=23 (GS, February2023)]. +* Critiques: [Price 2020](http://www.accessecon.com/Pubs/EB/2020/Volume40/EB-20-V40-I2-P132.pdf) direct replication of Niederle & Vesterlund study [n=60, citations=23 (GS, February2023)]. [Carlsson et al. 2020](https://doi.org/10.1016/j.joep.2020.102312) [adults in three Chinese cities, n = 854]. * Original effect size: Man more likely to select tournaments then women – probit regression coefficients in various regression models -.38 to -.16. -* Replication effect size: Price: A female in the replication has a 22.4 percentage point higher probability (58.0% versus 35.6%) of entering into the tournament than a female in NV (original study) after controlling for task performance; probit regression coefficient for the NV*Female interaction term from -0.65 (non-significant) to -0.99 (significant) [not replicated]. +* Replication effect size: Price: A female in the replication has a 22.4 percentage point higher probability (58.0% versus 35.6%) of entering into the tournament than a female in NV (original study) after controlling for task performance; probit regression coefficient for the NV*Female interaction term from -0.65 (non-significant) to -0.99 (significant) [not replicated]. Carlsson et al.: 42.8% of men vs 19.7% of women chose the tournament, _p_ < .001 (replicated). {{< /spoiler >}} * **Orgasm gap** (Orgasm equality). There is a gendered orgasm gap, with men experiencing orgasm more frequently than women in heterosexual sexual encounters.​ From cec54d1d999949f7d1dfb0fb176d696103984490 Mon Sep 17 00:00:00 2001 From: Lukas Wallrich Date: Fri, 13 Nov 2026 12:17:10 +0000 Subject: [PATCH 7/9] Reversals: last place aversion covers both original experiments A second direct replication (Martinangeli & Windsteiger 2021) also found no last-place aversion, so the entry is not NA. Its description now covers the original's gamble and redistribution results, and the status is mixed: two direct replications failed; a modified design and a queueing study found support. Co-Authored-By: Claude Opus 5.5 --- content/reversals/reversals.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/content/reversals/reversals.md b/content/reversals/reversals.md index bbadc352630..823dc594762 100644 --- a/content/reversals/reversals.md +++ b/content/reversals/reversals.md @@ -3030,13 +3030,13 @@ You can find a list of all effects we are working on [here](https://docs.google. * Replication effect size: Diederich et al.: Gain versus Loss effect on Risky option preference significant in two regression models, β=-0.206 and β=-0.303, respectively. (replicated). Kühberger: mean effect size for the 80 studies with Asian _d_=0.57 [0.53, 0.61] (replicated). Otterbring et al.: Study 1 - statistically significant effect of framing on participants’ choice of program (_b_ = 1.52, _Z_ = 4.80, _p_ < 0.001), such that a larger proportion of participants chose the risky program under conditions of negative (78.0%) compared to positive framing (44.0%); Study 2 - statistically significant effect of framing on participants’ choice of program (_b_ = 1.82, _Z_ = 11.13, _p_ < 0.001), such that a larger proportion of participants chose the risky program under conditions of negative (68.8%) compared to positive framing (26.6%) (replicated). Peterson and Tollefson: _d_ =0.26 [calculated from the reported chi-square and sample size, χ2 =17.41, N = 1021] (replicated). {{< /spoiler >}} -* **Last place aversion**. A phenomenon where individuals are averse to being in last place and choose gambles with the potential to move them out of last place that they reject when randomly placed in other parts of the distribution. +* **Last place aversion**. A phenomenon where individuals are averse to being in last place. In Kuziemko et al.'s experiments, people in last place chose gambles that could move them out of last place, which they rejected when randomly placed in other parts of the distribution, and people in second-to-last place were the least likely to give money to the person ranked just below them. {{< spoiler text="Statistics" >}} -* Status: NA (no follow-up tests the gambling result; a direct replication of the paper's redistribution game failed, and the queueing study is a conceptual extension) +* Status: mixed (two direct replications found no last-place aversion; a modified design and a queueing study found support) * Original paper: ‘[Last-Place Aversion”: Evidence and Redistributive Implications](https://doi.org/10.1093/qje/qjt035)’, Kuziemko et. al. 2014; laboratory experiment, N=84. [citations=358(GS, March 2023)]​. -* Critiques: [Camerer et al. 2016](https://doi.org/10.1126/science.aaf0918) [replication of Experiment 2, redistribution game with 6-person groups, n = 144]. [Buell 2021](https://doi.org/10.1287/mnsc.2020.3619) [Study 1: n=1144, Study 2-4: n=1203, citations= 30 (GS, March 2023)] +* Critiques: [Camerer et al. 2016](https://doi.org/10.1126/science.aaf0918) [replication of Experiment 2, redistribution game with 6-person groups, n = 144]. [Martinangeli and Windsteiger 2021](https://doi.org/10.1007/s10683-020-09682-8) [replication and a modified design that holds ranks fixed across rounds]. [Buell 2021](https://doi.org/10.1287/mnsc.2020.3619) [Study 1: n=1144, Study 2-4: n=1203, citations= 30 (GS, March 2023)] * Original effect size: Paper does not provide enough information to convert to effect size. Lottery experiment (Appendix Table 2): Relevant coefficient: "Last or fifth place", Coefficient value: 0.448, P-value: < 0.01. -* Replication effect size: Camerer et al.: marginal effect of second-to-last place on giving 0.045 (_p_ = .154) vs −0.116 (_p_ = .070) in the original (not replicated). Buell: Study 1: Observational analysis of customers queuing at a grocery store, where the author recorded the queue positions, wait times, and switching and abandonment behaviours of 1,144 customers. The results showed that being last in line doubled the probability of switching queues and quadrupled the chances of leaving the line altogether. The last-place indicator has a coefficient of 1.255 with a p-value < 0.05. This suggests that, holding all else constant, customers were 3.5 times more likely to switch queues when they were in the last place compared to having a single person waiting in line behind them. Studies 2-4: All were online experiments participants waited in a virtual queue for a chance to win a gift card. Show that being in last place increased 1)reduced wait satisfaction, increased abandonment rates 2) increased perceived value of the service, reduced 3) queue transparency moderated effects. +* Replication effect size: Camerer et al.: marginal effect of second-to-last place on giving 0.045 (_p_ = .154) vs −0.116 (_p_ = .070) in the original (not replicated). Martinangeli and Windsteiger: "fails to find evidence of last-place aversion in a replication of Kuziemko et al."; with ranks held fixed across rounds, support for a greater disutility from last place (abstract). Buell: Study 1: Observational analysis of customers queuing at a grocery store, where the author recorded the queue positions, wait times, and switching and abandonment behaviours of 1,144 customers. The results showed that being last in line doubled the probability of switching queues and quadrupled the chances of leaving the line altogether. The last-place indicator has a coefficient of 1.255 with a p-value < 0.05. This suggests that, holding all else constant, customers were 3.5 times more likely to switch queues when they were in the last place compared to having a single person waiting in line behind them. Studies 2-4: All were online experiments participants waited in a virtual queue for a chance to win a gift card. Show that being in last place increased 1)reduced wait satisfaction, increased abandonment rates 2) increased perceived value of the service, reduced 3) queue transparency moderated effects. {{< /spoiler >}} * **Ikea effect**. When compared to objectively similar goods not produced by themselves, consumers place a higher value on goods they have assembled. Consumers show a higher willingness-to-pay when they assemble products themselves. From 5e570147563179cd3380d71d18351f2d48cdcf13 Mon Sep 17 00:00:00 2001 From: Lukas Wallrich Date: Fri, 13 Nov 2026 12:19:21 +0000 Subject: [PATCH 8/9] Reversals: report the scarcity replication test and scope elderly priming Address review: O'Donnell et al.'s replication test is F(1, 530) = 3.59, p = .059 (n = 573 is the sample, not the test df), so "not significant" stands and the statistic is now shown. Elderly priming status no longer implies automated timing never showed slowing. Co-Authored-By: Claude Opus 5.5 --- content/reversals/reversals.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/reversals/reversals.md b/content/reversals/reversals.md index 823dc594762..3b2e3677258 100644 --- a/content/reversals/reversals.md +++ b/content/reversals/reversals.md @@ -176,7 +176,7 @@ You can find a list of all effects we are working on [here](https://docs.google. * **Elderly priming**. that hearing about old age makes people walk slower. {{< spoiler text="Statistics" >}} -* Status: not replicated (no effect with automated timing; slower walking appeared only when experimenters expected it) +* Status: not replicated (no effect with automated timing unless experimenters expected slower walking) * Original paper: '[Automaticity of social behaviour](https://scholar.google.com/scholar?cluster=3335859380278379099&hl=en&as_sdt=0,5&sciodt=0,5)', Bargh 1996; 2 experiments with Study 2a: n = 30, Study 2b: n = 30. [citations = 5938(GS, October 2021)]​. * Critiques:[ Doyen 2012](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0029081#s2) [n=120, citations=757(GS, October 2021)]. [Lakens 2017](https://psyarxiv.com/3m5y9) [meta analysis: citations = 21(GS, October 2021)]. [Pashler et al. 2011](https://web.archive.org/web/20170923093613/http://www.psychfiledrawer.org/replication.php?attempt=MTU%3D) [n=66, citations=21(GS, October 2021)]. * Original effect size: not reported. ​ @@ -3094,7 +3094,7 @@ You can find a list of all effects we are working on [here](https://docs.google. * Original paper: ‘[Spending on daughters versus sons in economic recessions](https://doi.org/10.1093/jcr/ucv023)’, Durante et al. 2015; within-subjects design, n=198. [citation=57(GS, November 2021)]​. * Critiques: [O’Donnell et al. 2021](https://doi.org/10.1073/pnas.2103313118) [n=573, citations=0(GS, November 2021)]. * Original effect size: _r_= .179. -* Replication effect size: O’Donnell et al.: _r_= -.082 (not significant). +* Replication effect size: O’Donnell et al.: _r_= -.082 (_F_(1, 530) = 3.59, _p_ = .059; not significant). {{< /spoiler >}} * **Scarcity effect - _Planning_**. Consumers who feel resource constrained shift to engage in relatively more priority planning, rather than efficiency planning. From ca8edc91773ee9874d0ac1e5c4f8f365b0507243 Mon Sep 17 00:00:00 2001 From: Lukas Wallrich Date: Fri, 13 Nov 2026 12:28:17 +0000 Subject: [PATCH 9/9] Reversals: 5-HTT entry no longer reports other genes as replications Chabris et al. 2012 (intelligence) and Farrell et al. 2015 (schizophrenia) test other candidate genes, so they move from the replication line to a labelled context note. Serretti et al. 2007, an unlinked review of the HTR2A receptor gene, is removed. The label rests on Culverhouse et al. 2017 and is unchanged. Co-Authored-By: Claude Opus 5.5 --- content/reversals/reversals.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/reversals/reversals.md b/content/reversals/reversals.md index 3b2e3677258..0a99ae07ed7 100644 --- a/content/reversals/reversals.md +++ b/content/reversals/reversals.md @@ -3567,9 +3567,9 @@ Critiques: [Gelman and Loken 2013](http://www.stat.columbia.edu/~gelman/research {{< spoiler text="Statistics" >}} * Status: not replicated * Original paper: ‘[Influence of Life Stress on Depression: Moderation by a Polymorphism in the 5-HTT Gene](https://www.science.org/doi/10.1126/science.1083968), Caspi et al. 2003; quasi-experimental analysis of how genotype (3 levels: s/s, s/l, l/l) and number of stressful life events (5 levels: 0, 1, 2, 3, 4+) across various categories (employment, financial, housing, health, relationship) that occurred after 21st birthday and before 26th birthday interacted to produce depression (measured by self-reported depression symptoms, probability of major depression episode, probability of suicide ideation/attempt, and informant reports of depression), 847 26-year-old participants from the Dunedin Study (a longitudinal study of 1037 non-Maori babies born between 1972-04-01 and 1972-03-31 in New Zealand). [citations = 6223 ([Dimensions, June 2022](https://app.dimensions.ai/details/publication/pub.1051420748))]​. -* Critiques: [Chabris et al. 2012](https://pubmed.ncbi.nlm.nih.gov/23012269/) [n=samples of 5,571, 1,759, and 2,441 individuals, citations=347(GS, April 2022)] for intelligence. [Culverhouse et al. 2017](https://www.nature.com/articles/mp201744) [n = 28,252 (21 studies), meta-analysis, citations = 173 ([Crossref](https://doi.org/10.1038/mp.2017.44), June 2022)]. [No good evidence](https://slatestarcodex.com/2019/05/07/5-httlpr-a-pointed-review/) that 5-HTTLPR is strongly linked to depression, insomnia, PTSD, anxiety, and more. S [Farrell et al. 2015](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4414705/pdf/nihms653267.pdf) [n=25 historical candidate genes for schizophrenia, citations=345(GS, April 2022)] for schizophrenia. Serretti et al. 2007 [review paper, n=NA, citations=178(GS, April 2022)] for the overall phenotypic profile of HTR2A variant carriers. +* Critiques: [Culverhouse et al. 2017](https://www.nature.com/articles/mp201744) [n = 28,252 (21 studies), meta-analysis, citations = 173 ([Crossref](https://doi.org/10.1038/mp.2017.44), June 2022)]. [No good evidence](https://slatestarcodex.com/2019/05/07/5-httlpr-a-pointed-review/) that 5-HTTLPR is strongly linked to depression, insomnia, PTSD, anxiety, and more. Context on other candidate genes (these do not test the 5-HTTLPR × stress effect): [Chabris et al. 2012](https://pubmed.ncbi.nlm.nih.gov/23012269/) [n=samples of 5,571, 1,759, and 2,441 individuals, citations=347(GS, April 2022)] for intelligence; [Farrell et al. 2015](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4414705/pdf/nihms653267.pdf) [n=25 historical candidate genes for schizophrenia, citations=345(GS, April 2022)] for schizophrenia. * Original effect size: N/A (unreported by Caspi; however, an OR for Caspi’s data was calculated by [Risch et al. 2009](https://jamanetwork.com/journals/jama/article-abstract/184107) and displayed in Figure 2C approximately 1.3). -* Replication effect size: Chabris et al.: We sought to replicate published associations between g and 12 specific genetic variants (in the genes DTNBP1, CTSD, DRD2, ANKK1, CHRM2, SSADH, COMT, BDNF, CHRNA4, DISC1, APOE, and SNAP25) using data sets from three independent, well-characterized longitudinal studies. Of 32 independent tests across all three data sets, only 1 was nominally significant. Culverhouse et al.: OR = 1.05 [0.94 - 1.16]. Farrell et al.: historical candidate gene literature did not yield clear insights into the genetic basis of schizophrenia. +* Replication effect size: Culverhouse et al.: OR = 1.05 [0.94 - 1.16]. {{< /spoiler >}}