Conspiracy Consequences for Whom?
By Timothy Dorr
Thesis supervisor: dr. Bert Bakker
While conspiracies are not a new phenomenon, they have become especially prominent on social media where they spread faster and farther than ever before. Also, their consequences are becoming more apparent with the world-wide COVID protests or the January 6th insurrection in the US. While conspiracies are probably not responsible for such protests by themselves, prior research suggests that they are contributing factors. Yet, prior experimental research has one major problem: they assume that everyone has an equal chance of being exposed to conspiracies. However, the social media realm is highly customizable with some exposing themselves to conspiracies (from hereon referred to as selectors) and some that avoid it where possible (from hereon referred to as non-selectors). Luckily, Brian Gaines provided a solution to this problem by advocating for the self-selection experimental design. The rest of this blog post will elaborate on why and how a self-selection design was used in my thesis to decompose the effects of conspiracies on government trust, voting intentions, and protesting intentions for selectors and non-selectors.
Conspiracy Consequences and Self-Selection Experimental Designs
The self-selection experimental design allows a researcher to decompose treatment effects between selectors and non-selectors, but why is that necessary in the first place? When investigating a stimulus which is prone to self-selection like conspiracies, it is likely that not everyone experiences conspiracy exposure equally often. Some might find conspiracies interesting and expose themselves a lot, while others find them irritating and thus avoid them. If conspiracies now have a heterogeneous treatment effect on these groups, meaning that the effect that conspiracy exposure has on people differs depending on whether they are selectors or non-selectors, then experimental results are limited in their generalizability. Experiments give us the average effect, yet only the self-selector effect is interesting as it translates to real world situations while the non-selector effect exists exclusively in the laboratory setting. If the self-selectors are affected much more than the non-selectors, then experimental results will underestimate the consequences that exposure to conspiracies actually brings about. Similarly, if the effect of non-selectors is stronger, then the results overstate the consequences.
But how can such a design be incorporated? To add a self-selection element to the experimental design, the experiment will need all participants to be randomly allocated to two arms: a self-selection arm and a forced exposure arm. The forced exposure arm is like classical experiments. Here, participants will be allocated either to the treatment or control group. In the self-selection arm however, the participants can choose whether they want to see the treatment or the control stimulus. The assignments as well as the expected means of each group is summarized in Figure 1 below.
The Self-Selection Experiment and Heterogeneity with Distributions and Expected Means for Response Variable Y
Note. This figure is taken out of Gaines, B. J., & Kuklinski, J. H. (2011). Experimental estimation of heterogeneous treatment effects related to self‐selection. American Journal of Political Science, 55(3), 724-736.
With this information, average effects can be calculated, but the treatment effects can also be decomposed for selectors and non-selectors. For an overview of how it works in detail, read the paper by Gaines and Kulinski.
Decomposed Results of Conspiracy Consequences
One shortcoming of decomposing results is that it introduces additional uncertainty and reduces sample sizes, meaning huge samples are required to reliably detect effects. Unfortunately, due to funding constraints, my study was severely underpowered and none of the decomposed effects seen below were significantly different from zero or each other.
Results from the decomposition of the effects of conspiracy exposure on voting intention, protesting intention, and government trust
||95% CI lower
||95% CI upper
||95% CI lower
||95% CI upper
However, the results consistently point towards the effect for selectors being much stronger than for non-selectors, which could be interpreted as suggestive evidence. Significance aside, if someone voluntarily exposes themselves to conspiracies , their voting intention is predicted to decrease by 12 percentage points, while it only decreases by 0.5 percentage points for those that would usually avoid it. Therefore, the effects for selectors, who actually experience such exposure regularly, is much larger than a traditional experiment would have suggested, which also means the negative consequences for the real world are much greater. Future research should thus decompose treatment effects when investigating consequences of conspiracy theories, and other phenomena which are prone to self-selection, to increase their external validity and have results that generalize to the real world.