-
Notifications
You must be signed in to change notification settings - Fork 0
/
How to Build Bipartisan Trust in Fa.txt
223 lines (162 loc) · 62.2 KB
/
How to Build Bipartisan Trust in Fa.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
How to Build Bipartisan Trust in Fact-Checking Sites: The Effects of Asymmetric Coverage on Source Credibility∗
Abstract
Unlike conventional journalism that achieves objectivity by giving “equal weights” to both sides of the debate, fact-checking takes a more interpretive and “weight of evidence” approach. One challenge is that the way fact-checking pursues objec- tivity allows for asymmetric coverage that covers one political party more heavily than the other. These imbalances may reflect genuine asymmetries in the preva- lence of misinformation, but can also undermine trust among partisans. Findings from a preregistered experiment indicate that, compared to symmetric coverage, uncongenial asymmetry (most articles challenge in-group) leads partisans to find a source less credible. Contrary to conventional belief, Democrats react more negatively to uncongenial asymmetry than Republicans. Congenial asymmetry (most articles challenge out-group) also causes partisans to perceive the source as less credible, especially when a portion of coverage challenges their own party on polarized topics. These findings suggest more symmetric coverage of political parties can foster more bipartisan trust in fact-checking.
Keywords: fact-checking, source credibility, journalistic norms, partisan moti- vated reasoning
∗The study was preregistered prior to data collection and was deemed exempt by the university Institutional Review Board.
Political fact-checking, a genre of news reporting dedicated to assessing the accuracy of political claims, aims to overcome the shortcomings of conventional media in addressing contemporary threats such as misinformation and partisan disagreements over facts (Graves 2016). For instance, conventional reporting that centers on “he said, she said” and “both sides of the story” falls short in helping citizens learn about the factual inaccuracies in political discourse and holding politicians accountable (Dobbs 2012). To address this concern, political fact-checking adjudicates the factual accuracy of political claims, and makes evidence-based, not balance-oriented, coverage decisions (Graves 2016). These approaches of fact-checking redefine the role of journalism from a dispassionate stenographer to an engaged arbitrator. Although fact-checking has potentials to help advance informed democracy, fact-checking sites have limited audience among the public, with a greater suspicion from Republicans that fact-checkers are biased (Guess, Nyhan and Reifler 2020; Walker and Gottfried 2019). Despite previous findings that exposure to fact-checking articles can improve public knowledge and mitigate misperceptions (Gottfried et al. 2013; Nyhan et al. 2020; Wood and Porter 2019), limited public trust and usage constrain the public impact of fact-checking. In search of ways to increase benefits and relevance of fact-checking, I identify reporting practices that likely
have prevented fact-checking outlets from earning broader trust.
Understanding when partisans likely trust fact-checking sites has important implications for building healthier democracies in two ways. First, source credibility has important demo- cratic consequences in deciding which information would shape voters’ candidate assessments or policy preferences (Swire-Thompson et al. 2020; von Hohenberg and Guess 2022). Second, partisan divisions on factual beliefs can seriously undermine the legitimacy of democratic processes and obstruct deliberation (Berlinski et al. 2021; Tong et al. 2020). Seeking ways to address this concern, this study identifies coverage decisions that can help partisans converge on evidence-based news sources, fostering more informed and collaborative democracies.
Most research on fact-checking has focused on the effects of exposure to individual fact- checks on factual beliefs and candidate appraisals (e.g., Amazeen et al. 2018; Nyhan et al.
2020; Thorson 2016; Wood and Porter 2019). Largely missing from the literature is a better understanding of factors that affect the credibility of fact-checking at the source level. To advance understanding, I focus on an attribute of news sources, specifically coverage traits, and explore the possibility that the manner in which fact-checking outlets cover political parties affects the public reputation of fact-checking sites as credible sources.
To understand the nature of fact-checking coverage, I examine how the norm of objec- tivity in U.S. journalism has evolved over time (Bennett 1996; Hamilton 2006). I show that fact-checking reflects how the objectivity norm has been redefined from a descriptive, “equal weights on all sides” approach to a more interpretive, “weights of evidence” approach. I pro- pose that this reformed notion of objectivity drives asymmetric coverage of political parties in fact-checking sites.
I draw upon partisan motivated reasoning and the criteria people use to assess one- sided news coverage to theorize the impact of asymmetric coverage on partisans’ source credibility perceptions. Because partisans tend to discredit information and news outlets that challenge their own group (Druckman and McGrath 2019; Kahan 2015), I expected that compared to symmetric coverage, coverage with uncongenial asymmetry (the majority of articles challenge one’s own party) would decrease perceived source credibility. Because prior research suggests that Republicans tend to be more resistant to uncongenial news and facts (Garrett and Stroud 2014; Jost et al. 2003), I expected that uncongenial asymmetry would lower perceived credibility to a greater extent among Republicans than Democrats. When coverage has congenial asymmetry (the majority of articles challenge the opposite party), the existing literature offers mixed guidance, because its ingroup-favorable content could improve source evaluations (Stroud 2011; Peterson and Iyengar 2021), yet its violation of audience expectations for balanced coverage might worsen source assessments (Allen 1991; Flanagin, Winter and Metzger 2020). To clarify, I examined whether congenial asymmetry would increase or decrease perceived source credibility among partisans.
The findings from my preregistered experiment suggest that fact-checking sites should
reconsider the reputational consequences of asymmetrical coverage of political parties. I find that, compared to symmetric coverage, asymmetric coverage in either direction harms an organization’s reputation as a credible source. First, compared to symmetric coverage, un- congenial asymmetry reduces source credibility perceptions among both partisan groups. Unexpectedly, I find that Democrats react more negatively to uncongenial symmetry com- pared to Republicans. Second, congenial asymmetry, despite its content being favorable to one’s party, also causes partisans to find a news source as less credible. Interestingly, results further suggest that Democrats likely find congenial asymmetry less credible particularly when a portion of coverage challenges their own party on highly polarized topics, whereas Republicans find congenial asymmetry less credible regardless of specific topics.
This study highlights the importance of coverage decisions of news outlets in building source credibility. While previous studies have focused on how partisans process individual fact-checks or news articles (Amazeen et al. 2018; Nyhan et al. 2020; Wood and Porter 2019) or a message from a source with known partisan slant (Traberg and van der Linden 2022), my work shows how the relative amount of news items that target either party affects source evaluations. This endeavor is particularly important given the nature of news outlets as experience goods, whose quality consumers can assess only by reading or observing the overall content of the outlet (Hamilton 2006). Thus, source assessment is unlikely to be achieved on the basis of just a single article. This inquiry also helps us understand how partisans would assess fact-checking sites upon visiting these sites. One of the main routes through which people get exposed to fact-checking is by directly visiting these sites, as captured by web traffic data (e.g., more than one miliion visitors per day to PolitiFact during the 2012 election; 7.4 million views per day to NPR.org’s fact-checking site during the 2016 election; Graves, Nyhan and Reifler 2016; Hassan et al. 2017). Furthermore, given the relative lack of familiarity with and use of fact-checking sites among the American public (Guess, Nyhan and Reifler 2020), it is valuable to experimentally test how the overall coverage of a relatively unfamiliar source shapes credibility assessments, which would strongly predict
whether the public would continue to use and learn from the source. While fact-checking is often accepted as a reputable form of journalism (Graves, Nyhan and Reifler 2016), my work demonstrates that fact-checkers’ coverage decisions driven by evidence-based arbitration may inhibit bipartisan trust. Instead, fact-checking sources need to exercise caution when evidence leads them to asymmetrically cover political parties and look for ways to further signal the impartiality, value, and rigor of their reporting.
Reforming Journalism through Fact-checking: The Norm of Objectivity The fact-checking movement reflects the evolution of journalistic norms, which refer to a set of rules that guide news content decisions, such as objectivity, transparency, accountability, and efficiency (Bennett 1996). Journalistic norms can be reconfigured in response to shifting
environments, such as the transformation of technology (e.g., the shift from paper to online news), growing distrust in the news media, and an increasingly fractured media landscape (e.g., the rise of partisan media, social media, and fabricated news) (Hayes, Singer and Ceppos 2007; Graves, Nyhan and Reifler 2016). To better understand the motivations behind the fact-checking movement, I specifically focus on how the norm of objectivity has been redefined over time in the U.S.
Ever since the partisan press of the 19th century was displaced by the objective journal- ism in the 20th century, the objectivity norm has cast journalists as independent of politics and as a “passive mirror” of society (Graves, Nyhan and Reifler 2016; Hamilton 2006; Ko- vach and Rosenstiel 2014). The independent media, dominant in the U.S. in the 1990s, was characterized by the emphasis on objectivity represented by the practice of giving “equal weights” on all sides and “he said, she said” reporting (Hiles and Hinnant 2014; Graves, Nyhan and Reifler 2016). To appeal to readers of diverse political affiliations and thereby increase profits, U.S. newspapers increasingly abandoned party affiliations, increased soft news coverage (e.g., entertainment, sports), and claimed a nonpartisan stance by covering public affairs in a balanced manner (Hamilton 2006). Under this norm, the broadcast media were heavily governed by “equal time” requirement to dedicate similar amount of airtime
to Democrats and Republicans (D’Alessio and Allen 2000). The objectivity norm under the objective news paradigm during the 1990s can be described as descriptive objectivity.
As the news environment became more polarized and diverse in the late 1900s, it became apparent that dispassionate, balanced coverage may not optimally inform voters about the issues and instead could confuse readers in terms of which claims are more valid (Budak, Goel and Rao 2016; Corbett and Durfee 2004). Starting in the late 1990s, the objectivity norm increasingly became more analytic and assertive, promoting the contextual and “weight of evidence” approach in news reporting (Fink and Schudson 2014; Hiles and Hinnant 2014). Under this “interpretive turn” in journalism, rather than simply reporting events or quoting public speeches, reporters increasingly offered contexts and interpretations (Barnhurst 2014). This has led to increasing awareness that false balance (i.e., the equal coverage of both sides when one perspective is overwhelmingly supported by evidence) likely misleads readers (Dixon and Clarke 2013). These trends can be characterized as the objectivity norm gradually evolving into interpretive objectivity.
Political fact-checking is one manifestation of this transition from descriptive objectivity to interpretive objectivity in U.S. journalism. In the early 2000s, fact-checking emerged as a response to the problems of the descriptive nature of conventional reporting (e.g., FactCheck.org in 2003; PolitiFact and Washington Post Fact Checker in 2007). One key example that highlights the shortcomings of traditional reporting is the conventional media’s failure to adequately fact-check the Bush administration’s claims about weapons of mass destruction (WMD) in Iraq in 2003. According to Michael Dobbs, the founder of Washington Post Fact Checker, this WMD episode “helped discredit the idea that reporters are merely messengers or stenographers” and fueled the rise of fact-checking (Dobbs 2012, p. 3). Different from conventional reporting, fact-checking requires reporters to not only quote public figures, but also interpret contexts and analyze evidence to assertively draw conclusions and point out falsehoods (Pingree, Brossard and McLeod 2014; Thorson 2018). In this sense, the fact- checking approach stands in stark contrast to conventional reporting, which avoided taking
sides and sought balanced coverage of both sides. For instance, the New York Times political editor Richard Stevenson stated that fact-checkers should “have the strength of character to call balls and strikes [. . . ] be willing to say that one side is right, and the other is wrong.” (Dobbs 2012, p. 13).
Through their evidence-based reporting aimed at interpretive objectivity, fact-checkers aspire to correct misperceptions across party lines. Glenn Kessler of the Washington Post Fact Checker said “What I love are the letters I get from readers—and it comes once, twice, three times a week—from readers that say, you know, ‘I was really thinking this, but you’ve convinced me otherwise’” (Graves 2016, pp. 187-188). This perspective is shared by Brooks Jackson of FactCheck.org, who said “sometimes we even get messages from people, like a guy will say, ‘Well I’m a Democrat, but I appreciate what you do because I want to know when my guys are lying to me.’ And there are people out there like that” (Graves 2016, p. 188). However, these idealized images of fact-checking audience do not align with most audiences in reality (Graves 2016). For example, partisans resist factual information that runs counter to their existing beliefs (Nyhan and Reifler 2010; Kahan 2015). Moreover, only a small proportion of the public actually visit fact-checking sites, while many people suspect the impartiality of fact-checkers (Guess, Nyhan and Reifler 2020; Walker and Gottfried 2019). Among many factors that could contribute to this gap between the goals and the performance of fact-checking, I consider coverage decisions on how often to cover either political party as one potential contributing factor that obstructs bipartisan trust in fact-checking sources.
Fact-checking Coverage Trait: Asymmetric Coverage of Political Parties One likely consequence of pursuing interpretive objectivity is asymmetric coverage where one political party is more often covered than the other. When overall coverage—the ag- gregation of individual fact-checks—is considered, fact-checking coverage is not constrained
by “equal weights” on both parties. Instead, it is influenced by the “weights of evidence” that allow reporters to disproportionately scrutinize one party than the other as needed. Moreover, because fact-checking coverage heavily focuses on monitoring the performance of
political authorities (Graves and Glaisyer 2012), most of their coverage corrects errors in what prominent political figures have said, rather than simply quoting their public state- ments. Illustrating these approaches, the mission statement of PolitiFact states that “We more often fact-check the party that holds power or people who repeatedly make attention- getting or misleading statements” (Holan 2018). In a similar vein, the Washington Post Fact Checker’s mission statement says “We fact check what matters—and what matters are people in power. When one political party controls the White House and both houses of Congress, it is only natural that the fact checks might appear too heavily focused on one side of the political spectrum” (Kessler 2017).
Asymmetric coverage, however, poses a dilemma for efforts to create bipartisan trust in fact-checking and help Democrats and Republicans converge on facts. While there is a need to occasionally diverge from symmetric coverage to accurately reflect evidence, asymmetric coverage may risk the loss of trust among partisans who perceive the coverage to be slanted or unfair. If this happens, the social value of political fact-checking significantly shrinks, because partisans who would benefit from fact-checking are likely to be alienated from fact- checking sites, reinforcing partisan divisions in perceptions of facts. This dilemma becomes more evident when we consider the reporting practices of professional fact-checking sites.
When fact-checking sites target one party more often than another, people can easily notice the asymmetry due to their tendencies to highlight inaccurate claims. Fact-checking sites more often rate partisan figures’ claims as “false” rather than “true” (e.g., among FactCheck.org’s fact-checks in 2017 -2019, 66% were negative (“false,” “partially false,” “very false”), only 8% were “partially true,” and 0% were “true”; Ferracioli, Kniess and Marques 2022), sometimes even mockingly (e.g., “Pants on Fire” rating of PolitiFact, “Pinocchios” ratings of Washington Post Fact Checker). Because the partisan targets in these corrections are usually explicit in headlines, partisans who encounter fact-checking coverage—on the front page of a fact-checking site, fact-checking posts or warning tags on social media—likely easily identify coverage asymmetry in one direction or the other.
Even though many professional fact-checking sites are committed to nonpartisanship and strive to apply the same standards to both parties,1 asymmetric coverage of political parties often takes place. For instance, in the early 2010s, PolitiFact was found to have corrected Republican claims three times more often than Democratic claims (Davis 2013; Ostermeier 2011). During the 2012 presidential election, among the fact-checks posted on Twitter by FactCheck.org, PolitiFact, and Washington Post Fact Checker, 42% were unfavorable to Republicans and 23% were unfavorable to Democrats (Shin and Thorson 2017). This asym- metry persisted in later years. Between 2017 and 2019, among the fact-checks published by FactCheck.org, 73% targeted Republicans, whereas only 24% targeted Democrats (Ferracioli, Kniess and Marques 2022).
In addition to prior work that examined fact-checking coverage across multiple years, my own data collection also indicates visitors of professional fact-checking sites have likely encountered asymmetric coverage at a given time period. To illustrate this point, I collected the entire set of fact-checking articles published by FactCheck.org and Washington Post Fact Checker during October 2016, June 2020, and September 2022.2 For each article, I collected data on the party that was challenged or validated, publication date, headline, deck summary, and topic (details in Section 1.2 of supplementary materials, Tables S1.6-S1.13).
1“We treat conservatives and liberals alike and apply exactly the same standards of ac- curacy to claims made by both sides.” (FactCheck.org n.d.); “We will strive to be dispas- sionate and non-partisan. The identity or political ties of the person or organization making a charge is irrelevant.” (Washington Post Fact Checker; Kessler 2017); The International Fact-Checking Network (IFCN)’s Code of Principles lists “a commitment to nonpartisanship and fairness” as the first principle (IFCN n.d.).
2October 2016 reflects fact-checking coverage during the most recent presidential elec- tion at the time of this study (Google Trends data show peaks of public interest in fact- checking during the month before the presidential election, Figure S1.1). June 2020 reflects fact-checking coverage when the experimental design was being finalized. October 2022 was originally selected to examine the month prior to election during the Biden administration. It was adjusted to September 2022, because Washington Post Fact Checker published only 3 fact-checks in October 2022, which was too few to examine distributions.
Figure 1: Percentage of Partisan Targets in Fact-checking Coverage
Note: Percentages are calculated out of the total number of fact-checks with partisan targets. Percentages of Republican-challenging and Democrat-challenging fact-checks may not sum up to 100% because there are additional categories (e.g., validate a claim). Table S1.6 presents the full results in tabular form.
As shown in Figure 1, in October 2016, among the fact-checks with partisan targets (e.g., politicians, partisan groups) in FactCheck.org, 60% challenged Republicans, whereas 28% challenged Democrats. The asymmetry was more prominent in Washington Post Fact Checker, where 73% challenged Republicans and only 19% challenged Democrats. In June 2020, the asymmetry was more prominent in FactCheck.org, where 91% challenged Re- publicans and only 9% challenged Democrats. As for Washington Post Fact Checker, 75% challenged Republicans and 25% challenged Democrats. In terms of additional categories not shown in Figure 1, occasionally a few fact-checks corrected both parties within an article or validated the accuracy of Democratic claims, but none validated Republican claims during these months. Another interesting observation is that fact-checking coverage rarely validated (once or never per month) but almost always critiqued political claims of interest. These re- sults indicate that fact-checking coverage leans toward identifying inaccuracies, rather than validating the truthfulness, of political claims.
These observed asymmetries that disfavor Republicans do not mean that the asymme- try in fact-checking coverage is inherently anti-Republican. In September 2022, after the presidency was transferred to the Democratic Party, FactCheck.org still leaned toward cor- recting Republicans (65%) more often than Democrats (35%). However, the asymmetry was
reversed in Washington Post Fact Checker, challenging Democrats (67%) more often than Republicans (22%). This shows that fact-checking coverage may also lean toward more heav- ily correcting Democrats. Moreover, one of the first politicians who cited a fact-checking site in public speech was a Republican politician, Dick Cheney, who cited FactCheck.org to sup- port a claim in the 2004 vice-presidential election debate. The recent asymmetry against Republicans is largely driven by circumstantial factors, such as the Republican Party being in power between 2017 and 2020 and some prominent Republican figures repeatedly mak- ing misleading claims.3 While most prior research shows fact-checking sites more often have targeted Republicans (e.g., Davis 2013; Ferracioli, Kniess and Marques 2022; Shin and Thor- son 2017), if circumstances change, for instance the Democratic Party comes to power as it did in 2020 or some prominent Democratic figures repeatedly make misleading claims, then fact-checking sites likely produce asymmetric coverage adverse to Democrats.
Source Credibility and Asymmetric Coverage of Political Parties While asymmetric coverage is at times necessary to accurately portray reality on the basis of evidence, the dilemma lies in how partisans assess a source based on its coverage. To examine how individuals evaluate and trust a source, I focus on source credibility, defined as
“the believability of a communicator” (O’keefe 20102, p. 181) or “audience [perception that] they would benefit from believing [the communicator]” (Lupia 2016, p. 87).4 As an impor- tant precondition of learning, persuasion, and belief formation, source credibility perceptions strongly determine whether partisans would accept or reject the information that the source provides (Berinsky 2017; Druckman and McGrath 2019; Lupia and McCubbins 1998).
One hurdle in earning bipartisan trust in fact-checking is hostile media perception, which
3Washington Post Fact Checker created a new rating category, “Bottomless Pinocchios,” in 2018 and assigned it to Donald Trump for repeatedly making misleading claims (Kessler and Clement 2018).
4In both definitions, receivers’ subjective perceptions, rather than objective traits, of a source determine the degree of perceived source credibility.
refers to partisans’ tendency to perceive neutral or balanced media reports to be biased against their own group or beliefs (Gunther and Schmitt 2004; Vallone, Ross and Lepper 1985). When media coverage is slanted, relative hostile media perception emerges, causing partisans to perceive greater bias in a source whose views do not align with their own (Coe et al. 2008; Gunther and Chia 2001). These hostile media perceptions imply that, absent convincing reasons to think otherwise, partisans are predisposed to suspect that fact-checking sources do not abide by their alleged nonpartisanship.
A major obstacle to bipartisan trust, particularly when fact-checking coverage more of- ten targets one’s own party than another (“uncongenial asymmetric coverage”), is partisan motivated reasoning, which refers to partisan tendencies to selectively reject uncongenial in- formation to protect their partisan identity or beliefs (Druckman and McGrath 2019; Kahan 2015). This tendency persists even when the source is an expert on a given issue (Kahan, Jenkins-Smith and Braman 2011; Nisbet, Cooper and Garrett 2015). One ramification of par- tisan motivated reasoning on news consumption is partisan selective exposure, which refers to partisans’ selective use of and trust in likeminded news sources (Stroud 2011). Selective exposure can further drive partisans to avoid and discredit news outlets and contents that challenge their own group or views (Bakshy, Messing and Adamic 2015; Garrett and Stroud 2014). In fact, partisans’ use of and trust in news sources are highly dependent on whether a source presents congenial political viewpoints (Peterson and Iyengar 2021). Drawing on evi- dence of partisan motivated reasoning, I propose that when the coverage of a source targets one’s own party at a greater rate, partisans likely perceive the coverage as a potential threat to the legitimacy of their group, triggering distrust in the source as a means to protect their partisan identity.
H1: Asymmetric coverage that more often challenges one’s own party (“uncongenial asymmetric coverage”) will reduce perceived source credibility among partisans, com- pared to symmetric coverage that similarly challenges each party.
Regarding partisan reactions to uncongenial asymmetry, prior studies largely suggest
Republicans would be more resistant to such coverage than Democrats. In studies on per- sonality traits, conservatives have been found to be more resistant to aversive experience and more intolerant of opposing views or other groups, compared to liberals (Farwell and Weiner 2000; Jost et al. 2003; Oxley et al. 2008). In the context of information processing, studies have found that Republicans tend to be more resistant to uncongenial news and facts than Democrats (Garrett and Stroud 2014; Nyhan and Reifler 2010; Shook and Fazio 2009). Drawing on these studies, I hypothesized that uncongenial asymmetry is likely to lower perceived credibility to a greater extent among Republicans than Democrats.
H2: Uncongenial asymmetric coverage will decrease perceived source credibility to a greater extent among Republicans, compared to Democrats.
When a source more heavily targets the opposite party (“congenial asymmetric cover- age”), there are two possible ways in which credibility assessments are affected. A first possi- bility is that congenial asymmetry would improve credibility assessments, because partisans selectively prefer and trust likeminded news and sources (Stroud 2011; Peterson and Iyengar 2021). It has also been found that people enjoy reading negative news about out-group, a tendency driven by their in-group favoritism (Ouwerkerk et al. 2018). Yet, the asymmetry itself, even if it is adverse to out-group, can still lower credibility for two reasons. First, peo- ple tend to find two-sided or balanced sources more credible than one-sided sources (Allen 1991; Mayweg-Paus and Jucks 2018). Second, perceiving a source to be biased in favor of a group can lower credibility even when the source is considered honest and expert (Wallace, Wegener and Petty 2020). The discounting hypothesis further suggests that a source that fails to meet audience expectations will cause the audience to reevaluate and “discount” the credibility of the source (Allen 1991). For instance, in a context where the audience expects non-partisan reporting from a given source (e.g., online encyclopedia), one-sided coverage could be perceived as an indicator of persuasive intent, likely violating expectations, and decrease perceived credibility of the source (Flanagin, Winter and Metzger 2020). This ex- pectation violation heuristic is especially powerful in credibility assessments of relatively
unfamiliar sources (Flanagin, Winter and Metzger 2020). Because professional fact-checkers proclaim nonpartisanship in their reporting (e.g., mission statements; FactCheck.org n.d.; Holan 2018) and given relatively low familiarity with fact-checking sites among the U.S. public (Graves 2016; Guess, Nyhan and Reifler 2020), congenial asymmetry can negatively affect source credibility perceptions. Given two possible theoretical expectations, I explore how congenial asymmetric coverage affects source credibility perceptions.5
RQ1: Does asymmetric coverage that more often challenges the opposite party (“con- genial asymmetric coverage”) increase or decrease perceived source credibility among partisans, compared to symmetric coverage?
A final focus of my inquiry pertains to source credibility perceptions under two different contexts, as a news source and as a source of policy advice. These two contexts partly stem from two different approaches to measuring source credibility in the literature, where one of them somewhat deviates from the theoretical concept of source credibility. Theoretically, source credibility is widely assumed to have two underlying dimensions (Hovland, Janis and Kelly 1953; Lupia and McCubbins 1998). According to Lupia (2016), perceived shared inter- est, or perceived trustworthiness, refers to the extent to which the listener and communicator want similar outcomes, whereas perceived expertise refers to the extent to which the speaker is knowledgeable about the consequences of the listener’s choice (pp. 87-88). However, because the literature lacks clear guidance on how to measure source credibility, source credibility has been often measured in ways not consistent with its two-dimensional concept.
One major way to measure source credibility perceptions is in the context of news sources, which is the main focus of this study. Under this context, the qualities of being accurate, fair, or complete are important traits expected for credible news sources. These expected values
5Given two potential theoretical expectations, I hypothesized the effects of congenial asymmetric coverage in both directions in my preregistration, which I learned only later that such predictions should better be registered as an exploratory research question. Because the underlying intention was to propose an exploratory question with unclear theoretical expectations, I present this inquiry as a research question.
of credible news informed the development of a news credibility scale (Gaziano and McGrath 1986; Meyer 1988). Although this scale, being one-dimensional, does not neatly fit with the two-dimensional assumption of source credibility, it has been widely adopted to measure perceived credibility of news messages or outlets (e.g., Flanagin and Metzger 2000; Tsfati 2010; Pingree et al. 2013; Turcotte et al. 2015). Given the focus on trust in news sources in this study, perceived news credibility is mainly used to test the proposed hypotheses.
Another important context is source credibility as a source of policy advice, or a pre- condition of persuasion, which more prominently brings the two dimensions of credibility into play. These two dimensions are important because credibility perception or persuasion is assumed to require non-zero, positive amount of shared interest and expertise perceptions from the communicator (Hovland, Janis and Kelly 1953; Lupia and McCubbins 1998). By examining how asymmetric coverage affects perceived shared interest and expertise, I further examine how overall coverage affects a source’s potential persuasive effects. Because there is a lack of empirical research that compares how partisans assess a source as a news source versus as a source of policy advice, I propose to explore how asymmetric coverage affects the perceptions of shared interest and expertise, compared to news credibility, among partisans.
RQ2: Do uncongenial and congenial asymmetries reduce perceived shared interest and expertise among partisans?
Study Design
To assess the effects of asymmetric coverage on source credibility, I conducted a survey ex- periment on August 10, 2020. Participants were recruited through Prolific, an online crowd- sourcing platform that has been found to provide higher quality data compared to alternative online platforms, demonstrated through better performance on attention checks, less dishon- est behavior, and its ability to reproduce existing results (Palan and Schitter 2018; Peer et al. 2017). Using Prolific’s prescreening data, I recruited an equal number of Democrats and Re-
publicans, 720 respondents in total (360 Democrats, 360 Republicans).6 My hypotheses and analysis plan were preregistered on AsPredicted.org prior to data collection.7
Experimental Treatments
Participants were told they would be presented with a list of headlines from an online news site. Among eight headlines, six challenged one of the two political parties (“partisan top- ics”) and two were neutral to political parties (e.g., health, finance). Partisan topics involved issues where political elites of both parties have made misstatements: abortion, black teen pregnancy, immigration, gun violence, Wall Street bailout, and US national debt (Wood and Porter 2019). For this reason, it was plausible to attribute either party as the source of misinformation to manipulate coverage asymmetry. As discussed earlier, the majority of fact-checking coverage challenges, rather than validates, target statements (Table S1.6). Reflecting this active adjudication, the headlines for partisan topics were designed to explic- itly challenge a partisan target as shown in Table 1. In actual fact-checking sites, a large number of headlines use languages of straightforward criticism of inaccuracy (e.g., “wrong,” ”misleading,” “incorrect”), and sometimes even employ derogatory language (e.g., “ridicu- lous,” “nonsensical”). As some critiques of fact-checking practice have noted, fact-checking coverage sometimes, although not frequently, suggests subjective assessments of a policy or phenomenon (Uscinski and Butler 2013, examples in Table S1.15).
Because the main purpose was to manipulate asymmetric coverage of political parties,
other features of fact-checking sites (e.g., rating scales, deck summary, issue-specific details) were eliminated8 and two of the headline wordings were more explicitly judgmental. This
6Using the typical two-step questionnaire, 56.3% were strong partisans, 37.2% were weak partisans, and 6.5% were partisan leaners. Partisan leaners were considered as partisans as they tend to hold partisan opinions as strong as weak and strong partisans (Petrocik 2009).
7The preregistration is available at: https://aspredicted.org/8T6 2BJ. The hypothesis wordings were refined to be more concise, compared to the preregistered versions.
8Although fact-checking headlines can be detailed and specific, I follow examples that are broadly worded to plausibly target either party (e.g., “[A Democrat]’s Misleading Debt
Partisan
Table 1: Headlines for Baseline and Treatment Conditions
• What [Democrats/Republicans] have wrong about the pregnancy rate among black teenagers
• [Democratic/Republican] National Committee pursues a policy for the worse on the deportation of illegal immigrants
• [Democratic/Republican] Senator misleads on which president signed the Wall Street bailout into law
• What [Democrats/Republicans] get incorrect about the number of abortions over time
• [Democratic/Republican] Party takes the wrong path to the policy on gun homicide
• [Democratic/Republican] governor mischaracterizes the causes of US debt
• Exercise can greatly reduce your risk of cancer and heart disease
• Google to spend $10 billion on offices, data centers in US this year
simplification helps remove confounding factors, strengthens the treatment, and better iso- lates the effects of coverage asymmetry on source assessments. These design choices reflect this study’s focus on clarifying the relationship between a source’s coverage decisions in terms of asymmetry and partisans’ credibility assessments, instead of perfectly mimicking fact-checking sites, whose reporting styles quite vary across specific outlets.
Two additional headlines unrelated to partisan controversies were included in the set of headlines for two reasons. First, through these additions, I intended to mitigate the percep- tion that the given news outlet was solely dedicated to partisan issues, which might otherwise reinforce partisan reactions. Second, the inclusion of neutral topics reflects reporting practices of conventional news outlets that run standalone fact-checking operations endorsed by the IFCN (e.g., Associated Press, USA Today, Daily Caller) or those that regularly produce arti- cles labeled as “fact-check” (e.g., ABC, New York Times), thus allowing the current study to provide implications with respect to a broader set of outlets that engage with fact-checking. There were three main experimental conditions: among six partisan topics, 1) five chal- lenged Republicans and one challenged Democrats (Republican-challenging); 2) five chal- lenged Democrats and one challenged Republicans (Democrat-challenging); or 3) three head- lines challenged each party (symmetric).9 To manipulate the coverage asymmetry, for six
Claims,” “[A Republican] Wrong on Murder Rate”; Table S1.7).
9There was a fourth condition that tested headline language effects (critical vs. neutral). The preregistration indicated that this condition was exploratory and that it would not be a
partisan topics, the bracketed part (e.g., “[Democratic/Republican]”) indicating the target was set to be either “Democratic” or “Republican” as shown in Table 2. Participants were considered as being assigned to uncongenial asymmetry if the majority of the headlines chal- lenged in-group (e.g., a Democrat assigned to Democrat-challenging asymmetry), whereas they were considered as assigned to congenial asymmetry if most headlines challenged out- group (e.g., a Democrat assigned to Republican-challenging asymmetry).
Table 2: Partisan Topics and Challenged Parties: Two Variations per Condition
Topic
Symmetric Coverage Replican-challenging
Democrat-challenging Asymmetry
1 2 1 2 1 2
Black teen pregnancy Rep Dem Rep Rep Dem Dem
Immigration Dem Rep Rep Dem Dem Rep
US debt Dem Rep Dem Rep Rep Dem
Abortion Rep Dem Rep Rep Dem Dem
Gun violence Dem Rep Rep Rep Dem Dem
Wall Street bailout Rep Dem Rep Rep Dem Dem
To ensure that the results would not hinge on the specific party-topic associations, par- ticipants were randomly assigned to one of the two party-topic variations per condition, as illustrated in Table 2. In the asymmetric coverage conditions, one headline with an opposite direction was designed to address either a highly controversial topic (immigration) or a less politicized one (US debt). In all variations, the headlines were ordered in a way that neutral topics were presented in between partisan topics to avoid either presenting six partisan topics in a row or presenting two neutral topics in a row. Further details about the experimental design are available in Section 1 of supplementary materials.
Measures
To measure perceived news credibility, after reading the headlines, participants were asked to indicate the degree to which they thought the website could be described as follows: “is fair,” “is accurate,” “is unbiased,” “tells the whole story,” and “can be trusted,” on a five-point
part of the main hypotheses and analyses. Critical language had minimal impacts on source credibility assessments compared to neutral language (results in Figure S3.1 and Table S3.3).
scale from “not at all” to “extremely” (Meyer 1988; Tsfati 2010; Pingree et al. 2013). The primary measure of news credibility perception was the composite score, constructed as the average, of the five items.
To measure perceptions of shared interest and expertise, the two underlying dimensions of source credibility, I adapted the questionnaires from Lupia and McCubbins (1998, p. 188). Perceived shared interest was measured as the degree to which participants perceived the authors of the website as agreeing with them on most political issues on a five-point scale from “never” to “always.” Perceived expertise was measured as the degree to which participants perceived the authors of the website as knowledgeable about how political decisions affect people on a five-point scale from “nothing at all” to “a great deal.”
Results
Effects of Asymmetric Coverage on News Credibility Perceptions
To analyze how coverage asymmetry affects perceived source credibility relative to the base- line condition of symmetric coverage, I used ordinary least squares (OLS) with robust stan- dard errors. The five items for the composite scale of source credibility loaded on a single underlying construct in factor analysis and had acceptable internal reliability (Cronbach’s α = .92; Bland and Altman 1997).10 In Table 3, the model estimates the effects of asymmetric coverage compared to symmetric coverage while allowing for the treatment effects to vary by partisan identity. Because it is hard to directly interpret interaction terms (Brambor, Clark and Golder 2006), I focus my discussion on the conditional average treatment effects (CATE) among each partisan group (e.g., Guess and Coppock 2020). In subsequent discussions, the treatment effects of uncongenial and congenial asymmetries assume symmetric coverage as the baseline condition.
Consistent with H1, uncongenial asymmetric coverage reduced perceived news credibility compared to symmetric coverage. As illustrated in Figure 2, this negative impact of uncon- genial coverage on perceived news credibility was present among both Republicans (–0.13,
10Factor analysis results and item-total correlations are available in Table S3.8.
Table 3: Asymmetric Coverage Effects on Perceived News Credibility, Shared Interest, and Expertise
Perceived News Credibility Perceived Shared Interest Perceived Expertise
Uncongenial -0.18∗∗∗ (0.03) -0.14∗∗∗ (0.03) -0.07∗ (0.04)
Congenial -0.05* (0.03) 0.07∗∗ (0.03) 0.10∗∗∗ (0.04)
Rep 0.05 -0.003 0.03
(0.03) (0.03) (0.03)
Uncongenial x Rep 0.05 0.08∗ -0.06
(0.05) (0.05) (0.05)
Congenial x Rep -0.05 -0.05 -0.09∗
(0.05) (0.05) (0.05)
Constant 0.38∗∗∗ (0.02) 0.42∗∗∗ (0.02) 0.44∗∗∗ (0.03)
N 540 539 540
Adjusted R2 0.08 0.07 0.05
Note: Entries are the ordinary least squares (OLS) regression coefficients with robust stan- dard errors are in parentheses. Uncongenial = 1 if ingroup-challenging asymmetry condition, 0 otherwise; Congenial = 1 if outgroup-challenging asymmetry condition, 0 otherwise. Rep
= 1 if Republican; =0 if Democrat. All variables were coded to range from 0 to 1. ∗p < .10;
∗∗p < .05; ∗∗∗p < .01.
p < .01) and Democrats (–0.18, p < .01).11 While I expected uncongenial asymmetry to reduce perceived credibility to a greater extent among Republicans than Democrats (H2), it was not the case in this study. Not only the magnitude of treatment effect was greater among Democrats (−0.13 for Republicans, −0.18 for Democrats), the average level of perceived news credibility under uncongenial asymmetry was significantly lower among Democrats than Re- publicans (Dem = 0.21; Rep = 0.31; t-test of difference in means, t = –2.79, p < .01). Further reinforcing this point, a tendency to more strongly discount uncongenial asymmetry than congenial asymmetry was found among Democrats but not among Republicans.12 This
11Treatment effects are calculated from Table 3. For instance, the treatment effect of uncongenial asymmetry compared to symmetric coverage is the coefficient estimates for [Uncongenial] for Democrats and [Uncongenial + Uncongenial×Rep] for Republicans. The subgroup analysis provides the same estimates of conditional treatment effects (Table S3.1).
12Section 3.4 of supplementary materials discusses an exploratory question of whether
Figure 2: Average Perceived News Credibility by Experimental Conditions
Note: Means and 95% confidence intervals by experimental conditions. Uncongenial = Ingroup-challenging asymmetric coverage; Symmetric = Symmetric coverage (baseline); Congenial = Outgroup-challenging asymmetric coverage. All variables were coded to range from 0 to 1. The estimates are derived from Table 3.
again implies that Democrats have a stronger tendency to discount the credibility of un- congenial asymmetry than Republicans. Overall, these findings indicate that partisans from both sides find a source less credible when the majority of its coverage challenges their own party, compared to when it evenly challenges both parties.
Congenial asymmetric coverage also had an effect of decreasing perceived news credibil- ity (RQ1). As shown in Figure 2, partisans who were given coverage where most headlines challenged the opposite party perceived the source to be less credible, relative to those who received symmetric coverage. The negative effects of congenial asymmetry on credibility per- ceptions were observed across both partisan groups, Republicans (–0.10, p < .01) Democrats (–0.05, p < .10). A further examination reveals that Democrats are more sensitive to the specific context of congenial asymmetry. As discussed in the study design, there were two randomized versions of headline content (with different topic-party associations) per condi- tion. In the congenial asymmetry condition, five headlines challenged the opposite party and
uncongenial asymmetry decreases perceived credibility to a greater extent than congenial asymmetry. It was the case among Democrats, but not Republicans.
one headline challenged one’s own party on either immigration or national debt.13 As shown in Figure 3, congenial asymmetry decreased credibility perceptions among Democrats when this single ingroup-challenging headline was on immigration (−0.14, p < .01), but not when it was about national debt (0.02, p = .56). In contrast, Republicans found a source with con- genial asymmetric coverage to be less credible, regardless the topic of ingroup-challenging headline (immigration: −0.12, p < .01; debt: −0.08, p < .05).
Figure 3: Average Perceived News Credibility under Congenial Asymmetry by Headline Con- tent Variations Compared to the Baseline Condition
Note: Means and 95% confidence intervals by experimental conditions. Symmetric = Symmetric coverage (baseline condition); Congenial/Debt = Congenial asymmetric coverage where 5 headlines challenge out-group, and 1 challenges in-group on national debt;
Congenial/Immigration = Congenial asymmetric coverage where 5 headlines challenge out-group, and 1 challenges in-group on immigration. All variables were coded to range from 0 to 1. Table S3.4 in supplementary materials presents these results in tabular form.
These findings suggests Republicans might perceive congenial asymmetry as more of a sign that the source is not reliable than as an endorsement of their group. Democrats, on the other hand, could perceive congenial asymmetry as credible as symmetric coverage when Democrats are challenged on less politicized issues (e.g., national debt). However, Democrats may find congenial asymmetry less credible when a portion of coverage challenges Democrats
13There was no statistically significant difference in source assessments between the two randomized versions in all other conditions and partisan groups (Tables S3.5-S3.7). The only exception was Democrats under congenial asymmetry as discussed here.
on highly polarized issues (e.g., immigration). Although there are limits to generalize this finding to other topics on the basis of this single study, because fact-checking sites heavily focus on political controversies,14 it is possible that at least a minority of their headlines challenge Democrats on controversial topics and cause Republican-challenging asymmetry to lower perceived credibility among not only Republicans, but also Democrats.
Effects of Asymmetric Coverage on Perceptions of Shared Interest and Expertise
Next, I consider how coverage asymmetry affects two underlying dimensions of source credi- bility, perceptions of shared interest and expertise (RQ2). Compared to symmetric coverage, uncongenial asymmetry decreased perceived shared interest among both Republicans (–0.06, p < .10) and Democrats (–0.14, p < .01). One finding relevant to the unexpected partisan difference in uncongenial asymmetry effects (H2) is that uncongenial asymmetry reduced perceived shared interest to a greater extent among Democrats than Republicans. As for perceived expertise, uncongenial asymmetry decreased perceived expertise among both Re- publicans (–0.12, p < .01) and Democrats (–0.07, p < .10) compared to symmetric coverage.
Figure 4: Average Perceived Shared Interest and Expertise by Experimental Conditions
Note: Means and 95% confidence intervals by experimental conditions. Uncongenial = Ingroup-challenging asymmetric coverage; Symmetric = Symmetric coverage (baseline); Congenial = Outgroup-challenging asymmetric coverage. All variables were coded to range from 0 to 1. The estimates are derived from Table 3.
14Professional fact-checking sites heavily focus on partisan topics, oftentimes more than 90% of their articles addressing statements made by partisan figures and groups (Table S1.6).
Congenial asymmetry, on the other hand, increased perceived shared interest among Democrats (0.07, p < .01) but had minimal impact among Republicans (0.02, p = .28), com- pared to symmetric coverage. Similarly, congenial asymmetry increased perceived expertise among Democrats (0.10, p < .01) but minimally affected perceived expertise among Repub- licans (0.01, p = .80). These results indicate that Democrats likely consider a source with congenial asymmetry to possess greater shared interests and expertise, while Republicans are more indifferent to symmetric coverage and congenial asymmetric coverage.15
Overall, Democrats were found to be more sensitive to the direction of coverage asym- metry when assessing shared interest and expertise of a source, compared to Republicans. Because shared interest and expertise are preconditions of persuasion (Lupia and McCubbins 1998), these findings suggest that compared to Republicans, Democrats are less likely to be persuaded by a source with uncongenial asymmetry, but more likely to be persuaded by a
source with congenial asymmetry.
Discussion
This study finds that asymmetric coverage of political parties often found in professional fact-checking sites, although it is driven by evidence-based news coverage decisions, can have an unintended consequence of undermining bipartisan trust. Compared to symmetric coverage that corrects each party at a similar rate, asymmetric coverage lowered perceived source credibility among both partisan groups. Uncongenial asymmetry, where a majority of headlines challenge one’s own party, reduced perceived news credibility, shared interest, and expertise among both Democrats and Republicans. Contrary to popular belief, Democrats more negatively assessed a source with uncongenial asymmetry than Republicans. Conge-
15An unexpected yet interesting finding is that congenial asymmetry reduced perceived news credibility compared to symmetric coverage among both partisan groups, yet increased perceived shared interest and expertise among Democrats, but not Republicans. Given the importance of shared interest and expertise in persuasion, this pattern suggests Democrats, but not Republicans, are more likely to be persuaded by the messages and corrections from a source with congenial asymmetric coverage despite its lower credibility as a news source.
nial asymmetry, where most headlines challenge the opposite party, also reduced perceived news credibility among both partisan groups. Democrats found congenial asymmetry as less credible particularly when a portion of coverage challenged their own party on a highly politi- cized issue. Interestingly, congenial asymmetry triggered Democrats, but not Republicans, to perceive greater shared interest and expertise from the source.
Because uncongenial asymmetry reduced perceived credibility among both partisan groups, it is evident that both are motivated to protect their partisan identity by distrusting a source that heavily challenges one’s own group. Congenial asymmetry, on the other hand, caused both partisan groups to find the source to be less credible as a news source, indicating that the violations of audience expectations for balanced coverage dominantly determined source assessments. Congenial asymmetry is more likely to reduce perceived news credibility when the coverage contains in-group challenge on politicized topics, as shown in Democrats’ reac- tions. Democrats, but not Republicans, perceived greater shared interest and expertise from a source with congenial asymmetry, suggesting Democrats would be more likely to be per- suaded by messages delivered by such sources. These findings suggest that Democrats have greater tendencies to distrust uncongenial asymmetry and favor congenial asymmetry, while being more sensitive to the specific context of asymmetric coverage in source assessments.
Overall, these findings imply that by producing asymmetric coverage, fact-checking sites run the risk of losing trust among not only Republicans but also Democrats who are often thought to be favorable to fact-checking. These results echo the concerns that some jour- nalists have in their reluctance to embrace the arbitration model of fact-checking for fear that it will harm public perceptions of their objectivity, particularly if their corrections favor one party over the other (Dobbs 2012; Thorson 2018). My study suggests that this fear that fact-checking practice could harm source reputations can be mitigated by pursuing more symmetric coverage of political parties.
I propose several potential explanations for why Democrats more negatively react to uncongenial asymmetry and more sensitive to headlines that challenge their group, although
further research is is needed to fully understand this phenomenon. First, because Repub- licans tend to hold lower baseline trust in the news media than Democrats (Pennycook and Rand 2019), there could be a floor effect that limits the degree to which asymmetric coverage decreases perceived source credibility among Republicans. Republicans also might already perceive their group to be disfavored by the media or fact-checkers (Shin and Thorson 2017), which would lower their baseline trust when asked to assess a news source. However, in this study, perceived source credibility under the baseline condition was similar between Democrats and Republicans, requiring further investigation and alternative explanations. Second, Democrats and Republicans might hold different perceptions of reality. Given the relatively greater amount of Republican misstatements covered in fact-checking (Ferraci- oli, Kniess and Marques 2022) and the prevalence of conservative unreliable news outlets (Pennycook and Rand 2019), Democrats may perceive the reality to be tilted toward more Republican misstatements in recent years. If that is the case, Democrats could perceive un- congenial asymmetry as inaccurately reflecting reality and find the source to be not credible. A final possibility is that the existing theories on partisan personality traits and information processing (Jost et al. 2003; Garrett and Stroud 2014) need refinement because Democrats could be more resistant to adverse stimuli than Republicans under certain contexts.
What kind of practical recommendations can this research offer to professional fact-
checkers? As discussed earlier, there is a dilemma for producing asymmetric coverage to achieve interpretive objectivity, because asymmetric coverage is sometimes necessary to ac- curately reflect evidence but jeopardizes public trust. My work does not intend to encourage professional fact-checkers to pursue balance for the sake of balance. Instead, my findings suggest that asymmetric coverage poses an obstacle to earning bipartisan trust. While ad- hering to their evidence-oriented coverage decisions, fact-checkers should extend their efforts to better communicate their motivation and non-partisan practice to the public: how they overcome shortcomings of conventional journalism; how they achieve transparent, nonparti-
san, and rigorous reporting;16 and how their non-partisan principles could sometimes lead to asymmetric coverage. Absent these extra efforts, their asymmetric coverage, despite its merits and occasional need, likely alienates partisans from both sides.
Another practical recommendation that this study generates is that fact-checking sites, or any other evidence-based sources, can build public trust by pursuing symmetric coverage of political parties. Even when the reality has an imbalance in the amount of misstatements produced by different parties, there could be ways to signal symmetric coverage while avoid- ing “false balance” that artificially imposes balance regardless of evidence. For instance, even when there is an asymmetry in partisan misstatements in the short run, fact-checking sites can keep track of the relative amount of fact-checked statements from each party to show a rough balance in the long run.17 They can also consider sharing the pool of statements that they have considered and examined, which could be more balanced than the set of statements that they ultimately publish as fact-checks. Another approach could be to explicitly present both parties as the key targets and keep threads of fact-checks targeted at each party, to demonstrate attention to both.18
Like any single study, I conducted this one in a particular context. Several aspects of this context may affect its generalizability. For example, the sample for this study was recruited through an online crowdsourcing platform. Because the sample tends to be younger and more educated compared to the general population (Table S2.1), further research is needed to evaluate the extent to which the results generalize to different populations. There are a
16The IFCN Code of Principles describes how fact-checking sites are monitored to abide by a set of rules for nonpartisan, transparent, and evidence-based reporting (IFCN n.d.).
17A Canadian fact-checking site, FactsCan (currently inactive), displayed a pie chart track- ing the share of fact-checks targeted at different political parties. This practice—setting tar- gets by party to roughly balance with recent popular vote—is explicitly rejected by the U.S. fact-checkers (Graves 2018).
18LeadStories, a U.S. fact-checking site, keeps the “Blue Feed” and “Red Feed” icons with partisan symbols (a donkey and an elephant) on the top of its website, explicitly signaling that they consider both parties as major targets.
few design limitations that can be addressed in future research. First, in designing exper- imental stimuli, I employed one version of asymmetric coverage, where five versus one out of eight headlines targeted either party. Future work can examine different combinations of asymmetric coverage. Second, there were two party-topic variations for each condition to minimize the chance that the results hinge on the specific party-topic associations. However, in asymmetric coverage conditions, only two out of six partisan topics could be associated with different parties, thus not entirely ruling out the influence of specific topic-party asso- ciations. In future work, the party-topic associations can be fully randomized to allow all partisan topics to be equally likely to be associated with either party. Additionally, two of the six headlines on partisan topics had relatively more opinionated language because I intended to make the asymmetry more explicit. Future study can employ strictly factual language in all headlines to keep the language and tone similar across headlines.
In closing, this study demonstrates that bipartisan trust in fact-checking can be fa- cilitated under certain coverage decisions, further enabling the rigor and unique value of fact-checking to benefit society. It shows that asymmetric coverage in either direction causes a loss of trust among not just one side of partisan spectrum, but among partisans across the aisle. While previous studies have focused on how partisans process individual fact-checking articles (e.g., Nyhan et al. 2020; Wood and Porter 2019), my work focuses on how overall coverage, or a collection of articles, affects partisans’ initial evaluations of a source. This en- deavor is important because most Americans are still relatively unfamiliar with fact-checking sites (Guess, Nyhan and Reifler 2020) and because credibility perception is the first step to- ward learning, persuasion, and continued use of those outlets (Druckman and McGrath 2019; Lupia 2016). By taking this approach, this study clarifies conditions under which partisans can converge on trusted news sources. By identifying ways to build bipartisan trust in fact- checking sites, my hope is that this study can inform the work of journalists, educators, and policymakers in their efforts to create evidence-based news sources that help citizens find common factual ground, make informed decisions, and achieve democratic accountability.