To participate, you may either post a contribution to an existing discussion by selecting the thread for that topic (and then click on "Post Reply") or start a new thread by clicking on "New Topic" below.
For instructions on how to follow a discussion thread by email, click here.
University of Pennsylvania
- Posts: 6
- Joined: Thu Apr 07, 2016 4:50 pm
Indeed, this is arguably a more holistic process for encouraging transparency, while also adding expectations of originality and logical coherence. COMMON SENSE already obliges researchers to do what they can to be convincing to audiences that they must assume to be skeptical given the looming threat of rejection from reviewers and the possibility of critical appraisals of their published work. And, some type of "policing" is already evident in common-sense understanding of different forms of social scientific research, and that adding new layers of procedures and regulations requires uniform undersandings of what constitutes "knowledge" or "truth." As for those inclined to knowingly circumvent ethical standards and obligations, adding new standardized procedures simply shift the loci and techniques of evasion, as is amply evident from cases in other disciplines (psychology and economics, for example) where transparency procedures have long been in existence but where some scholars nonetheless sought to evade detection of dishonest or problematic practices (and succeeded in doing so for a period). For more intrinsically diverse disciplines like political science, the main point of vulnerability in a knowledge claim varies for different styles of research aimed at different intellectual objectives and even different kinds of audiences -- and for each of the resulting research products, scholars have long been aware of the things they need to do to lay out convincing knowledge claims, as this is the entire point of their professional identities and commitments.
In purely statistical work relying on data-sets involving intrinsically quantitative data (e.g. GDP growth rates, migration statistics, voter turnout), it is in the interest of most researchers to tout their data-sets and codes to gain recognition for their approach, and repeated refusals by scholars to share their data with colleagues interested in replicating their work carry mounting reputational costs. More problematic are data-sets that require the researchers to quantify observations that do not originally appear as numeric quantities (e.g. assigning a value on a continuous variable to capture the extent of democratization or the strength of identification with an ethnic group), as here the most critical locus of the problem has to do with the initial CODING of historical events, institutional attributes and social processes so that these can be treated as quantitative data. For qualitative researchers relying on archival research, the discussion in the text and footnotes allow authors to anticipate some of the challenges and lay out their assumptions about contested historiography or their motivations for selecting from the range of available primary sources (as self-aware historians have sought to do to make their arguments more convincing). For interpretive work relying on close-range observations of human subjects, there are not only issues with ensuring the privacy and safety of the subjects themselves (a point repeatedly stressed by ethnographers) but also a more fundamental issue of capturing the moment of the observation within a spatio-temporally bound context. For the latter, the issue becomes one of whether similar acts of observation and interpretation might yield alternative narratives within other contexts that appear to be similar - and it is out of the resulting debates that we seek to establish the value of a particular narrative. Unless we assume that we live in a world of final truths that all of us define and approach in the same way, there is no reason to insist on a uniform definition of transparency or a uniform view of the purpose of replication.
Short of that kind of unified consensus, which has eluded us for centuries for a good reason, the very act of convincing skeptics among their primary audiences has a built-in incentive to make different steps and components of one's research clear to readers, and this obligation is carried forward in different ways in different disciplines and in different styles of research in our own discipline. Against this backdrop, an effort to create and impose uniform procedures across journals that have historically showcased diverse approaches and arguments for diverse audiences will create unevenness in submissions, acceptance rates, and costs in terms of time and resources. At least as importantly, these procedures will simply shift the locus of "dishonesty" for the small minority of researchers who consciously seek to conceal some form of "malpractice." In some ways, the DA-RT effort is reminiscent of the creation of long, elaborate, complicated tax codes that make tax evaders simply find different loopholes, while leaving the rest to pay the costs by having to wade through more complicated forms and expend more energy in filing their taxes.
London School of Economics
- Posts: 6
- Joined: Thu Apr 07, 2016 3:33 pm
Jacques Hymans (University of Southern California)
University of Southern California
effectively how this set of "solutions" does not address the "problems" that the APSA and many political science journal editors believe are endangering the discipline's credibility. The imposition of new procedural practices to regulate "data
access," assure "analytical transparency," and legitimate "production transparency" in the generation of the journal science
in the discipline exceeds, as Rudy notes, "Common Sense."
On the one hand, his remarks in this posting about political science today point out how less and less of
a shared intellectual project exists in common across the discipline, which has been developing in different ways on various planes since the late 1960s to early 1970s. The limited moves toward "methodological pluralism" in the 1990s and 2000s
opened the doors for developing a more common sense of the discipline; yet, it appears too many embedded intellectual and institutional interests in profession disliked what they beheld as these doors opened. The DA-RT process provides a complex
regulatory apparatus for policing these "pluralist methodological" turns in a fashion that reaffirms the conventional norms of hypothetico-deductive model of scientific reasoning at the core of a thousand "Scope and Methods in Political Science" courses, while at the same time affirming the aspirations of the network of journal editors to believe they are becoming political, social, or governmental physicists.
On the other hand, however, Rudy's appeal to "Common Sense" also discloses how the DA-RT process to affirm "good science," "transparent research," and/or "analytical rigor" might be likened to the public relations efforts of major corporations, which are entangled in unsustainable, ecologically degrading or environmentally destructive commodity production chains. With new seals of sustainability, ecological respect or environmental awareness, these firms "green wash" or "green wrap" their product lines in a fashion that occludes the sloppy, irrational, and/or dangerous by-products of their work.
Anxious to protect their increasingly questioned intellectual and institutional sources of normative authority, professional control, and academic legitimacy, the DA-RT supportive individuals and groups in discipline import the regulatory practices of many biomedical, natural or physical sciences to reinforce their professional power, position, and privilege. Yet, such efforts to "good wash" or "good wrap" the social science research process, as they understand it, are a misstep. They are neither producing the best outcomes endorsed in their ethical manifestos nor are they widely celebrated by those they aspire to regulate. Instead they are creating new roadblocks for many researchers, endangering the vocation of scholarship in many sub-fields, and resealing many of the once slightly open doors in the profession where more critical, creative, and cutting-edge work could contribute to everyone's intellectual awareness. Meanwhile, the deeper flaws and darker failures of the discipline are neglected, ignored or forgotten.