Page 1 of 1

How the Rule of Transparency Can Become Self-Destructive for Political Science

Posted: Fri Dec 23, 2016 5:29 pm
by Bill Kelleher
Marcus Kreuzer et al. pose several thoughtful and central questions that deserve careful consideration if the DA-RT “transparency” challenge to non-positivistic methods of political science inquiry is to be fully understood. (See their post introducing this thread at viewforum.php?f=13)

Kreuzer also reminds us that DA-RT proponents have already had the APSA formally adopt a rule of methodology to govern the work of all political scientists. He writes,
As articulated in the 2012 APSA Guide to Professional Ethics in Political Science, “Researchers have an ethical obligation to facilitate the evaluation of their evidence-based knowledge claims through data access, production transparency and analytic transparency so that their work can be tested and replicated.” (See his post “Setting the Stage” at viewtopic.php?f=13&t=134)

But I argue that for Interpretive Political Science, which uses multiple methods, “testing and replication” are the key terms of which to beware.

In a letter dated 1-7-16, 20 APSA presidents expressed their concern over some of the implications for research of the Journal Editors Transparency Statement (JETS). The letter was sent to the editors of 27 journals who signed the statement. But the ideal of the replicability of research was not an object of critical examination. Indeed, they praised JETS for upholding the ideal as a part of “the long efforts to improve research transparency and facilitate replication in political science. Many of us have strongly supported these efforts.” Which efforts they characterize as “the replication and transparency movement,” at ... uidelines/

This letter and that ethics percept show how deeply ingrained positivistic assumptions about science are in our profession’s ethos. The assumptions are that transparency is important as a way to facilitate replication, and replication is important as a way to test and evaluate the validity of research knowledge claims. These notions are derived from a positivistic idealized model of natural science, which Clarke and Primo call the Hypothetical-Deductive Model (see my article in Perspectives on Politics*).

Consider the term “transparency” as having two significant uses. As an ethical ideal, transparency implies honesty, and no one can deny the importance of honesty in science. But as a methodological ideal, insofar as transparency implies the replicability of studies in political science, it is deeply problematic.

For the DA-RT positivists, professional ethics requires that journal articles describe their methods, and have an appendix and/or stored information that would facilitate replication. But this principle is a threat to multi-method Interpretive Political Science, both because subjects have privacy rights, and because interpretation relies very much on empathy. Empathic interpretation is a highly personal research practice. Hence, it is generally too unique to be replicated. It requires a different set of standards for verification. (For more on how unique interpretive studies are threatened by the replicability requirement see my proposed Community Transparency Statement for Interpretive Political Science on this site at viewtopic.php?f=21&t=195#p800)

Of course, fundamental to any science is the principle of Reason requiring that all studies be methodical. In line with this principle, knowledge claims from interpretive studies can be made “transparent” by discussing the methods used. Even if such studies cannot be replicated, they can be criticized. The criticism of the methods and criticism of the knowledge claims derived therefrom can be conducted both prior to publication and thereafter. Along with the experts, any informed persons can make their critiques of a study known. Through this criticism, interpretive knowledge claims can be validated in such degree as there is consensus within the community of experts. Because unique studies cannot, by definition, be replicated, replication is an inappropriate professional standard.

As I have mentioned, at viewtopic.php?f=18&t=200#p846, Alice Goffman’s study, On the Run, is an excellent example of empathic interpretation. She is fully transparent about the methods of her study. While her study is too dependent on her unique abilities and circumstances to ever be exactly replicated, her methods and findings can be criticized and validated in whole or in part.

Under the rule of replicability, a study like Goffman’s would have to be denied recognition as important for political science due to its uniqueness. Because it would exclude important knowledge, this is a self-destructive rule rather than a helpful rule for our profession.

Why is Goffman's study an exemplar of method and of important knowledge for political science? The answer is readily given within the context of David Easton’s definition of political science as the study of the political system, particularly as to how a political system is able to persist over time.^ Goffman’s study was of an aspect of the US criminal justice system (CJS). The CJS is a subsystem of the political system. It can be understood as having the function of aiding or facilitating the persistence of the political system.

Goffman’s study exposed some deleterious unintended consequences of a CJS public policy; namely, a major city’s “get tough on crime” policing policy. As knowledge for political science, her study raises the question as to whether the continuation of such a policy aids or threatens the persistence of the political system. Such knowledge is vital to the political system. If its policies are headed towards self-destruction, then it should be made aware of that. In this way, political science studies can serve as an alert system for the political system. Goffman’s study started out as a pure research project without expectation as to its results. The results were a discovery made in the course of the study.

Whether or not the city involved corrects its CJS public policy, Goffman’s study illustrates Easton’s vision of political science as being empirical, methodical, and able to produce knowledge which is of vital importance to the political system.

One of the unintended consequences of the positivistic insistence that replicability is the key to validating political science knowledge is the continuing decline of esteem for the profession among political professionals and the public. Our work is often criticized as heavy on method but light on heft. (See my discussion of this in my paper on Good Work in Political Science at )

Therefore, the APSA ought to jettison the DA-RT project as a threat to the best interests of the profession. Additionally, APSA resources should be used to inform its membership of the vital role Interpretive Political Science can play in furthering the interests of the profession, especially as to its esteem in society.

William J. Kelleher, Ph.D.

*A Model Discipline, and How “Good Work” Hurts Political Science William Kelleher Journal: Perspectives on Politics / Volume 13 / Issue 2 / June 2015 DOI: Published online: 18 June 2015, pp. 446-448 ... 8C738B9BA5

^David Easton, A Systems Analysis of Political Life; A Framework for Political Analysis; The Political System

Re: How the Rule of Transparency Can Become Self-Destructive for Political Science

Posted: Sun Jan 01, 2017 9:14 pm
by Guest
The argument of Khelleher’s submission that the adoption of DA-RT would be a form of self-destruction is quite dramatic. The more than 800 views demonstrate the interest in, and apprehension from this possibility. I think the extension of time for further deliberations proved to be a good idea. It really increased the number of contributions especially to areas members care about the most. My general impression- like everyone else I wait for the result’s summary- is that the deliberations enhanced, with crucial qualifications, the case for DA-RT in qualitative research. I especially benefited from the conversations in “presentation of evidence”, finessing case selection, and outlining the range of vulnerabilities in marginalized populations. The deliberations highlight the depth of reflection that goes into the research process with all the attendant vagaries. The submissions are combined with a refreshing frankness as to what remains simply tentative. This behind the scenes look at knowledge production reassured me, as I think it did others, that one is not alone in this difficult, self-chosen (perhaps from the individual perspective self-destructive) task.
I have read other pieces of Kelleher and I tend to agree with some of what he says, but not in the conclusion of this specific submission. He is right, qualitative work is sadly undervalued and is now asked to bare “all”, most or some of its data. Qualitative data exist mostly in narrative form, not hidden by operational variables ready for statistical play. In addition, it is rather true that "positivistic assumptions about science" are deeply ingrained in the field. Others have called this tendency 'physics envy'. But, one may say that the qualitative (ists) have a form of 'anthropology envy' with their stress on ethnography. Furthermore, other fields such as literary criticism may have psychoanalytic and even political envy. The way the words politics and ideology hang on the titles of so many works in the humanities, one can say that there is a great deal of "political envy". The word envy, even in jest, connotes at least some unsavory intent while the central issue pertains as to how a field or sub-field can maintain its integrity even with so much cross-borrowing among and internal to specific disciplines. For example, the more recent emphasis on political interpretations of literature and film may prove to be crucial for a more refined understanding of the drama, farce, comedy and tragedy of politics, analyzed and appreciated from a politically centered perspective. Although empathy plays an important role in interpretative political science, as Kelleher stresses, but so does distance, a capacity for self-reflection, a tuned conscience, among other character traits, proper attitudes, and virtues crucial for rational thought.
In political science, multi-method approaches, with method often standing for theoretical perspective, require a form of organization appropriate to the aims of the particular study. Ideally, each work has multiple theoretical and empirical dimensions that could be integrated while remaining open to later developments. Integration is a key concept appearing in several fields and studies adjacent to political science. I have recently encountered it in the work of Sanjay Subrahmanyam, who credits the late American scholar Joseph F. Fletcher (a Harvard professor who passed too early) with being his guide in “Explorations in Connected History”, by connected he means integrated. Also, the work of Sarah Stroumsa, at the Hebrew University of Jerusalem, uses integrative approaches in her field of Jewish-Muslim relations in the Medieval era. Multi-method approaches may constitute a unique combination (constellation, in Adorno’s term) appropriate to its object of study, but that does not translate into absolute uniqueness that nullifies comparisons, and thereby duplication. Kelleher argues for jettisoning DA-RT “Because unique studies cannot, by definition, be replicated, replication is an inappropriate professional standard” (admittedly the very sentence that prompted this response). Kelleher is more to the point when he states that some unique studies cannot be replicated but only criticized and only partially validated, which is a good start from my perspective. I discussed my work on Ibn Khaldun in the last issue of CLIO where I argue that the duplication of findings, in this case between the 14th century and modern research is key to knowledge claims. More generally, repetition, even of the most destructive kind, is an inescapable part of social processes. It would indeed be self-destructive to not use our increasingly complex methodologies to pursue such patterns.
Lillian Farhat
Independent Scholar