Substantive Dimensions of the Deliberations
We encourage contributors to the Discussion Board to publicly identify by registering and logging in prior to posting. However, if you prefer, you may post anonymously (i.e. without having your post be attributed to you) by posting without logging in. Anonymous posts will display only after a delay to allow for administrator review. Contributors agree to the QTD Terms of Use.
Instructions
To participate, you may either post a contribution to an existing discussion by selecting the thread for that topic (and then click on "Post Reply") or start a new thread by clicking on "New Topic" below.
The transition to Stage 2 of the deliberations is currently underway but will take some time to complete. In the meantime, we very much welcome additional contributions to the existing threads in this forum.
For instructions on how to follow a discussion thread by email, click here.
-
Margaret Keck
Johns Hopkins University - Posts: 2
- Joined: Fri Apr 08, 2016 8:22 am
No place for my work in this debate
Post Reply
-
Alan Jacobs
University of British Columbia - Posts: 38
- Joined: Fri Feb 26, 2016 9:59 pm
[Steering Committee] No place for my work in this debate
You write in your post of a disconnect between the DA-RT framework and your work. When I read this, I immediately thought about how transparent I find your work. One particular way in which your work is unusually open – I am thinking here of your recent book Practical Authority, with Rebecca Abers – is in how explicitly you discuss the methodological difficulties that you confronted, the failure of initial hypotheses to find support in the empirical record, and the way in which the research strategy and the argument emerged as the investigation unfolded (including, in an extensive “Methodological Narrative” appendix). We often talk about case-study research as being rich with opportunities for inductive discovery and as a process that typically involves a back-and-forth movement between theoretical development and empirical analysis (as, of course, does much quantitative research). But in actual writeups of research findings, we do not often see the kind of transparency about that process that your book displays. It is also not a form of openness that has featured much in recent transparency discussions.
Would you be willing to comment on how and why you and Abers decided to be pursue this type of transparency in the book?
Post Reply
-
Guest
Re: No place for my work in this debate
Beyond what I have seen posted already, I hope that the APSA Section for Qualitative and Multi-Method Research will pay particular attention to the problems associated with archival research in the US and overseas, if it has not already. In addition, I hope the section will consider the issues associated with archival materials in foreign languages (as Dr. Keck points out). Complying with disciplinary citation standards in political science has already been a problem with archival evidence. The DA-RT framework makes it even more of one in several dimensions.
Post Reply
-
Margaret Keck
Johns Hopkins University - Posts: 2
- Joined: Fri Apr 08, 2016 8:22 am
Re: No place for my work in this debate
One further comment: There's a certain irony in the DA-RT discussion on archiving, which is focused on making research materials available to other social scientists through specialized sites, pre-sifted and translated and ready to be re-analyzed. I am currently engaged in finding archival homes for thousands and thousands of documents and other ephemera collected in the course of multiple research projects. These documents will be archived in libraries, where scholars and other people who are searching for materials in their subject areas can find them - in collections known to contain materials of this kind or via WorldCat. Some will be scanned and made available online. These will be original materials, in their original language, not translations. Although organizing papers for archiving is a lot of work, it's really useful, as anyone who has ever worked in a well-organized archive knows.
Post Reply
-
Guest
Re: No place for my work in this debate
For many of us, especially those who have been writing for years, these debates seem so disconnected from the realities of research. And we have plenty of other (inter-) disciplinary outlets if we chose to opt out of dysfunctional political science. Unfortunately, the price will be paid by emerging scholars, who will waste a lot of time and energy trying to digitize and archive everything in their research.
For those of us who work on topics and in environments where the focus is controversial (academically and/or politically) many sorts of "transparency" are either obvious (= must position oneself within theoretical debates which are equally ideological as academic) or dangerous (= immediate risks to interlocutors or possibly in the indeterminate future, because no one can accurately predict a coup or other type of policing). In my most recent book, I spent a few pages explicitly outlining these choices; it needed no more or less.
For those who have compiled resources which are truly a treasure trove, there may be better ways to archive them (as Mimi is doing). Even mundane resources can be valuable, albeit not worth institutional resources to scan/preserve. For instance, recently, I have been finding individual emerging scholars working on topics related to my earlier research who have been eager to receive packets of materials that I accumulated over the years but will never use again. I've hung onto them for decades on the off-chance someone might challenge my inferences--honestly, I'm not making that up--which NO ONE EVER did. It's time to triage, not post to a repository.
Post Reply
-
ReginaBateson
Re: No place for my work in this debate
To the extent that DART and JETS is about creating public goods to be used by other scholars, I feel that qualitative researchers in political science are largely being asked to duplicate work already done by oral historians, archivists, and librarians. Either our notes and ephemera are "scratch work" not worth disseminating because they would make little sense to others, or they are valuable primary sources -- in which case it would seem to make more sense to archive our papers with established institutions.
I am all for thorough, detailed citations to primary sources. But especially when using documents that are already in a well-organized archive--perhaps even a digital archive!--it seems redundant (and potentially a violation of the archive's policies) to ask us to scan or photograph those documents (in full? in part?) and re-post them in what would be, essentially, yet another archive just for political scientists. Such a standard would require considerable unnecessary work by qualitative scholars, while also forcing us to work well outside our areas of expertise. Most of us are not trained as archivists or oral historians. Why pretend to be?
Regina Bateson
MIT
Post Reply
-
Guest
Re: No place for my work in this debate
This seems to me what we could propose to strive for in certain subfields, instead of the blunt fact checking approach of Da-art.
Post Reply
-
Sarah Parkinson
Johns Hopkins University - Posts: 12
- Joined: Mon Apr 18, 2016 4:32 pm
Re: No place for my work in this debate
Is what Keck suggests fully possible outside of books? Is it/should it be incentivized in article-length scholarship? What are the implications for scholars at different points in their careers?
Post Reply
-
Sean Yom
Temple University - Posts: 3
- Joined: Wed Apr 20, 2016 10:43 pm
Re: No place for my work in this debate
separkinson wrote:I wonder what more people think of Mimi Keck's idea of "walking" readers through research; as she and others noted, it underscores a very different approach to transparency than the definition to which many subscribe. It also relates to some of Joe Soss's comments at last year's APSA, which is that many researchers engaging in intensive fieldwork frequently want more space in publications to discuss the stories behind and processes/decisions undergirding their research.
Is what Keck suggests fully possible outside of books? Is it/should it be incentivized in article-length scholarship? What are the implications for scholars at different points in their careers?
I would guess, Sarah, that some scholars would not want to be completely transparent (or let's call it honest) in divulging to their readers, and students, what precise sequence of steps they undertook (and the mistakes they made) in order to end up with the publishable manuscript in article or book form. We want readers to be interested in the sausage, not how it's made. DA-RT is like the FDA ordering a manufacturer to tell consumers where it is getting the ingredients for the sausage, but that's it. Consumers still don't know, precisely, how it's being made in that opaque factory behind locked doors.
On the other hand, many of us do want to be more transparent, but publishing norms in article or book form prevent us from disclosing all of these messy mental steps and random empirical mishaps that occurred along the way of crafting our scholarship.
Post Reply
-
Tim Buthe
HfP/Technical Univ of Munich & Duke University - Posts: 32
- Joined: Fri Feb 26, 2016 11:39 pm
Re: No place for my work in this debate
It may be that many readers are only interested in the sausage and not the "sausage-making," i.e., information about the research process. And for popular books or intro-level textbooks focusing on the key research findings may be fine. But it seems to me that we as scholars--at least those of us who embrace the "scientific" ambitions of Political Science (appropriately broadly understood)--must be interested both in learning about other researchers' process and sharing information about our own. Such information is essential for learning from each other (including the mistakes). And it is impossible to really interpret and assess the research findings without knowing how they were obtained.
An important question then is: How can we provide the most important and most useful information without unduly high costs to readability? Marc Trachtenberg, in his article in the QMMR symposium on Transparency in Qualitative and Multi-Method Research (http://ssrn.com/abstract=2652097), not only identifies this tension between increasing transparency and writing for a broad audience; he also makes some suggestions for overcoming/reducing that tension, yet much more can and should surely be said about that topic.
Post Reply
-
Sarah Parkinson
Johns Hopkins University - Posts: 12
- Joined: Mon Apr 18, 2016 4:32 pm
Re: No place for my work in this debate
If we're going to stick with the food metaphor, I wonder if talking about the proverbial distinctions between: 1.) Sharing exact ingredients; 2.) Providing a general "recipe" (the kinds of things that we do to gain access, the general and likely context variant types of "ingredients" used, general strategies of generating evidence, and ways of engaging with data), and; 3.) Sharing knowledge about what it takes to "make sausage" (meat grinder) versus "bake a cake" (mixer) would be useful.
Lunchtime.
Post Reply
-
James Heilman
University of Massachusetts-Amherst - Posts: 3
- Joined: Tue Apr 26, 2016 8:33 am
Re: No place for my work in this debate
1) I like the idea of walking the reader through the research process. I wonder if our in-the-field practices will change so that we can more easily make our research process transparent for articles than we currently do. Training for graduate students would have to include some practices that field researchers could make use of so that when they start writing they are able to easily take field notes and make them transparent for an article or appendix to an article.
2) Some people in this thread have mentioned that one option is to not publish in journals that sign onto DART. As a graduate student I am concerned about how this could effect how others evaluate my work. If many of the top journals implement transparency requirements that are extra-onerous or impossible to fulfill for some types of qualitative research, then I might have to publish in journals that could be well recognized by peers in my sub-discipline but might not be as well respected by the political science discipline at large. Therefore, this could put me at a competitive disadvantage when I am on the job market. Since hiring committees are not discipline specific, I strive to publish in top-tier journals so that when people on a hiring committee debate my CV there cannot be an argument about the quality of the journals I have published in. Furthermore, once I am a faculty member working towards tenure, I will continue to want to publish in widely respected journals in order to make my tenure review as smooth as possible. I recognize that as new transparency standards are enforced the reputations of journals will change in ways we cannot predict. I hope that a sufficient number of journals that are respected throughout the discipline and publish a wide range of research topics and methodologies remain and that the discipline does not lose they type of journals.
Post Reply
-
Joe White
Case Western Reserve University, Political Science - Posts: 1
- Joined: Thu May 19, 2016 2:22 pm
Re: No place for my work in this debate
Confidential interviews are gained on the basis of trust relationships. I promise full confidentiality. I am referred to people by those whom I have interviewed before. I go back to people whom I have interviewed before, because I want to understand change and can ask them to compare situations. I draw my own conclusions about who can be trusted in what ways, and then I have to provide information about my sources that is clear enough to provide some sense of their ability to speak to an issue and yet vague enough so that they cannot be precisely identified. They can have very good reasons not to want to be identified.
Here is an example: I did some work on budgeting during the GWBush years, and in particular decision-making within the executive branch. That involved interviewing within and about OMB. In what I wrote, I made claims about how the administration made decisions about budget totals. One source I called something like, "a highly ranked OMB political official" - you know the usual locutions.
The most reasonable true identity for such a person would be the OMB Director. The good news is, there were four of them during the administration. The bad news is, one is a sitting U.S. Senator who might get nominated for Vice President; a second is a long-serving former Governor who was frequently mentioned as a potential candidate for president and still might be some day; a third is out of politics for now but is running a significant interest group and might well go into a GOP administration even if he doesn't run for office again; and the fourth probably will never be a candidate but could be, oh, nominated for Secretary of the Treasury or go back to being White House Chief of Staff. If my source happened to be an OMB Director, am I really supposed to share my interview, with identifying information? But if I don't give the identifying information, how does anyone else know how well-positioned the person was to describe what I quote him describing?
I just don't see how DA-RT could possibly work reasonably with elite interviewing. I recognize the problem with asking people to trust me. Heck, I've constantly wished I could identify my sources - for example, when a politician makes an argument in public that s/he has directly contradicted in an interview. It bothers me to be sitting on more than a thousand pages of interviews that others don't get to see (well, once or twice I have shared material, with many strong promises of confidentiality, with people I think I know well enough to trust; but even that makes me nervous). But I talk to staff people who are not free to be public. I talk to bureaucrats who are in difficult situations balancing demands from their political bosses and Congress. I talk to advocates about how they pursue their objectives, which might not be quite the same as their public self-presentations I ask people about how they do their jobs, and ask for examples, and the examples would identify them or at least greatly narrow the possibilities.
Yes, you could conclude that I do "journalism" which is not "political science." But if the discipline forces its members to meet strict standards of sharing and transparency, then only journalists will be in a position to do some of the most basic investigations of political behavior. That's nuts.
Post Reply