I.3. Power and Institutions
We encourage contributors to the Discussion Board to publicly identify by registering and logging in prior to posting. However, if you prefer, you may post anonymously (i.e. without having your post be attributed to you) by posting without logging in. Anonymous posts will display only after a delay to allow for administrator review. Contributors agree to the QTD Terms of Use.
Instructions
To participate, you may either post a contribution to an existing discussion by selecting the thread for that topic (and then click on "Post Reply") or start a new thread by clicking on "New Topic" below.
For instructions on how to follow a discussion thread by email, click here.
-
Tim Buthe
HfP/Technical Univ of Munich & Duke University - Posts: 32
- Joined: Fri Feb 26, 2016 11:39 pm
Power and the Institutionalization of Research Transparency/Openness/Explicitness
Post Reply
-
Guest
Re: Power and the Institutionalization of Research Transparency/Openness/Explicitness
One statement of this topic I have seen appears to set up a binary between "rules with enforcement" and "norms with empowering capacity-building or enaabling pedagogy."
I think it is more helpful to think in terms of 8 possibilities, which I will outline with a decision tree-type notation:
no capacity-building pedagogy 1
enforcement capacity-building pedagogypedegogy 2
rules
enabling no capacity-building pedagogy 3
capacity-building pedagogy 4
no capacity-building pedagogy 5
enforcement capacity-building pedagogypedegogy 6
norms
enabling no capacity-building pedagogy 7
capacity-building pedagogy 8
Rules need followers, and hence enabling; if rules are supported only by enforcement the enforcement gets overwhelmed; internalizing norms enables, but norms also need enforcement against those who won't internalize. So my own take is that we are really talking about a combination of 4 and 8 because people will need to learn stuff to be able to understand and follow rules and norms.
MJ Peterson
University of Massachusetts Amherst
Post Reply
-
Jane Mansbridge
Harvard Kennedy School - Posts: 8
- Joined: Sat Apr 23, 2016 4:53 pm
Re: Power and the Institutionalization of Research Transparency/Openness/Explicitness
The less well-resourced the institution the scholar works in, the more that scholar is underfunded.
Having to incur costs to convert field notes or interviews into the appropriate forms for deposit and having to pay fees for depositing these notes and interviews in a qualified depository might well prove impossible for such researchers.
I worry not only that these added costs will fall disproportionately on the least well-resourced scholars (who are often closer to settings that merit qualitative research), but also that these costs will discourage younger scholars from taking qualitative approaches. Choosing a dissertation topic, they may find it much easier and attractive just to use a database already in the public domain. This predictable dynamic discourages innovative research, making it harder for new questions to come on the political science agenda.
Post Reply
-
Tasha Fairfield
LSE - Posts: 17
- Joined: Mon Sep 05, 2016 4:05 pm
Re: Power and the Institutionalization of Research Transparency/Openness/Explicitness
jane mansbridge wrote:I worry not only that these added costs will fall disproportionately on the least well-resourced scholars (who are often closer to settings that merit qualitative research), but also that these costs will discourage younger scholars from taking qualitative approaches. Choosing a dissertation topic, they may find it much easier and attractive just to use a database already in the public domain. This predictable dynamic discourages innovative research, making it harder for new questions to come on the political science agenda.
Post Reply
-
Guest
Re: Power and the Institutionalization of Research Transparency/Openness/Explicitness
I would like to echo this concern that the costs of these new transparency standards will lead young scholars (in particular, but not only) to choose easier and lower-cost methods and questions that can be answered with those methods. This is already happening, with more and more new PhDs doing survey experiments, which can be outsourced to a survey research firm and whose external validity can be questionable, instead of spending months in one or more countries doing interviews or archival research. The latter is increasingly devalued, and the DART transparency standards promise to further accelerate this trend, to the detriment of the discipline's ability to accumulate real knowledge about complex problems.
Post Reply
-
Michelle Jurkovich
University of Massachusetts Boston - Posts: 6
- Joined: Sun Apr 24, 2016 10:05 am
Re: Power and the Institutionalization of Research Transparency/Openness/Explicitness
jane mansbridge wrote:In my experience, many qualitative researchers do not have grants (or at least extensive grants). They do much of their research on their own time, using their own personal monetary resources. They may receive a travel grant, but rarely a grant that actually covers all of the expenses for a field trip, let alone the time required to write up the results
I'd like to echo Jane Mansbridge's point above. Transcribing interviews has been a far more expensive and time consuming process than I'd expected and certainly not one I would have been able to do if I hadn't been fortunate to receive financial support from my institution. My transcriptions are not for public viewing (part of the agreement I made with interviewees in securing informed consent) and the cost and time would have been even greater if I'd needed to worry about making them viewable by the public (i.e. removing identifiers), which again, I can't do as that was not part of the agreement I made with participants. Leaving aside all ethical and human subject concerns (which are many!!!!) if journals are not going to provide the funding for this process it will place a severe financial hardship on qualitative scholars---many of whom will simply not be able to submit to these journals if there is such a high financial burden to do so.
Post Reply
-
Sandra Resodihardjo
Radboud University - Posts: 3
- Joined: Thu Apr 28, 2016 8:42 am
Re: Power and the Institutionalization of Research Transparency/Openness/Explicitness
(1) respondents will be less likely to agree to an interview if they know the transcript of the interview will be made public
(2) true anonimity can never be guaranteed when 'sanitizing' transcripts from personal characterizations. There was only one person who was, for example, Home Secretary when prison escape X happened so the transcript can be easily linked to the respondent.
And if respondents do agree, they might censure themselves as they know that their statements will be made public. Consequently, it will be more difficult for researchers to do their research.
Moreover, this proposal raises numerous ethical and legal questions, including
(a) questions about national data protection/privacy laws which may vary per country and thus affect researchers differently (i.e. making it impossible to publish in US journals)
(b) re-use of data even though respondents only agreed that their data could be used by the original researcher (i.e. they agreed to be interviewed for a certain research, with a specific research question)
The question is: what are we hoping to achieve by all of this? More transparancy? Better research? The follow up question is: do the proposed solutions really help to achieve this?
I'm not in favour of creating a one size fits all format for research - as it will definitely not fit all sizes. It's better to set a minimum level of requirement for all research (clear research/methodology section; proper referencing as citations can still be sorely lacking - and yes that is partially due to word limit) and then add additional information based on what is needed. So if you do qualitative research and opted for inductive content analysis of interviews, it's not the interviews that need to be published online, it is the complete codebook with an explicit explanation of what you did in each round of coding. Depending on the type of research, these codebooks can be quite lengthy. If you opted, for instance, for deductive media analysis, the codebook needs to be published online as well. In both cases, researchers need to explain in their methodology section what they did, followed by a reference to the online codebook for those who want to replicate the study.
Post Reply
-
Mneesha Gellman
Emerson College - Posts: 11
- Joined: Thu Apr 07, 2016 8:20 pm
Re: Power and the Institutionalization of Research Transparency/Openness/Explicitness
I also have concerns about what this financial burden of transcription means in terms of widening the inequality gap between faculty at different kinds of institutions, and with different profiles. At risk of oversimplifying, will white, male R1 faculty get the grants that allow them to "do it all," while female LAC faculty continue to make these trade offs? The "burden of proof" for research risks exacerbating an already problematic equity divide in academia. Mix this with a methodological referendum like DART, and the dangers to a certain profile to which I happen to belong (female LAC faculty doing qualitative research with ethnic minority social movement participants in sometimes politically unstable places) seem rife.
resodihardjo wrote:I concur with those stating that the proposed policies for opennes and accessability of data favour quantitative research and make it less likely for students and reseachers to opt for qualitative research (either by design or because they no longer have access to data). It is, however, not the cost of transcribing that I think is the most problematic (as transcribing interviews is often (but not always) needed for a proper analysis of the interviews), but the access to respondents if researchers are forced to make their interviews public. After all,
(1) respondents will be less likely to agree to an interview if they know the transcript of the interview will be made public
(2) true anonimity can never be guaranteed when 'sanitizing' transcripts from personal characterizations. There was only one person who was, for example, Home Secretary when prison escape X happened so the transcript can be easily linked to the respondent.
And if respondents do agree, they might censure themselves as they know that their statements will be made public. Consequently, it will be more difficult for researchers to do their research.
Moreover, this proposal raises numerous ethical and legal questions, including
(a) questions about national data protection/privacy laws which may vary per country and thus affect researchers differently (i.e. making it impossible to publish in US journals)
(b) re-use of data even though respondents only agreed that their data could be used by the original researcher (i.e. they agreed to be interviewed for a certain research, with a specific research question)
The question is: what are we hoping to achieve by all of this? More transparancy? Better research? The follow up question is: do the proposed solutions really help to achieve this?
I'm not in favour of creating a one size fits all format for research - as it will definitely not fit all sizes. It's better to set a minimum level of requirement for all research (clear research/methodology section; proper referencing as citations can still be sorely lacking - and yes that is partially due to word limit) and then add additional information based on what is needed. So if you do qualitative research and opted for inductive content analysis of interviews, it's not the interviews that need to be published online, it is the complete codebook with an explicit explanation of what you did in each round of coding. Depending on the type of research, these codebooks can be quite lengthy. If you opted, for instance, for deductive media analysis, the codebook needs to be published online as well. In both cases, researchers need to explain in their methodology section what they did, followed by a reference to the online codebook for those who want to replicate the study.
Post Reply
-
Ecaterina Locoman
Rutgers University - Posts: 1
- Joined: Tue Nov 29, 2016 5:23 pm
Re: Power and the Institutionalization of Research Transparency/Openness/Explicitness
There is also the question of AVAILABILITY of documents: one site which I visited for my field research did not allow me, as a researcher, to take photos of the documents – I could only read and take written notes. If I use some of these sources and quotes in a future article, how am I supposed to show “proof” of the original document where I took the quote?
As some of the scholars above wrote, we, as researchers using qualitative tools, are put at a disadvantage in rapport with the colleagues using quantitative methods: not only it is taking me more than 12 months to read, translate and annotate all these documents from other languages into English , I have to also worry about making them available. I have colleagues in graduate school who deliberately do not use qualitative methods, because they want to graduate quickly, find a job and not spend years doing their PhDs.
Post Reply
-
Guest
Re: Power and the Institutionalization of Research Transparency/Openness/Explicitness
That being said, I think that it is important to acknowledge that qualitative data sharing can also empower people in our profession (like me) who have limited access to resources for field research, archival visits, and so on. Let me give an example. Some other political scientists working in my research area conducted >100 interviews that they subsequently published online in the form of an oral history project several years ago. I -- and other researchers -- have made great use of that oral history project's interview transcripts to answer different research questions. I have also used interview transcripts housed in oral history projects produced by international organizations several other times in the past. When I haven't had the ability to get overseas because I didn't have access to enough research money -- and when I had trouble as a graduate student getting people to respond to my e-mails with requests for interviews -- this kind of data sharing really helped me.
So yes, transcribing interview notes or making archival notes available takes a lot of resources. And yes, doing that is often impossible from a ethical or legal perspective. (And yes, for researchers working in an ethnographic tradition, the idea of sharing ones interview notes to promote "replication" or "transparency" or whatever is absurd.) Yes to all of that. HOWEVER, making available interview transcripts (or archival documents) -- ONLY when feasible, ethical, and legal, of course -- can also really help under-resourced and junior scholars. This gain should also be acknowledged by those who are concerned about equity issues in our profession.
I'm not sure what the solution is. If we reward researchers who provide the public good of this kind of data access, then we are more likely to reward the researchers who are already well-resourced. Then again, if we reward researchers who provide the public good of this kind of data access, then I think more people will do it, and I do think there is something real that under-resourced scholars can gain from that kind of data access.
Post Reply
-
Vicky Murillo (Columbia University)
Re: Power and the Institutionalization of Research Transparency/Openness/Explicitness
The less well-resourced the institution the scholar works in, the more that scholar is underfunded.
Having to incur costs to convert field notes or interviews into the appropriate forms for deposit and having to pay fees for depositing these notes and interviews in a qualified depository might well prove impossible for such researchers.
I worry not only that these added costs will fall disproportionately on the least well-resourced scholars (who are often closer to settings that merit qualitative research), but also that these costs will discourage younger scholars from taking qualitative approaches. Choosing a dissertation topic, they may find it much easier and attractive just to use a database already in the public domain. This predictable dynamic discourages innovative research, making it harder for new questions to come on the political science agenda.[/quote]
This is Vicky Murillo @ Columbia University (unable to login after registration):
In response to Jane Mansfield comments, which echo those of Peter Hall in the CP newsletter, I want to add another issue to the discussion.
Even if the journal or depository pays for the cost of conversion of the interviews or notes into digital format, which I believe should be the case to make it equally affordable to all scholars, it seems to me that the issue of time and the issue of contextual accuracy would still be pending.
First, contextual accuracy is crucial in the interpretation of data either quantitative or qualitative. We have talked about the obvious methods of triangulation and relying on multiple sources but sometimes even if a 3 or 4 interviews coincide, these may not be the adequate person to interview or all of them may be misrepresenting reality. It seems to me that beside the obvious check of blind review in journals, there should be some mechanisms to involve reviewers from the region when the relevant work is on comparative politics. Reviewers from the region would hopefully be less tied by internal academic networks and more akin to the interpretation of whether the choice of actors to be interviewed or their representation of reality may be wrong. Currently, American academy relies on data (quantitative and qualitative) collected by US trained scholars and we have to accept that is true even when local scholars may have made very different choices in terms of which data is relevant.
Second, the one year limit in turning qualitative data for repositories is not reflective of the real time it takes to analyze this data given other obligations in my personal experience. It seems to me that having finished the project (book for instance) is the time that is right for the scholar to make his/her data available (and as I said the cost should be paid by repositories of making the data digital) and it is in her/his interest to do it so that others can cite his/her work and check his/her findings to contribute to the scholarly debate.
Post Reply
-
Robin Turner
Butler University - Posts: 3
- Joined: Fri Apr 08, 2016 5:02 am
Re: Power and the Institutionalization of Research Transparency/Openness/Explicitness
These data transparency costs are far from proportionate given the limited utility of a word-for-word transcript read without understanding not only of the research context (see Vicky Murillo, this section, Aili Tripp II.C, http://tinyurl.com/gwtox3v) but also of the particular interpersonal interaction between that researcher and participant (see Tim Pachirat, I.1, http://tinyurl.com/j9wp6a2).I see research as "a specifically relational and deeply intersubjective enterprise" (Tim) in which no data is truly raw or unfiltered. Present disciplinary norms call for scholars to state "what work we've done, and how" (Amanda Fulmer, II.2, http://tinyurl.com/z6wcjc4) but don't require everyone to address positionality. I see this disregard as a greater threat to valid research inference than narrowly defined data transparency (my work in progress).
Like many QTD deliberation participants, I am deeply concerned that DART will harm qualitative inquiry by further marginalizing qualitative/interpretive scholarship and damaging the interactions with research subjects/participants through which many political scientists generate data.
Post Reply
-
Timothy Luke
Virginia Polytechnic Institute & State University - Posts: 1
- Joined: Wed Aug 31, 2016 6:25 am
Re: Power and the Institutionalization of Research Transparency/Openness/Explicitness
"correct transparency practices," or "examination board certification" of how "good science" is performed with the discipline
as recognized by professional organizations, associations for college/university accreditation, and state councils of education.
Once again, the desires of many anxious social scientists all to eager to prove their equality with biomedical, natural or
physical sciences could be affirmed by requiring members in the discipline to submit to these trials of legitimacy.
Post Reply