II.2. Evidence from researcher interactions with human participants
To participate, you may either post a contribution to an existing discussion by selecting the thread for that topic (and then click on "Post Reply") or start a new thread by clicking on "New Topic" below.
For instructions on how to follow a discussion thread by email, click here.
LBJ School, U. of Texas
- Posts: 2
- Joined: Fri Nov 18, 2016 5:06 pm
Transparency may add noise, obstruct valid inference
My own interview research has been mainly with elites in recent or ongoing internal conflicts – i.e., the political and military elites of rebel/revolutionary/liberation movements and of the states they oppose. In my experience, the patterns of frankness of interviewees fall into three broad categories:
- Very frank from the start of the interview. This is typical of older, retired officials who have little to gain or fear.
- Initially reticent and parroting party line, but over time, as the interviewer demonstrates expertise and sympathy, becoming more frank.
- Parroting party line and refusing to open up throughout. This is typical of current and aspiring politicians.
As a result, transcripts typically are a mix of the following:
- Falsehood followed by Truth.
The job of the expert interviewer is two-fold: First, to elicit as much truth as possible from the interviewees using techniques that have been explained in the academic literature and elsewhere. Second, to separate the truth from the falsehood, using a variety of techniques that rely on the interviewer’s expertise of the case and research method. For example, if I know you lied in your last assertion, then I should take your next assertion with a grain of salt. Or if I hear a change in your tone of voice, or if you start blinking rapidly, or if you abruptly end the interview, it may signal a switch from honesty. By employing such techniques, the expert interviewer is separating the signal from the noise – and thus aiding the cumulation of knowledge.
The proposed requirement to make public the transcripts of interviews would effectively add back the noise to the signal. One can imagine a future researcher looking at the transcripts – without benefit of the interviewer’s case expertise, method expertise, and face-to-face presence with the interviewee – and reaching a very different conclusion based on weighing the noise and signal equally. Indeed, the future researcher could employ an ostensibly more “scientific” method, such as computerized textual analysis, to make inferences that would be far less valid – due to the well know problem of “garbage in / garbage out.”
This may not be the biggest concern with the proposal, but it is a substantial one, because it means the proposal could have an effect opposite to its stated intention.