a working group with journal editors
Posted: Wed May 25, 2016 11:30 am
While QTD is a discussion that aims to be broader than the discussion around DART and JETS, I think it is still important that part of the discussion engages DART and JETS because people are coming to these boards in part motivated by these changes and thanks to the protest movement that highlighted the discussion. Thus I want to add some of my thoughts about QTD insofar and what maybe could be helpful going forward with respect constructing together a proposal to make JETS clear and operational that journals can adopt and some changes in the wording of JETS when required.
I think that the discussion around JETS has insofar well captured the extreme 'realizations' of the possible scenarios and realities facilitated by JETS. On one hand QTD has clearly captured a variety of legitimate fears regarding potential extreme consequences of JETS, on the other the article of Elman and Lupia in the CP newsletter (here) provides a "nothing will change" counter message that combined with some of the older discussions around active citations as precondition for data replication (here) instead of engaging fears just dismisses them as miss-construed miss-understanding or reactions from luddites that have something to hide or do not know anything about 'good' methodology.
This is fairly normal given that QTD offered the first space for open discussion and thus vented all the things that had been badly compressed before. The discussion that preceded QTD simply did not do enough to engage the public. QTD with zero funding and a tiny team of volunteers has reached more than a 1500 people, many not in R1 universities, proving that the claim that nobody wants to discuss these things is empirically wrong.
For JETS purposes we have mapped well the extremes, now I think we should focus on the things the lie in between or we risk to not offer any operational solution.
For example, and the following are just 4 examples:
1) Active citations
what if active citations did not require to scan anything, but simply allowed people to have more space for a citation? When I cite my interviews, or a document that I think is important for my argument, I am always super-constrained by the word limit of the paper. My citations end-up being a one liner. If I had a hyperlink in which I could cite a larger portion of the document or interview that would be quite useful. Maybe I could even have an entire page to explain the context in case the point I am making is not linked to a specific smoking gun, but a series of interviews. This approach envisions the hypertext of an active citation as a "peak under the hood" additional space for explaining things for the curious reader. These hyperlinked texts would be totally under the control of the author and would have nothing to do with a data replication effort that as many colleagues have argued in QTD is difficult to apply as soon as we try to be a bit more nuanced. So my point is can we simply transform active citations in a space for more explanations that does not require any mandatory costly scanning/uploading? I think we can, and the discussion on this board about the meaty footnote (if I am not miss-understanding see here) goes in such direction and I think it is an example of operational suggestions that is not trapped in the extremes that exemplifies how I would recommend we go forward to build together a proposal to operationalize/amend JETS.
2) Human subject protection
I think that nobody in their right mind would require to endanger human subjects and nobody would ask to violate IRB compliance. I think that all editors agree with this and it is just an unfortunate wording of JETS that makes things unclear. We should propose a simple amendment that makes things clear. My personal sense is that this crucial heated debate stems from a miss-understanding so it is probably the easiest to solve on one level, on the other it will require a change in JETS wording, and given how defensive the proponents of JETS have been insofar it might not be as easy and it might require another petition or protest. Going forward I think that one operational frame could be "we all agree that human subjects come first, can we add a line in JETS that highlight such concept? It's a bit vague and we do not want to send a vague message on such important subject" or something like that. I know that there have been previous attempts to do such thing and I am proposing to just do another one. But my point is that such attempt should come from the understanding that nobody wants to endanger human subjects and it is a shared value, so it is just the vagueness we need to resolve not some profound ethical and divisive debate. Maybe we do not need to change the wording of JETS, but we just need to ask the journals to add a strong operationalization of such rules in their bylaws.
3) Editors role & engaging more the editors
On one hand it is obvious that the editor of a journal has complete control on what gets published. On the other why reify such obvious operational thing in JETS? What is the reason to do that? This is one of the things I do not understand of JETS and I would like an explanation. Going forward I think we need to involve as many editors as possible in this conversation. I have noticed one or two posts from editors. Maybe we should have a working group dedicated to editors and asking questions to editors. They are not some strange aliens, they are colleagues and many must be as confused as we are about JETS.
4) Clarifying once and for all that there is no mandatory added costs for qualitative research
A lot of people in these boards have repeated that their research would be impossible due to added costs. My sense is that nobody wants that and actually journals would like more qualitative research being produced and have a hard time getting more. At the same time JETS is so vague that the fears of our colleagues should not be dismissed and we need a clear understanding about what will be required operationally. So this links with point 3, we need to talk to editors, without such exchange we risk to be trapped in a non operational discussion. I do not think any editor would require uploading my badly scrabbled irrelevant field note, all the transcripts of my interviews etc. etc. So what would they require in practice?
In conclusion, I propose we build 1 working group that will focus on engaging the editors and operationalizing/amending JETS. This way in all the other working groups we can focus on more relevant and interesting discussions.
I think that the discussion around JETS has insofar well captured the extreme 'realizations' of the possible scenarios and realities facilitated by JETS. On one hand QTD has clearly captured a variety of legitimate fears regarding potential extreme consequences of JETS, on the other the article of Elman and Lupia in the CP newsletter (here) provides a "nothing will change" counter message that combined with some of the older discussions around active citations as precondition for data replication (here) instead of engaging fears just dismisses them as miss-construed miss-understanding or reactions from luddites that have something to hide or do not know anything about 'good' methodology.
This is fairly normal given that QTD offered the first space for open discussion and thus vented all the things that had been badly compressed before. The discussion that preceded QTD simply did not do enough to engage the public. QTD with zero funding and a tiny team of volunteers has reached more than a 1500 people, many not in R1 universities, proving that the claim that nobody wants to discuss these things is empirically wrong.
For JETS purposes we have mapped well the extremes, now I think we should focus on the things the lie in between or we risk to not offer any operational solution.
For example, and the following are just 4 examples:
1) Active citations
what if active citations did not require to scan anything, but simply allowed people to have more space for a citation? When I cite my interviews, or a document that I think is important for my argument, I am always super-constrained by the word limit of the paper. My citations end-up being a one liner. If I had a hyperlink in which I could cite a larger portion of the document or interview that would be quite useful. Maybe I could even have an entire page to explain the context in case the point I am making is not linked to a specific smoking gun, but a series of interviews. This approach envisions the hypertext of an active citation as a "peak under the hood" additional space for explaining things for the curious reader. These hyperlinked texts would be totally under the control of the author and would have nothing to do with a data replication effort that as many colleagues have argued in QTD is difficult to apply as soon as we try to be a bit more nuanced. So my point is can we simply transform active citations in a space for more explanations that does not require any mandatory costly scanning/uploading? I think we can, and the discussion on this board about the meaty footnote (if I am not miss-understanding see here) goes in such direction and I think it is an example of operational suggestions that is not trapped in the extremes that exemplifies how I would recommend we go forward to build together a proposal to operationalize/amend JETS.
2) Human subject protection
I think that nobody in their right mind would require to endanger human subjects and nobody would ask to violate IRB compliance. I think that all editors agree with this and it is just an unfortunate wording of JETS that makes things unclear. We should propose a simple amendment that makes things clear. My personal sense is that this crucial heated debate stems from a miss-understanding so it is probably the easiest to solve on one level, on the other it will require a change in JETS wording, and given how defensive the proponents of JETS have been insofar it might not be as easy and it might require another petition or protest. Going forward I think that one operational frame could be "we all agree that human subjects come first, can we add a line in JETS that highlight such concept? It's a bit vague and we do not want to send a vague message on such important subject" or something like that. I know that there have been previous attempts to do such thing and I am proposing to just do another one. But my point is that such attempt should come from the understanding that nobody wants to endanger human subjects and it is a shared value, so it is just the vagueness we need to resolve not some profound ethical and divisive debate. Maybe we do not need to change the wording of JETS, but we just need to ask the journals to add a strong operationalization of such rules in their bylaws.
3) Editors role & engaging more the editors
On one hand it is obvious that the editor of a journal has complete control on what gets published. On the other why reify such obvious operational thing in JETS? What is the reason to do that? This is one of the things I do not understand of JETS and I would like an explanation. Going forward I think we need to involve as many editors as possible in this conversation. I have noticed one or two posts from editors. Maybe we should have a working group dedicated to editors and asking questions to editors. They are not some strange aliens, they are colleagues and many must be as confused as we are about JETS.
4) Clarifying once and for all that there is no mandatory added costs for qualitative research
A lot of people in these boards have repeated that their research would be impossible due to added costs. My sense is that nobody wants that and actually journals would like more qualitative research being produced and have a hard time getting more. At the same time JETS is so vague that the fears of our colleagues should not be dismissed and we need a clear understanding about what will be required operationally. So this links with point 3, we need to talk to editors, without such exchange we risk to be trapped in a non operational discussion. I do not think any editor would require uploading my badly scrabbled irrelevant field note, all the transcripts of my interviews etc. etc. So what would they require in practice?
In conclusion, I propose we build 1 working group that will focus on engaging the editors and operationalizing/amending JETS. This way in all the other working groups we can focus on more relevant and interesting discussions.