There’s an interesting debate going on over at openGlobalRights.  Drawing on their recent Social Problems article, Neve Gordon and Nitza Berkovitch provocatively accuse human rights quantitative scholars of “concealing social wrongs” by using quantitative cross-national data that does not account for the disproportionately high voter disenfranchisement among African Americans.  Todd Landman and Chad Clay, two scholars known for their use/production of quantitative human rights data respond to Gordon and Berkovitch, saying that their piece ignores much quantitative human rights scholarship that is not at the cross-national level, fails to understand the coding decisions and methodology behind cross-national human rights data, and misses what we’ve learned from existing studies.  It’s a great discussion and one I’m going to make sure my human rights students all read.

I’m going to take a slightly different approach here in responding to Gordon and Berkovitch, two scholars, I should note, that I have learned a lot from.  I think this particular piece, however, is completely disingenuous: there is nothing special about qualitative analysis that necessarily implies that a researcher will observe/record/code group differences in the protection of human rights within a country.

Qualitative analysis does not necessarily equate to disaggregation.  Similarly, quantitative analysis – even the use (gasp!) of cross-national datasets – does not imply that a human rights scholar “ignore[s]” within-country differences in rights enjoyment, thus “play[ing] a role in concealing…institutional racism.”  That’s a ginormous leap and one, quite frankly, that should rub all scholars – regardless of methodology or approach – the wrong way.  First off, I’m reminded of one of the slides I use in my intro-level methodology class.  It’s a line buried in a piece by Gary King and Eleanor Neff Powell:

“all research is qualitative, and a subset is also quantitative.”

For any scholar – with any methodological leaning –  to study within-country differences in protection of certain human rights requires that the scholar (a) has a research question that speaks to this phenomenon, (b) uses a unit-of-analysis appropriate for this question, and (c) uses coding rules that lead to this information being observed and recorded.  This process is identical for quantitative and qualitative scholars!  To simply assume that qualitative researchers have some sort of leg-up in studying these things is bogus: if I’m asking that research question, I can follow the same procedure to observe and record the information I need, either in a qualitative or quantitative way. Frankly, I wouldn’t use either of the datasets Gordon and Berkovitch refer to (CIRI or Freedom House) in looking at that research question because –surprise, surprise – those datasets aren’t designed for that research question.  Similarly, if I was approaching a project on within-group differences in rights enjoyment qualitatively, I wouldn’t adopt the country as my unit of analysis and use coding rules that don’t reflect the information I need.  This isn’t rocket science; it’s just Research Methods 101.

In reviewing their claims, I have a couple of other questions for Gordon and Berkovitch.  First, as qualitative scholars, did they interview anyone from Freedom House or CIRI?  This would have likely provided information contradictory to the claims they are trying to make in this piece. The information Chad Clay provides in his rebuttal would have probably been gathered if Gordon and Berkovitch had interviewed Chad Clay or any other CIRI director as part of their background research for this piece.  Interviews could have also helped Gordon and Berkovitch process trace the decision-making process for the coding rules of the CIRI indicator they are trying to critique.  Even a detailed look at CIRI’s coding guide would have provided the exact international laws that the CIRI electoral self-determination indicator captures. Also, did Gordon and Berkovitch look at the US’s position on other indicators by CIRI or Freedom House?  In 2011, for example, the US’s performance across all CIRI indicators wasn’t even in the top 30 countries.  Without providing this information, it appears that Gordon and Berkovitch are cherry-picking an indicator that doesn’t fit their view of what could be gathered with their unwritten coding rules and a qualitative sub-national approach.

I’m sympathetic to the idea that all human rights scholars need to think about changes in human rights norms and differences in rights enjoyment between groups within countries.  That doesn’t mean, however, that the whole field needs to study these things or that our research – on completely different research questions – is somehow invalid because Gordon and Berkovitch think quantitative scholars are missing this particular question.   Don’t throw out the baby with the bathwater.

I’m sympathetic to the cause Gordon and Berkovitch are championing but, quite bluntly: the 90s called, they’d like their quantitative-qualitative debate back.