Comments on: Intercoder Reliability Worked Examples/Formulae http://dfreelon.org/2009/10/30/intercoder-reliability-worked-examples/ On communication technologies, politics, and methods Fri, 03 Feb 2012 20:48:14 +0000 hourly 1 http://wordpress.org/?v=3.1.1 By: Toi http://dfreelon.org/2009/10/30/intercoder-reliability-worked-examples/#comment-728 Toi Mon, 17 May 2010 14:51:28 +0000 http://dfreelon.org/?p=56#comment-728 Hi Deen, I'm a qualitative researcher for a large non-profit. We're coding some video from kids' video diaries. Everything in ReCal seems to be working well EXCEPT when we're in near complete agreement, and there is only one divergent rating. So for example, with three coders, we can have 10 different cases for the same variable. So out of the 30 judgments, we can have 29 1s and one 2, and the output on the Krippendorff is a fat 0. I must admit that I am not well-versed in stats, so, this is a little out of my realm. All of the other statisticians we have on staff have never heard of Krippendorff's alpha, so they can't really help explain why this is the case. Can you help? Thanks, Toi Hi Deen,

I’m a qualitative researcher for a large non-profit. We’re coding some video from kids’ video diaries.

Everything in ReCal seems to be working well EXCEPT when we’re in near complete agreement, and there is only one divergent rating. So for example, with three coders, we can have 10 different cases for the same variable. So out of the 30 judgments, we can have 29 1s and one 2, and the output on the Krippendorff is a fat 0.

I must admit that I am not well-versed in stats, so, this is a little out of my realm. All of the other statisticians we have on staff have never heard of Krippendorff’s alpha, so they can’t really help explain why this is the case.

Can you help?

Thanks,

Toi

]]>