Comments on: ReCal2: Reliability for 2 Coders http://dfreelon.org Assistant professor, American University School of Communication Sat, 05 May 2012 05:00:23 +0000 hourly 1 http://wordpress.org/?v=3.1.1 By: Maria http://dfreelon.org/utils/recalfront/recal2/#comment-2578 Maria Sun, 22 Jan 2012 14:36:28 +0000 http://dfreelon.org/?page_id=17#comment-2578 i am testing intercoder reliability on 15 variables most of the variables will be coded as 0 or 1 however 2 of the variables have 3 possible answers coded as 1 or 2 or 3 can these latter 2 variables be tested in the same way? thanks! i am testing intercoder reliability on 15 variables

most of the variables will be coded as 0 or 1

however 2 of the variables have 3 possible answers coded as 1 or 2 or 3

can these latter 2 variables be tested in the same way?

thanks!

]]> By: Nicole http://dfreelon.org/utils/recalfront/recal2/#comment-2488 Nicole Fri, 06 Jan 2012 11:50:44 +0000 http://dfreelon.org/?page_id=17#comment-2488 Thank you! This tool is very easy to use, a PhD student's dream!!! Thank you! This tool is very easy to use, a PhD student’s dream!!!

]]> By: Karolina http://dfreelon.org/utils/recalfront/recal2/#comment-1758 Karolina Sun, 14 Aug 2011 11:45:11 +0000 http://dfreelon.org/?page_id=17#comment-1758 I just would like to say thank you! It was like magic that solves unsolved problem. I just would like to say thank you! It was like magic that solves unsolved problem.

]]> By: dfreelon http://dfreelon.org/utils/recalfront/recal2/#comment-1704 dfreelon Fri, 29 Jul 2011 14:38:06 +0000 http://dfreelon.org/?page_id=17#comment-1704 Congratulations! You have (re-)discovered an error in SPSS. It occurs when one coder uses at least one category/variable value that the other doesn't. But there is no algebraic reason kappa can't be calculated in this situation. Apparently there used to be a macro available on the SPSS website that would do so that is no longer available; see <a href="http://spssx-discussion.1045642.n5.nabble.com/Re-Kappa-missing-rows-colums-td1075694.html" rel="nofollow">this thread</a>. Also, the <em><a href="http://cran.r-project.org/web/packages/irr/index.html" rel="nofollow">irr</a></em> package in R will calculate kappa for this kind of data with no errors, and that was written by Real Statisticians. So the ReCal value is correct, and you can double-check it in <em>irr</em> if you wish. Congratulations! You have (re-)discovered an error in SPSS. It occurs when one coder uses at least one category/variable value that the other doesn’t. But there is no algebraic reason kappa can’t be calculated in this situation. Apparently there used to be a macro available on the SPSS website that would do so that is no longer available; see this thread. Also, the irr package in R will calculate kappa for this kind of data with no errors, and that was written by Real Statisticians. So the ReCal value is correct, and you can double-check it in irr if you wish.

]]> By: SLH http://dfreelon.org/utils/recalfront/recal2/#comment-1703 SLH Fri, 29 Jul 2011 09:33:10 +0000 http://dfreelon.org/?page_id=17#comment-1703 Hi, I have found this tool extremely useful - thanks very much! A question though: you advise double checking the results which I have done for a few variables using SPSS. Using the Cohen's Kappa test as an example, for many variables the results come out exactly the same as produced by ReCal. However, there have been a few instances where SPSS says that Kappa can't be calculated because it 'requires a 2-way table in which the values of the first variable match the second'. However, ReCal has produced me a Kappa value. Has anyone come across this before and advise if I'm doing something wrong, either in SPSS or in ReCal? Thanks. Hi, I have found this tool extremely useful – thanks very much!

A question though: you advise double checking the results which I have done for a few variables using SPSS. Using the Cohen’s Kappa test as an example, for many variables the results come out exactly the same as produced by ReCal. However, there have been a few instances where SPSS says that Kappa can’t be calculated because it ‘requires a 2-way table in which the values of the first variable match the second’. However, ReCal has produced me a Kappa value. Has anyone come across this before and advise if I’m doing something wrong, either in SPSS or in ReCal? Thanks.

]]> By: John http://dfreelon.org/utils/recalfront/recal2/#comment-747 John Sun, 25 Jul 2010 21:39:45 +0000 http://dfreelon.org/?page_id=17#comment-747 One word. Awesome! This program gets the job done. Good work and keep bring it. One word. Awesome! This program gets the job done. Good work and keep bring it.

]]>