EDIT 06/30/10: This document described below has been superseded by a new version available in the recently published journal article “ReCal: Intercoder Reliability Calculation as a Web Service.” No changes have been made to the substance of the equations themselves, but the examples have been clarified based on referees’ suggestions.

For those of you interested in calculating intercoder reliability by hand, I have created a PDF document that contains formulae and worked examples for all of the coefficients ReCal offers. They are: percent agreement, Scott’s pi, Cohen’s kappa, Krippendorff’s alpha, and Fleiss’ kappa. If you find any errors or simply don’t understand something, please don’t hesitate to let me know either in comments or in an email.

I have just introduced a new feature to ReCal called “Export Results to CSV.” The purpose of this feature is to offer a new format for the program’s output—until today, the only output option offered was HTML. Users now have the option of saving their results to a formatted CSV file suitable for viewing in a spreadsheet application such as Excel.

Using this new function is easy. Simply submit your data file(s) to ReCal as usual, and near the bottom of the usual HTML results page you’ll see a button labeled “Export Results to CSV.” Click this button, and your output CSV file should pop up as a download. Give this file an appropriate name (this is important, as the default filename is “output.csv”) and save it to your hard drive. You should now be able to open your results file in any spreadsheet application. (Though it is technically possible, I highly recommend not trying to open these output files in statistical applications such as SPSS or STATA, as they most likely will not display the data properly.) When you export after having accumulated the results of multiple data files via the Save Results History option, the resulting output file will contain all of that data.

If you experience any problems or have any questions about exporting your results, please leave a comment below.