research resources

In this section you’ll find a collection of online scholarly resources subdivided by category that I’ve found useful in my research. Use the linked table of contents below for easy navigation.

Intercoder reliability calculators

Content analysis resources

Internet/CMC research tools

Internet/CMC research journals

ReCal (percent agreement, Scott’s pi, Cohen’s kappa, Krippendorff’s alpha, & Fleiss’ kappa). Hosted on this site, ReCal calculates more coefficients than any other online calculator currently available (that I know of) and has been used thousands of times by researchers and professionals representing over 90 colleges, universities, and university-affiliated hospitals worldwide. It currently calculates the following coefficients for two or more coders: percent agreement, Scott’s pi, Cohen’s kappa, Fleiss’ kappa, and Krippendorff’s alpha.

PRAM (link dead as of 10/27/09; to my knowledge there are no alternate links available). Intercoder reliability calculator for Windows XP which calculates percent agreement, Scott’s Pi, Cohen’s Kappa, Lin’s Concordance, Holsti’s R, Spearman’s Rho, and the Pearson correlation (the latter four of which ReCal does not calculate). May or may not work with Windows Vista.

Krippendorff’s alpha macros for SPSS and SAS by Andrew Hayes. These enable Krippendorff’s Alpha calculation in SPSS and SAS.

Cohen’s Kappa Calculator by Jeroen Geertzen. Calculates Cohen’s kappa for multiple coders and two or more variable values. Uses Krippendorff’s generalization of CK for multiple coders whereas ReCal calculates average pairwise CK.

Cohen’s Kappa Calculator by Justus Randolph. Calculates two versions of CK, and also uses a different data input format than ReCal. It’s written in Java so it may run a little slow on some computers.

Cohen’s Kappa Calculator by NIWA. CK calculator by the National Institute of Water & Atmospheric research in New Zealand. Only accepts two coders working with binary variables.

General

Content Analysis Overview (Writing@CSU) - excellent introduction to the method

Content Analysis Research Methods (Shoemaker) - offers lots of useful info

The Content Analysis Guidebook Online (Neuendorf) - the online companion to her excellent book offers a number of resources

Yoshikoder (Sourceforge) - haven’t used this, but looks like an interesting open-source computer-assisted content analysis application

Intercoder reliability

Intercoder Reliability in Content Analysis (Lombard et al.) - great summary of what intercoder reliability is, why it’s important, and its various forms

Krippendorff’s Alpha Reliability (Krippendorff) - Krippendorff himself explains how to calculate his reliability statistic under any set of parameters

• Cyberpsychology & Behavior
• First Monday
• Information, Communication & Society
• The Information Society
• International Journal of Internet Research Ethics
• Internet Research
• Journal of Computer-Mediated Communication
• Journal of Information Technology & Politics
• New Media & Society
• Social Science Computer Review