research resources

In this section you’ll find a collection of online scholarly resources subdivided by category that I’ve found useful in my research. Use the linked table of contents below for easy navigation.

Intercoder reliability calculators

Content analysis resources

Internet/CMC research tools

Internet/CMC research journals

Intercoder reliability calculators

ReCal (percent agreement, Scott’s pi, Cohen’s kappa, Krippendorff’s alpha, & Fleiss’ kappa). Hosted on this site, ReCal calculates more coefficients than any other online calculator currently available (that I know of) and has been used thousands of times by researchers and professionals representing over 90 colleges, universities, and university-affiliated hospitals worldwide. It currently calculates the following coefficients for two or more coders: percent agreement, Scott’s pi, Cohen’s kappa, Fleiss’ kappa, and Krippendorff’s alpha.

PRAM (link dead as of 10/27/09; to my knowledge there are no alternate links available). Intercoder reliability calculator for Windows XP which calculates percent agreement, Scott’s Pi, Cohen’s Kappa, Lin’s Concordance, Holsti’s R, Spearman’s Rho, and the Pearson correlation (the latter four of which ReCal does not calculate). May or may not work with Windows Vista.

Krippendorff’s alpha macros for SPSS and SAS by Andrew Hayes. These enable Krippendorff’s Alpha calculation in SPSS and SAS.

Cohen’s Kappa Calculator by Jeroen Geertzen. Calculates Cohen’s kappa for multiple coders and two or more variable values. Uses Krippendorff’s generalization of CK for multiple coders whereas ReCal calculates average pairwise CK.

Cohen’s Kappa Calculator by Justus Randolph. Calculates two versions of CK, and also uses a different data input format than ReCal. It’s written in Java so it may run a little slow on some computers.

Cohen’s Kappa Calculator by NIWA. CK calculator by the National Institute of Water & Atmospheric research in New Zealand. Only accepts two coders working with binary variables.

Lin’s Concordance Calculator by NIWA. Calculates Lin’s Concordance (interval/ratio data) for two coders only. Accepts the same column-row data arrangement as ReCal, albeit not through CSV files.

Content analysis resources


Content Analysis Overview (Writing@CSU) – excellent introduction to the method

Content Analysis Research Methods (Shoemaker) – offers lots of useful info

The Content Analysis Guidebook Online (Neuendorf) – the online companion to her excellent book offers a number of resources

Yoshikoder (Sourceforge) – haven’t used this, but looks like an interesting open-source computer-assisted content analysis application

Intercoder reliability

Intercoder Reliability in Content Analysis (Lombard et al.) – great summary of what intercoder reliability is, why it’s important, and its various forms

Krippendorff’s Alpha Reliability (Krippendorff) – Krippendorff himself explains how to calculate his reliability statistic under any set of parameters

CMC research tools
One of the most important tasks in CMC research involving web texts is to ensure that the objects of study remain unaltered throughout the analysis. Sound content archiving practices are essential to account for the possibility that web pages may change or disappear without warning. The following is a comparative table of four free web archiving tools I have used along with my usage notes. Under the “Hosting Type” column, “Remote” indicates that your web archives are hosted online by the archiving service; “Local” means that the tool downloads the web archives to your hard drive.

Tool Hosting Type Notes
Furl Remote
  • Requires browser toolbar install
  • Sometimes has difficulty saving logged-in content
  • Can be somewhat slow at times
Iterasi Remote
  • Only works with Windows at the moment
  • Currently in beta
UnMHT Local
  • Firefox add-on that allows users to save in MHT format, which compresses all of a given webpage’s content (HTML, images, etc.) into a single file
  • You can easily create a web folder full of MHT files for access by more than one person; see example here
  • IE users can read MHTs natively, but Firefox users cannot read them without installing UnMHT
Scrapbook Local
  • Firefox add-on that reproduces local copies of webpages on your hard drive
  • Each web page is saved in a folder with its constituent files (html, images, etc.) intact, making remote sharing difficult
  • Provides strong indexing, search, and annotation functions for saved material

Journals that primarily publish internet/CMC and related research

Leave a Reply

Your email address will not be published. Required fields are marked *