In this section you’ll find a collection of online scholarly resources subdivided by category that I’ve found useful in my research. Use the linked table of contents below for easy navigation.
Intercoder reliability calculators
Internet/CMC research journals
Intercoder reliability calculators
ReCal (percent agreement, Scott’s pi, Cohen’s kappa, Krippendorff’s alpha, & Fleiss’ kappa). Hosted on this site, ReCal calculates more coefficients than any other online calculator currently available (that I know of) and has been used thousands of times by researchers and professionals representing over 90 colleges, universities, and university-affiliated hospitals worldwide. It currently calculates the following coefficients for two or more coders: percent agreement, Scott’s pi, Cohen’s kappa, Fleiss’ kappa, and Krippendorff’s alpha.
PRAM (link dead as of 10/27/09; to my knowledge there are no alternate links available). Intercoder reliability calculator for Windows XP which calculates percent agreement, Scott’s Pi, Cohen’s Kappa, Lin’s Concordance, Holsti’s R, Spearman’s Rho, and the Pearson correlation (the latter four of which ReCal does not calculate). May or may not work with Windows Vista.
Krippendorff’s alpha macros for SPSS and SAS by Andrew Hayes. These enable Krippendorff’s Alpha calculation in SPSS and SAS.
Cohen’s Kappa Calculator by Jeroen Geertzen. Calculates Cohen’s kappa for multiple coders and two or more variable values. Uses Krippendorff’s generalization of CK for multiple coders whereas ReCal calculates average pairwise CK.
Cohen’s Kappa Calculator by Justus Randolph. Calculates two versions of CK, and also uses a different data input format than ReCal. It’s written in Java so it may run a little slow on some computers.
Cohen’s Kappa Calculator by NIWA. CK calculator by the National Institute of Water & Atmospheric research in New Zealand. Only accepts two coders working with binary variables.
Lin’s Concordance Calculator by NIWA. Calculates Lin’s Concordance (interval/ratio data) for two coders only. Accepts the same column-row data arrangement as ReCal, albeit not through CSV files.
Content analysis resources
General
Content Analysis Overview (Writing@CSU) – excellent introduction to the method
Content Analysis Research Methods (Shoemaker) – offers lots of useful info
The Content Analysis Guidebook Online (Neuendorf) – the online companion to her excellent book offers a number of resources
Yoshikoder (Sourceforge) – haven’t used this, but looks like an interesting open-source computer-assisted content analysis application
Intercoder reliability
Intercoder Reliability in Content Analysis (Lombard et al.) – great summary of what intercoder reliability is, why it’s important, and its various forms
Krippendorff’s Alpha Reliability (Krippendorff) – Krippendorff himself explains how to calculate his reliability statistic under any set of parameters
CMC research tools
One of the most important tasks in CMC research involving web texts is to ensure that the objects of study remain unaltered throughout the analysis. Sound content archiving practices are essential to account for the possibility that web pages may change or disappear without warning. The following is a comparative table of four free web archiving tools I have used along with my usage notes. Under the “Hosting Type” column, “Remote” indicates that your web archives are hosted online by the archiving service; “Local” means that the tool downloads the web archives to your hard drive.
| Tool | Hosting Type | Notes |
|---|---|---|
| Furl | Remote |
|
| Iterasi | Remote |
|
| UnMHT | Local |
|
| Scrapbook | Local |
|
Journals that primarily publish internet/CMC and related research
- Cyberpsychology & Behavior
- First Monday
- Information, Communication & Society
- The Information Society
- International Journal of Internet Research Ethics
- Internet Research
- Journal of Computer-Mediated Communication
- Journal of Information Technology & Politics
- New Media & Society
- Social Science Computer Review