ReCal2: Reliability for 2 Coders
ReCal2 (”Reliability Calculator for 2 coders”) is an online utility that computes intercoder/interrater reliability coefficients for nominal content analysis data coded by two coders. (A version for 3+ coders is also available.) Here is a brief feature list:
- Calculates four of the most popular reliability coefficients: percent agreement, Scott’s Pi, Cohen’s Kappa, and Krippendorff’s Alpha.
- Can calculate reliability for multiple variables at a time
- Accepts any range of possible variable values
- Results should be valid for nominal data coded by two coders (other uses are not endorsed, and accurate results are not guaranteed in any case — trust but verify!)
If you have used ReCal2 before, you may submit your data file for calculation via the form below. If you are a first-time user, please read the documentation first. You should also read ReCal’s very short license agreement before use.
- Data should be nominal
- Data for each variable should represent two coders’ judgments on the same units of analysis
- All codes must be represented numerically
- Input file must be formatted properly
- If you are calculating reliability for multiple variables at once, all variables must contain the same number of units of analysis (see below)
To format your data for ReCal2 analysis, follow these instructions:
- For each variable, make sure that each of your content analysis code values is represented by a unique number. E.g. 0 = absent, 1 = present, 99 = N/A. Your file must contain no characters other than numeric digits—no letters, no dashes, no decimal points, only digits.
- In Excel, SPSS, or another spreadsheet-like program, create a new file.
- Into the first two columns of your new file, enter the first and second coder’s data respectively, one unit of analysis per row, ensuring that each row represents the same unit of analysis. If you wish to calculate reliability for multiple variables, insert the second data pair into the third and fourth columns, the third pair into the fifth and sixth, etc. The screenshot below shows a file containing three variables, each with a corresponding pair of code data—A-B, C-D, and E-F.
- Do not include any header information–the first cell of each column should be each coder’s first code.
- There should be no missing data; ReCal2 will generate an error otherwise.
- If you are calculating reliability for multiple variables at once, all variables must contain the same number of units of analysis (i.e. all variables must end on the same row in your spreadsheet). If you need to calculate reliabilities for variables with different numbers of units of analysis, they must be entered into ReCal2 separately.
- Save this file in CSV/Comma-Separated Values format—ReCal will not work with files saved in proprietary formats such as Excel, SPSS, etc., but these applications can export to CSV—and choose “comma” as the column or field delimiter (if applicable). Click through any warning messages that may pop up. The file should have a “.csv” extension. Your file is now ready for analysis; use the file selection box above to locate it on your hard drive.
- Here is the full example file from which the screenshot above was taken. It contains three variables across six columns (two columns per variable) and 20 units of analysis. In a spreadsheet program it will look like a normal spreadsheet, but a web browser or text editor will display it as a series of comma-separated numbers.
If you’re having trouble getting ReCal2 to work with your data, please let me know by leaving a comment below or sending me an email. Feel free also to leave any general questions or comments regarding ReCal2 below in comments.
Wow! This simple tool instantly makes content analysis a more desirable and easier method! Thank you!