ReCal2: Reliability for 2 Coders

ReCal2 operates under the following requirements and assumptions:

• Data should be nominal
• Data for each variable should represent two coders’ judgments on the same units of analysis
• All codes must be represented numerically
• Input file must be formatted properly
• If you are calculating reliability for multiple variables at once, all variables must contain the same number of units of analysis (see below)

To format your data for ReCal2 analysis, follow these instructions:

1. For each variable, make sure that each of your content analysis code values is represented by a unique number. E.g. 0 = absent, 1 = present, 99 = N/A. Your file must contain no characters other than numeric digits—no letters, no dashes, no decimal points, only digits.
2. In Excel, SPSS, or another spreadsheet-like program, create a new file.
3. Into the first two columns of your new file, enter the first and second coder’s data respectively, one unit of analysis per row, ensuring that each row represents the same unit of analysis. If you wish to calculate reliability for multiple variables, insert the second data pair into the third and fourth columns, the third pair into the fifth and sixth, etc. Because ReCal2 requires paired data for each variable, the number of columns in each data file submitted to it must be even. The screenshot below shows a file containing three variables, each with a corresponding pair of code data—A-B, C-D, and E-F.
   • Header information may be included in the first row of the file. If ReCal detects any letters on this row, it will discard the entire row and begin its analysis on the second row.
   • There should be no missing data; ReCal2 will generate an error otherwise.
   • If you are calculating reliability for multiple variables at once, all variables must contain the same number of units of analysis (i.e. all variables must end on the same row in your spreadsheet). If you need to calculate reliabilities for variables with different numbers of units of analysis, they must be entered into ReCal2 separately.
4. Save this file in comma-Separated values (.csv), semicolon-separated values (also .csv), or tab-separated values format (.tsv)—simply changing the file extension to “.csv” or “.tsv” will not work; the file needs to be “saved as” CSV in whatever spreadsheet or stat program you’re using—and choose “comma” as the column or field delimiter (if applicable). Click through any warning messages that may pop up. The file should have a “.csv” or “.tsv” extension. Your file is now ready for analysis; use the file selection box above to locate it on your hard drive.
5. Here is the full example file from which the screenshot above was taken. It contains three variables across six columns (two columns per variable) and 20 units of analysis. In a spreadsheet program it will look like a normal spreadsheet, but a web browser or text editor will display it as a series of comma-separated numbers.

If you’re having trouble getting ReCal2 to work with your data, first check the FAQ/troubleshooting page, and if you don’t find the answer to your question there, send me an email. Feel free also to leave any general questions or comments regarding ReCal2 below in comments.