ReCal: reliability calculation for the masses
ReCal (”Reliability Calculator”) is an online utility that computes intercoder/interrater reliability coefficients for nominal content analysis data. It is compatible with Excel, SPSS, STATA, OpenOffice, Google Docs, and any other database, spreadsheet, or statistical application that can export comma-separated (CSV), tab-separated (TSV), or semicolon-delimited data files. Select one of the following two links to get started:
I need to compute reliability for two coders only. (ReCal2)
I need to compute reliability for three or more coders. (ReCal3)
- 10/31/09: For those of you interested in calculating intercoder reliability by hand (or double-checking ReCal’s results), I have made available a set of worked examples for percent agreement, Scott’s Pi, Cohen’s kappa, Fleiss’ kappa, and Krippendorff’s alpha.
- Vsit the ReCal FAQ/troubleshooting page if you have questions or are experiencing difficulty getting ReCal to work with your data.
ReCal is continuously updated to correct the errors users encounter, so if it doesn’t work for you today, check back in a few days; I might have fixed the problem. ReCal’s source code was last updated on 10/11/2009. To date, ReCal (2 and 3 combined) has been successfully executed a total of 
times by persons other than the author.
Wow! This simple tool instantly makes content analysis a more desirable and easier method! Thank you!
This was very simple to use, and (I think) it worked beautifully. Thanks for building it and making it available.
Wonderful tool, and I’ll recommended it to others. ReCal is especially helpful for data in an Excel spreadsheet, because Excel has no easy way for calculating intercoder reliability.
I’m using this to create some examples for the research methods class I’m teaching. Thanks for providing such a helpful tool!
It’s quite useful. Thanks a lot!
This is so useful. I was looking everywhere for a decent app, and to have it web-based is just great!
Thanks for helping me to beat deadline on a big (for me, anyway) conference paper.
A very useful product, but I would strongly encourage you to give users a viable option for exporting the results. I have tried every way know to man and I just can get the data into a useful (reportable) format.
Thanks. This was extremely helpful. So far I have only used the tool with sample data but will return when the second coder has finished.
very good job!
thank you so much! be sure that i will cite the link in my Phd thesis.
keep on your useful work!
regards,
Chrysi Rapanta
this was so easy to use- thank you! It saved me alot of work!
This was absolutely amazing (and absolutely free); so quick and simple and the guidelines were excellent and easy to follow. Thank you SO much!
A really useful tool. Many thanks
Many thanks for providing this service. Infact, this is great service for those who intensely need it. Once again thank you very much and please try more to help others in the same way.
WISAL
ASS Professor, KUST
Absolutely brilliant!
Outstanding service. Easy to use program with clear and concise output. Many thanks.
Very handy! Thanks for making this available.
It certainly saves me lots of sleepless nights looking for the solution. Timely for my final touches on the thesis. Certainly an excellent invention.And thanks for earlier reply to my email.
Great tool…this is first time I am using ReCal and I have only words of admiration for it!
So simple yet so great. I will definitely reference it in paper I am ready to publish.
Thanks a lot,
o
This is very easy and quick reliability test.
Thanks for your tool!!
Thank you so much for making this available to frantic students! Wonderfully helpful and easy to use!! Appreciatively, Michele
Thank you so much for a great tool. But, I hope you can help me clear up a discrepancy I’ve noticed in my results for variables that have the same number of agreements/disagreements. For example, variable 1 has 26 agreements and 1 disagreement. So does variable 3. So does variable 5. Yet, the results for variable 1 are: 96.3% agreement and Scott’s pi of 0.924. The results for variable 3 are: 96.3% agreement and Scott’s pi of 0.914. The results for variable 5 are: 96.3% agreement and Scott’s pi of 0.886. Can you please tell me why the Scott’s pi is different for each variable when all the raw data for them is the same (ie same number of agreements and disagreements)? This scenario has occurred on three separate occasions when I’ve submitted my .csv files for analysis.
Best wishes
Dianne
EDIT: I have answered this question.
Ok, I am stumped. How can I have a percent agreement of .97 and a Scott’s Pi of-.015? I have two coders coding either Yes (1) or No (0) for the presence of a variable. What am I doing wrong. I find when calculating by hand I get similar results (off by a decimal or so). When using RECAL or calculating Scotts Pi with more than two categories, I don’t get negative Scotts Pi when the percent agreement is high.
Thanks so much for sharing your program and answering my question if you have the time.
Happy Holidays!
Sonya
EDIT: I have answered this question.
easy and convenient to use. It is faster than SAS MACRO that I am using.
Great job!!!
Bob
Thanks for this great tool , before I visited this website I used PRAM and the macro of Krippendorf in SPSS. But this tool is indeed faster and very handy! My ompliments to you! I shall recommend this website to toher researchers. Eric
I will use this site for testing my content analysis results during my phd research! The topic of my research is quantitative content analysis of student’s reflective writings in Teacher Education. Eric
Very usefull. Thank you for your effort on making content analysis an easier job. I will tell everyone about this tool. Keep up the good work!
Dear Mr. Freelon,
you helped us a lot. We have a huge amount of data
out of different studies in which we analysed children’s
stories about their pain experiences concerning an
accident by bicycle, headaches, abdominal pain and
getting a vaccination by the doctor. These stories
were all analysed by 2 independent coders and we
already knew, that only calculating the percentage
agreement is not enough. Then we read the paper of
Prof. Lombard and got to know, that you developped
a programm, doing all necessary calculations to
present the intercoder - reliability as requested.
We can’t thank you enough for this great work.
It helps us and atleast about 20 students
who are working in our project.
Thank’s a lot. Others will write you, as soon as
they use your programm.
Best regard
Gaby Ostkirchen
Ivana Tolic