ReCal (”Reliability Calculator”) is an online utility that computes intercoder/interrater reliability coefficients for nominal, ordinal, interval, or ratio-level data. It is compatible with Excel, SPSS, STATA, OpenOffice, Google Docs, and any other database, spreadsheet, or statistical application that can export comma-separated (CSV), tab-separated (TSV), or semicolon-delimited data files. Additional technical details, formulae, examples, and evidence of ReCal’s accuracy are available in the peer-reviewed article “ReCal: Intercoder Reliability Calculation as a Web Service.” Please cite this article whenever you include ReCal’s output in a publication.
ReCal consists of three autonomous modules each specialized for different types of data. The following table will help you select the module that best fits your data. (If you do not know whether your data are considered nominal, ordinal, interval, or ratio, please consult this Wikipedia article to find out more about these levels of measurement.)
| Level of measurement | N of coders | Use |
| Nominal | 2 coders only | ReCal2 (includes percent agreement, Scott’s pi, Cohen’s kappa, and nominal Krippendorff’s alpha) |
| Nominal | 3 or more coders | ReCal3 (includes pairwise percent agreement, Fleiss’ kappa, pairwise Cohen’s kappa, and nominal Krippendorff’s alpha) |
| Ordinal, interval, or ratio | Any N of coders | ReCal OIR (includes ordinal, interval, and ratio Krippendorff’s alpha) |
Please visit the ReCal FAQ/troubleshooting page if you have questions or are experiencing difficulty getting ReCal to work with your data. If you still have questions please contact me directly.
ReCal’s source code was last updated on 06/21/2010. To date, ReCal (2, 3, and OIR combined) has been successfully executed a total of 
times by persons other than the developer.
Wow! This simple tool instantly makes content analysis a more desirable and easier method! Thank you!
This was very simple to use, and (I think) it worked beautifully. Thanks for building it and making it available.
Wonderful tool, and I’ll recommended it to others. ReCal is especially helpful for data in an Excel spreadsheet, because Excel has no easy way for calculating intercoder reliability.
I’m using this to create some examples for the research methods class I’m teaching. Thanks for providing such a helpful tool!
It’s quite useful. Thanks a lot!
This is so useful. I was looking everywhere for a decent app, and to have it web-based is just great!
Thanks for helping me to beat deadline on a big (for me, anyway) conference paper.
A very useful product, but I would strongly encourage you to give users a viable option for exporting the results. I have tried every way know to man and I just can get the data into a useful (reportable) format.
Thanks. This was extremely helpful. So far I have only used the tool with sample data but will return when the second coder has finished.
very good job!
thank you so much! be sure that i will cite the link in my Phd thesis.
keep on your useful work!
regards,
Chrysi Rapanta
this was so easy to use- thank you! It saved me alot of work!
This was absolutely amazing (and absolutely free); so quick and simple and the guidelines were excellent and easy to follow. Thank you SO much!
A really useful tool. Many thanks
Many thanks for providing this service. Infact, this is great service for those who intensely need it. Once again thank you very much and please try more to help others in the same way.
WISAL ASS Professor, KUST
Absolutely brilliant!
Outstanding service. Easy to use program with clear and concise output. Many thanks.
Very handy! Thanks for making this available.
It certainly saves me lots of sleepless nights looking for the solution. Timely for my final touches on the thesis. Certainly an excellent invention.And thanks for earlier reply to my email.
Great tool…this is first time I am using ReCal and I have only words of admiration for it! So simple yet so great. I will definitely reference it in paper I am ready to publish. Thanks a lot, o
This is very easy and quick reliability test.
Thanks for your tool!!
Thank you so much for making this available to frantic students! Wonderfully helpful and easy to use!! Appreciatively, Michele
Thank you so much for a great tool. But, I hope you can help me clear up a discrepancy I’ve noticed in my results for variables that have the same number of agreements/disagreements. For example, variable 1 has 26 agreements and 1 disagreement. So does variable 3. So does variable 5. Yet, the results for variable 1 are: 96.3% agreement and Scott’s pi of 0.924. The results for variable 3 are: 96.3% agreement and Scott’s pi of 0.914. The results for variable 5 are: 96.3% agreement and Scott’s pi of 0.886. Can you please tell me why the Scott’s pi is different for each variable when all the raw data for them is the same (ie same number of agreements and disagreements)? This scenario has occurred on three separate occasions when I’ve submitted my .csv files for analysis. Best wishes Dianne
EDIT: I have answered this question.
Ok, I am stumped. How can I have a percent agreement of .97 and a Scott’s Pi of-.015? I have two coders coding either Yes (1) or No (0) for the presence of a variable. What am I doing wrong. I find when calculating by hand I get similar results (off by a decimal or so). When using RECAL or calculating Scotts Pi with more than two categories, I don’t get negative Scotts Pi when the percent agreement is high.
Thanks so much for sharing your program and answering my question if you have the time.
Happy Holidays!
Sonya
EDIT: I have answered this question.
easy and convenient to use. It is faster than SAS MACRO that I am using.
Great job!!!
Bob
Thanks for this great tool , before I visited this website I used PRAM and the macro of Krippendorf in SPSS. But this tool is indeed faster and very handy! My ompliments to you! I shall recommend this website to toher researchers. Eric
I will use this site for testing my content analysis results during my phd research! The topic of my research is quantitative content analysis of student’s reflective writings in Teacher Education. Eric
Very usefull. Thank you for your effort on making content analysis an easier job. I will tell everyone about this tool. Keep up the good work!
Dear Mr. Freelon, you helped us a lot. We have a huge amount of data out of different studies in which we analysed children’s stories about their pain experiences concerning an accident by bicycle, headaches, abdominal pain and getting a vaccination by the doctor. These stories were all analysed by 2 independent coders and we already knew, that only calculating the percentage agreement is not enough. Then we read the paper of Prof. Lombard and got to know, that you developped a programm, doing all necessary calculations to present the intercoder – reliability as requested.
We can’t thank you enough for this great work. It helps us and atleast about 20 students who are working in our project.
Thank’s a lot. Others will write you, as soon as they use your programm.
Best regard Gaby Ostkirchen Ivana Tolic
Dear Mr. Freelon, thank you for your helpful tool. I am member of the study-group of Mrs. Dr. Ostkirchen.
After calculating the reliabilities we hat categories with bad results. That’s why we startet an analysis of the mistakes. This helped us to improve our category system. Best regard Hildegard Lüdecke
Thank you, Deen! What a fantastic program — made my intercoder reliability calculation easy. Thanks as well for answering my questions via email. You’re the best!
Hi There, Thanks for making this tool available as it provides a quick and easy way to work out reliability.
Well done!
Paul, South Australia
Dear Mr. Freelon, would it be possible, you send us your opinion on our problem? Since we calculate intercoder-reliability for different sub-studies of our project with your programme we easily get the reliabilty-results, including the amount of coder-differences. We learnt that our coding-system ameliorated over the time, and we started to use your programme to help us to sharpen up our category-system by adding examples and by reformulating the rules. Hildegard did a complete analysis of the mistakes (disaggrements) found by ReCal2 and up to now 5 mistakes are remaining. In her thesis she wants to present the first calculation with about 60 disagreement, than a table with all commented disaggrements and then she executes a new reliability analysis and of course nearly all categories show an aggrement of 100%. Can we do it like this? Or do you propose another way? We find it very necessary to sharpen up our system through the process we explained above. An expert of methods like you, has he any arguments against this procedure? Thanking you in anticipation for you soon reply we remain with best wishes Gaby and Hildegard
Thank you! This tool was immensely useful for content analysis research. I was about to resort to calculating krippendorf’s alpha by hand. You saved me hours of time!
I’m a somewhat cynical person that truly believes, “If it sounds too good to be true, it probably is.” So I was waiting for the catch with this website tool. There wasn’t one. THANK YOU!!! What a great service.
Thank you so much. This was immensely helpful with my research. What a wonderful tool!
This tool is great. I’ll definitely be sharing this with colleagues. It’s an easy to use solution to calculating reliability. Thanks!
Thank you so much for developing this – it is super cool and I have found it incredibly useful over the past few days. To other users – it has a quick learning curve (just a few tries to get used to the data formatting requirements), but it is worth it. Reliability tests that used to take hours are literally done in about 10 minutes.
Thank you! This has helped my research so much and you can see the quality care that you have put into this on the website.
ReCal is simply amazing! Many of the other tools are not so user-friendly and some not available. ReCal made it for me within second. It only require careful formating of data and you are there in minutes!I find ReCal very useful and i am going to extend this knowledge to others. Thanks a lot!
Thank you for creating this program. It worked well and saved hours of time. The frustration with all the methods of checking intercoder reliability that take into account agreement based on chance is that they tend to be very conservative when events are rare. It would be nice to include Perreault and Leigh’s measure which tends to be more liberal. Of course, some critics say it is too liberal. I think it’s a good idea to include multiple measures of reliability, at least one that tends conservative and one that tends liberal.
Very useful. Used it when my SPSS license died and I needed an analysis right away.
Thank you, thank you, thank you! Just found ReCal and it made my life so much easier.
This saved us so much time and energy.
Simply awesome! My professor and collegues are all using this. Thank you for this great gift!
Vivi Xie
I cant tell how useful this website has been for my research!!! I am doing my PhD and this software was just TERRIFIC!!!!! No more headache looking for calculators..much better than SPSS that i am using which only offers Kappa…
Thank you so much for making this utility, I used recal to calculate the results for my thesis experiment and mentioned ReCal and the article.
Two words: Thank. you.
Welcome!
Thank you thank you thank you sooo much. I am absolutely hopeless with numbers and this makes sense even to me!!! :’) really thanks.
Great job guys! Thanks for the enormous help!
What a great tool…thank you
I found ReCal very very useful. Easy to use. Just the tool for quick and efficient work. Unfortunately, the tool does not raise the inter rater reliability itself
.
Thanks a million.
Fantastic site.
So glad to find this. We’re pretesting some questionnaires at our partner sites here in Cambodia. Your reliability tool was a great find, and saved me a lot of time! Thank you!
Thank goodness for ReCAL! A real tremendous help! Thanks again!
Fantastic site and great concept! Well done
Thank you very much!!! This is great.