ReCal: reliability calculation for the masses

posted September 22nd, 2008 by dfreelon

ReCal (”Reliability Calculator”) is an online utility that computes intercoder/interrater reliability coefficients for nominal content analysis data. It is compatible with Excel, SPSS, STATA, OpenOffice, Google Docs, and any other database, spreadsheet, or statistical application that can export comma-separated (CSV), tab-separated (TSV), or semicolon-delimited data files. Select one of the following two links to get started:

I need to compute reliability for two coders only. (ReCal2)

I need to compute reliability for three or more coders. (ReCal3)

ReCal is continuously updated to correct the errors users encounter, so if it doesn’t work for you today, check back in a few days; I might have fixed the problem. ReCal’s source code was last updated on 10/11/2009. To date, ReCal (2 and 3 combined) has been successfully executed a total of times by persons other than the author.

28 comments for “ReCal: reliability calculation for the masses”

  1. Wow! This simple tool instantly makes content analysis a more desirable and easier method! Thank you!


  2. This was very simple to use, and (I think) it worked beautifully. Thanks for building it and making it available.


  3. Wonderful tool, and I’ll recommended it to others. ReCal is especially helpful for data in an Excel spreadsheet, because Excel has no easy way for calculating intercoder reliability.


  4. I’m using this to create some examples for the research methods class I’m teaching. Thanks for providing such a helpful tool!


  5. It’s quite useful. Thanks a lot!


  6. This is so useful. I was looking everywhere for a decent app, and to have it web-based is just great!

    Thanks for helping me to beat deadline on a big (for me, anyway) conference paper.


  7. A very useful product, but I would strongly encourage you to give users a viable option for exporting the results. I have tried every way know to man and I just can get the data into a useful (reportable) format.


  8. Thanks. This was extremely helpful. So far I have only used the tool with sample data but will return when the second coder has finished.


  9. very good job!

    thank you so much! be sure that i will cite the link in my Phd thesis.

    keep on your useful work!

    regards,

    Chrysi Rapanta


  10. this was so easy to use- thank you! It saved me alot of work!


  11. This was absolutely amazing (and absolutely free); so quick and simple and the guidelines were excellent and easy to follow. Thank you SO much!


  12. A really useful tool. Many thanks


  13. Many thanks for providing this service. Infact, this is great service for those who intensely need it. Once again thank you very much and please try more to help others in the same way.

    WISAL
    ASS Professor, KUST


  14. Absolutely brilliant!


  15. Outstanding service. Easy to use program with clear and concise output. Many thanks.


  16. Very handy! Thanks for making this available.


  17. It certainly saves me lots of sleepless nights looking for the solution. Timely for my final touches on the thesis. Certainly an excellent invention.And thanks for earlier reply to my email.


  18. Great tool…this is first time I am using ReCal and I have only words of admiration for it!
    So simple yet so great. I will definitely reference it in paper I am ready to publish.
    Thanks a lot,
    o


  19. This is very easy and quick reliability test.


  20. Thanks for your tool!!


  21. Thank you so much for making this available to frantic students! Wonderfully helpful and easy to use!! Appreciatively, Michele


  22. Thank you so much for a great tool. But, I hope you can help me clear up a discrepancy I’ve noticed in my results for variables that have the same number of agreements/disagreements. For example, variable 1 has 26 agreements and 1 disagreement. So does variable 3. So does variable 5. Yet, the results for variable 1 are: 96.3% agreement and Scott’s pi of 0.924. The results for variable 3 are: 96.3% agreement and Scott’s pi of 0.914. The results for variable 5 are: 96.3% agreement and Scott’s pi of 0.886. Can you please tell me why the Scott’s pi is different for each variable when all the raw data for them is the same (ie same number of agreements and disagreements)? This scenario has occurred on three separate occasions when I’ve submitted my .csv files for analysis.
    Best wishes
    Dianne

    EDIT: I have answered this question.


  23. Ok, I am stumped. How can I have a percent agreement of .97 and a Scott’s Pi of-.015? I have two coders coding either Yes (1) or No (0) for the presence of a variable. What am I doing wrong. I find when calculating by hand I get similar results (off by a decimal or so). When using RECAL or calculating Scotts Pi with more than two categories, I don’t get negative Scotts Pi when the percent agreement is high.

    Thanks so much for sharing your program and answering my question if you have the time.

    Happy Holidays!

    Sonya

    EDIT: I have answered this question.


  24. easy and convenient to use. It is faster than SAS MACRO that I am using.

    Great job!!!

    Bob


  25. Thanks for this great tool , before I visited this website I used PRAM and the macro of Krippendorf in SPSS. But this tool is indeed faster and very handy! My ompliments to you! I shall recommend this website to toher researchers. Eric


  26. I will use this site for testing my content analysis results during my phd research! The topic of my research is quantitative content analysis of student’s reflective writings in Teacher Education. Eric


  27. Very usefull. Thank you for your effort on making content analysis an easier job. I will tell everyone about this tool. Keep up the good work!


  28. Dear Mr. Freelon,
    you helped us a lot. We have a huge amount of data
    out of different studies in which we analysed children’s
    stories about their pain experiences concerning an
    accident by bicycle, headaches, abdominal pain and
    getting a vaccination by the doctor. These stories
    were all analysed by 2 independent coders and we
    already knew, that only calculating the percentage
    agreement is not enough. Then we read the paper of
    Prof. Lombard and got to know, that you developped
    a programm, doing all necessary calculations to
    present the intercoder - reliability as requested.

    We can’t thank you enough for this great work.
    It helps us and atleast about 20 students
    who are working in our project.

    Thank’s a lot. Others will write you, as soon as
    they use your programm.

    Best regard
    Gaby Ostkirchen
    Ivana Tolic


Leave a comment

all content © dfreelon.org 2008. Design credits: Arcsin (template), aNieto2k (Wordpress theme), dfreelon (additional theme modifications)
fucking girls Alicia Tease ball honeys Backroommilf BIG booty Alix Lakehurst sinfulcomics milf soup pimpin milflessons big mouthfuls milflessons milf soup Working Latinas juicy round ass Puma Sweede facialfest milfsoup Marquetta Jewel Sativa Rose blowjobninjas bangbus fucking teen bang bus Spanish Diosa milflessons Newbie Black Backroom milf bang bros fucking pussies pushing up daisies Fuck Team Five milfsoup monstersofcock bangbus newbieblack Abby Skyy Bait Bus Bait Bus bangbus bangbros milfsoup Big Tits, Round Asses big mouthfuls Esperanza Gomez ballhoneys Shy Love bigtitsroundasses workinglatinas sweet pussy Celeb Nudity fuckteamfive crazy ass fucking cock hungry ass parade milf soup fuck team five Catholic school girls squirting Ahryan Astyn facialfest bangbus Allie Perdue Sophie Dee bigmouthfuls Tugjobs Jessi Summers big mouthfuls latina lips Vivian West Britney Blew Busty Milf Alexa Benson bang bus Fuck Team Five milfsoup Fuck Team Five celeb king gothic chick bangbus Alanna Ackerman fuck videos ass parade Alexis Silver bangbros Allie Foster Alayah Sashu big mouthfuls eva angelina big bubble butt couple pussies