A few people have asked recently how we go about selecting the cases for our rapid reporting module and how we create our exam packets. Rather than answer the same question 20 times, I thought I’d take the time to write an article about the process.
How do we choose the cases?
All cases are hand selected by one of our three founders. Generally they are stumbled upon during our day-to-day job of reporting plain films although we are occasionally offered examples of classic cases by our colleagues. The majority of the cases have been collected over a number of years.
It may be hard to believe but one of the most difficult aspects of building the rapid reporting module is finding good normal cases. We’ve had very few “complaints” overall about our images but when we do, it’s often because a tiny subtle abnormality has sneaked in under the guise of a normal film. Things like cervical ribs, excessive degenerative change, etc are common culprits. It’s for this reason that we now actively exclude normal variants from our cases.
Once we have the cases, what happens next?
Every case that is added to our database goes into a special area of our database called “quarantine”. Whilst in this area, the question is not shown to users of the site until removed. Every question is double read by a post-FRCR radiologist and many of the cases have been triple read for accuracy. If there is any doubt about the quality of the image or the diagnosis then the question is removed and you (the end user) never see the question.
How do you make your packets?
We are very proud of how we do this. Rather than simply randomise 50% normal cases and 50% abnormal cases, we use a custom algorithm that I wrote. First, the algorithm randomises how many normal and abnormal cases will be in each packet of 30 questions. Generally harder packets have more normal cases (although this is not always true). As each case is given an arbitrary difficulty rating (1 = easy, 5 = very difficult) the algorithm will then determine the ratios of discriminator images (i.e. very hard) to easy cases to include in the packet. The algorithm will also try to balance the packet in terms of body region. Finally, the order of the cases is jumbled up and released to the wild for your enjoyment.
Hopefully this gives some insight into how much effort goes into making our packets fair and balanced. We at Revise Radiology firmly believe that our packets are as balanced as possible. It’s for all of the complicated processes described above that we call them curated.
So many exciting things are happening here at Revise Radiology. We have five packets in quarantine at present and are currently triple-checking them for accuracy. Once done, they will become available to all users free of charge. Watch this space!