A diverse mix of trainees and physicians demonstrated moderate interobserver agreement when sorting prostate cancer cases into categories laid out by the Prostate Imaging Reporting and Data System (PI-RADS), demonstrating resiliency to varying levels of education attained by users. However, a persistently "OK" agreement rate may indicate systemic problems with PI-RADS.