Elon Musk urges users to submit X-ray, PET and MR images to xAI chatbot Grok
The world’s richest man is urging patients and providers to try submitting their X-ray, PET and MR images to his company’s artificial intelligence chatbot for analysis.
Elon Musk shared the request on X.com (formerly Twitter before he bought it) Tuesday, asking users to share where “Grok gets it right or needs work.”
“This is still early stage, but it is already quite accurate and will become extremely good,” Musk, who owns Twitter, xAI, Tesla and SpaceX, wrote Oct. 29 to his nearly 203 million followers.
Radiologists and other physicians responded to the request Tuesday. French rad Dr. Thibaut Jacques input the last MR image in his queue, displaying adhesive capsulitis of the shoulder, which he deemed as “level: easy, typical pattern.” He said the results were mostly “not true,” with Grok’s output “too generic” and lacking a diagnosis.
Laura Heacock, MD, a breast radiologist, deep learning researcher and associate professor at NYU, also responded. She previously ran a series of breast mammograms, ultrasounds and MRIs through GPT-4, sharing the results on Twitter. Heacock repeated the experiment with Grok, to disappointing results.
The AI model thought a breast MRI was an image of the brain on the first try. On the second case, Grok correctly identified that it was looking at a breast MRI (which contained a “very obvious" left breast cancer).
“However, Grok seemed unable to tell me anything more than a description of the ‘brightness’ evenness across the image,” Heacock added. “Correct identification, no points for diagnosis. Half point for probably realizing the left breast is abnormal.”
After several more attempts with different medical images, Heacock shared her final assessment:
- “Grok is OK at recognizing breast radiology image types, with some exceptions.
- Grok is wrong at diagnosing breast imaging, although that capability may just be locked.
- I prefer the explanations (even if wrong) to GPT-4v.
- Real radiologists >>>>> Grok radiologist.”
“All the single images I gave it were sufficient context for a medical student or an early radiology trainee to make the diagnosis,” Heacock added later. “Grok is not suitable for medical diagnosis in its current form. I agree that may change in the future, but that wasn't the intent of this test.”
“Nice work here,” responded Doug Lake, MD, a radiologist based in Ames, Iowa. “These are slam dunk, easy cases and it’s fascinating the tech sometimes struggles to recognize the modality and can’t nail down easy diagnoses that a medical student would get It will get better, but it has a long way to go.”
Marty Stempniak has covered healthcare since 2012, with his byline appearing in the American Hospital Association's member magazine, Modern Healthcare and McKnight's. Prior to that, he wrote about village government and local business for his hometown newspaper in Oak Park, Illinois. He won a Peter Lisagor and Gold EXCEL awards in 2017 for his coverage of the opioid epidemic.