As AI becomes more common in hospitals, medical physicists may play a growing role in ensuring these systems remain safe, reliable and trustworthy.