Abstract
Simulating an individual's disease course requires connecting patient history to future events, clinical timing and disease morphology. Longitudinal prediction and multimodal representations exploit this information, but organizing future readouts around a common patient state and advancing trajectories without real intermediate observations remain central challenges. Here we introduce Saga, a multimodal patient world model that learns a shared state from patient histories to predict clinical events and generate multimodal images. Generated events and times are appended to the history and re-encoded to advance prediction. We studied 461,370 patients with 12 cancer types across 112 centres. Modality-specific paired evaluations linked future images to observed follow-up. With historical images held fixed, removing patient conditioning reduced event F1 from 0.716 to 0.591 and increased lesion-burden error from 11.8% to 21.3%. In free rollouts, eight-step event F1 was 0.672, compared with 0.553 for a longitudinal Transformer evaluated from the same origins, and timing error was lower by 8.7 days. In the internal test set, one-year progression AUROC was 0.882, with values of 0.871 and 0.856 in temporal and geographical external evaluations. These results show that shared patient states connect future phenotypes across observational scales with event-driven trajectory progression, sustaining comparative predictive advantages during rollout and discrimination in external clinical evaluation. Saga makes patient-level disease simulation a multimodal prediction task that can be tested against observed follow-up.
Abstract (translated)
URL
https://arxiv.org/abs/2610.33054
