The Complete Timeline of AI in Medical Imaging: From AlexNet to Foundation Models

TL;DR

• Every major leap in medical imaging artificial intelligence (AI) followed a change in data, not a change in architecture. AlexNet needed ImageNet. CheXNet needed ChestX-ray14. CheXzero needed MIMIC-CXR’s paired reports.

• As of September 2026, the US Food and Drug Administration (FDA) had authorized more than 1,600 AI-enabled medical devices for marketing in the United States.

• Foundation models moved the bottleneck downstream. They did not remove the bottleneck. Fine-tuning data is now more specific than it was before.

• External validation studies show models consistently underperform on external data: median area under the curve (AUC) typically dropped by roughly 0.03, with specificity falling by as much as 24 percentage points.

.

‍

Introduction

Every major advance in medical imaging AI since2012 has followed a change in data: what was available, at what scale and in what form. The usual account is about architectures, with convolutional networks giving way to transformers and transformers giving way to foundation models. That account is accurate but incomplete. The architecture changes came after the data changes. This timeline traces five eras, from AlexNet in 2012 to the first FDA clearances of foundation model-powered triage software, and the data each one depended on.

‍

Era 1: Proof of Concept (2012–2015)

2012: AlexNet wins ImageNet

Krizhevsky, Sutskever, and Hinton’s convolutional network reached a top-5 error rate of 15.3% at the ImageNet Large Scale Visual Recognition Challenge, against 26.2% for the runner-up.¹ The margin, not the method, is what changed the field. Convolutional networks had existed for decades, but this was the first demonstration that learned features could decisively beat hand-engineered ones at scale.

The medical imaging community’s reaction was skepticism, and the objection was specifically about data. ImageNet had millions of labeled natural images. Radiology had nothing comparable: no large annotated corpora, no shared benchmarks, no obvious path to either.

2015: U-Net

Ronneberger and colleagues published an encoder-decoder architecture with skip connections at the Medical Image Computing and Computer Assisted Intervention conference, originally designed for microscopy cell segmentation.² It was built explicitly for the low-data regime that medical imaging was stuck in, and it became the default segmentation backbone for the next decade.

2015: ResNet

Residual connections made very deep networks trainable. He et al. trained residual networks up to 152 layers deep, and an ensemble of them cut ImageNet top-5 error to 3.57%.³ Combined with transfer learning, this gave medical imaging teams a practical workaround: pre-train on natural images, then fine-tune on the few thousand clinical studies you could actually assemble.

‍

‍

Era 2: Clinical Validation (2016–2018)

This is when the field stopped publishing accuracy numbers and started publishing clinical comparisons.

‍2016: Diabetic retinopathy

Gulshan et al. published in JAMA (December 13, 2016) a deep learning algorithm for detecting referable diabetic retinopathy. It was developed on a retrospective set of 128,175 retinal images, each graded between three and seven times.⁴ The grading effort is the headline the field underweighted at the time.

2017: Dermatology and pathology

Esteva et al. reported dermatologist-level skin lesion classification in Nature (February 2017).⁵ Ehteshami Bejnordi et al. published the CAMELYON16 results in JAMA (2017), assessing deep learning algorithms for lymph node metastasis detection in breast cancer.⁶

2017: The first hospital-scale imaging dataset

Wang et al. released ChestX-ray8 (later expanded to ChestX-ray14) at CVPR, an order of magnitude larger than any comparable public chest radiograph dataset.⁷ Within months, CheXNet (Rajpurkar et al., November 2017) used it to report radiologist-level pneumonia detection.⁸

‍

The dataset came first, then the result. That order repeats throughout this timeline.

‍

2018: First AI triage authorization

The FDA granted De Novo authorization to Viz.ai’s ContaCT (now Viz LVO), computer-aided triage software that analyzes computed tomography (CT) images of the brain for indicators of a large vessel occlusion.⁹ When it finds a suspected occlusion, it sends a notification to a neurovascular specialist. It does not replace a full evaluation, and the specialist still reviews the images on a clinical workstation.⁹

2018: First autonomous AI diagnostic

On April 11, 2018, the FDA authorized IDx-DR (now LumineticsCore) via the De Novo pathway.¹⁰ Its results did not require review by a specialist, which meant diabetic retinopathy screening could happen in a primary care office. The FDA based its decision on a clinical study of retinal images from 900 patients with diabetes at 10 primary care sites.¹⁰ IDx-DR correctly identified more than mild diabetic retinopathy 87.4% of the time, and correctly identified patients without it 89.5% of the time.¹⁰

For context on how young this all is: the first radiology device on the FDA’s AI-enabled device list dates to 1998. It was the M1000 ImageChecker, a mammography computer-aided detection system from R2 Technology, later acquired by Hologic.¹¹

‍

Era 3: Scale, Standardization, and the Label Bottleneck (2019–2022)

2019: MIMIC-CXR

Johnson et al. released MIMIC-CXR, a public database of about 377,000 chest radiographs from about 228,000 studies, paired with their free-text radiology reports.¹² The pairing mattered more than the size: it made image-text training possible in radiology at all.

2021: nnU-Net

Isensee et al. published a self-configuring segmentation framework in Nature Methods that automatically adapted preprocessing, architecture, and training to each dataset.¹³ It became the benchmark that new methods had to beat, and it made competent segmentation reproducible by teams without deep tuning expertise.

2021: CLIP and the contrastive turn

Radford et al. showed that image-text pairs scraped at scale could produce transferable visual representations without categorical labels.¹⁴ For medicine, the implication was immediate: hospitals had been generating paired image-and-report data for decades as a byproduct of clinical care.

2022: CheXzero

Tiu et al. published in Nature Biomedical Engineering (September 2022) a model trained on chest X-rays with no explicit annotations, learning directly from the accompanying reports.¹⁵ On external validation it matched board-certified radiologists on several pathologies and generalized to conditions it had never been explicitly trained to detect.

‍

Evidence snapshot

The constraint on medical imaging AI had never really been algorithmic. It was the cost of expert annotation. CheXzero demonstrated that the constraint could be routed around, provided there was access to enough paired clinical data.

‍

Era 4: Foundation Models Arrive (2023–2025)

April 2023: Segment anything

Meta released the Segment Anything Model (SAM), the first general-purpose segmentation foundation model.¹⁶ Applied directly to medical images, it proved unstable: strong on some targets, poor on others.¹⁷ That turned out to be the more instructive result. General-domain pre-training does not transfer cleanly to clinical imaging.

January 2024: MedSAM

Ma et al. published in Nature Communications a version fine-tuned on medical data, reporting a large improvement over SAM on medical segmentation.¹⁸ Their scaling analysis found that increasing the training set toward one million images materially improved performance on both internal and external validation.

2024: Specialization by modality

RAD-DINO explored image-only medical encoders without text supervision. CheXagent (January 2024) introduced an instruction-tuned chest X-ray foundation model with its own benchmark. MAIRA-1 and MAIRA-2 advanced grounded report generation. Merlin extended vision-language pre-training to 3D CT.¹⁹

2024–2025: Generalist medical models

Google’s Med-Gemini demonstrated cross-domain multimodal capability. MedGemma (Sellergren et al., 2025) released open weights, lowering the barrier for teams that could not pre-train from scratch.²⁰

February 2025: Regulation catches up

Aidoc received FDA clearance for its rib fracture triage solution, a computer-aided triage device built on CARE1, version 1 of its Clinical AI Reasoning Engine (CARE) foundation model.²¹ The company described it as the first clearance of its kind for a foundation model-powered clinical AI solution.²¹

Era 5: Where Things Stand in 2026

January 21, 2026: Comprehensive foundation-model triage

Aidoc received FDA clearance for an abdominal CT triage solution that combines 11 newly cleared indications with 3 previously cleared ones in a single workflow, powered by its CARE foundation model.²² In the pivotal study submitted to the FDA, the 11 new indications reported a mean sensitivity of 97% and a mean specificity of 98%.²²

2026: The regulatory picture

As of September 2026, the FDA reports that it has authorized more than 1,600 AI-enabled medical devices for marketing in the United States.²³ The agency publishes the full list, including each device’s lead review panel, and updates it periodically. The FDA notes that the list is not a comprehensive resource of all AI-enabled devices.²³

Metric	Number	Source
FDA-authorized AI-enabled medical devices (as of September 2026)	1,600+	U.S. FDA, September 202623
External validation: median AUC drop	~0.03	Systematic review, 2022 to June 202524
External validation: max specificity drop	Up to ~24 pp	Systematic review, 2022 to June 202524

Sources: U.S. Food and Drug Administration, Artificial Intelligence-Enabled Medical Devices, September 2026;²³ systematic review of external validation studies, 2022 to June 2025.²⁴

External validation: the gap that remains

A systematic review of internal-versus-external validation studies in CT, magnetic resonance imaging (MRI), and X-ray (2022 to June 2025) found that models consistently underperform on external data. Median AUC dropped by roughly 0.03, but specificity fell by as much as 24 percentage points. Multicenter training was among the factors associated with better generalization.²⁴

‍

The Through-Line: Every Leap Was a Data Leap

Read the timeline again and the pattern is hard to miss.

AlexNet needed ImageNet. CheXNet needed ChestX-ray14. CheXzero needed MIMIC-CXR’s paired reports. MedSAM’s own scaling results pointed toward a million images. The 2026 foundation models need more than that, and they need it across scanners, protocols, geographies, and patient populations. The specificity drop on external data is what limits a model at a second hospital.²⁴

The architectures kept changing. The bottleneck did not.

There is a second pattern worth naming. Each era’s data requirement was qualitatively different, not just larger:

The third requirement is not a bigger version of the first. It is a different problem, and it is the one teams are solving right now.

‍

What This Means If You Are Building Today

If you are training or fine-tuning an imaging model in 2026, three implications follow from the timeline:

1. Pre-training scale is table stakes; fine-tuning diversity is the differentiator. The FDA list shows what gets cleared. The external validation literature shows what breaks. The distance between those two is almost always a training distribution that did not include the deployment site.

2. Foundation models moved the bottleneck downstream; they did not remove it. MedSAM’s improvement over SAM was a fine-tuning story. Aidoc’s multi-indication clearance was a fine-tuning story. What changed is that the data you need is now more specific, not less.

3. Regulatory-grade data is a distinct category. A dataset that supports a preprint does not necessarily support a 510(k) or a De Novo submission. Provenance, de-identification rigor, and demographic representation are submission requirements, not nice-to-haves.

‍

How Segmed supports foundation model development

The three implications above describe a data problem, and it is the one Segmed was built for. Segmed’s data network includes 150 million studies and spans 2,800+ healthcare partner sites, with sourcing from all 50 U.S states. Modalities includes CT, MRI, X-ray and ultrasound as well as PET/CT, PET/MR, DEXA, mammography, echocardiography and fluorescence. Imaging studies link to multimodal data,including electronic health records and pathology slides.

Breadth across sites, devices and geographies iswhat supports generalization. The external validation literature is consistent:specificity degrades when the deployment environment does not resemble thetraining distribution, and multicenter training is among the factors associatedwith better generalization [24].

For teams working toward a submission rather than a preprint, the requirements change again. Provenance, de-identification methodology, demographic representation and version control become documentation obligations. Segmed’s datasets are de-identified inaccordance with HIPAA requirements and built with those documentation trailsintact. Segmed is ISO 27001:2022 certified. More than 50 medical devices validated on Segmed data network have received FDA clearance.

Ready to explore real-world data for your foundation model? Connect with us to discuss your data requirements or submit a project feasibility inquiry to evaluate modality and cohort availability.

‍

Frequently Asked Questions – F.A.Q.

‍

What was the first FDA-authorized autonomous AI diagnostic device?

IDx-DR (now LumineticsCore) received De Novo authorization from the FDA on April 11, 2018.¹⁰ It was the first AI device whose results did not require review by a specialist. The FDA based its decision on a clinical study of retinal images from 900 patients with diabetes at 10 primary care sites. The device correctly identified more than mild diabetic retinopathy 87.4% of the time, and correctly identified patients without it 89.5% of the time.¹⁰

‍

What percentage of FDA AI-enabled device authorizations are in radiology?

Through Q1 2026, radiology accounted for 1,164 of 1,524 total FDA AI-enabled device authorizations, 76% of the total.²³ In Q1 2026 alone, 69 of 92 authorizations (75%) were radiology devices.²³

‍

What is the external validation problem in medical imaging AI?

A systematic review of studies from 2022 to June 2025 found that imaging AI models consistently underperform when applied to data from outside their training environment.²⁴ Median AUC dropped by roughly 0.03, and specificity fell by as much as 24 percentage points. Multicenter training was associated with better generalization.²⁴

‍

What was CheXzero and why was it significant?

CheXzero was a model trained on chest X-rays with no explicit annotations, learning instead from the text of accompanying radiology reports.¹⁵ Published in Nature Biomedical Engineering in September 2022, it matched board-certified radiologists on several pathologies and generalized to conditions it had never been explicitly trained to detect. It demonstrated that the expert annotation bottleneck could be routed around using clinical data hospitals already generate.

‍

What is CLIP and how did it change medical imaging AI?

CLIP, published by Radford et al. at the International Conference on Machine Learning in 2021, showed that image-text pairs scraped at internet scale could produce transferable visual representations without categorical labels.¹⁴ For medicine, this was significant because hospitals generate paired image-and-report data as a routine byproduct of clinical care. That data could now, in principle, be used for training without the expense of expert annotation.

‍

What is the difference between a general foundation model and a medical one?

When Meta’s SAM was applied directly to medical images, its performance was unstable: strong on some targets, poor on others.¹⁷ MedSAM, fine-tuned on medical data and published in Nature Communications in January 2024, reported a large improvement.¹⁸ General-domain pre-training does not transfer cleanly to clinical imaging. Fine-tuning on domain-specific data, at sufficient scale and diversity, remains necessary.

‍

What makes a dataset “regulatory-grade” for imaging AI?

A regulatory-grade dataset is not simply large or well-annotated. FDA’s draft guidance on AI-enabled device software asks marketing submissions to address data collection, reference standard, annotation, independence of data, and representativeness.²⁵ A dataset that supports a preprint does not automatically meet these requirements.

‍

‍

References

1. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. In: Proceedings of NIPS; 2012.

2. Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation. In: Proceedings of MICCAI; 2015.

3. He K, et al. Deep residual learning for image recognition. In: Proceedings of CVPR; 2016.

4. Gulshan V, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316(22):2402-10.

5. Esteva A, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542(7639):115-8.

6. Ehteshami Bejnordi B, et al. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA. 2017;318(22):2199-210.

7. Wang X, et al. ChestX-ray8: hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: Proceedings of CVPR; 2017.

8. Rajpurkar P, et al. CheXNet: radiologist-level pneumonia detection on chest X-rays with deep learning. arXiv:1711.05225. 2017.

9. U.S. Food and Drug Administration. FDA permits marketing of clinical decision support software for alerting providers of a potential stroke in patients [press release]. 2018 Feb 13. Available from: https://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/ucm596575.htm. Product name (Viz LVO, formerly Viz ContaCT): Viz.ai [press release]. Business Wire; 2021 Aug 4. Available from: https://www.businesswire.com/news/home/20210804005405/en

10. U.S. Food and Drug Administration. FDA permits marketing of artificial intelligence-based device to detect certain diabetes-related eye problems [press release]. 2018 Apr 11.

11. U.S. Food and Drug Administration. FDA Talk Paper T98-38: M1000 ImageChecker. 1998 Jun 29. Available from: https://webarchive.library.unt.edu/eot2008/20090118171524/http://www.fda.gov/bbs/topics/ANSWERS/ANS00881.html. Hologic. Hologic to acquire R2 Technology, Inc. [press release]. 2006 Apr 24.

12. Johnson AEW, et al. MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports. Sci Data. 2019;6:317. doi:10.1038/s41597-019-0322-0

13. Isensee F, et al. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat Methods. 2021;18:203-11.

14. Radford A, et al. Learning transferable visual models from natural language supervision. In: Proceedings of ICML; 2021.

15. Tiu E, et al. Expert-level detection of pathologies from unannotated chest X-ray images via self-supervised learning. Nat Biomed Eng. 2022;6(12):1399-406.

16. Kirillov A, et al. Segment anything. arXiv:2304.02643. 2023.

17. Huang Y, et al. Segment anything model for medical images? Med Image Anal. 2024;92:103061. doi:10.1016/j.media.2023.103061

18. Ma J, et al. Segment anything in medical images. Nat Commun. 2024;15:654.

19. Chen Z, et al. CheXagent: towards a foundation model for chest X-ray interpretation. arXiv:2401.12208. 2024. Pérez-García F, et al. Exploring scalable medical image encoders beyond text supervision (RAD-DINO). Nat Mach Intell. 2025. doi:10.1038/s42256-024-00965-w. Hyland SL, et al. MAIRA-1: a specialised large multimodal model for radiology report generation. arXiv:2311.13668. 2023. Bannur S, et al. MAIRA-2: grounded radiology report generation. arXiv:2406.04449. 2024. Blankemeier L, et al. Merlin: a vision language foundation model for 3D computed tomography. arXiv:2406.06512. 2024.

20. Yang L, et al. Advancing multimodal medical capabilities of Gemini (Med-Gemini). arXiv:2405.03162. 2024. Sellergren A, et al. MedGemma technical report. arXiv:2507.05201. 2025.

21. Aidoc. Aidoc secures landmark FDA clearance for first foundation model-powered clinical AI solution of its kind [press release]. PR Newswire; 2025 Feb 18. Available from: https://www.prnewswire.com/il/news-releases/aidoc-secures-landmark-fda-clearance-for-first-foundation-model-powered-clinical-ai-solution-of-its-kind-302378437.html

22. Diagnostic Imaging. FDA clears CT-based AI triage platform from Aidoc. 2026 Jan 21. Available from: https://www.diagnosticimaging.com/view/fda-clears-ct-based-ai-triage-platform-aidoc

23. U.S. Food and Drug Administration. Artificial intelligence-enabled medical devices [Internet]. FDA; [updated 2026 Sep 22; cited 2026 Nov 9]. Available from: https://www.fda.gov/medical-devices/digital-health-center-excellence/artificial-intelligence-enabled-medical-devices

24. Assessing the generalizability of artificial intelligence in radiology: a systematic review of performance across different clinical settings. Ann Med Surg. doi:10.1097/MS9.0000000000004166. PMID: 41377327.

25. U.S. Food and Drug Administration. Artificial intelligence-enabled device software functions: lifecycle management and marketing submission recommendations [draft guidance]. Silver Spring (MD): FDA; 2025 Jan 7. Docket No. FDA-2024-D-4488. Available from: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/artificial-intelligence-enabled-device-software-functions-lifecycle-management-and-marketing

‍

‍

Related Resources

Pre-Training Data Is What Decides If Your Model Generalizes

Regulatory and Privacy Challenges for Real-World Imaging Data-Driven Foundation Models

Multimodal medical AI: beyond the image

Segmed: Large-scale data for radiology AI foundation models, from pretraining to FDA clearance