Large language models are rapidly becoming widespread in clinical decision support, patient communication, and documentation. An evidence-based, balanced assessment for the clinician who is aware of modes of failure (hallucination, evidence-datedness, concealment, bias) as well as capacities.
⚕️ Target audience: This content is aimed at healthcare professionals and is for educational/current-information purposes. AI tools offer clinical decision support; It does not replace clinician responsibility and approved clinical workflows.
Large language models (LLM - deep learning models trained on very large text corpora, producing human-like text by probabilistically predicting the next word) came to the clinical agenda with the public launch of ChatGPT in November 2022. Obstetrics and maternal-fetal medicine are a natural application ground for this technology, as a field that carries both intense patient communication and heavy documentation burden. But these same characteristics — producing fluid, persuasive but unverified text — bring with them unique risks. This article discusses the realistic uses of LLMs in obstetric practice, together with the numerical findings of published evaluation studies and patterns of failure.
LLMs are based on neural network structures called "transformers" such as GPT (Generative Pre-trained Transformer). The model completes a text word by word based on statistical patterns in the training data; it models linguistic possibility, not medical “understanding.” This distinction is clinically critical: the model produces the most likely continuation, not the truth. Correct and incorrect answers can be presented in the same confident style.
Modern models have now evolved into large multi-modal models (LMM) that can process images, audio and video as well as text. In its guidance published in 2024, the World Health Organization (WHO) identifies five broad areas of use for LMMs in health: diagnosis and clinical care, patient-oriented use, administrative/clerical work, medical education, and scientific research. The main conceptual limit of LLMs is that the training data is “frozen” at a specific date and the model cannot verify its source of information in real time.
Initial evaluations published in Obstetrics showed that LLMs can provide surprisingly consistent answers to structured questions. While Grünebaum and colleagues (Am J Obstet Gynecol, 2023) reported that ChatGPT provided largely fluent, knowledgeable, and significant error-free answers to obstetrics-gynecology questions, they clearly emphasized that the model was unable to reliably use and cite data post-2021—which carries the potential to mislead.
When Allahqoli et al (Gynecol Obstet Invest, 2023) presented 30 cases from six obstetrics-gynecology fields to the model and expert, ChatGPT correctly managed 90% (27/30) of the cases. The important limitation, noted by the authors themselves, is that the facts were taken from a textbook; This may have created a bias in favor of the model (test data being similar to training data). The external validity of these single-center, small-sample studies is limited.
In a more structured context, Gumilar and colleagues (Comput Struct Biotechnol J, 2025) compared AI language models in the interpretation of cardiotocography (CTG — recording of fetal heart rate and uterine contraction) images. ChatGPT-4o outperformed other models (Gemini Advanced 57.14; Copilot 47.29) and junior physicians (61.57) with a score of 77.86 out of 100; approached senior physicians (80.43) without a statistically significant difference (p > 0.05). Although this result is promising, it should be noted that the study was conducted on only seven images, with five evaluators, and is not a prospective test integrated into the actual clinical workflow.
Clinical perspective: The common message of these studies is that LLMs may add value alongside expert judgment, as an initial draft or checklist, rather than replace it. None provides prospective validation based on patient outcomes; claims of numerical superiority apply to artificial test conditions.
Perhaps the most practical use of LLMs is to produce understandable informational texts for the patient. In Horgan et al.'s clinician guide to the practice of maternal-fetal medicine (Am J Obstet Gynecol MFM, 2024), four board-certified MFM specialists evaluated five patient information brochures produced by ChatGPT: the accuracy median was 4.8 out of 6, and the completeness median was only 2.2 out of 3. The authors highlighted that some leaflets omitted clinically important information for counseling and the source could not be verified; As a result, they positioned the tool not as a resource for the public to use freely, but as an adjunct to clinical practice and as a draft generator that must be subject to expert approval.
Peled et al. (Int J Gynaecol Obstet, 2024) had 20 specialists evaluate responses to 15 obstetric questions frequently asked by pregnant women. Overall, 75% of responses received favorable ratings (≥4); accuracy was the strongest dimension (mean 4.2; 80% favorable), whereas completeness (mean 3.8; only 46.7% favorable) and safety (mean 3.9; 53.3% favorable) were markedly weaker. The authors' clear warning: caution is essential for questions involving maternal or fetal safety. Similarly, a recent editorial (Recker et al., Arch Gynecol Obstet, 2025) describes LLMs as "a digital companion for informed decision-making" — with the caveat that they cannot assess individual health status and cannot replace medical consultation.
The most rapidly adopted clinical LLM application is ambient AI scribes—tools that listen to the patient-clinician encounter and automatically draft clinical notes (e.g., Nuance/Microsoft DAX Copilot). They hold substantial promise for reducing documentation burden in high-volume outpatient specialties such as obstetrics and gynecology.
In a pilot study of 48 physicians at Stanford Health Care (Shah et al., JAMIA, 2025), ambient AI scribe produced statistically significant reductions in task burden (−24.42; p < 0.001) and burnout (−1.94; p < 0.001). In a companion study (Ma et al., JAMIA, 2025), LLM-supported scribe reduced daily documentation time by 6.89 minutes and total time spent in the electronic health record by 19.95 minutes/day; however, the tool was used in only 55.25% of the interviews and there was significant variability between users.
As the scale grows, the picture becomes more balanced. In an evaluation of 100 clinicians at Sutter Health (Stults et al., JAMA Netw Open, 2025), although mental workload and rushed-pace scores decreased significantly, the reduction in burnout did not reach statistical significance (p = 0.12) and the time saved per note was less than 1 minute per encounter. Benefit varied by specialty and user (highest satisfaction in primary care).
Security warning (documentation): Auto-generated notes are not error-free. Biro et al.'s validation study (J Med Internet Res, 2025) revealed that errors that could pose a security risk in AI scribe technology should be systematically evaluated. Each note produced should be reviewed and corrected by the clinician before signing.
The most critical clinical risk of LLMs is hallucination: when the model confidently produces information that is not factual, a bogus reference, or a guideline recommendation that does not exist. A controlled study by Williams et al (JAMA Intern Med, 2025) embodies this: In 100 inpatient episodes, LLM-generated discharge summaries were found to be comparable and equally preferable to physician notes in terms of overall quality; however, LLM grades contained more unique errors per abstract (mean 2.91 versus 1.82), and these errors included inaccuracies, omissions, and hallucinations. Although the overall potential for harm is low, it was emphasized that use without human approval is unsafe. In obstetrics, incorrect production of a dose, a gestational age threshold, or a contraindication can lead to direct patient harm.
Freezing training data at a specific date results in the model not knowing current guidelines (e.g., revised screening thresholds, new drug safety alerts). Additionally, a standard LLM cannot cite actual sources; The "references" it produces are often fabricated. Therefore, every LLM output should be validated against an authoritative and up-to-date source (guide, UpToDate, primary literature).
Entering patient data into a public, consumer-grade LLM interface is a serious risk of privacy violation. Personal Data Protection Law (KVKK) in Türkiye and General Data Protection Regulation (GDPR) in Europe protect health data as special (sensitive) data. WHO's LMM guidance includes data governance, consent and privacy as core principles. Personally identifiable patient information should not be entered into unaudited cloud-based models; enterprise use requires data processing agreements and local/secure distribution.
LLMs learn and reproduce social biases in training data. A systematic review examining demographic disparities in medical LLMs (Int J Equity Health, 2025) found bias in 22 of 24 studies reviewed (91.7%); gender bias was reported in 15 of 16 studies (93.7%), and racial/ethnic bias was reported in 10 of 11 studies (90.9%). Given the already existing racial and socioeconomic disparities in maternal mortality, the risk of an unvalidated model widening these gaps is particularly important for obstetrics.
Fluent and confident output may lead the clinician to abandon critical evaluation and accept the model's recommendation without question (automation bias). This is one of the most insidious risks when used correctly and shows why the "human-in-the-loop" principle is indispensable.
Most of the current obstetrics literature consists of small-sample, single-center, mostly retrospective or artificial test-based evaluation studies. While foundational studies in general medicine (e.g., Med-PaLM; Singhal et al., Nature 2023) have shown models to achieve high accuracy on medical question banks, human evaluation of the same studies has revealed models lagging behind clinicians and revealing safety and bias vulnerabilities. There is not yet a prospective, randomized LLM study measuring patient outcomes in obstetrics. Performance metrics (scores, percentages) are sensitive to test conditions and questions selected; their generalization to different populations, languages, and clinical contexts has not been demonstrated. Because models also change rapidly from version to version, the result obtained with a particular version should not be read as a permanent truth.
The balanced conclusion is that LLMs are a real tool of efficiency and communication in obstetric practice, but within a controlled, bounded and human-approved framework. Practical principles:
The regulatory and ethical framework in this area (WHO LMM guidance, reporting standards, institutional postures) is detailed in the Ethics, Bias, Validation and Regulation paper. For topics related to the image processing side of LLMs (e.g. CTG/ultrasound interpretation supports), see the sections Artificial Intelligence in Cardiotocography and Artificial Intelligence in Fetal Ultrasonography.
Last updated: . Sources have been confirmed with their publication dates and current versions.