Note: OpenAI leads the general AI space, but AI companies are developing deep research tools and experimenting with AI-powered academic searching in support of research. Perhaps you have faculty or students asking you to present these tools to classes.
Health professionals ask me all the time, What's happening with artificial intelligence (AI) in knowledge synthesis (KS) in 2026? A 2026 scoping review found that 65 AI tools and 25 machine learning models already automating parts (or all) of evidence synthesis workflows. However, most studies measure AI performance and only ~4% evaluate actual workload reduction in KS.
A variety of AI tools are used for systematic reviews or evidence synthesis but with highly variable and often mixed results. There are more and more tools all the time, including for crafting search strategies, identifying relevant articles or resources, and aiding in data screening, extraction, and synthesis. AI-based tools are adept at drafting plain language summaries of papers. Still, AI tools should be tested at various stages of a review, understanding their potential limitations and risks for all knowledge synthesists and expert searchers. When using AI tools that lack a rigorous evidence base and prior evaluation, the emerging consensus seems to be that ...they should be paired with established, validated human-in-the-loop methods. Additionally, ethical, copyright, and intellectual property considerations must be addressed, especially when uploading data or full-text articles to AI platforms. This practice, which now seems to be routine for some, introduces copyright infringement and misuse of intellectual property. Speak to your librarian first.
Julie Glanville on the role of AI search tools in knowledge synthesis
Julie Glanville, an independent consultant, does a deep dive into using AI tools to build and run searches for systematic reviews, as part of the AI Nurses Network webinar series. https://www.ai-nurses.com/
Bottom line: "For title and abstract screening and literature searching, various methods and tools (including AI) aim at improving the efficiency of SR production. However, only few studies have addressed the influence of these methods and tools in real-world workflows. Few studies exist that evaluate methods or tools supporting the remaining tasks. Additionally, while validity outcomes are frequently reported, there is a lack of evaluation regarding other outcomes".
Alshami, A.; Elsayed, M.; Ali, E.; Eltoukhy, A.E.E.; Zayed, T. Harnessing the Power of ChatGPT for Automating Systematic Review Process: Methodology, Case Study, Limitations, and Future Directions. Systems 2023, 11, 351. https://doi.org/10.3390/systems11070351
Bottom line: Authors explore the use of ChatGPT in (1) preparation of Boolean research terms and article collection, (2) abstract screening and articles categorization, (3) full-text filtering and information extraction, and (4) content analysis to identify trends, challenges, gaps, and proposed solutions. Results are probably better in 2025 than when this paper was written in 2023, although caution is advised.
Blaizot, A, Veettil, SK, Saidoung, P, et al. Using artificial intelligence methods for systematic review in health sciences: A systematic review. Res Syn Meth. 2022; 13(3): 353-362. doi:10.1002/jrsm.1553
Bottom line: The review discusses automated tools and platforms that use artificial intelligence (AI) approaches; it evaluates the reported benefits and challenges in using such methods. Authoers report on the usage of Rayyan Robot Reviewer EPPI-reviewer; K-means; SWIFT-review; SWIFT-Active Screener; Abstrackr; Wordstat; Qualitative Data Analysis (QDA); Miner and NLP and assess the quality of the reviews.
Clark J, Barton B, Albarqouni L, et al. Generative artificial intelligence use in evidence synthesis: A systematic review. Research Synthesis Methods. Published online 2025:1-19. https://doi.org/10.1017/rsm.2025.16
Bottom line: Current evidence does not support GenAI use in evidence synthesis without human involvement or oversight. However, for most tasks other than searching, GenAI may have a role in assisting humans with evidence synthesis.
Harasgama S, Pearce H, Appel C, Loftus L, Painter H, Kuhn I, Karpusheff J, Ceesay A, Ford J. Artificial Intelligence Tools for Automating Evidence Synthesis: Scoping Review. J Med Internet Res. 2026 Mar 30;28:e81597. doi: 10.2196/81597. https://pubmed.ncbi.nlm.nih.gov/41911537/
Botom line: This review comprehensively captures the broad, evolving suite of AI automation tools available to support evidence synthesis, leveraged by increasingly complex AI approaches that range from traditional ML to LLMs. The notable shift toward studying general-purpose generative AI tools reflects how these technologies are actively transforming evidence synthesis practice. The lack of studies in our review comparing different AI approaches for specific automation stages or evaluating their effectiveness pragmatically represents a significant research gap. Optimal tool selection will likely depend on the review topic and methodology and researcher priorities. While they offer potential for reducing workload, ongoing evaluation to mitigate AI bias and to ensure the integrity of reviews is essential for safeguarding evidence-based decision-making.
Bottom line: ...a must-read for librarians involved in knowledge synthesis and those who do deduplication to support it.
Kebede, MM, Le Cornet, C, Fortner, RT. In-depth evaluation of machine learning methods for semi-automating article screening in a systematic review of mechanistic literature. Res Syn Meth. 2023; 14(2): 156-172. doi:10.1002/jrsm.1589
Bottom line: "The authors aimed to evaluate the performance of supervised machine learning algorithms in predicting articles relevant for full-text review in a systematic review. Any machine learning approaches in title/abstract screening should be investigated further toward refining these tools and automating their implementation"
Bottom line: "This paper identified that LitSuggest, Rayyan, Abstractr, BIBOT, R software, RobotAnalyst, DistillerSR, ExaCT and NetMetaXL have potential to be used for the automation of systematic reviews. However, they are not without limitations. The review also identified other studies that employed algorithms that have not yet been developed into user friendly tools. Some of these algorithms showed high validity and reliability but their use is conditional on user knowledge of computer science and algorithms."
Khraisha Q, Put S, Kappenberg J, Warraitch A, Hadfield K. Can large language models replace humans in systematic reviews? Evaluating GPT-4's efficacy in screening and extracting data from peer-reviewed and grey literature in multiple languages. Res Syn Meth. 2024; 1-11. doi:10.1002/jrsm.1715
Bottom line: "Although our findings indicate that, currently, substantial caution should be exercised if LLMs are used to conduct systematic reviews, they also offer preliminary evidence that, for certain review tasks delivered under specific conditions, LLMs can rival human performance in terms of screening and extracting data."
Lieberum J-L, Töws M, Metzendorf M-I, Heilmeyer F, Siemens W, Haverkamp C, Böhringer D, Meerpohl JJ, Eisele-Metzger A, Large language models for conducting systematic reviews: on the rise, but not yet ready for use – a scoping review, Journal of Clinical Epidemiology (2025). https://www.jclinepi.com/article/S0895-4356(25)00079-4/fulltext
Bottom line: This is an excellent paper. ".. LLMs should be used with caution and limited to specific SR tasks under human supervision. Despite the apparent technical simplicity of implementing LLMs in literature searching, their evaluation – authors rated half of the approaches as non-promising – seems to show a range of limitations. LLMs seem to support in study selection and data extraction and appeared more favorable, with by far most (study selection) or at least a slight majority (data extraction) of the described application forms rated as promising, and the rest categorized as neutral." Dean: a key paper delivered by outstanding librarian colleagues. SR lit searching n=15; 41% of 37 papers; searching most frequent SR step aided by LLMs; also, most reported as non-promising (n=8); rated as non-promising.
Mahuli, S., Rai, A., Mahuli, A. et al. Application ChatGPT in conducting systematic reviews and meta-analyses. Br Dent J 235, 90–92 (2023). https://doi.org/10.1038/s41415-023-6132-y
Bottom line: Explores using ChatGPT for conducting Risk of Bias analysis and data extraction from a randomised controlled trial.
Ovelman, C., Kugley, S., Gartlehner, G., & Viswanathan, M. (2024). The use of a large language model to create plain language summaries of evidence reviews in healthcare: A feasibility study. Cochrane Evidence Synthesis and Methods, 2(2), e12041. https://onlinelibrary.wiley.com/doi/abs/10.1002/cesm.12041
Bottom line: "LLMs show promise for assisting in PLS creation but likely require human input to ensure accuracy..."
Qureshi, R., Shaughnessy, D., Gill, K.A.R. et al. Are ChatGPT and large language models “the answer” to bringing us closer to systematic review automation?. Syst Rev 12, 72 (2023). https://doi.org/10.1186/s13643-023-02243-z
Bottom line: "Our experience from exploring the responses of ChatGPT suggest that LLMs show some promise for aiding in SR-related tasks, but the technology is in its infancy and needs development. We advise great caution be taken by non-content experts in using these tools due to much of the output appearing, at a high level, to be valid, while much is erroneous and in need of active vetting."
van Dijk SHB, Brusse-Keizer MGJ, Bucsán CC, et al.Artificial intelligence in systematic reviews: promising when appropriately used. BMJ Open 2023;13:e072254. doi: 10.1136/bmjopen-2023-072254
Bottom line: Suggests how to conduct a transparent and reliable SR using AI tool ‘ASReview’ for title and abstract screening.
