Automated indexing may be defined as indexing the subject content of papers by means of a computer with some human intervention or oversight, or none at all. (Giustini et al., 2025). In 2005, Gay et al wrote that “[the Medical Text Indexer (MTI)] is the embodiment of the automated methods developed at NLM... and has been used to support human indexers at the NLM since September 2002. We refer to this processing as semi-automatic indexing in contrast to the fully automated indexing provided by MTI for some meetings abstracts collections”.
Automated indexing encompasses semi-automated and fully automated processes, depending on the level of human curation involved. According to Ruiz and Aronson (2009), automatic indexing is a form of text categorization in which documents are assigned terms from a controlled vocabulary by machines to summarize their subject content. Increasingly, automated indexing has incorporated computational approaches such as algorithms (hence, algorithmic indexing), natural language processing, and artificial intelligence (AI). "...Automated subject indexing is then machine-based subject indexing where human intellectual processes ... are replaced by techniques based on computational approaches". (Golub, 2021).
Historical background
Automated indexing has its roots in the early automation efforts of the 1950s, when researchers first began to apply computational methods to document analysis, indexing, and information retrieval.
Luhn (1957) was among the first to demonstrate that statistical properties of language such as word frequency and distribution could be used to identify significant terms in indexing, laying the groundwork for machine-based approaches (seelink). Probabilistic indexing emerged in the late 1950s, marking interest from deterministic to uncertainty based models. Maron and Kuhns (1960) introduced the idea that the relationship between documents, terms, and relevance could be expressed probabilistically, where terms could be weighted according to their likelihood of relevance; their work proposed that terms could be weighted according to their estimated ability to discriminate between relevant and non-relevant documents, establishing an early foundation for probabilistic indexing and automated retrieval systems.
Golub (2021) characterized automated indexing as the computational replacement of human cognitive processes, where algorithms perform subject or conceptual analysis, and term assignment. Golub's entry highlights the evolution from manual, human-centered indexing towards automated techniques that replicate or approximate these intellectual tasks using computational methods. (See also: Golub, 2019).
Automated (or semi-automated) compared to human indexing?
A commonly-stated goal of state-of-the-art automated indexing is to mimic human indexing. The principal challenge in automating the indexing process lies in extracting an exhaustive yet precise set of controlled terms that accurately represent the subject content of each document - ideally at the level of a trained human indexer with expert-level judgment.
"...Research on automatic subject indexing began with the availability of electronic text in the 1950s (Luhn 1957; Baxendale 1958; Maron 1961) and continues to be a challenging topic... For a historical overview of automatic indexing, see Stevens (1965) and Sparck Jones (1974) covering the early period of automatic indexing, and Lancaster (2003, 289-292) for the later one. A related term is machine-aided indexing (MAI) or computer-assisted indexing (CAI) where it is the human indexer who decides, based on a suggestion provided by the computer (see, for example, Medical Text Indexer (U.S. National Library of Medicine 2016)). A similar approach is applied by Martinez-Alvarez, Yahyaei, and Roelleke (2012) who propose a semi-automatic approach in which only those predictions likely to be correct are processed automatically, while more complex decisions are left to human experts to decide."
Semi-automated indexing combines machine-generated recommendations with human expertise to improve the efficiency, scalability, and consistency of indexing while maintaining the quality provided by human indexers. With the NLM's MTI, algorithms identify candidate MeSH terms from article text and related records, while human indexers review, validate, modify, and supplement these suggestions. Human curation remains essential for interpreting complex biomedical concepts, resolving ambiguity, applying appropriate specificity, identifying false positives and missed concepts, and ensuring alignment with the structure and evolving terminology of controlled vocabularies. As automated indexing has advanced toward machine learning and neural network approaches, the role of human indexers has shifted from assigning every descriptor manually toward supervising automated outputs, managing exceptions, and maintaining the reliability of the indexing. Semi-automated indexing therefore represents a human-in-the-loop model in which automation enhances productivity while expert judgment safeguards accuracy and quality.
NLM continues to evaluate emerging technologies to improve indexing performance but challenges remain, particularly as novel biomedical concepts enter the literature. In 2021, the average time to index articles by human indexers was 145 days, excluding bibliographic processing. By 2022, NLM had implemented a fully automated indexing program; in this model, human review is retained for selected subject areas, while other records are reviewed on a sampling basis. By 2025, the move to automated indexing reduced indexing time to approximately one business day.
What is fully automated indexing in MEDLINE?
Automated indexing in MEDLINE is increasingly referred to as algorithmic indexing (seeAmar-Zifkin et al., 2025). The MTI of 2002 employed its own algorithmic approaches, but was always intended to primarily support human indexers by generating indexing recommendations. As publication volumes increased, NLM progressively relied more heavily on automated approaches, deep learning and computational methods. In 2022, first-line indexing for all MEDLINE records was performed by MTI Auto (MTIA), with human review focused mainly on specialized areas such as gene- and protein-related records. By 2025, NLM had transitioned to the MTIX, a neural network system designed to improve scalability, consistency, and MeSH indexing accuracy. The MTIX now performs the majority of routine indexing tasks in MEDLINE, while human indexers (and "curators") continue to provide oversight, quality assurance, and expert review for new, complex, ambiguous, or high-priority records. Despite decades of MTI development and an extensive body of research examining its enhancements, no prior scoping or systematic review has synthesized the large body of evidence on MTI-related research prior to Giustini et al, 2026).
Drivers, benefits and challenges of automated indexing
NLM followed a deliberate, staged trajectory from 2002; first, MTI as a decision support tool, gradual assessment recommending full automation with human curation in 2022 and 2024 (Cid et al, 2025; Krithara et al, 2023).
Quantifiable benefits are obvious for timeliness and throughput; concrete, NLM authored data on direct labour or budget reductions are not fully elucidated in the scholarly literature, but cost per article figures are mentioned in papers (Amar-Zifkin et al 2025; Mao et al, 2017).
Automated indexing has not eliminated human indexers or their work: one third of records still receive human curation, concentrated in complex or high value content, and multiple publications document indexing deficiencies (see below).
Medical text indexer (MTI) and MEDLINE
The Medical Text Indexer (MTI) is NLM's automated indexing system for MEDLINE, and one of the most significant achievements in large-scale automated indexing by a national library; its development reflects decades of sustained research, evaluation, and refinement. However, the MTI was envisioned as an automated tool that supported, rather than replaced, the expertise and oversight of highly skilled indexers.
NLM introduced the MTIX (Medical Text Indexer–NeXt Generation) in 2024, replacing MTI-Auto and incorporating machine learning and neural network methods to assign MeSH terms to biomedical articles. Key advantages of MTIX include substantially improved speed and scalability.
Trained on millions of MEDLINE citations published between 2007 and 2022, MTIX analyzes article titles, abstracts, and journal metadata to recommend MeSH terms with high recall (e.g., >94% for disease detection) and strong precision (e.g., ~87% for disease categories).
MTIX supports both semi-automated and fully automated workflows, significantly reducing the burden on human indexers while maintaining indexing standards. Nevertheless, despite an overall F-score of approximately 0.74, estimated error rates remain substantial—ranging from one-third to one-half in some analyses (Amar-Zifkin et al., 2025; Askin et al., 2025).
Neural networks underpin the MTIX enabling rapid, large-scale indexing within one business day; since 2020, nearly a million new citations were indexed at NLM, and nearly 1.4 million papers added in 2022 alone. These numbers will probably increase over time, and in 2026 are thought to approach or even exceed 1.5 million citations. See PubMed search for 2025.
While human oversight remains essential for quality assurance, NLM’s AI-driven systems support public tools such as MeSH on Demand.
Since 2020, NLM has incorporated transformer technology into the MTIX called Bidirectional Encoder Representations from Transformers (BERT)–based models (e.g., BioBERT and PubMedBERT). These models support “First-Line” and “Full-Text” predictors, improving recall for rare MeSH terms and reducing human workload. Domain-specific pretraining is critical, as general-purpose models lack the biomedical vocabulary and contextual sensitivity required for accurate MeSH prediction. For specialized tasks like gene entity recognition, BERT achieved F-scores of 0.92 (precision 0.94; recall 0.90), reducing false positives by 15%.
Despite these advances, human indexers remain essential for correcting errors and ensuring the quality and consistency of MEDLINE records. Often, curation occurs after the MTIX assigns MeSH terms within one business day, but may or may not include access to the full-text of the articles indexing by NLM https://support.nlm.nih.gov/kbArticle/?pn=KA-05326
Automated indexing from MTI (2002), MTIFL (2011) to MTI-Auto (2019,2022)
Rules-based systems such as the Medical Text Indexer (MTI) (2002) relied on human-authored instructions (e.g., “based on NLM policy, assign the most specific MeSH term”). Rules are often derived from synonym mappings and “See/Use” references in MeSH. For example, if a paper contained the phrase “heart attack,” MTI would recommend assigning the MeSH heading Myocardial Infarction.
In 2011, the MTI First Line (MTIFL) became the first-line indexer for 14 journals, expanding to 51 journals by 2015. MTI-FL improved the precision of MeSH recommendations presented to human indexers by applying new filters (see Tsatsaronis et al, 2015]. It incorporated Journal Descriptor Indexing (JDI) and journal subsets to assign weights to candidate MeSH terms. Bayesian classifiers, trained on approximately four million MEDLINE citations, were used to predict 122 discipline categories (e.g., Cardiology and Medical Genetics). By leveraging statistical associations among journal-level metadata, textual features, and MeSH assignments, the MTIFL achieved higher precision and recall compared with the earlier MTI.
The third generation MTI, MTIA (Auto) (2019), determined which MeSH terms should apply to a given MEDLINE record by: identifying uncommon or specific terms in that article’s title and abstract; finding MeSH for those terms; gathering MeSH which have been assigned to other records with similar uncommon or specific terms from within MEDLINE; and ranking these terms and deciding which to apply to the record. The MTIA used several processes to rank terms. (For more information about the MTIA algorithm, seeAmar-Zifkin et al., 2025).
More recently, the MTIA and MTIX (neXt) (2024) were shown to have difficulties with newer terminologies and MeSH terms, evolving language, or complex phrasing which led to missed or incorrect MeSH assignments. By 2024, machine learning systems using neural networks had emerged as more adaptive and important alternatives for integration. MTIX was trained on millions of MEDLINE records (2007–2022), allowing it to learn linguistic and semantic patterns rather than relying on fixed rules.
Rules-based systems (2002–2022) functioned effectively for many years but required continual updating and human intervention, using various methods such as journal descriptor indexing (JDI) leveraging semantic types (STs) and other methods were widely tested. (See also Automatic Indexing by Discipline and High-Level Categories). As the biomedical literature expanded, machine learning approaches proved more capable of addressing linguistic variation and semantic nuance. Even so, human indexers continue to amend incorrectly indexed records as of 2026.
MTIX of 2024
The MTIX, introduced in 2024, replaced the MTIA (Auto) (2019, 2022), which was the result of decades of research and building upon the legacy system. Rules-based methods including earlier versions such as MTI, MTIFL, and MTIA relied on hand-crafted rules and heuristics rather than learning directly from the data in MEDLINE citations. These systems applied predetermined assignments based on MEDLINE indexing policies, as well as directives embedded in see references and scope notes within the MeSH vocabulary.
For many years from 2002-2022, the MTI and its variants matched exact keywords in article titles and abstracts to candidate MeSH terms and applied pattern-based rules (e.g., assigning the MeSH term Hip Fractures when phrases such as “fracture of the hip” appeared). Additional rules were used to assess relevance, including word-frequency thresholds and other heuristic semantic techniques.
By contrast, MTIX employed data-driven machine learning methods that have dramatically improved indexing efficiency. The MTIX also leverages neural networks to learn complex semantic relationships between biomedical text and Medical Subject Headings (MeSH), enabling more accurate and scalable indexing than earlier rule-based systems. By training on millions of MEDLINE citations, these neural architectures capture contextual meaning and synonymy that cannot be encoded through hand-crafted rules. As a result, MTIX achieves faster indexing turnaround while maintaining high recall and precision across diverse biomedical domains. As of 2026, article citations are typically indexed within one day of receipt in NLM’s indexing system. In practical terms, most articles from MEDLINE-indexed journals now appear in PubMed with assigned MeSH terms within one business day. https://www.nlm.nih.gov/bsd/indexfaq.html#descriptor
Missing MeSH (False Negatives) terms and tags — automated indexing may not "see" concepts (in the full-text, for example) and therefore may not assign relevant MeSH terms and check tags that would be obvious to a human indexer.
Amar-Zifkin et al assessed a sample of MEDLINE records (using MTIA) from February–March 2023; 47 % of records had inadequacies in indexing, such as missing significant concepts, use of overly general headings, or misassignments, confirming substantial false negatives and reduced recall. “Musings on MeSH” reported Amar-Zifkin et al concluded that 47 % of records had minor or major MeSH issues, which would indeed affect retrieval. Still relying solely on human indexing is no longer practical, and continuous refinement of the algorithm underscores NLM's commitment to accuracy.
Extra (Spurious) MeSH Terms (False Positives)
Askin et al., in their JMLA article “Filtering failure: the impact of automated indexing in Medline on retrieval of human studies for knowledge synthesis,” said that indexing often includes irrelevant terms or omits obviously relevant ones re: human studies. Concerns about check tag errors such as gender biases favouring “Male” over “Female” underscore the problem of false positives in machine learning models. Chen et al. (2023) noted a frequent misuse or omission of check tags (e.g., gender or age).
Overly General or Inaccurate Publication Types - errors in publication types were often hierarchically related e.g., tagging as Historical Article instead of more accurate Biography, or Clinical Trial when it’s a Clinical Study. PT errors affect search precision and filtering in search filters and knowledge synthesis.
NLM's MeSH 2025 Update showed that NLM makes adjustments to Publication Types—such as introducing “Network Meta-Analysis” or “Scoping Review.” Automated indexing systems (like MTIX) may lag or misclassify when publication types change or are too general.
Menke et al, 2025 report that "full-text features, enhanced document representations, and fine-tuning optimizations improve publication type and study design indexing."
Limited Context: Missing Populations or Methods Details — MTIX relies on titles and abstracts (plus metadata in journal and pubyear) rather than full text. It can miss details such as populations or methodology, commonly found in full text.
Amar-Zifkin et al. note that MTIX (2025) was trained on citations to 2022 and used titles, abstracts, and metadata, not full-text content. “Musings on MeSH” blog states that automated indexing is based only on title and abstract, meaning details found deeper in full text—such as population descriptors or methodology—can be missed. Chen et al. said the MTI tended to rank “Male” check tag more highly than “Female,” and frequently omitted “Aged” check tag—reflecting how terms can be missed.
New or Drifted MeSH Terms - MTIX’s training data covers citations up to 2022, so new MeSH terms or those evolved in meaning (“drifted”) may not be recognized or applied. NLM addresses this by adding examples of new or drifted terms for MTIX retraining, but gaps still exist.
NLM reported that MTIX “needs new training data” in order to recognize new MeSH terms or drifted terminology, indicating gaps if new concepts emerge post-training. NLM's MeSH 2025 Update shows revisions (e.g., additions of AI-related headings, publication type changes) are being made to the vocabulary. MTIX’s older training means it may miss or misapply these new terms.
Health sciences librarians (HSLs) may wish to consider how automated indexing is reshaping search practices and MEDLINE instruction. Understanding MTIX and its AI-driven features suggests a growing need to test and refine search strategies that combine MeSH and free-text terms to ensure comprehensive retrieval—particularly for very recent, partially indexed, or non-indexed literature. HSLs may also play an important role in communicating the fundamentals of automated indexing to users, sharing emerging best practices with colleagues, and explaining the implications of these changes for search precision and recall in MEDLINE.
This raises several questions for practice and professional reflection:
How will automated indexing influence our search strategies in support of knowledge synthesis (KS) and our users—if at all?
In what ways might HSLs’ searching evolve as they develop a deeper understanding of MTIX and its AI features?
What pivots are HSLs making in MEDLINE instruction and in the design of expert search strategies?
How are librarians responding to user questions about MeSH assignment in MEDLINE, such as “How are MeSH terms assigned?”
Feel free to share your comments, experiences, and concerns.
Dean Giustini
UBC Biomedical Librarian
dean.giustini@ubc.ca
Epp C, Askin N, Ostapyk T. Still a filtering failure? Automated indexing using MTIX versus MTIA and its impact on human study filtering for knowledge synthesis. J Can Health Libr Assoc. 2025 Aug 1;46(2):53–62.
Lancaster FW. Indexing and Abstracting in Theory and Practice. 2nd ed. Champaign, IL: University of Illinois Graduate School of Library and Information Science; 1998.
Note: Please use your critical reading skills while reading entries. No warranties, implied or actual, are granted for any health or medical search or AI information obtained while using these pages. Check with your librarian for more contextual, accurate information.
