Automated indexing has been defined as “indexing the subject content of papers by means of a computer, either with some human intervention and oversight, or none at all”. (Giustini et al., 2025). As of 2025, automated indexing is carried out using a range of computational approaches, including algorithms (hence, algorithmic indexing), natural language processing, and artificial intelligence (AI). Automated indexing encompasses both semi-automated and fully automated processes, depending on the level of human curation involved.
According to Ruiz and Aronson (2009), automatic indexing is a form of text categorization in which documents are assigned terms from a controlled vocabulary by machines in order to summarize their subject content.
Automated (or semi-automated) compared to human indexing?
A commonly stated goal of state-of-the-art automated indexing is to mimic human indexing. The principal challenge lies in extracting an exhaustive yet precise set of controlled terms that accurately represent the subject content of each document—ideally at the level of a trained human indexer. While the National Library of Medicine (NLM) continues to evaluate emerging technologies to improve indexing performance, persistent challenges remain—particularly as novel biomedical concepts enter the literature. In fiscal year 2021, the average time required to index articles fully reviewed by human indexers was 145 days, excluding bibliographic processing. By 2022, NLM had implemented a fully automated indexing program for MEDLINE using its Medical Text Indexer (MTI-Auto). Under this model, human review is retained for selected subject areas, while other records are reviewed on a sampling basis. The move to automated indexing reduced indexing time to approximately one business day.
What is automated indexing in MEDLINE?
Automated indexing in MEDLINE is increasingly referred to as algorithmic indexing (seeAmar-Zifkin et al., 2025). Within the MTI framework, algorithms play a central role in NLM’s indexing workflow. In 2022, first-line indexing for all MEDLINE records was performed by MTIA, with human curation largely limited to gene- and protein-related records. By 2025, NLM had transitioned to MTIX, which is based on neural networks.
According to the Encyclopedia of Knowledge Organization, algorithmic indexing has historically referred to search-engine environments in which automation is essential due to scale. Similarly, semantic indexing assumes that terms appearing in related documents share conceptual meaning; associations between co-occurring terms are computed, and concepts are extracted from a large corpus of materials. Semantic AI–based indexing scales efficiently and underpins modern web search engines. While algorithms have been shown to improve indexing consistency, they do not eliminate inconsistency entirely. Automated indexing is therefore not an “objective” process: it reflects the perspectives and biases embedded in the texts and data on which it is trained. Reliance on large volumes of raw text introduces its own forms of imprecision and unreliability.
Medical text indexer (MTI) and MEDLINE
The Medical Text Indexer (MTI) is the automated indexing system developed by the National Library of Medicine (NLM) for MEDLINE and represents one of the most significant achievements in large-scale automated indexing by a national library. Its development reflects decades of sustained research, evaluation, and refinement.
In 2024, MTIX (Medical Text Indexer–NeXt Generation) replaced MTI-Auto, incorporating machine learning and neural network–based methods to assign MeSH terms to biomedical articles. Key advantages of MTIX include substantially improved speed and scalability. Trained on millions of MEDLINE citations published between 2007 and 2022, MTIX analyzes article titles, abstracts, and journal metadata to recommend MeSH terms with high recall (e.g., >94% for disease detection) and strong precision (e.g., ~87% for disease categories).
The MTIX supports both semi-automated and fully automated workflows, significantly reducing the burden on human indexers while maintaining indexing standards. Nevertheless, despite an overall F-score of approximately 0.74, estimated error rates remain substantial—ranging from one-third to one-half in some analyses (Amar-Zifkin et al., 2025; Askin et al., 2025).
Neural networks underpinning MTIX enable rapid, large-scale indexing—critical given that nearly 1.4 million papers were added to MEDLINE in 2024 alone. While human oversight remains essential for quality assurance, NLM’s AI-driven systems support public tools such as MeSH on Demand.
Since 2020, NLM has incorporated Bidirectional Encoder Representations from Transformers (BERT)–based models (e.g., BioBERT and PubMedBERT). These models support “First-Line” and “Full-Text” predictors, improving recall for rare MeSH terms and reducing human workload. Domain-specific pretraining is critical, as general-purpose models lack the biomedical vocabulary and contextual sensitivity required for accurate MeSH prediction.
Despite these advances, human indexers remain essential for correcting errors and ensuring the quality and consistency of MEDLINE records.
Automated indexing from MTI (2002) to MTI-Auto (2022)
Rules-based systems such as Medical Text Indexer–Automated (MTIA) rely on human-authored instructions (e.g., “based on NLM policy, assign the most specific MeSH term”). Rules are often derived from synonym mappings and “See/Use” references in MeSH. For example, if a paper contained the phrase “heart attack,” MTI would assign the MeSH heading Myocardial Infarction.
While precise, rules-based approaches are rigid. New terminology, evolving language, or complex phrasing often led to missed or incorrect MeSH assignments. By 2024, machine learning systems using neural networks had emerged as more adaptive alternatives. MTIX was trained on millions of MEDLINE records (2007–2022), allowing it to learn linguistic and semantic patterns rather than relying on fixed rules.
Rules-based systems (2002–2022) functioned effectively for many years but required continual updating and human intervention. As biomedical literature expanded in scale and complexity, machine learning approaches proved more capable of addressing linguistic variation and semantic nuance. Even so, human indexers continue to amend incorrectly indexed records.
MTIX of 2024
MTIX, introduced in 2024, replaced MTIA (Auto) (2019, 2022), which was a legacy rules-based system. Rules-based methods—including earlier versions such as MTI, MTI-FL, and MTIA—relied on hand-crafted rules and heuristics rather than learning directly from data in MEDLINE citations. These systems applied predetermined assignments based on MEDLINE indexing policies, as well as directives embedded in see references and scope notes within the MeSH vocabulary.
For example, MTI matched exact keywords in article titles and abstracts to candidate MeSH terms and applied pattern-based rules (e.g., assigning the MeSH term Hip Fractures when phrases such as “fracture of the hip” appeared). Additional rules were used to assess relevance, including word-frequency thresholds and other heuristic semantic techniques.
By contrast, MTIX employs data-driven, machine learning–based methods that have dramatically improved indexing efficiency. The MTIX also leverages neural network–based models to learn complex semantic relationships between biomedical text and Medical Subject Headings (MeSH), enabling more accurate and scalable indexing than earlier rule-based systems. By training on millions of MEDLINE citations, these neural architectures capture contextual meaning and synonymy that cannot be encoded through hand-crafted rules. As a result, MTIX achieves faster indexing turnaround while maintaining high recall and precision across diverse biomedical domains. As of 2026, article citations are typically indexed within one day of receipt in NLM’s indexing system. In practical terms, most articles from MEDLINE-indexed journals now appear in PubMed with assigned MeSH terms within one business day. https://www.nlm.nih.gov/bsd/indexfaq.html#descriptor
Missing MeSH (False Negatives) terms and tags — automated indexing may not "see" concepts (in the full-text, for example) and therefore may not assign relevant MeSH terms and check tags that would be obvious to a human indexer.
Amar-Zifkin et al assessed a sample of MEDLINE records (using MTIA) from February–March 2023; 47 % of records had inadequacies in indexing, such as missing significant concepts, use of overly general headings, or misassignments—confirming substantial false negatives and reduced recall. “Musings on MeSH” reported Amar-Zifkin et al concluded that 47 % of records had minor or major MeSH issues, which would indeed affect retrieval. Still relying solely on human indexing is no longer practical, and the continuous refinement of the algorithm underscores NLM's commitment to accuracy.
Extra (Spurious) MeSH Terms (False Positives)
Askin et al., in their JMLA article “Filtering failure: the impact of automated indexing in Medline on retrieval of human studies for knowledge synthesis,” said that indexing often includes irrelevant terms or omits obviously relevant ones re: human studies. Concerns about check tag errors — such as gender biases favoring “Male” over “Female” — underscore the problem of false positives in machine learning models. Chen et al. (2023) noted a frequent misuse or omission of check tags (e.g., gender or age).
Overly General or Inaccurate Publication Types — errors in publication types were often hierarchically related — e.g., tagging as Historical Article instead of more accurate Biography, or Clinical Trial when it’s a Clinical Study. PT errors affect search precision and filtering in search filters and knowledge synthesis.
NLM's MeSH 2025 Update showed that NLM makes adjustments to Publication Types—such as introducing “Network Meta-Analysis” or “Scoping Review.” Automated indexing systems (like MTIX) may lag or misclassify when publication types change or are too general.
Menke et al, 2025 report that "full-text features, enhanced document representations, and fine-tuning optimizations improve publication type and study design indexing."
Limited Context: Missing Populations or Methods Details — MTIX relies on titles and abstracts (plus metadata in journal and pubyear) rather than full text. It can miss details such as populations or methodology—commonly found in full text.
Amar-Zifkin et al. note that MTIX (as of early 2025) was trained on citations up to 2022 and primarily uses titles, abstracts, and metadata, not full-text content. “Musings on MeSH” blog states that automated indexing is based only on title and abstract, meaning details found deeper in full text—such as population descriptors or methodology—can be missed. Chen et al. said the MTI tended to rank “Male” check tag more highly than “Female,” and frequently omitted “Aged” check tag—reflecting how terms can be missed.
New or Drifted MeSH Terms — MTIX’s training data covers citations up to 2022, so new MeSH terms or those evolved in meaning (“drifted”) may not be recognized or applied. NLM addresses this by adding examples of new or drifted terms for MTIX retraining, but gaps still exist.
NLM reported that MTIX “needs new training data” in order to recognize new MeSH terms or drifted terminology—indicating gaps if new concepts emerge post-training. NLM's MeSH 2025 Update shows revisions (e.g., additions of AI-related headings, publication type changes) are being made to the vocabulary. MTIX’s older training means it may miss or misapply these new terms.
Health sciences librarians (HSLs) may wish to consider how automated indexing is reshaping search practices and MEDLINE instruction. Understanding MTIX and its AI-driven features suggests a growing need to test and refine search strategies that combine MeSH and free-text terms to ensure comprehensive retrieval—particularly for very recent, partially indexed, or non-indexed literature. HSLs may also play an important role in communicating the fundamentals of automated indexing to users, sharing emerging best practices with colleagues, and explaining the implications of these changes for search precision and recall in MEDLINE.
This raises several questions for practice and professional reflection:
How will automated indexing influence our search strategies in support of knowledge synthesis (KS) and our users—if at all?
In what ways might HSLs’ searching evolve as they develop a deeper understanding of MTIX and its AI features?
What pivots are HSLs making in MEDLINE instruction and in the design of expert search strategies?
How are librarians responding to user questions about MeSH assignment in MEDLINE, such as “How are MeSH terms assigned?”
Feel free to share your comments, experiences, and concerns.
Dean Giustini
UBC Biomedical Librarian
dean.giustini@ubc.ca
Note: Please use your critical reading skills while reading entries. No warranties, implied or actual, are granted for any health or medical search or AI information obtained while using these pages. Check with your librarian for more contextual, accurate information.
