{| id="mp-topbanner" style="width:98%; background:#cee8e6; margin-top:-1em; border:1px solid #ccc;"
| style="width:58%; color:#000;" |
[[File:Visualization-of-algorithms-vs-artificial-intelligence-vs-machine-learning-vs-deep.png|thumb|300px|<center>'''[https://www.researchgate.net/profile/Johannes-Vrana/publication/339997962_The_NDE_40_Key_Challenges_Use_Cases_and_Adaption/links/5e71e0f692851c93e0aa4f35/The-NDE-40-Key-Challenges-Use-Cases-and-Adaption.pdf Algorithms vs. artificial intelligence vs. machine learning vs. deep learning (Author: Johannes Vrana, CC BY-ND 4.0)</center>]]]

==Compiled by==
* Dean Giustini, UBC Biomed librarian, [https://directory.library.ubc.ca/people/view/120 dean.giustini@ubc.ca]

==Updated==
* 7 September 2026 | Part of '''[[Knowledge Synthesis (KS) & AI Search Wiki 2026]]''' & '''[https://wiki.ubc.ca/User:DeanGiustini A to Z Listing]'''

==See also==
*'''[[Artificial intelligence (AI) glossary of terms]]''' | '''[[AI-powered searching - a definition]]'''
* '''Note''': '''[https://openai.com OpenAI]''' (and thus '''[https://chatgpt.com/ ChatGPT]''') leads the general AI space, but AI companies such as '''[https://claude.ai/new Claude by Anthropic]''' and '''[https://gemini.google.com/ Google Gemini]''' are developing '''[[deep research]]''' tools and others such as '''[[Elicit.com]]''' and '''[[Perplexity]]''' are experimenting with AI-powered academic searching in support of research.
* '''[[Which companies are behind AI search tools?]]'''
* ''How will AI tools affect our traditional bibliographic databases? Will we see GenAI being put into our search platforms? Can we stop it?''
* '''[[Algorithms]]''' | '''[[Automated indexing]]''' | '''[[Which companies are behind AI search tools?]]'''
* '''[[Large language models (LLMs)]]''' | [[Machine learning]] | [[Neural networks]] | [[Prompt engineering]] | [[Reasoning models]]
* ''AI-powered search tools:'' '''[[Asta (agentic AI by Ai2)]], [[Consensus]], '''[[Elicit.com]], [[Open Evidence]], [[otto-SR]], [[Perplexity]], [[PubMed.ai]], [[Undermind.ai]]'''
* '''Also''': This open textbook (or wiki channel) is intended to help librarians and other information professionals learn about AI. It is not, in itself, meant to be seen '''''as promotion of AI''.''' If anything, the goal is harms mitigation or ''harms reduction''.

==Introduction==
'''Vector-based searching''' (semantic searching or embedding-based searching) is an information retrieval method that finds and ranks results according to the semantic similarity between a search query and documents, rather than relying primarily on exact keyword or free text matches. It represents queries and documents as numerical vectors, or embeddings, and retrieves results by measuring the similarity between those vectors, typically using metrics such as cosine similarity.

'''''Traditional keyword searching''''' relies primarily on lexical matching, where results are retrieved when the words or phrases in a query match terms in a document or bibliographic record. Controlled-vocabulary searching, however, is different: it uses standardized subject terms or concepts assigned to records, allowing a search to retrieve relevant records even when the exact words used in the query do not appear in the record. Vector-based searching takes a different approach by using '''[[machine learning]]''' models to represent the semantic meaning of queries and documents.

'''In vector-based systems,''' documents and queries are converted into numerical embeddings and positioned within a high-dimensional vector space. A high-dimensional vector space represents content using many numerical dimensions or features. An embedding is therefore a list of numbers that encodes aspects of the content's meaning. Documents and queries with similar meanings tend to be positioned closer together in this space, allowing systems to retrieve conceptually related content even when it uses different words, synonyms, or paraphrases.

'''In short:''' keyword searching emphasizes matching words, controlled-vocabulary searching emphasizes matching standardized concepts, and vector-based searching emphasizes matching semantic representations of meaning.

== How vector-based searching works ==
'''Vector-based searching typically involves four stages:'''

=== 1) Embedding ===
Content such as documents, sentences, images, or audio is converted into numerical representations called '''vector embeddings''' using a machine learning model. Queries are embedded using the same model.

=== 2) Indexing ===
Embeddings are stored in a specialized index or '''vector database'''. To enable fast retrieval at scale, systems commonly use '''Approximate Nearest Neighbor''' (ANN) algorithms.

=== 3) Querying ===
When a user submits a query, it's "transformed" into a vector embedding.

=== 4) Similarity matching ===
The system calculates similarity between the query vector and stored vectors using distance metrics such as:
* Cosine similarity https://en.wikipedia.org/wiki/Cosine_similarity
* Euclidean distance https://en.wikipedia.org/wiki/Euclidean_distance
* Dot product https://en.wikipedia.org/wiki/Dot_product

Results are ranked by closeness in the vector space.

== Comparison with keyword search ==
{| class="wikitable" style="width:100%; text-align:center; border:2px solid #ccc; border-collapse:collapse;"
|-
! style="background-color:#A2D5F2;" | ''Feature''
! style="background-color:#FFC9DE;" | Keyword Search
! style="background-color:#B8E0D2;" | Vector-Based Search
|-
| style="background-color:#FFF4C2;" | Matching method
| Exact word "lexical" matching
| Semantic similarity
|-
| style="background-color:#FFE4E1;" | Synonym handling
| Limited
| Strong
|-
| style="background-color:#FFFACD;" | Sensitivity to phrasing
| High
| Low
|-
| style="background-color:#E0F7FA;" | Context awareness
| Minimal
| High
|-
| style="background-color:#FDEBD0;" | Multilingual capability
| Limited
| Often supported
|}

== Models used ==
'''''Vector-based searching''''' typically relies on '''encoder models''', which generate embeddings rather than text. Common examples include:
* '''[[Bidirectional Encoder Representations from Transformers (BERT)]]'''
* Sentence-BERT https://sbert.net/
* E5 (EmbeEddings from bidirEctional Encoder rEpresentations) https://www.pinecone.io/learn/the-practitioners-guide-to-e5/
* MiniLM e.g., https://www.educative.io/answers/what-is-all-minilm-l6-v2-model

These models differ from large language models (LLMs), which are designed primarily for text generation rather than semantic encoding.

== Applications ==
Vector-based searching is widely used in the following search and information retrieval systems:
* Web search engines;
* Academic and biomedical databases;
* [[Retrieval augmented generation (RAG)]] systems;
* Recommendation systems;
* Chatbots and question-answering systems;
* Image and multimodal search

== Hybrid search (ensemble) approaches ==
Many modern search systems implement hybrid search techniques, combining vector-based (semantic) search with traditional keyword or Boolean search approaches. This ensemble method leverages the strengths of both: vector-based retrieval improves semantic recall by capturing meaning and synonymy, while keyword-based methods provide lexical precision, exact matching, and reliable filtering.

By integrating approaches, hybrid search systems can return results that are both contextually relevant and textually exact, improving overall retrieval quality in complex information environments such as academic databases, search platforms, and modern AI-powered search tools.

== Advantages ==
* Improved recall for semantically related content;
* Robust handling of synonyms and paraphrases;
* Better support for natural-language queries;
* Cross-lingual and multimodal capabilities.

== Limitations ==
* Higher computational costs;
* Reduced transparency compared to keyword matching, "opaque" and black box effects;
* Potential semantic false positives;
* Dependence on model quality and training data.

==Librarian perspectives==
''Librarians tend to view vector-based searching as a pragmatic and generally positive development'', but with some important cautions related to search transparency and reproducibility. An important, emerging perspective is that hybrid search is an improvement over single-method retrieval, because it combines:
* Vector/semantic search, which helps surface relevant materials even when terminology differs (useful for interdisciplinary research, clinical synonyms, or evolving language)
* Keyword/Boolean search, which preserves precision, explicit control, and repeatability

From a health and academic librarianship standpoint (e.g., biomedical databases such as PubMed/MEDLINE), our systems already combine controlled vocabularies (e.g., MeSH terms) with keyword searching. We see modern hybrid systems as an extension of longstanding retrieval principles rather than a completely new idea. Further, vector-based searching and embeddings are viewed as a useful but imperfect augmentation of traditional retrieval systems. The consensus is that it works best when it is transparent, well-documented, and paired with explicit search strategies, especially in research contexts like '''[[systematic reviews]]''' where rigour and reproducibility matter.

==References==
'''Note''': I have read widely on this topic, and will be populating this section with an extensive bibliography to support the entry. This is a complex topic so thank you for your patience while I write this entry for librarians and information professionals. Some content was informed by the Wikipedia entry: https://en.wikipedia.org/wiki/Vector_database and https://learn.microsoft.com/en-us/azure/cosmos-db/vector-database and '''What is Vector search"?''' https://learn.microsoft.com/en-us/training/modules/improve-search-results-vector-search/2-vector-search

* [https://esj.eastasouth-institute.com/index.php/esiscs/article/view/938 Chowdhury T. Semantic Search with Vector Database: A Comprehensive Review of Models, Indexing and Applications. The Eastasouth Journal of Information System and Computer Science. 2026 Mar 12;3(03):323-36.]
:* ''The use of semantic search with the help of vector databases has become an impressive paradigm of retrieving the pertinent information by offering the contextual and conceptual sense of the information searching more than using the conventional methods of keyword searching. This paper provides an in-depth overview of the models of vector representation, transformer-based semantic encoders, and technologies of vectors database that jointly allow efficient and error-free semantic search.''
* [https://arxiv.org/abs/2602.17856 Ghanadian H, Kamali A, Tekieh MH. Enhancing Scientific Literature Chatbots with Retrieval-Augmented Generation: A Performance Evaluation of Vector and Graph-Based Systems. arXiv preprint arXiv:2602.17856. 2026 Feb 19.]
:* ''This paper investigates the enhancement of scientific literature chatbots through retrieval-augmented generation (RAG), with a focus on evaluating vector- and graph-based retrieval systems. The proposed chatbot leverages both structured (graph) and unstructured (vector) databases to access scientific articles and gray literature, enabling efficient triage of sources according to research objectives. To systematically assess performance, we examine two use-case scenarios: retrieval from a single uploaded document and retrieval from a large-scale corpus. Benchmark test sets were generated using a GPT model, with selected outputs annotated for evaluation. The comparative analysis emphasizes retrieval accuracy and response relevance, providing insight into the strengths and limitations of each approach. The findings demonstrate the potential of hybrid RAG systems to improve accessibility to scientific knowledge and to support evidence-based decision making.''
* [https://arxiv.org/pdf/2402.01763 Jing Z, Su Y, Han Y. When large language models meet vector databases: A survey. In: 2025 Conference on Artificial Intelligence x Multimedia (AIxMM) 2025 Feb 3 (pp. 7-13). IEEE.]
* Rusum GP, Anasuri S. Vector Databases in Modern Applications: Real-Time Search, Recommendations, and Retrieval-Augmented Generation (RAG). International Journal of AI, BigData, Computational and Management Studies. 2024 Dec 30;5(4):124-36.
* Salsabilla N, Wiharja K. Implementation of Semantic Search Based on Vector Database for Personal Documents. In2025 International Conference on Advancement in Data Science, E-learning and Information System (ICADEIS) 2025 Feb 3 (pp. 1-6). IEEE.
* [https://dl.acm.org/doi/abs/10.1145/3662010.3663448 Sanca V, Ailamaki A. Efficient Data Access Paths for Mixed Vector-Relational Search. In: Proceedings of the 20th International Workshop on Data Management on New Hardware 2024 Jun 10 (pp. 1-9).]

==Disclaimer==
* '''Note''': Please use your critical reading skills while reading entries. No warranties, implied or actual, are granted for any health or medical search or AI information obtained while using these pages. Check with your librarian for more contextual, accurate information.