Postings list

The postings list is a data structure commonly used in information retrieval (IR) systems to store indexing information about a corpus. It is central to the design and efficiency of search engines and database management systems that need to retrieve information rapidly.

At the bare minimum, a postings list is associated with a term from a document and records the places where that term appears. Each term found in documents within a corpus is mapped to a corresponding postings list containing information such as the documents the term appears in and often the positions within those documents.^[1]

Structure

A postings list consists of posting elements, sometimes referred to as postings. Each posting typically contains:

A document identifier (DocID), which uniquely identifies a document in the corpus.
Frequency information (Term Frequency), indicating how often the term appears within the document.
Position information, indicating where in the text the term appears.
Additional metadata may include fields such as document titles, headings, or other relevant document-specific information.

The exact structure of a postings list can vary based on its application, with some using linked lists, arrays, or more complex data structures like skip lists to optimize for different types of searches.

During a search query, the IR system retrieves postings lists for each term in the query to determine which documents contain the terms and how relevant those documents could be based on the frequency and positions of the terms.

Variants

Some variants of postings lists include:

Inverted index: A form of postings list that points from terms to documents.
Impact-ordered postings: Lists where postings are ordered by the weight or "impact" of the term in the document.
Positional postings lists: Enhanced postings lists that include position information for phrase queries and proximity searches.

References

^ Büttcher, Stefan; Clarke, Charles L. A.; Cormack, Gordon V. (2016). Information retrieval: implementing and evaluating search engines (First MIT Press paperback ed.). Cambridge, Massachusetts London, England: The MIT Press. ISBN 978-0-262-52887-0.

[1] Büttcher, Stefan; Clarke, Charles L. A.; Cormack, Gordon V. (2016). Information retrieval: implementing and evaluating search engines (First MIT Press paperback ed.). Cambridge, Massachusetts London, England: The MIT Press. ISBN 978-0-262-52887-0.

[1]