Wikipedia talk:Administrator intervention against vandalism/Homoglyph detection heuristic

In connection with a discussion on the Talk page about § Homoglyph vandalism, the topic of defining a heuristic to decide what and how to flag suspicious cases arose. This page contains a response from Chat GPT to the question of defining such a heuristic.

Heuristic for homoglyph detection

Character set deviation

Flag any word or text string that contains a mixture of characters from multiple Unicode scripts (e.g., Latin, Greek, Cyrillic) when the majority of the word is composed of characters from a single script.

Example: If a word is mostly Latin script but contains a single Cyrillic letter, flag it (e.g., Tеst where the e is Cyrillic).

Script context consistency

Flag any text where the detected Unicode script for a word or phrase does not align with the context of the surrounding text. This includes:

An isolated use of a non-Latin script character within an otherwise Latin script sentence.

A switch to another script that does not match the dominant script of the article's language.

Unusual glyph substitutions in known words

Maintain a dictionary of high-frequency words in the target language (e.g., English for English Wikipedia) and flag occurrences where homoglyph substitutions occur in these words.

Example: If the word "The" appears as "Тhe" (Cyrillic Т), it should be flagged because "Тhe" is not a valid word in English.

Suspicious single character words

Flag single characters that resemble homoglyphs of commonly used characters in the article's language.

Example: A single Cyrillic Т appearing in isolation in an English article should be flagged.

Location-based checks

Prioritize flagging homoglyphs in sensitive or prominent locations within articles, such as:

• Titles or headings.
• Proper nouns (names, places, etc.).
• Internal links (e.g., URLs or anchor text).

Edit-based analysis

Compare the character scripts in an article before and after an edit. Flag edits where a previously single-script section now contains mixed scripts or where script changes involve known homoglyphs.

Word-length anomalies

Flag cases where a homoglyph substitution results in a word or phrase that matches neither the standard orthography of the target language nor a plausible foreign-language insertion.

Example: If "hello" is changed to "hеllo" (Cyrillic е), flag it because "hеllo" is not valid in any relevant language context.

Frequency of script appearance

Flag articles where non-dominant scripts (for the article’s language) appear unusually frequently, especially in short bursts or isolated instances.

Program considerations

Whitelist valid multiscript words

Maintain a whitelist of known proper nouns, terms, or transliterations that combine characters from multiple scripts legitimately (e.g., names like "São Тomé" or brands with mixed scripts).

Human review emphasis

All flagged cases should be queued for manual review by editors to mitigate false positives while ensuring suspicious cases are not missed.

Precision-recall balance

This heuristic emphasizes high recall (catching most instances of homoglyph vandalism), accepting a manageable level of false positives, as these can be filtered out during human review.

Tuning – Adjust the strictness of flagging based on the frequency and types of false positives encountered during testing.

By implementing this heuristic, the program could effectively flag potential homoglyph vandalism while minimizing the risk of overlooking genuine content.