User:Alexis Jazz/Factotum/Mumbo jumbo

History

Being disappointed with DiscussionTools which Enterprisey suggested as the successor to reply-link and DiscussionTools having broken reply-link, I had a problem. I fixed the immediate issue by forking reply-link and un-breaking it. But as great as Reply-link is, it also had many issues and I could think of dozens of useful features for it.

Initially I thought I'd take Reply-link and improve it. After looking at the source code for a while, it became apparent that wasn't such a good idea. Forking DiscussionTools was never on the table: it's an extension and as a user you can't sideload those. Besides, DiscussionTools has.. issues. Serious issues. It's flawed by design. (they are building a shadow ledger, I'll write more about this if there's interest)

So development started around 6 December 2021. The initial idea was simple: a major part of Reply-link was signature detection. Signature detection is a PITA, but DiscussionTools has signature detection. So, take advantage of that, make a simple form, done.

Now just about everything from that assumption turned out to be completely wrong..

The first test edit appears to have been [1] on 13 December 2021. Initially Factotum was developed purely for discussions: commenting, creating new sections and many related features. But some of those features proved to be very practical: custom inserts that allow you to create your own buttons to insert some particular text you often use, regular expressions, and.. it was fast. As Factotum doesn't require navigating away from the current page, the form can open rather quickly. This made it so that I wanted to use Factotum for regular edits as well, so full page editing was added. Fun fact: I use Factotum to update Factotum. Section editing was a matter of time at that point, and despite being unaware of the existence of User:BrandonXLF/QuickEdit at the time, I gave Factotum practically the same functionality as QuickEdit. Which isn't a bad thing.

During development it became more and more clear that I was fighting with the metadata DiscussionTools provides. (remember, the signature detection PITA) It's.. all wrong. A nightmare to work with, it overcomplicated my code and caused bugs. What seemed like an attractive shortcut turned out to be nothing but a millstone. And another truth surfaced: DiscussionTools is wasteful. Very, very wasteful and depending on it was irresponsible. So it was time to take the plunge and I gave Factotum its own signature detection functionality so Factotum could stand on its own.

Nearly 4 months have passed and many bugs and features later, Factotum became a Swiss Army knife, taking notes from Reply-link (not least requested features that were never implemented), AutoWikiBrowser, Convenient Discussions and Flow.

Known limitations and known unknowns

There's not much that's too serious here beyond the exotic timestamp thing. Please report it if the timestamps on your favorite wiki aren't being detected.

Timestamp support is tracked at /Timestamp support.

• There are some limitations for multiline templates/tags. This is a wikitext limitation, not specific to Factotum. In most cases Factotum will apply a workaround. If truenewline is present on your wiki a better workaround will be applied.
• For comments that span multiple lines in wikitext only the last line (with the signature) can be edited.
  • When placing a comment with Factotum the entire comment will exist on a single line in wikitext with a few exceptions like new sections created through an inputbox. The latter was done to aid people who answer edit requests and similar cases where someone is expected to edit a comment someone else posted.
  • It will be converted back to multiline when editing it with Factotum.
  • You only notice this when trying to edit a comment you didn't place with Factotum (or a new section created through an inputbox) or when you try to edit a comment you placed with Factotum using another editor.
• Sectionless transclusion: if a discussion has no section header and it's transcluded and links are added using the "legacy" strategy (plain wikitext signature), Factotum will not work. If a comment was placed using Factotum (with locator), it'll work. A BCL link (with the right parameters set) will work. An InputBox will work. This kind of limitation exists for all scripts, gadgets and extensions, but here it's documented, unlike here. There are a few ways to solve this, but this should be done server-side. DiscussionTools attempted to get around this limitation at a massive cost which also hits pages without such comments and they still didn't succeed completely, so that wasn't one the ways to solve it.
• Only one reply form can be opened simultaneously. (per window/tab) This is by design as opening multiple reply forms at once can easily lead to confusion and the other reply buttons are repurposed as mention buttons if you already have a form open. During development this also became something I counted on, so this limitation can't be lifted easily.
• Non-WMF wikis: largely untested. Should be okay in theory. Some dependencies on ParserFunctions.
• Editing as anon: reply form opens and anoneditwarning loads correctly, dry runs appear fine. (testers?)
• Factotum assumes the presence of localStorage for various things. Having localStorage frequently expire (like every session) isn't expected to be an issue, but it's untested what happens if your browser doesn't support it at all. (which would mean you disabled it on purpose) Would possibly result in a JS error.
• Factotum uses Echo emulation to show notifications about new replies. As it's emulation, it isn't perfect. Most notably, you need to visit the wiki where you subscribed to something to get a notification. There are two reasons for this: I don't it consider it acceptable to make a dozen API requests while you're browsing Wikipedia (even if that action is throttled to once every 10 minutes) but also because subscriptions are stored locally, so while you browse English Wikipedia Factotum can't access your subscriptions on Wiktionary. Technically subscriptions could be made global, but you'd hit the 64K limit for preferences even sooner. You know how Bill Gates famously never said that "640K ought to be enough for anybody"? 640K actually WOULD be enough, but we only got a tenth of that. Once you have the notification it'll show on any wiki. After pressing the existing "Mark all as read" link or following a link from a notification, the notification in question (all of them in case of "Mark all as read") from Factotum will be gone. Not marked as read, just gone. You'd run out of storage space real quick if I didn't as the Echo emulation also uses a preference which is limited to 64K. To my knowledge Factotum is the only userscript that can use the notification area at all. If phab:T306211 gets implemented this will all get better, but it's beyond my control.
• When posting a comment near the end of a minute (like 14:59:59) using parse-in-place preview (default), the displayed timestamp may differ from the posted timestamp. This is just cosmetic. It would be difficult to fix. Factotum can't generate wiki-style timestamps. It knows one when it sees one, it can extract the time from a wiki-style timestamp but it has no clue how to generate one from scratch. This being a total edge case with no real ramifications, there's no reason to put any effort into this. For wikis using hh:mm format it could technically be feasible to make it so that this could only happen at 23:59, but again, who cares.
• Parse-in-place appears to be failing in some instances where it's actually expected to work. (currently observed when editing comments) You may see the full page content (instead of just the comment) temporarily turning purple, this means Factotum didn't select the correct HTML element. The fault (which could have multiple causes) is currently detected and Factotum falls back to reparsing the entire page which is a bit slower, but at least nothing breaks.

Underscores in internal links

Factotum rewrites internal links, replacing underscores with spaces. 99% of the time this is correct and an improvement as it's the result of users copy/pasting page names (with or without anchors) from their address bar. Very rarely however it is intended, like Jéské Couriano's signature which uses the ASCII art/smiley v^_^v to link their talk page. If one wanted to link an article by its stylized name when the stylized name includes an underscore (like Watch_Dogs) it would also be preferred to keep the underscore. It's recommended to replace the _ character with its HTML code: _ as this makes it easy to differentiate between unintended and intended underscores in internal links.

Not my fault!

• On Minerva desktop there are problems with echo emulation. (showing subscriptions in the general notification area) MediaWiki bug. phab:T306737. Don't read that task. You'll get severely disillusioned with developer attitude.
• Can't distinguish between a null edit and an edit conflict when not editing a full page. MediaWiki bug. phab:T299809. Completely safe to read as in 4 months no WMF developer has commented on it.

Compatibility with other scripts

• Loading Factotum before Convenient Discussions completely breaks Convenient Discussions. Loading Factotum after Convenient Discussions appears to damage Factotum's ability to detect signatures in some cases.
• BrandonXLF's QuickEdit fails to load when collapsible sections are enabled in Factotum. QuickEdit also causes broken things when using the mobile site with collapsible sections, without Factotum. QuickEdit just clashes with collapsible sections in general. A fix (see fork) has been submitted to BrandonXLF. Note: Factotum has an option for full section editing (opt-in) which is virtually identical to QuickEdit.
• Any script that changes up signature timestamps and is loaded before Factotum will cause Factotum's legacy signature detection to fail. This includes WP:LOCO. This cannot be fixed. Either load your script well after Factotum has loaded or (if all you wanted was to see signature timestamps in local time) use the functionality Factotum provides for that. For WP:LOCO specifically Factotum includes a workaround.
• At a very specific width, when the #firstHeading title is exactly as long as the viewport allows without becoming two lines, the full page edit/new section icons can jump "under" the #firstHeading element, potentially overlapping other elements. I am fucking done with this. I can't fix it. Well I could, but the only way I managed was with a <table> to force everything into place. Unfortunately tables are considered a sin and it would remove the title text from the CSS :before which would probably have unforeseen consequences. Flex, inline-flex, justify-items/self/content, float, text-align, margin:auto, NOTHING WORKS.

Templates Factotum can use

Template	Used for	Title aquired from	Fallback for wikis with ParserFunctions without a Wikidata sitelink
Wikipedia:AutoWikiBrowser/Typos (not really a template but the page title is obtained the same way)	Guess..	d:Q6585066	"Project:RETF" if it exists, title configurable in settings
Wikipedia:Citemap.json (not really a template but the page title is obtained the same way)	Guides the conversion of Web2Cit data into the right citation template with correct parameter names.	d:Q112131235	None yet, will be "Project:Citemap.json" if it exists
{{Truenewline}}	Enables multiline SyntaxHighlight and <pre> for indented comments	d:Q111996190	Template named "truenewline" if it exists
{{Smiley}}	Used for smileys when BBCode is enabled	d:Q6021960	Template named "smiley" if it exists
{{Atops}}	Used when "Wrap section in closed/archived block" is checked to substitute the local variant of {{Atop}}.	d:Q112108086	Template named "atops" if it exists, otherwise HTML fallback
{{Abots}}	Used when "Wrap section in closed/archived block" is checked to substitute the local variant of {{Atop}}	d:Q112108089	Template named "abots" if it exists, otherwise HTML fallback
{{Tq}}	Used to encapsulate selected text that gets included as a quote with a mention when that feature is enabled.	d:Q112199474 d:Q11394737	Template named "tq" if it exists, otherwise HTML fallback

Atops and abots are used because Atop/Abot parameters are not standardized across projects.

Helper script to generate Citemaps: User:Alexis Jazz/Factotum/MakeCitemap.js. You'll need to add the postfix for your language/project for template documentation pages.

LocalStorage use

The following can be stored in your localStorage:

• FTTBasicLang (JSON): cached translation of essential elements that was imported from MediaWiki. Only languages you've actually used. Also includes the timezone and copyright warning of the wiki. Is refreshed when Factotum gets an update.
• FTTLang: (JSON): cached translation of other elements that was imported from User:Alexis Jazz/Factotum-extra.js. Only Dutch and German and only if you've used either of those. Is refreshed when Factotum gets an update.
• FTT (JSON): your settings (optional, settings can also be stored in your preferences)
• FTTDrafts (JSON): unsent comments. Individual unsent comments expire after 7 days.
• ENOM: cached edit notices (shared with WP:ENOM)
• FTTSubs (JSON): contains topic subscription data. Also used for the opt-in feature of automatically collapsing sections that contain no new comments since your last visit.
• FTTThanks (JSON): keeps track of what comments you've thanked so their heart changes color. Tracks up to 500 thanks, removes the oldest entries if you exceed that.

FTTDrafts get deleted when you save your preferences and have message recovery disabled. There's a button to erase all localStorage items.

Translations

Translations for most of the basic interface elements are imported from MediaWiki. This doesn't include Factotum-specific messages which are mostly found in the settings form.

To translate Factotum, see User:Alexis Jazz/Factotum-extra.js. Just translate de qqq object. To load an existing partial or outdated translation, use the script provided there. Also provide a translation for "editing comment" (not capitalized) and ""Removed comment" (capitalized). These two can occur in edit summaries and are stored elsewhere as they follow the project language instead of the user interface language. Technical details: FTT.wikiMsgsObj.mul.editCommentSummary ("[[LINK|editing comment]]USER") and FTT.wikiMsgsObj.mul.rmCommentSummary ("Removed commentUSER").

Locator

Some details on what I call the locator. It's.. a span tag. With an ID. That encapsulates the signature. Here's an example:

<span id="AJ:1648348630791:User_talkFTTCLNAJ-test" class="FTTCmt">[[User:AJ]] 02:37, 27 March 2022 (UTC)</span>

You might be curious why this thing exists and why Factotum uses it by default. Legacy signatures (as I call them, the famous four tildes) are.. a bit of a nightmare:

• They are not guaranteed to be unique as their precision is limited to minutes, and one could easily post multiple replies in one minute. Some people even post multiple responses in the same edit.
• They are not properly machine-readable. The current timestamp on Portuguese Wikipedia is "03h06min de 27 de março de 2022 (UTC)". On Dutch Wiktionary it's "27 mrt 2022 05:09 (CEST)", on Farsi Wikipedia it's "‏۲۷ مارس ۲۰۲۲، ساعت ۲۰:۲۸ (UTC)" and on English Wikipedia it's "03:09, 27 March 2022 (UTC)". On Thai Wikibooks it's "10:10, 27 มีนาคม 2565 (+07)". Yeah, they live in the future. There truly is no standard here. This is one way to deal with it..
• There's usually policy that a signature must contain a link to the user page or user talk page. Again, not exactly machine readable. Is it user or user talk? Or Benutzer? Or Special:Contributions? (which is what anons get anyway, but a few users also use it) Nightmare. NIGHTMARE. NIGHTMARE.
• As if that wasn't bad enough, some projects (like ptwiki) have wgExtraGenderNamespaces.
• No way to detect if the signature is transcluded from another page. If it is, the origin is unknown.
• Even just finding signatures on a page is difficult. They have no CSS class or whatever.

Factotum (and all of its competitors that work with the wikitext PageContentModel) has workarounds for all these issues. Every developer who faced this has been on the verge of being admitted to the loony bin. And these are just that: workarounds. A nightmare to create, a mess to maintain, never perfect and difficult to get good performance out of.

And this is where the locator comes in:

• Millisecond precision guarantees uniqueness as one user can't make multiple edits within the same millisecond.
• Milliseconds since Unix epoch. No more timezones, no daylight saving, no commas, no upside-down crow's feet instead of numbers, no month names, no month abbreviations, no religious-calendar-based year, no fucking bullshit!!
• Username is username. Is username. Very username.
• Page title included. Sectionless transclusion is no longer an issue.
• Has a CSS class!!
• Total backwards compatibility for existing tools and bots as it encapsulates a legacy signature.

As for the small amount of text the locator adds to wikitext: that could actually be a negative amount. The backwards compatibility means the locator is additive. It wouldn't have to be: it's technically trivial (Factotum supported it at one point, it was removed to simplify testing since nobody is going to adopt it anytime soon) for the locator to be templated. So the locator could be {{BwlCmt|AJ:1648348630791:User_talkFTTCLNAJ-test}} with a possible second parameter for custom signatures. The entire signature with local date etc. could be generated by the template out of that.

In an ideal world, the four tildes could substitute that, and if multiple signatures are posted in a single edit with source editing the MediaWiki software should ensure the timestamps differ by 1ms so they'd still be unique. My fear is that the WMF developers are going to steamroll over all of this and implement something similar but extremely flawed all by themselves: "Some features may involve introducing new wikitext. Although, any changes to wikitext will be limited to those that enable new features that benefit contributors. Features like replying to specific comments or watchlisting particular discussions.". It's like the sword of Damocles.

Random stuff

Notes that don't fit elsewhere.

DT being strange

What the hell happened at Wikipedia:Articlesfordeletion/WillSmithassaultofChrisRockattheOscars (diff ~1080099955)? No, that's not Factotum, that's DiscussionTools, but always good to analyze to avoid falling into the same trap. But what the hell? It inserted an empty newline, which should NEVER happen to begin with. (judging from the HTML, screen readers are getting royally screwed here) This is followed by a colon-asterisk indentation (thanks to the lack of verbosity of DiscussionTools there's no telling who Super Goku V was replying to, something that was fixed in Factotum on 10 December 2021, 4 days after it's inception, no joke), it ends with a signature (as expected) but then what follows? A second comment with colon-colon-asterisk indentation (different from the first comment!), in the same edit, with, again, a signature? What.. how?

Follow-up: Super Goku V tried to post two replies at once. This would inevitably lead to issues, but DT isn't handling it very well with the change in indentation.

How to trigger an error

Open settings and press "bug?" in the bottom right corner, next to the init time/version. Can be useful to generate the detailed status report if you encounter an issue that doesn't trigger the error reporter.

Modules

Factotum has some support for modules/addons: little pieces of code to add specific functionality. These can, for example, be added to your common.js. Load order doesn't matter as long as they are available when you need them. Here are a few examples that should help to understand how to create a useful module. See User:Alexis Jazz/Factotum/Modules for more.

Add "Hello" button which triggers an mw.notify with "hello world"

if ( typeof window.FTTModules == 'undefined' ) { window.FTTModules=[]; }
window.FTTModules.push({'load':['afterOpenForm'],'buttonLabel':'Hello','buttonFunc':function(){mw.notify('hello world');}});

Add "shout" button which CAPITALIZES selected text

if ( typeof window.FTTModules == 'undefined' ) { window.FTTModules=[]; }
window.FTTModules.push({'load':['afterOpenForm'],'buttonLabel':'shout','selectionFunc':function(a){return a.toUpperCase();}});

Replace the encapsulation for quotes added from selected text

if ( typeof window.FTTModules == 'undefined' ) { window.FTTModules=[]; }
window.FTTModules.push({'load':['beforeOpenForm'],'beforeOpenFormFunc':function(){FTT.basicmsgs.tq='';FTT.wikiMsgs.quoteOpen='{{tq2|1=';FTT.wikiMsgs.quoteClose='}}';}});

Insert username when starting a new section on a user talk page

if ( typeof window.FTTModules == 'undefined' ) { window.FTTModules=[]; }
window.FTTModules.push({'load':['afterOpenForm'],'afterOpenFormFunc':function(){
	if ( FTT.PRMOpened.type == 'newsection' && ! FTT.PRMOpened.preload && mw.config.get('wgNamespaceNumber') == 3 && !mw.config.get('wgPageName').match(/\//) ) {
		FTT.UITextInput.setValue(mw.config.get('wgRelevantUserName')+', ')
	}
}});

Replace "GROUPS" in the text with your user groups (autoconfirmed etc)

if ( typeof window.FTTModules == 'undefined' ) { window.FTTModules=[]; }
window.FTTModules.push({'load':['processComment'],'processCommentFunc':function(text,int){text = (text||'');FTT.MD.MyGroups = '';for (var int=0;int<mw.config.get('wgUserGroups').length;int++){if ( mw.config.get('wgUserGroups')[int] != '*' ){FTT.MD.MyGroups = FTT.MD.MyGroups + mw.config.get('wgUserGroups')[int] + ', ';}}return text.replace(/GROUPS/g,FTT.MD.MyGroups.replace(/, $/,''));}});

Stuff you'd think I would have borrowed, but didn't

Icons

• 
• 
• 
• 
• 
• 

Icons used in the Factotum UI are all original. You'd think it would be easier to take some existing freely licensed icons right? Well, you'd be right. Why draw a link icon when File:Mismatch finder link icon SVG.svg is a perfectly nice icon? Because of the license. That SVG is CC BY-SA 4.0 and I feel I can't provide proper attribution for that. I'm not sure it would legally be sufficient to credit the author in the source code or on a page like this one. The end user would never see it. And shockingly, I wasn't able to find much in the way of suitable public domain icons. Besides, I thought I could do it in fewer bytes. I also didn't use the icons that are provided by OOUI, for several reasons: adding the OOUI icons to the load requirements substantially bumped up the loading time in some cases for reasons I couldn't explain. They would also prevent screenshots of Factotum from being public domain. The edit marker icon is vaguely inspired by previous revisions of File:OOjs UI icon edit-ltr.svg.

WYSIWYG editor

VisualLight is a very basic editor with no relation to VisualEditor or any other editor for that matter. [2] suggested WYSIWYG stuff could easily be inserted with mw:OOUI, but sadly that's not the case.

Signature detection

You'd think I would have ripped that from DiscussionTools, signaturedetector.js, reply-link or ConvenientDiscussions.. but I didn't.

Subscriptions

You can't do subscriptions client-side, that's madness. It's just a wrapper for DiscussionTools, right? Wrong! But it is madness.

JS minifier

A minified version hasn't been deployed yet, but UglifyJS couldn't mangle my keys, and my keys are very long and descriptive. So I made AJSJSMangler. Not currently recommended for general use, though.