User talk:Evad37/Script modules

Possible extension: Version control

Script module importation could be extended to include version control. This would necessitate each script module having an associated json page mapping version numbers to old revision ids. The importScriptModule function, if given a version number as a second parameter, could first import the script module's json page, and add the oldid= parameter with the appropriate value to the url for loading the script. There should be a way of specifying partial version numbers, and/or greater-than/less-than a version number, similar to how npm does it. The exportScriptModule function would also need to know the version number, and store the exported function/object at Window.exportScriptModule[name][version]. Perhaps the json pages could be maintained by bot. - Evad37 [talk] 03:02, 26 January 2019 (UTC)

Start collecting

Regardless of if this is implemented or not, it would be useful to have a collection of such "script modules" (if implemented, they form the basis of the "namespace," and if not, they could still serve its purpose). Accordingly, if anyone has a script that they want to add, please post it here (I'll think of a better system soon) so that we can keep track of them. Thanks, --DannyS712 (talk) 04:44, 26 January 2019 (UTC)

I've got a bunch of helper functions in XFDcloser and my other scripts that are general enough to be reused (some of them I've straight copy/pasted between my scripts). Very simple functions are probably better off in libraries like User:Evad37/Module/extra.js, while bigger/more-complicated ones I would make into their own module, e.g. User:Evad37/Module/multiButtonConfirm.js - Evad37 [talk] 00:29, 28 January 2019 (UTC)

Comments

I think this is a good idea, but have a couple of points:

• With the existing syntax, I make out that not just functions, even classes can be exported? If so, that's awesome.
• Perhaps you could extend the syntax to match that of mw.loader.using, allowing multiple imports to be declared in an array, so that $.when need not be used?
• Is it necessary to restrict the import of scripts to the ones stored in the mediawiki namespace? While it does improve security, I don't that's a very big consideration since till now there hasn't been a single attack that vandalised the mediawiki space (right?). Most people are not interface admins. It would help people to maintain their script-modules if they aren't stored in mediawiki space.

SD0001 (talk) 05:41, 26 January 2019 (UTC)

• Yes, anything that javascript can return can be exported (except undefined, but there wouldn't be much point in exporting that)
• Yeah, that should be easy enough to do. Will take a look at coding it tomorrow.
• I think there have been cases of malicious code being inserted into the sitewide common.js on other wikis by comprised admin accounts, which is part of the reason interface admins were introduced. Requiring an edit request and a quick check by an int-admin doesn't seem too much of a burdern, especially since this check is explicitly not a full code review. I expect most proposed edits will probably just be rubber-stamped, hence the comment about them not being much better than the average userscript. The other advantage of requiring int-admins to make the edits is that the syntax (single call to Window.exportScriptModule, all module code within the IIFE) is more likely to be enforced / less likely to be inadvertently broken.

- Evad37 [talk] 06:36, 26 January 2019 (UTC)

[allow] multiple imports to be declared in an array –  Done - Evad37 [talk] 07:53, 27 January 2019 (UTC)

Pseudo-namespace

I find it interesting that you want to create a pseudo-namespace for these script modules, when the review process for them are the same as other global scripts in technical terms. It seems that the purpose here is to prevent users from loading the rest of the MediaWiki namespace into their user scripts. What use cases are you expecting where we want to block a normal user from loading a sitewide script into their own user script? Deryck C. 19:56, 26 January 2019 (UTC)

@Deryck Chan: fyi blocked users can't edit their own common.js, etc --DannyS712 (talk) 20:19, 26 January 2019 (UTC)

...therefore the issue of blocked users is irrelevant to my question about whether a pseudo-namespace is necessary. Deryck C. 20:22, 26 January 2019 (UTC)

@Deryck Chan: Sorry, I misunderstood your question. I read expecting where we want to block a normal user from as "expecting where blocked users would be..." or something like that. But, what do you mean by prevent users from loading the rest of the MediaWiki namespace into their user scripts? I'm confused by what you are referring to... --DannyS712 (talk) 20:29, 26 January 2019 (UTC)

@DannyS712: In Evad37's description, the reason for having a pseudo-namespace is so that interface admins can constrain what scripts can be loaded into user scripts and what scripts can't. Why do we want to stop users from loading MediaWiki namespace scripts in general, given that all of them have been reviewed by IAdmins? Deryck C. 20:32, 26 January 2019 (UTC)

@Deryck Chan: I didn't know that we stop that... maybe because the MW namespace in general is for fully fledged scripts/gadgets/skins, etc, while this would be for reusable components? --DannyS712 (talk) 20:39, 26 January 2019 (UTC)

@Deryck Chan: the review process for them are the same as other global scripts in technical terms – the review process is meant to be a very low bar to cross, basically just "not evil", as opposed to gadgets which have more criteria and require a proposal to gather consensus.
It seems that the purpose here is to prevent users from loading the rest of the MediaWiki namespace into their user scripts – The purpose is to only allow script modules to be loaded using Window.importScriptModule. Other types of MediaWiki scripts can be loaded using conventional means, such as how XFDcloser has a importScript('MediaWiki:Gadget-morebits.js'); line, and later on uses new Morebits.simpleWindow(...etc... .
What use cases are you expecting where we want to block a normal user from loading a sitewide script into their own user script? – When using Window.importScriptModule is going to return a rejected promise, because you tried to load a random script that doesn't use the Window.exportScriptModule('name', IIFE); syntax
Why do we want to stop users from loading MediaWiki namespace scripts in general, given that all of them have been reviewed by IAdmins? – We don't, but only the special script modules using the correct syntax can be imported using Window.importScriptModule. Other scripts have to be imported by other means.
One other benefit of using a pseudo-namespace is making them easy to find using Special:PrefixIndex. And it is similar to gadgets being prefixed by "Gadget-". Plus like Danny just said, this is just for components to be used within other scripts. - Evad37 [talk] 23:15, 26 January 2019 (UTC)

@Evad37 and Deryck Chan: to get a better idea of what this would include, I suggest we combine some examples of snippets that we already reuse individually. See my post above. --DannyS712 (talk) 23:38, 26 January 2019 (UTC)

Agree. Better to make a copy of User:Evad37/extra.js (or to go one step further, MediaWiki:Gadget-morebits.js) that uses the exprtScriptModule syntax, for the sake of demonstration. SD0001 (talk) 04:39, 27 January 2019 (UTC)

@SD0001: Yeah, but I don't know how to convert it... --DannyS712 (talk) 05:02, 27 January 2019 (UTC)

Here are some sample script modules:

• User:Evad37/Module/parseAllTemplates.js
• User:Evad37/Module/extra.js
• User:Evad37/Module/multiButtonConfirm.js

Converting a regular user script to a script module isn't too hard:

1. Add the line Window.exportScriptModule('moduleName', (function() { to the top
2. Remove any additions to the Window object, and add a return statement at the bottom with whatever should be exported
3. Add the line })()); to the bottom
4. Clean up comments/documentation

- Evad37 [talk] 00:22, 28 January 2019 (UTC)

Problem being solved?

What's the problem being solved here? It seems to add a decent amount of complexity...for what gain exactly?

Creating a shared library seems like a decent idea, but I don't see much advantage to creating small individual modules compared to a single grabbag of stuff. I would also suggest looking at what Commons has (see c:Special:Gadgets, the lib* ones), and whether we can leverage that.

HTH, Legoktm (talk) 01:06, 28 January 2019 (UTC)

@Legoktm: The idea came out of a discussion on my talk page User_talk:Evad37#Template_parser_script. One of the main things, really, is an easy way to use other people's code in a user script, with a promise to ensure it is loaded before it is used, and without reloading something that another script has already loaded. Currently this requires code like

var fooIsLoaded = Window.foo !== undefined;
$.when(
    fooIsLoaded ||
    $.getScript('https://en.wikipedia.org/w/index.php?title=User:Example/foo.js&action=raw&ctype=text/javascript')
).then(function() {
    // code that uses Window.foo
})

whereas I would like to simply write

importScriptModule('foo').then(//... foo is ready to be used, and was not reloaded if already loaded

The second thing would be to have a bit of an increase in security, by storing reusable code in MediaWiki: namespace – but with a lightweight ("not evil") approval process, so its still relatively easy for users to publish and update such code.

The advantage of individual modules would be you only load what you need, e.g. not every script needs to parse wikitext. Having modules act as libraries for particular tasks or otherwise-related bits of code would make sense, rather than lots of small modules needing lots of network requests to load. (The demo modules are intentionally tiny as they are for proof-of-concept, rather than representing the size of a real module). - Evad37 [talk] 02:56, 28 January 2019 (UTC)

@Legoktm: The proposal is basically to create an equivalent of mw.loader.using for stuff other than ResourceLoader modules. I wonder why mw.loader.using doesn't work for external scripts, while mw.loader.load does? That aside, the Commons libraries seem to be great, especially c:MediaWiki:Gadget-libWikiDOM.js. Since they are gadgets, do they get loaded on all pages? Otherwise, looking at a search result, they seem to be unused. SD0001 (talk) 09:10, 28 January 2019 (UTC)

No, those gadgets are not enabled by default, and due to a permissions trick, won't show up in Special:Preferences for users to enable. But they are normal ResourceLoader modules, so any gadget or user script can depend upon them with mw.loader.using. That's what I'd recommend doing instead of inventing a parallel system, especially since it's already planned to use the MediaWiki namespace. Legoktm (talk) 08:12, 29 January 2019 (UTC)

@Legoktm: How can a "user script ... depend upon them with mw.loader.using"? It seems to always return a rejected promise with an Unknown dependency error (when used with anything other than the modules listed at mw:RL/CM). - Evad37 [talk] 09:00, 29 January 2019 (UTC)

What module did you try to load? For example, if I wanted to ensure navpopups was loaded, I can do: mw.loader.using('ext.gadget.Navigation_popups').done(() => console.log('success!'));, and see "success!" printed. You can use mw.loader.getState() to see what the current state is, 'ready' for loaded, 'registered' for not loaded, and null if the module doesn't exist. Legoktm (talk) 17:03, 29 January 2019 (UTC)

Well, I didn't realise that a ext.gadget. prefix was needed – that should really be documented on mw:RL/CM, and possibly elsewhere. But in any case, I was trying to load MediaWiki:Gadget-morebits.js – still doesn't work, probably because it doesn't have it's own line on MediaWiki:Gadgets-definition (loading twinkle using mw.loader.using('ext.gadget.Twinkle') works as expected).

Assuming int-admins are willing to add script modules/libraries as off-by-default, hidden gadgets without too much fuss, then that will take care of the bulk of this proposal. Something like mw.loader.using but for any userscript could still be useful – basically a wrapper for $.getScript, but taking a wiki page name like (importScript), or an array of page names (like mw.loader.using), and keeping track of what's already been loaded (by other scripts) to prevent unnecessary requests being sent. - Evad37 [talk] 00:55, 1 February 2019 (UTC)

Yep, morebits would need its own gadgets-definition line. What you're asking for is phab:T27962, which is waiting for someone to work on it. Legoktm (talk) 18:04, 1 February 2019 (UTC)

That bug doesn't look too hard, I'll have a go at making a patch. - Evad37 [talk] 00:58, 2 February 2019 (UTC)

https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/487566/ - Evad37 [talk] 06:13, 2 February 2019 (UTC)

You're awesome. Legoktm (talk) 05:38, 6 February 2019 (UTC)

Example modules

These aren't in the right format, but some scripts of mine that would be useful as "script modules" IMO:

• User:DannyS712 test/page.js get and set the contents of a wikitext page (interact with it as a string)
• User:DannyS712 test/JSON.js get and set the contents of a JSON page (interact with it as a JSON object)

--DannyS712 (talk) 07:17, 23 February 2019 (UTC)

Not sure if you know this, but there is an existing mw.Api ResourceLoader module, which also makes editing easy with its postWithToken method, and to set the User-Agent header once (when creating an mw.Api object), rather than for each request. What I've started doing in XFDcloser is making some helper functions to make it easy to extract the wanted data from the returned JSON:

var arrayFromResponsePages = function(response) {
	return $.map(response.query.pages, function(page) { return page; });
};

var pageFromResponse = function(response) {
	return arrayFromResponsePages(response)[0];
};

var makeErrorMsg = function(code, jqxhr) {
	var details = '';
	if ( code === 'http' && jqxhr.textStatus === 'error' ) {
		details = 'HTTP error ' + jqxhr.xhr.status;
	} else if ( code === 'http' ) {
		details = 'HTTP error: ' + jqxhr.textStatus;
	} else if ( code === 'ok-but-empty' ) {
		details = 'Error: Got an empty response from the server';
	} else {
		details = 'API error: ' + code;
	}
	return details;
};

I think a module/library of function like this, perhaps with some that make the appropriate input object from just the page name, would be a good idea. That would simplify code to something like:

// import `apiLib` from script module/library
var api = new mw.Api( { ajax: { headers: { 'Api-User-Agent': /* your info here */ }	} } );
// ...
api.get( apiLib.query("Page title") )
.then(apiLib.pageWikitextFromResponse)
.then(function(pageWikitext) { /* ... do stuff ... */ });

As for the script module formatting, rather than re-invent the wheel it's probably better to just use the exisitng method of attaching stuff to the window object. Plus loading scripts with a callback (promise) is about to get much easier, once the patch I made for phab:T27962 gets deployed (likely next week). - Evad37 [talk] 00:33, 1 March 2019 (UTC)