Wikipedia talk:XTools/Archive 1

Archive 1

Page history

Hi @Hedonil, Cyberpower678, TParis, X!, and Technical 13::

I'm having ongoing problems with page history (this). Usually, nothing happens: the "results" are just a blank white page. Sometimes (rarely, usually after reloading several times) it appears, but then it's not actually what I want.

Is the old version still around somewhere? All I really want is the list of most frequent contributors (which this buries in a bunch of other stuff), and especially to be able to get the list of the most frequent contributors for a particular timespan, like the last 6 months, or during a particular year (which this doesn't appear to do at all, but the old one did). Whatamidoing (WMF) (talk) 02:11, 4 June 2015 (UTC)

If you would like to try and use the only on the GitHUb repo has a copy of it in its history. It's quite a resource hogger though.—^cyberpower_Chat:Offline 03:51, 4 June 2015 (UTC)

More maintainers

A watchlist notification called for people with PHP or Linux experience, so I'm responding here. I have around 10 years of PHP experience (large and small projects, backend processing, embedding, etc.) and about 15 years of Linux experience (servers, desktops, embedded systems, networking, databases, etc.), so I might be able to help. Please let me know what would be the actual duties and pending tasks, and we'll discuss. — Dsimic (talk | contribs) 08:48, 3 July 2015 (UTC)

As above, five years of PHP experience to an experienced level on minor and major projects. Willing to help out. Alex|The|Whovian 14:40, 3 July 2015 (UTC)

Hello @Dsimic and AlexTheWhovian: We are looking to completely rewrite the xtools suite and have it running on a dedicated instance, apart from tool labs. You have the current source code to go off of, but it relies on an old version of PHP, and is structually unsound making it difficult to debug. We are looking to adapt a MVC paradigm approach. The exact road map and method of execution is unclear, but as a starting point, we need to onboard some dedicated developers who are eager to be a part of something big, that will be a service to nearly all Wikimedia Foundation wikis. If you are interested, please email us (xtools at lists.wikimedia.org), outlining your relevant experience. Please use an informative subject line "your username - PHP developer / sysadmin" or just PHP developer, etc. Thank you! — MusikAnimal ^talk 15:29, 3 July 2015 (UTC)

Ugh, just have a look at public_html/articleinfo/index.php – that's a true mess of mixed HTML and PHP. For this kind of a web application I might suggest using something like a simple templating engine (Smarty would be very neat) instead of taking the full MVC approach; something like that might be much more flexible in a long run. — Dsimic (talk | contribs) 20:02, 3 July 2015 (UTC)

Excellent, we'd love to hear more ideas like this. Step one is figuring out our new approach. So you think a framework is over doing it? Anyway I thought the same about the current codebase... as a Rails developer seeing the backend and frontend all in the same file is no bueno, and seems like most syntax highlighters wouldn't be too fond if it either — MusikAnimal ^talk 20:43, 3 July 2015 (UTC)

Mixing PHP (backend) and HTML (frontend) is almost always an extremely bad thing even for very small codebases. Just had a more detailed look at public_html/articleinfo/index.php, and Smarty templates seem to have been already used, see the line 93 for example... Though, having a look at modules/WebTool.php reveals that Smarty is actually no longer used, and has been replaced with some custom quasi-templating code that made it possible to keep existing Smarty-style code, see the line 1043 for example. It would be really useful to know why that happened back at the time, as there might be some unforeseen aspects of the project preventing efficient use of a templating engine. — Dsimic (talk | contribs) 21:06, 3 July 2015 (UTC)

I'm afraid those questions may go unanswered... The primary developer of the code you see has long left the project and is unresponsive. This is the reason why we've had so much trouble maintaining the tools and ensuring stability. We're unfamiliar with the code and it's clearly difficult to debug. We eventually decided a complete rewrite is the most favourable. Much of the code was written by a mix of authors, patched over time and time again, which might explain the mix of technologies used that you are seeing. — MusikAnimal ^talk 21:22, 3 July 2015 (UTC)

What's dealt with seems to be an example of undead code. :) However, I'd still find that not using an MVC framework and resorting to a templating engine such as Smarty might be better. The only few things a framework might help with would be OAuth logins, associated session management, and some input data validation. However, implementing OAuth logins isn't that complicated, while there doesn't seem to be much of the input validation. — Dsimic (talk | contribs) 21:40, 3 July 2015 (UTC)

(lurking) You mean X!? He just recently edited.
 — Berean Hunter (talk) 21:45, 3 July 2015 (UTC)

No, User:Hedonil. As I understand it he rewrote much of what X!, TParis and others wrote, but I'm not certain. — MusikAnimal ^talk 22:16, 3 July 2015 (UTC)

I have been programming since the days of the ZX81 and would like to have a look at xtools' code just out of personal interest, but I cannot realistically commit any time for doing anything on it - I am still "on call" for other PHP / Python / JavaScript projects on t'interweb and having another one to support is the last thing I need. So I can't do much more than a moral support. Ritchie333 ^{(talk) (cont)} 09:16, 5 July 2015 (UTC)

Label for article namespace in edit counter is in Korean

In the edit counter, the label for article namespace in the edit counter is in Korean. Seems to repro in other languages too (I tried Finnish). Regards, Orange Suede Sofa (talk) 20:14, 4 July 2015 (UTC)

Yeah, I've also noticed that. — Dsimic (talk | contribs) 21:30, 4 July 2015 (UTC)

I mentioned this last month at either the help desk or the VP (sorry I can't remember which one). Nobody there new why this happened or whether there was any rhyme or reason for it. I took a wild guess that it might be a holdover from an April Fools Day prank. The headings at the "wmflabs.org/supercount" do not use this Korean so whatever the reason that it is there now it would be nice if it could be changed back to English. MarnetteD|Talk 19:53, 5 July 2015 (UTC)

It has something to do with the internalization service we are using. E.g. it's not written in the xTools code directly, in fact that text used to be in German before it decided to be Korean. Hopefully we'll be able to iron it out with our rewrite of the code. Sorry! — MusikAnimal ^talk 21:05, 5 July 2015 (UTC)

Thanks for taking the time to explain this MusikAnimal. It is appreciated. MarnetteD|Talk 21:13, 5 July 2015 (UTC)

Alternative tools

Hi, going by the theory of the more the merrier [and conveniently ignoring too many cooks spoil the broth], I was just wondering, just like we have the Supercount tools as an alternative to X!s edit counter, can we have APPER's German tool ported as a long term alternative tool? --Rsrikanth05 (talk) 15:14, 6 July 2015 (UTC)

This was being discussed at User talk:Cyberpower678/Archive 25#Page histories. I'm not sure where we are on this — MusikAnimal ^talk 16:37, 6 July 2015 (UTC)

I'm aware of that, but from the nature of the discussion, it seems like it is being looked at as more of an ad-hoc solution. --Rsrikanth05 (talk) 19:22, 6 July 2015 (UTC)

Looking for source code on GitHub

/data/project/intuition/src/Intuition/ToolStart.php

Please link to the "Intuition" code. Thanks, Wbm1058 (talk) 19:11, 6 July 2015 (UTC)

Found it: https://tools.wmflabs.org/intuition/

https://github.com/Krinkle/intuition — Wbm1058 (talk) 20:20, 6 July 2015 (UTC)

Debugging help requested

I'm trying to run articleinfo (Page history) under Windows 7.

I'm getting Fatal error: Class 'Redis' not found in C:\php\WebTool.php on line 948 see code here

Where is Class 'Redis' defined?

And what's the deal with this code at the bottom of that code file? Why would Redis be "not available"?

/**
 * dummy class, if redis is not availabe
 * @author Hedonil
 */
class RedisFake{
   function get(){
      return false;
   }
   function set(){
      return false;
   }
   function setex(){
      return false;
   }
   function expire(){
      return false;
   }
   function close(){
      return false;
   }
}

I see:

• Redis, a data structure server
• http://redis.io/
• https://github.com/antirez/redis
  • an in-memory database that persists on disk
  • a data structures server. What this means is that Redis provides access to mutable data structures via a set of commands, which are sent using a server-client model with TCP sockets and a simple protocol. So different processes can query and modify the same data structures in a shared way.
  • PHP extension for interfacing with Redis

So do I need this for the application to work, or does it just enable more efficient operation? Wbm1058 (talk) 18:10, 7 July 2015 (UTC)

Not needed, I believe. Cyberpower678 said it could simply be turned off... but the function that does this is written incorrectly, apparently. Redis is a key-value store data server that shouldn't be too hard to set up if you're up for it, but I've never tried on Windows. Not sure how this PHP extension works either, so maybe I'm too optimistic. You could also consider creating an Ubuntu virtual machine with VirtualBox. As I understand the file system is essentially an NFS, so you could still use your IDE but would just run the app in the VM rather than on your local. — MusikAnimal ^talk 18:25, 7 July 2015 (UTC)

Thanks to that RedisFake, it can be. By redirecting the Redis class to RedisFake and fixing, the code can be moved to a location without Redis. My thanks to Wbm1058 for discovering this.—^cyberpower_{Chat:Limited Access} 18:29, 7 July 2015 (UTC)

Just comment out the Redis code that attempts to connect, and we're fine. Just goto $redis = new RedisFake(); straight away. That's what it does when the connection attempt fails.

Now I hit my next roadblock. Notice: Undefined index: QUERY_STRING in C:\php\WebTool.php on line 712

Deprecated: Function split() is deprecated in C:\php\WebTool.php on line 712 line 712

Per Deprecated features in PHP 5.3.x, split() is deprecated in version 5.3 – use preg_split() instead -- hopefully that's easy enough to avoid the need to back down to running PHP 5.2 – Wbm1058 (talk) 20:01, 7 July 2015 (UTC)

$argv = split('&', $_SERVER['QUERY_STRING']); → $argv = preg_split('/&/', $_SERVER['QUERY_STRING']); fixes the deprecated function. Wbm1058 (talk) 23:33, 7 July 2015 (UTC)

As a side note, split() still works despite the fact that PHP says it's a deprecated function. Also, have a look at explode(), which will work faster as in this particular case there's no need for using regular expressions. — Dsimic (talk | contribs) 23:59, 7 July 2015 (UTC)

But QUERY_STRING is still undefined. What's going on in this checkSpider function?

function checkSpider()

function checkSpider(){ global $redis, $perflog; $lang = isset($_REQUEST['lang']) ? $_REQUEST['lang'] : null; $wiki = isset($_REQUEST['wiki']) ? $_REQUEST['wiki'] : null; $wikifam = isset($_REQUEST['wikifam']) ? $_REQUEST['wikifam'] : null; $project = isset($_REQUEST['project']) ? $_REQUEST['project'] : null; $uagent = isset($_SERVER["HTTP_USER_AGENT"]) ? $_SERVER["HTTP_USER_AGENT"] : null; $reqUri = isset($_SERVER["REQUEST_URI"]) ? $_SERVER["REQUEST_URI"] : null; $article = isset($_REQUEST['article']) ? $_REQUEST['article'] : null; $page = isset($_REQUEST['page']) ? $_REQUEST['page'] : $article; $user = isset($_REQUEST['user']) ? $_REQUEST['user'] : null; $argv = preg_split('/&/', $_SERVER['QUERY_STRING']); $reqStr = 'xtspidercheck003' . $reqUri . $uagent; $reqMin = (int)$redis->incr( $reqStr ); if ( $reqMin > 10 || in_array( $wikifam, array(".wikibooks.org" ) ) || in_array( $article, array("Digital_journalism", "Talk:Monte_Vista_High_School_(Danville,_California)","Template:WPRe" ) ) || in_array( $user, array( "112.198.79.177", "2.91.35.107", "Carpodacus","Ambroix") ) || in_array( $page, array( "Category:AppleScript_Programming","AppleScript_Programming/Mail_alert") ) || in_array( $uagent, array("Java/1.6.0_34","Java/1.7.0_51", "-","Python-urllib/2.7", "Magic Browser2", "Magic Browsers","Magic Browser") ) || count($argv) > 15 || ( $uagent == "Mozilla/5.02" && $lang == "th" ) ){ $redis->expire( $reqStr , 1800 ); header("HTTP/1.1 403 Forbidden"); echo "Forbidden: checkSpider. (timeout 10 min) .Please inform the tool maintainer if this isn't correct."; file_put_contents('/data/project/xtools/spider.log', gethostname()."\t$reqUri\t$uagent\n", FILE_APPEND ); $this->__destruct(); } else{ $redis->expire( $reqStr , 60 ); } #file_put_contents('/data/project/xtools/uagent.log', gethostname()."\t$lang\t$uagent\n", FILE_APPEND ); /** * It's a game that sucks, dude ! */ #$perflog->add('checkSpider', 0, $argv ); } line 720 ... Digital journalism? Talk:Monte Vista High School (Danville, California)? Template:WPRe? Is this for debugging, or what?

If this is just related to Redis, can we just make this function a no-op? Wbm1058 (talk) 23:55, 7 July 2015 (UTC)

The whole thing with parsing the query string is totally unneeded as the standard $_GET array can be used instead. — Dsimic (talk | contribs) 00:05, 8 July 2015 (UTC)

OK, thanks. I just commented out that block of code for now; will come back to it later if necessary.

Cp678, I just got the "Class 'NumberFormatter' not found" error and resolved it by enabling extension=php_intl.dll in my php.ini file. There were also about eight ic*.dll files I had to copy into my php directory.

Now I'm getting a boatload of warnings related to mysqli. I assume I have to give it a login id and password somewhere? I'm running this from inside Windows PowerShell. Wbm1058 (talk) 20:22, 8 July 2015 (UTC)

Or should I run this by linking to a file on my C:\ drive from within my Chrome browser? If so, how do I set up such a file? Wbm1058 (talk) 20:37, 8 July 2015 (UTC)

Is your app configured to connect to the labs database? — MusikAnimal ^talk 22:33, 8 July 2015 (UTC)

TO connect to the database you need to create an SSH double tunnel to the labs environment and straight to the database host. Then you need to use your replica.my.cnf file issued to you when you gained access to labs. Only then can you let it access the DB.—^cyberpower_Chat:Online 23:01, 8 July 2015 (UTC)

I see. Is wikitech:Help:access where I go to get started with that? Wbm1058 (talk) 21:10, 9 July 2015 (UTC)

Yes and Help:Getting Started — MusikAnimal ^talk 21:27, 9 July 2015 (UTC)

My memory has been jogged. I see that I had created a Wikitech account back in December 2013 to ask a question about X!'s count. Now I remember that. I see that Secure Shell explains the acronym SSH. Wbm1058 (talk) 13:15, 13 July 2015 (UTC)

Getting a little hung up on the next step, to generate an SSH key. I see that all the instructions seem to assume that I'm running Linux. Hmm:

• how to generate public key from windows command prompt
• Microsoft bringing SSH to Windows and PowerShell (June 2, 2015)
• Looking Forward: Microsoft Support for Secure Shell (SSH) (June 3, 2015)

  "The team is in the early planning phase"

So is PuTTY (http://www.putty.org/) my best option? Wbm1058 (talk) 14:36, 13 July 2015 (UTC)

• How to generate an SSH key on Windows using PuTTY?

Hmm, from that StackOverflow it looks like you will need extra software to generate a key. I have no idea how secure shell would work on Windows... I'm guessing even with Putty or whatever you still can't SSH in from the command line, which begs the question of how your PHP app is going to use secure shell, since it won't know about Putty (or maybe it will?). Perhaps that MySQL extension provides the necessary functionality to connect via SSH, or maybe you'll need another Windows-specific PHP extension. If you're able to get this up and running on Windows that's fantastic, but I personally would recommend installing VirtualBox with a Ubuntu distribution. *nix systems are very developer-friendly and you should be able to get things set up quickly. I don't mean to deter you from your preferred environment, though! — MusikAnimal ^talk 14:52, 13 July 2015 (UTC)

I just downloaded the simple puttygen.exe Windows app (PuTTY key generator) and used it to generate private and public keys, and pasted the public key to HERE. Now I need to figure out what to do with it. I assume I can use PuTTY as my "command prompt window" as evidently PowerShell doesn't support this yet. – Wbm1058 (talk) 16:47, 13 July 2015 (UTC)

So, I have access to "Labs", but I still need to obtain access to "Tool Labs". I assume that PuTTY will get me into "Bastion SSH access", but I haven't tried that yet. I'll wait for my Tools Access Request to be approved. Wbm1058 (talk) 17:20, 13 July 2015 (UTC)

Yep, you uploaded the keys to the right place. Wait for your tools access to be granted, then try to ssh in with ssh username@tools-login.wmflabs.org. You may get some notice about unknown hosts, just hit whatever you have to in order to proceed. Then you can see your Labs MySQL credentials with cat ~/replica.my.cnf. The next part is figuring out how to get your PHP app to connect to the database through the SSH tunnel... maybe with the connection open through Putty and your credentials defined in the app it will just work, I'm not sure. E.g. if the PHP app is looking for a local MySQL database and 127.0.0.1:3306 (default port) is routed to the labs db through the SSH tunnel, you might be all set — MusikAnimal ^talk 17:31, 13 July 2015 (UTC)

Edit Counter feature request

Under the "Month Counts" section of the edit counter, each month appears with the number of edits made in that month and a bar showing a visual representation of those edits by namespace — except for months where no edits were made by the user. For example I made no edits in 2013-10 so that month does not appear, and the chronology jumps from 2013-09 to 2013-11. Would it be possible to show all months, with a zero for those months where no edits were made? -=# Amos E Wolfe ^talk #=- 21:16, 28 July 2015 (UTC)

Hello! Interestingly, I actually prefer the "no zero-edit months" layout, and there must be more people with different tastes. Thus, it would be the best to have a checkbox that selects between the two behaviors. — Dsimic (talk | contribs) 21:31, 28 July 2015 (UTC)

AmosWolfe this alternate edit counter "http://tools.wmflabs.org/supercount/index.php?user=AmosWolfe&project=en.wikipedia.org&toplimit=10" includes the months with zero edits. You need to click on the "Monthly Stats" tab near the top to get to them. I put your username in the string so feel free to copy/paste (without the quote marks) it to where it can be useful for you. MarnetteD|Talk 14:36, 8 September 2015 (UTC)

WikiHistory live

Cyberpower678 has set up WikiHistory, available (probably temporary location) at toollabs:xtools/wikihistory. More to come soon, hopefully. Cheers - L235 (t / c / ping in reply) 01:14, 22 July 2015 (UTC)

Great news, Cyberpower678! Is the authorship data piece of this functional? See this comparison:

• German Note the Text Share column and the pie chart showing each editors' share of the text, as calculated by APPER's Author determination algorithm
• English "Attention: The data for authorship is loading. Depending on the data and the size of the article, this may take a while. The page does not need to be reloaded. The data will automatically appear."

I tried this page several hours ago, and it still says the data is loading. I figured that this would be the trickiest part of this application to port and get up and running. Wbm1058 (talk) 13:15, 22 July 2015 (UTC)

Apparently I have not yet. My mistake. I'll work on it.—^cyberpower_Chat:Online 13:18, 22 July 2015 (UTC)

Great, looking forward to seeing it working hopefully soon. Wbm1058 (talk) 13:22, 22 July 2015 (UTC)

@Cyberpower678, Wbm1058, L235: Awesome! Can't wait! :-) --Atlasowa (talk) 13:24, 8 September 2015 (UTC)

I'm getting close to getting it set up. I'm still having issues getting the text share part of it working.—^cyberpower_Chat:Online 15:18, 8 September 2015 (UTC)

Cyberpower, it's wonderful to be able to click on page information --> revision history statistics, and be taken to WikiHistory. Thank you for setting this up. Sarah ^(talk) 16:26, 8 September 2015 (UTC)

xTools down

getting 404ed when going to toollabs:xtools or any "articles created" links... PS. Now it says "301 Moved Permanently This tool has moved to a new location. You will be redirected to tools.wmflabs.org/xtools-articleinfo/index.php? shortly." and then autorediredting to toollabs:xtools-articleinfo. --.js (((☎))) 01:45, 22 January 2016 (UTC)

Meaning of deleted edits

Some users are worried about the meaning of "Deleted edits" in the edit counter and ask about it on help pages. See for example Wikipedia:Help desk#Deletions since I began reviewing new articles (permanent link). I know it's the standard term but it would alleviate some concerns if the text linked to a brief explanation that deleted edits include all edits to pages which were later deleted for any reason. PrimeHunter (talk) 10:46, 26 April 2016 (UTC)

xTools edit counter error

Under the semi-automated edits heading of the edit counter for my user, it says that I have performed zero edits using HotCat which is definately wrong. Is it a bug? NikolaiHo^☎️ 04:12, 12 December 2016 (UTC)