User talk:BilledMammal/ARBPIA activity statistics

Including article talk SPAs

@BilledMammal: ECP requires 500 edits from outside the CTOP, your data set will by definition exclude any SPA with less than 1000 edits. It also excludes SPAs who continue to make non-ECP edits. With that in mind, would you consider also showing editors who have more than half of their article talk page replies in the CTOP? RAN1 (talk) 16:40, 27 August 2024 (UTC)

I’ll try to get that for you BilledMammal (talk) 11:13, 28 August 2024 (UTC)

Please clarify

(a) Does "edits" mean only article space, or talk space as well?
(b) Does "since 2022" mean "since 1/1/2022" or "since 1/1/2023"?
Zero^talk 09:16, 28 August 2024 (UTC)

• Talk space as well
• Since 1/1/2022

BilledMammal (talk) 09:39, 28 August 2024 (UTC)

Thanks. There is a big difference between article edits and talk page participation, so this makes the information less useful. Also, it would be useful to know what the total number of ARBPIA edits by all editors was. Zero^talk 09:58, 28 August 2024 (UTC)

Which would you find more informative? Personally, I don’t see why it would be useful to split them? BilledMammal (talk) 10:00, 28 August 2024 (UTC)

It's interesting when you split odd and even namespaces though e.g. since 2022-01-01, in total, not just the topic area...

• FourPi made 93.76% of their revisions in even namespaces, and 6.24% in odd, whereas...
• Selfstudier made 38.18% of their revisions in even namespaces, and 61.82% in odd.

So FourPi stays quiet and edits whereas as Selfstudier discusses. This is one of the many interesting properties of accounts.

Sean.hoyland (talk) 10:32, 28 August 2024 (UTC)

That doesn't quite work - it misclassifies noticeboards and AFD's - but it is interesting.

However, I'm not sure how informative it is, without evidence indicating that there are circumstances in which an editor would be expected to talk. BilledMammal (talk) 10:36, 28 August 2024 (UTC)

Yes, it gets a bit fiddly if you want visibility at namespace resolution or even higher resolution. I guess the extent to which revisions need to be put into different classes depends on how the information will be used and the specific objective e.g. whether an ARBECR revert is the same as a talk page comment for a given objective. Sean.hoyland (talk) 11:03, 28 August 2024 (UTC)

Some people spend most of their effort in discussion and some people edit mostly in article space. Treating them both the same is obviously misleading. There are also other reasons why the raw counts have to taken with a grain of salt. For example, some people make their edits in little pieces, some make them all at once. Zero^talk 13:13, 28 August 2024 (UTC)

Some people spend most of their effort in discussion and some people edit mostly in article space.

For the purpose of assessing whether an editor is functionally a SPA, what is the difference?

For example, some people make their edits in little pieces, some make them all at once.

That’s why the percentage is provided; it is reasonable to expect that editors will behave the same way inside and outside the topic area. BilledMammal (talk) 21:38, 28 August 2024 (UTC)

BilledMammal, more than a quarter of all your edits are made using automated tools, and another uncalculated chunk appear to be fast clean up edits. That means your stats on this table will by definition look very different from other editors, even though automated edits take almost no time at all, versus the time required for a detailed talk page comment on a contentious topic. The percentage of time you spend on ARBPIA edits will thus be much greater than what these stats will suggest, as currently shown. Onceinawhile (talk) 22:29, 28 August 2024 (UTC)

That’s only counting main space, while this extends to talk space - and even excluding all of those, I still wouldn’t meet the requirements of this list.

In addition, most of those are page moves, done as a result of closing RM’s. They aren’t really equivalent to an AWB edit. BilledMammal (talk) 22:36, 28 August 2024 (UTC)

BM, I need to ask you again for the total of all ARBPIA edits during the period covered by your counts. Zero^talk 01:25, 29 August 2024 (UTC)

I’m in the process of getting that for you - I realised the number I had readily available missed edits by editors who had made less than 500 edits total in the period. BilledMammal (talk) 01:37, 29 August 2024 (UTC)

397,857 edits since 01/01/2022. BilledMammal (talk) 08:41, 29 August 2024 (UTC)

So by your count, the table with your analysis is missing 80% of all relevant edits. Seems to confirm that the vast majority of edits to the area are made by smaller accounts – albeit subject to seeing the full list of accounts without the 50% filter.

I suspect the 80% would be even higher if this was limited to mainspace edits only. Onceinawhile (talk) 09:12, 29 August 2024 (UTC)

Made by non-SPA editors, but yes?

I think a lot of editors are misunderstanding the point of this table. The point is to attempt to answer Black Kite's question about SPA editors. This does that. BilledMammal (talk) 11:23, 29 August 2024 (UTC)

Sorry, that number is actually wrong - I was looking at the wrong sheet. The real one is 431,132. BilledMammal (talk) 13:44, 29 August 2024 (UTC)

I just realised this number would exclude bots. If you want to include bots, Sean.hoylands figure is more accurate. BilledMammal (talk) 13:14, 3 September 2024 (UTC)

My numbers are almost always wrong in some way that I don't discover usually until slightly after I post them. As a sanity check, right now, I get 473212 post-2022 revisions excluding bots, 509911 including bots. I don't really know whether I've excluded all the bots because I'm not sure how to do that with 100% precision. Anyway, I'm doing this at the moment. Let me know if there is a better way. I was surprised to discover when I hit an error that the mb4 can be necessary is rare circumstances if the query is hitting enough rows and stumbles across something that needs it.

and convert(ar.actor_name using utf8mb4) not like '%bot'
and convert(ar.actor_name using utf8mb4) not like '%bot III'
and convert(ar.actor_name using utf8mb4) not like '%(bot)%'

Sean.hoyland (talk) 14:19, 3 September 2024 (UTC)

<- FWIW BilledMammmal, and feel free to ignore this because I probably would, but if I were you and doing this, I would ask myself some questions like

• How can I avoid subjectivity and magic numbers as far as possible?
• Is there an agreed definition of 'SPA'? No, so maybe I should ignore that and just present raw counts without any personal classification schemes.
• Is the single-account model useful from a statistical perspective? It's useful as a way to look at honest editors who comply with SOCK, and the properties of individual sock accounts, but a single-account view misses so much activity because many people do not fit the single-account model. They spread their topic area revisions across multiple accounts and make a significant number of edits, often more than honest single-accounts people. It is a good strategy for the dishonest advocate.
• Why have I picked specific magic numbers e.g. 2022-01-01, SPA >50%, namespaces (0, 1) as 'requirements for the list'. Does this capture what I want to capture?
• Does 50% make sense when there is an order of magnitude difference between 50% of 2000 and 50% of 20000, and people who use multiple accounts do not often cross the 50% threshold (actually I have stats for that somewhere for the topic area for all the checkuser blocks).
• I (you) have made ~2893 edits within the 'topic area' in total, ~2724 since 2022-01-01. It's a lot. Much more than many other accounts on the list. Why am I not on the list? I must be missing something with my magic number choices.
• Do the results address the question asked with respect to 'Sub-5000-edit accounts' (another magic number) active in the topic area e.g. why haven't the results picked up newish accounts active in the topic involved in recent edit wars etc.? There are many making a lot of edits.

I could go on, that's just some random thoughts, and I get how annoying it can be when people do this. My attitude to 'notes' is often - no one's stopping you from doing it yourself. So, no need to reply. Sean.hoyland (talk) 05:15, 29 August 2024 (UTC)

Regarding #2, there isn’t - but there is agreement on who is not an SPA. It makes sense to exclude such editors, to ensure the evidence is concise and relevant.

Regarding #3, if you can provide me with a list of masters and sock puppets in the topic area, I can merge their figures

Regarding #4:

1. I picked 2022 because I had to cut it off somewhere. Two and a half years ago seems long enough to be viable
2. I picked 50%, because editors who spend less than 50% of their time in a topic area are obviously not in the scope of Black Kites question. Exactly where the line could be above that is debatable, but editors can debate that themselves
3. I picked namespace 1 and 0 because this analysis can’t be applied to other namespaces, with the possible exception of template.

Regarding #5, yes. The purpose is to identify editors who may be SPA’s on this topic area; while the number of edits can determine their overall impact, they don’t change whether the editor is an SPA

Regarding #6, remember that the purpose is to determine SPA’s. I’m not one of those, being far more involved with other topics such as notability and page moves than I am with ARBPIA. It makes sense that I’m not on the list.

I’ll note that in other cases I’ve been careful to pick my definitions so that I am included, but in this case that wasn’t a viable option - I was simply too far away from the lines.

Regarding #7, I believe it does. The question is regarding SPA’s; it’s possible that those new accounts, for all the disruption they are causing, are not SPA’s. I’ve got a different tool in mind to identify edit warring, but that’s still on the drawing board.

With the exception of #3, I’ve considered all of these already. BilledMammal (talk) 05:38, 29 August 2024 (UTC)

If it is the case that "there is agreement on who is not an SPA" (I don't know whether there is ever agreement on anything), let me ask you this.

• For a given time period t, would you consider sock-A with 1000 revisions in PIA out of 2000 to be different from sock-B with 1000 revisions in PIA out of 4000?
• If so, what is the difference between them? The impact on the topic area is the same, 1000 revisions each.
• What is the notion of SPA-ness telling us about these accounts, other than something about the difference in the extent to which they dilute their PIA edits, standard operating procedure for many recidivist socks?
• Is it helpful or unhelpful to apply different labels to things that are the same from an impact on PIA perspective? Sean.hoyland (talk) 17:44, 29 August 2024 (UTC)

Total edits

If the objective is to try to measure SPA-ness, perhaps the namespace for the total shouldn't be limited to 0 and 1 e.g. Marokwitz made 1722 revisions since 2022-01-01 by my count which is significantly different from the 1275 used for the calculation. Sean.hoyland (talk) 11:53, 28 August 2024 (UTC)

That’s done because we can’t determine whether edits in other namespaces are within scope. Including it would result in systematic undercounting, BilledMammal (talk) 21:35, 28 August 2024 (UTC)

hmmmm, doesn't it mean systematic overcounting? If Marokwitz had made 200 edits in namespace (0,1) and 102 of those were in the topic area out of their total of 1722, is an SPA-ness value of 51% accurately describing them? I don't think so. Sean.hoyland (talk) 05:25, 29 August 2024 (UTC)

Marokwitz wouldn’t be on the list, if that was the case. It’s part of the reason I included the 500 edit requirement.

And no; I’ve spot checked a number of the editors, and their involvement in project space is either trivial or in line with their participation in the topic area. Exceptions would exist - for example, as discussed below, myself - but such exceptions don’t appear to apply to any of the editors on the list.

With that said, if we discover any that are, we can either manually exclude them, or prominently note the exception, whichever you would prefer. BilledMammal (talk) 05:52, 29 August 2024 (UTC)

Selected names and allocated alignment

Where did the list of names come from? Odd that the creator didn’t add themselves to the list for example. There are definitely some other names that should be there. My experience is that there are many more pro-Israeli accounts than pro-Palestinian accounts, although many of the former have shorter editing histories.

How were the proposed alignments arrived at? Some of them look wrong to me.

Onceinawhile (talk) 22:19, 28 August 2024 (UTC)

Editors are included if they have made more than 500 edits in that time, and they have made 50%+ inside the topic area.

Alignments were determined by reviewing edits. However, as I said in on the page:

My assessments may be incorrect; editors who disagree with my assessment of an editor should feel free to make a WP:BOLD edit to correct it, with an edit summary explaining why. If I or another editor disagree we can discuss on the talk page.

BilledMammal (talk) 22:25, 28 August 2024 (UTC)

What tool did you use to make these two assessments – both the 500/50% and the alignment? These are the key judgements, and I don’t see the value in this work if the key assessments are made subjectively. Onceinawhile (talk) 22:31, 28 August 2024 (UTC)

I explained my methodology in the page. What aspect is unclear? BilledMammal (talk) 22:37, 28 August 2024 (UTC)

What tool did you use, so I can replicate the calculations? Onceinawhile (talk) 22:40, 28 August 2024 (UTC)

I produced this using a custom Python script. Unfortunately, it’s not really replicable, but you can verify many of the figures by looking at xtools.

The assessments were done manually, and if you disagree with them I encourage you to take advantage of my offer to make a bold correction. BilledMammal (talk) 22:42, 28 August 2024 (UTC)

OK so we cannot check it without recreating it from scratch. That’s a shame.

Could you please provide the long list of editor accounts that you put through your script? Conscious that many did not qualify. Onceinawhile (talk) 22:48, 28 August 2024 (UTC)

Every editor account that has edited ARBPIA since 2022. I can provide the full list if helpful? BilledMammal (talk) 22:51, 28 August 2024 (UTC)

Yes please - that would be helpful. Onceinawhile (talk) 23:05, 28 August 2024 (UTC)

Perhaps to make it a manageable list, just those with more than 100 edits in the topic area. And without the 50% filter. We can then sort that by most active editors, to check that it is working.

Is it possible to also split the data between mainspace and talk?

Thank you.

Onceinawhile (talk) 23:08, 28 August 2024 (UTC)

I can’t split it like that, but I’ll try to get you the full list in the next few days. BilledMammal (talk) 23:11, 28 August 2024 (UTC)

@Onceinawhile: See User:BilledMammal/ARBPIA activity statistics complete BilledMammal (talk) 13:22, 29 August 2024 (UTC)

Thank you @BilledMammal: interesting table. Could you also run this since the start of October 2023 until today (i.e. for 11 months, instead of the 32 months currently being shown)? My sense is that the picture changed a little since October with the influx of new editors to the area. Onceinawhile (talk) 18:16, 29 August 2024 (UTC)

I might be able to do a little better than that. I’ll provide something over the weekend. BilledMammal (talk) 12:15, 30 August 2024 (UTC)

@Onceinawhile: I've updated User:BilledMammal/ARBPIA activity statistics complete; I think it should answer all of your questions, except the one regarding the talk/article split which I haven't been able to include.

Zero0000, I think it also addresses your request? There are some inaccuracies with assessed ECP date, but they are rare and generally obvious - for example, yourself. BilledMammal (talk) 05:02, 2 September 2024 (UTC)

Top edits

Regarding the point above about automated or similarly quick edits skewing the figures, I find the attached instructive.

We could do this for all the editors on the list. Of their top edited articles, talk pages, wikipedia and template pages, what % are in scope of ARBPIA.

To take BilledMammal as an example, given they made the effort to create this analysis, a look at their top edits shows the vast majority of their detailed editing and discussion focus being on ARBPIA topics.

Onceinawhile (talk) 22:44, 28 August 2024 (UTC)

You’re welcome to do so - although I’m not sure I’ll call less than half (28/60, looking at article, talk, and template - Wikipedia is harder to classify, but is probably 1/20) a vast majority.

However, the issue I see is that it isn’t representative - I think it is important to consider pages editors have made a lower number of edits to when determining just how focused an editor is on a topic area. BilledMammal (talk) 22:51, 28 August 2024 (UTC)

Your results are skewed by cherrypicking template, where you make very few edits, and treating all equally.

73% of your edits to your top 20 most edited articles are ARBPIA.

Onceinawhile (talk) 23:04, 28 August 2024 (UTC)

I included three of the four namespaces you listed - and the one I excluded, Wikipedia, skewed it in the opposite direction, as I have made more edits to my top 20 there than in the other three put together, and very few of them have been in relation to ARBPIA.

Regardless, if we’re going to look at edit count, shouldn’t we at least look at total edit count rather than a relatively arbitrary subset? BilledMammal (talk) 23:09, 28 August 2024 (UTC)

The value of this list

BM, twice you have referred to Black Kite's request as your reason for various choices. However, Black Kite asked for "sub-5000-edit accounts which are basically SPAs on the PIA area" while you included many others. So you should stop citing Black Kite as your excuse for your arbitrary decisions. You also classified them according to your own judgement of their POV which, given your own very strong POV which every ARBPIA editor knows, is a highly dubious exercise. If only the sub-5000 members of the list were presented, the table would look entirely different. Now that it transpires that only 20% of ARBPIA edits are included, the only safe conclusion is the SPAs and socks that make a large fraction of ARBPIA edits are mostly not in the list at all. Zero^talk 11:57, 29 August 2024 (UTC)

The only aspect I didn't closely follow Black Kite's request was in not limiting the list to editors with less than 5000 edits, as that limitation made little sense to me - although I still provided that information.

Regarding the classification:

My assessments may be incorrect; editors who disagree with my assessment of an editor should feel free to make a WP:BOLD edit to correct it, with an edit summary explaining why. If I or another editor disagree we can discuss on the talk page.

If you think any are wrong, please do follow that.

Not so simple. As Nableezy likes to tell us, the division is not really between pro-Israel and pro-Palestinian. In fact the pro-Israel POV that abounds is stronger and much more common than a strong pro-Palestinian POV. A more accurate description, ignoring outliers, is between pro-Israel and not-pro-Israel. Zero^talk 14:21, 29 August 2024 (UTC)

I disagree with that view; there are indisputably editors with a pro-Palestinian POV, just as there are indisputably editors with a pro-Israeli POV.

Of course, having a POV only becomes an issue when it results in POV pushing - and it has here, on both sides, although not all editors have engaged in it. I’ve been particularly impressed by Vice regent. BilledMammal (talk) 14:42, 29 August 2024 (UTC)

I agree that they are pro-Palestinian from where you are standing. But the caveat is the issue. Zero^talk 15:24, 29 August 2024 (UTC)

Now that it transpires that only 20% of ARBPIA edits are included, the only safe conclusion is the SPAs and socks that make a large fraction of ARBPIA edits are mostly not in the list at all.

Can you explain how you came to that conclusion?

This list contains ~19% of edits. Of the remaining ~81%, ~85% were by editors with more than 500 edits since 2022, but with less than 50% of their edits in the topic area. In other words, unless you're saying an editor can be an SPA while having less than half their edits in the topic area, I don't think your conclusion is right.

It is possible that there are a number of socks, but:

1. They weren't in the scope of the question asked
2. If you provide me a list of socks, I can provide you what percentage of edits were made by them.

BilledMammal (talk) 12:27, 29 August 2024 (UTC)

We are getting there, but as someone else mentioned there is a problem due to the EC rule. New editors with between 500 and 1000 edits will automatically have less than 50% in ARBPIA because the first 500 were not allowed to be in ARBPIA. Those first 500 edits will bias every new editor away from 50%. A new account with 2000 edits will need 67% in ARBPIA to meet the criterion. With 3000 edits, 60%. I see a lot of editors in the 1000-3000 range. If you want a rule that takes care of this problem better, only count edits starting with the first ARBPIA edit in the time period. That would also fix another source of misleading data: editors who only joined ARBPIA after Oct 7 last year. There are a huge number. Just going by editor counts, the situation since Oct 7 is quite different from before, but by combining it with a larger time period before Oct 7 you are smoothing it away. Zero^talk 14:21, 29 August 2024 (UTC)

only count edits starting with the first ARBPIA edit in the time period

I might be able to do that; I’ll look into it. BilledMammal (talk) 14:43, 29 August 2024 (UTC)

Thinking more about this, it might not work. Very often editors make an ARBPIA edit or two before being informed of the 30/500 rule and that will spoil it. Is "first ARBIA edit after getting EC" too difficult? Zero^talk 15:12, 29 August 2024 (UTC)

Plus, edit requests.

Is "first ARBIA edit after getting EC" too difficult?

Thinking more about it, I might be able to make it work, or something close enough. I’ll try over the weekend. BilledMammal (talk) 15:17, 29 August 2024 (UTC)

Extreme tunnel vision

I think we should all take a step back, exit this tunnel vision and avoid asking for reforms on a non-reformable statistic that does not even tackle the question at hand. This is a quantitative analysis that does not take any qualitative aspect into consideration, for example the quality of sources used, the quality of text inserted, the level of each editor's collaborative-ness and good faith, etc. Even if we were to cast a qualitative analysis aside, there are many fundamental flaws within this quantitative analysis that makes it moot:

1- If I am interested in Jordanian articles, almost everything I edit will be counted according to the methodology: Al-Maghtas, a site in Jordan; Alia Toukan, the former Queen of Jordan; the Arab Bank, a Jordanian company, are linked to Palestine Wikipedia project. Habis Majali, a Jordanian general, is under ARBCOM. The Decapolis, a Greco-Roman ten city league most of which was in Jordan, is linked to Israel Wikipedia project. Hell, even Moses is linked to Israel WP project. You get the point.

2- If I am operating a bot for simple edits across multiple articles or if I am filing AE reports every fortnight, while my most important edits are under ARBPIA, then of course that editor will get a low "score".

And finally, WP:SPA is an essay that can be edited by anyone, whose current definition of an SPA includes editing "one very narrow area or set of articles, or whose edits to many articles appear to be for a common purpose." Obviously, the Israel and Palestine WP projects and ARBPIA are not "very narrow," nor does editing in them is necessarily sharing a common purpose.

There is no need to waste more time on this statistic or on discussing it. Makeandtoss (talk) 11:52, 4 September 2024 (UTC)

Only one of the articles you list is considered by this analysis as within the topic area - Habis Majali, and as he fought several wars against Israel, it makes sense that he is.

The rest is more subjective/complicated, and as you aren’t interested in discussing this I won’t go into it. BilledMammal (talk) 12:24, 4 September 2024 (UTC)

Even if Habis Majali is the only article, then we must agree that this is not a "very narrow" scope. Second, only now I realized the "and" criteria, but my point is still valid. Arabic, Fertile Crescent, Holy Land articles are linked to both Palestine and Israel WP projects. An entire topic area is not a "very narrow" area. Makeandtoss (talk) 12:42, 4 September 2024 (UTC)

I think if a new editor came along and was focused solely on ARBPIA, we would - and correctly so - call them a SPA.

How many articles - and more importantly, how many edits - do you think are incorrectly classified as being within the topic area? If it’s at least 95% accurate - and my review suggests it is - then I think it’s accurate enough to be useful. BilledMammal (talk) 12:51, 4 September 2024 (UTC)

I wouldn't because ARBPIA is huge, spanning at least a century of articles everywhere in the Eastern Mediterranean, and to a lesser degree the entire world. Not even the Israel-Hamas war would even be considered a "very narrow" area either given the huge amount of coverage and articles.

The statistics are not accurate enough to be useful since the scope is huge. Even if it was narrower, WP:SPA is still an essay, and there is no qualitative analysis to show systemic lack of neutrality and disruption. Makeandtoss (talk) 13:11, 4 September 2024 (UTC)

there is no qualitative analysis to show systemic lack of neutrality and disruption

This table isn’t intended to show that. BilledMammal (talk) 13:15, 4 September 2024 (UTC)

Then its of no use; WP:SPA "..are expected to contribute neutrally..". Makeandtoss (talk) 13:22, 4 September 2024 (UTC)

SPA=bad is a potentially problematic premise for me. Bots are SPAs. I aspire to be a bot-like SPA, apparently not as well as I had hoped according to the stats. There is also an issue of granularity/limited resolution which seems a bit difficult to address. I mean for example that if I could be bothered to generate some kind of objective function for my revision history to measure performance, I would want to be able to look at something like the ratio of 'policing' edits (reverts of nonsense disguised as edit requests etc.) vs other stuff, things like that. Anyway, baby steps. Sean.hoyland (talk) 13:37, 4 September 2024 (UTC)

That is the implication being made here. But anyway, the root here is shaky, because the criteria is not "very narrow". Makeandtoss (talk) 13:53, 4 September 2024 (UTC)

I get that there is a lack of trust, people think about motive. Personally, I don't care about that. BilledMammal is a smart guy. They are doing something to quantify and understand things. This is better than nothing. Presenting evidence and people ripping it to shreds, repeat n times, is normal for me. I do agree the root is shaky. Bots are SPAs. But if I said, I found an SPA and presented a bot, the response would be, not that kind. So, for me it is a bit ill-defined and subjective. What exactly is the search for, because it is SPA+something. Sean.hoyland (talk) 14:10, 4 September 2024 (UTC)

I would not overestimate or underestimate the intelligence of any editor here. All initiatives are prone to criticism, including this one. If the whole premise is that ARBPIA as a topic is "very narrow," then that premise crashes down at the slightest scrutiny. Makeandtoss (talk) 15:11, 4 September 2024 (UTC)

Regarding "the level of each editor's collaborative-ness and good faith". Talk page revision counts are perhaps the only readily available proxy for that information. Reporting article and talk page revision counts separately could illuminate things and add information, which should be an objective of this effort. Combining article and talk page revision counts obscures information, which should not be an objective of this effort. Sean.hoyland (talk) 19:18, 4 September 2024 (UTC)