Talk:Opinion polling on Scottish independence/Archive 2
![]() | This is an archive of past discussions. Do not edit the contents of this page. If you wish to start a new discussion or revive an old one, please do so on the current talk page. |
Archive 1 | Archive 2 | Archive 3 | Archive 4 | Archive 5 |
Techne UK Poll 29/30 June 2022
There has been ongoing discussion as to the correctness or otherwise of including this poll on this page and especially in the main table. I would welcome others views on this matter as myself and another editor seem to have reached an impasse. This poll is a sub set of 502 taken from a larger poll, as such I don't believe it should be included here. As you are probably aware sample size is crucial in all polling, the British Polling Council has developed across the industry that for all National polling a sample of 1000 produces the best results, from their website "There is no, “minimum”, sample size for a poll which is acceptable, but around one thousand has become the established norm for a nationwide opinion poll in Great Britain." https://www.britishpollingcouncil.org/faqs-by-members-of-the-public/ On this page we have had recent changes where a Yougov poll for These Islands was removed because they were subsets of only 501 and 519 Across wikipedia 1000 has become the acceptable norm for a National poll https://en.wikipedia.org/wiki/Opinion_polling_for_the_2021_Senedd_election https://en.wikipedia.org/wiki/Opinion_polling_for_the_2019_United_Kingdom_general_election https://en.wikipedia.org/wiki/Opinion_polling_for_the_2021_Scottish_Parliament_election https://en.wikipedia.org/wiki/Opinion_polling_for_the_next_Scottish_Parliament_election I believe that this sub set does not match the criteria for being a poll demonstrated across this page, other wiki pages on UK polling and in fact the BPC The old adage about " just because you can do something does not mean we should" and to include this subset lowers teh standard within this page and that would be regrettable. I would welcome views from others as to this issue Soosider3 (talk) 11:04, 9 July 2022 (UTC)
- The poll specifies that it is weighted for Scotland and TechneUK is a member of the British Polling Council. It is not a subsample.
- The only reason not to include the poll is that the sample size is 501, meaning the margin of error is 4% instead of the standard 3%, however this sits within a 95% confidence interval, so I think it should be included with that caveat. AlloDoon (talk) 12:48, 10 July 2022 (UTC)
- Hi thanks for the reply
- I think there are more reasons than just sample size.
- 1/This is not a full poll but rather a subset of a larger poll, this is evident by Techne themselves in the data tables where it clearer identifies that this is a subset of 501 from a bigger report of some 1600 https://www.techneuk.com/wp-content/uploads/2022/07/R24-UK-2022-7-1-DATA.pdf
- 2/ Techne do appear to be a member of BPC and I have no concerns about the compliance of their full poll
- 3/BPC have long established that a sample size of at least 1000 is required to produce accurate and comparable data
- the larger poll would have been of sufficient size to have confidence in its weighing across a whole range of categories, this can not be said for a subset such as this.
- 4/ Most National polls under BPC have a well established requirement to have a sample size of at least 1000, this is evidenced across a wide range of places, on wikipedia I would ask you to look at these and explain why every one of the polls recorded have a sample size of 1000.
- https://en.wikipedia.org/wiki/Opinion_polling_for_the_2021_Senedd_election
- https://en.wikipedia.org/wiki/Opinion_polling_for_the_2019_United_Kingdom_general_election
- https://en.wikipedia.org/wiki/Opinion_polling_for_the_2021_Scottish_Parliament_election
- https://en.wikipedia.org/wiki/Opinion_polling_for_the_next_Scottish_Parliament_election
- 5/ In polling as in most statistics 95% accuracy would not be acceptable, or at least be a matter of disappointment that such a margin of error had been allowed to happen, it introduces a significant increase in the margin of error and makes it unsafe to do comparisons with other higher standard results.
- 6/ statistically the increase from less than 3% to 4.4% is a huge increase, it increases the error margin by some 43%, yep the increase from 3% to 4.4% is an increase of 43% in teh margin of error
- 7/ if this entry stands it introduces a drop in standards that could have unexpected consequences, do we now allow subsets or polls with only 500 sample, because that is the benchmark you would lower this page to, that would be a great shame as it has been a reliable source on polling to do with Independence.
- I really do believe this would be a retrograde step, however I can be convinced if a coherent argument can be made for including subsets and/or small sample sizes. The onus is really on you to explain why you believe this departure from established norms is best for this page. Soosider3 (talk) 18:28, 10 July 2022 (UTC)
- We have, you just don't accept the points that have been made. WP:DROPTHESTICK. Jmorrison230582 (talk) 18:40, 10 July 2022 (UTC)
- No you really haven't explain a coherent rationale for including this data. You have expressed a desire but in doing so have ignored the clear factual evidence that you are wrong. Precedence is well established across the industry, we do not treat subsets as polls and 1000 is the industry norm for national polling, the evidence for that is overwhelming.
- The data set of 1642 in the full poll clearly shows that this is a subset of 501, there is no other explanation for it. You are suggesting that this is a full poll within another full poll, that's illogical, inconsistent and nonsensical.
- You have also failed to explain why you wish to abandon accepted practice not just on Wikipedia for National Polling but long established best practice and established norm for the British Polling Council.
- So as I said the onus is on you to explain this change you wish from established practice and norms.
- Regrettably I may have to take this to dispute resolution, I have convinced 2 other editors of teh correctness of my position but seem unable to move you and one other. Soosider3 (talk) 22:07, 10 July 2022 (UTC)
- We have, you just don't accept the points that have been made. WP:DROPTHESTICK. Jmorrison230582 (talk) 18:40, 10 July 2022 (UTC)
- First of all, you claimed that the Techne poll shouldn't be included because they aren't a BPC member. In fact, the BPC website lists Techne as a member. Since then you have insisted that opinion polls must have a sample size of 1,000, even though there is no such requirement. I have also pointed out articles where polls with smaller sample sizes are used, e.g. 2022 Wakefield by-election. Your other argument is that it is a sub-sample of a larger poll, but this is not uncommon either. Firms poll Great Britain as a whole, then boost the Scottish part of the sample to get a representative sample of that population as well. Jmorrison230582 (talk) 05:39, 11 July 2022 (UTC)
- Good Morning
- I fear you may not be focusing on the key matters here, membership of BPC was based on a statement on Techne website, which i linked to, you provided evidence from BPC website and I conceded the point, however my prime focus has been and continues to be the size of the sample and it being a subset, which is clearly at odds with all other national polling.
- You create a false comparison when trying to compare constituency polling for a by election to national polling.
- We seem to be having a discussion based largely on semantics, you say there is no "requirement" for a 1000 of a sample and I argue that the established "norm" across the polling industry is a sample of 1000 for a national poll.
- I ask that you look at this page and ask yourself why almost all the samples for a full poll are 1000 and does a sample of 501 really fit in with this? and then do similar with other national polling across the UK
- So far 5 editors have expressed a view 3 supporting my view and 2 supporting yours.
- Your position is unsustainable, flys in the face of established practice and BPC expectation. Please withdraw this entry Soosider3 (talk) 08:47, 11 July 2022 (UTC)
- First of all, you claimed that the Techne poll shouldn't be included because they aren't a BPC member. In fact, the BPC website lists Techne as a member. Since then you have insisted that opinion polls must have a sample size of 1,000, even though there is no such requirement. I have also pointed out articles where polls with smaller sample sizes are used, e.g. 2022 Wakefield by-election. Your other argument is that it is a sub-sample of a larger poll, but this is not uncommon either. Firms poll Great Britain as a whole, then boost the Scottish part of the sample to get a representative sample of that population as well. Jmorrison230582 (talk) 05:39, 11 July 2022 (UTC)
- Please point out these supposed comments from other users in favour of your position. I can't see any. All I can see is you just droning on about the same poll. Jmorrison230582 (talk) 09:32, 11 July 2022 (UTC)
- See Talk between MSLavender, Mako001 and myself, initially both, correctly pulled me up about my lack of following correct etiquette, after explanation they both were content to go along with my revision.
- Regrettably we seem to have reached an impasse and there is little point in continuing this 'discussion but before I go.
- It is not a good look when someone is unable to support their point of view with a coherent fact based argument, despite mountains of evidence against it, then resorting to petty comments about "droning on" and "boring you"
- I will remove the entry, once again and if reinstated will look to dispute resolution to resolve Soosider3 (talk) 10:36, 11 July 2022 (UTC)
- Where is this discussion? It is not on this page, its archive or your talk page. All I can see is that @HMSLavender: and @Mako001: both warned (diff 1. diff 2) you about removing content without explanation. To say that they were "content to go along with my revision" with your position is a gross misrepresentation of their edits. Jmorrison230582 (talk) 10:49, 11 July 2022 (UTC)
- @Soosider3: - @Jmorrison230582: is quite right.
- As the poll sits within a 7% margin of error, the confidence interval is still above 95% and the poll is valid. TechneUK is a member of the British Polling Council, and therefore they are required to adhere to established practices to publish public opinion polls in the UK. If you want to open this issue to further discussion then please WP:RFC, or otherwise WP:DROPTHESTICK. AlloDoon (talk) 15:23, 11 July 2022 (UTC)
- You complete ignore that this published on this page is not the full poll of 1602 but a subset of only 501. My concern is not with Techne who I am sure perform their tasks professionally, my concern is as said several times. This is not a poll but a subset and as such does not sit with any other polling on this page. That is a sustainable and easily verified fact not opinion
- Could I ask if you would do 2 things, look at this page and ask yourself why there are no other examples of such a small sample, genuinely reflect on this and consider the established norm across the industry, as per BPC. Having done so please explain why you wish to change that long established standard for this entry Soosider3 (talk) 15:36, 11 July 2022 (UTC)
- Where is this discussion? It is not on this page, its archive or your talk page. All I can see is that @HMSLavender: and @Mako001: both warned (diff 1. diff 2) you about removing content without explanation. To say that they were "content to go along with my revision" with your position is a gross misrepresentation of their edits. Jmorrison230582 (talk) 10:49, 11 July 2022 (UTC)
- Please point out these supposed comments from other users in favour of your position. I can't see any. All I can see is you just droning on about the same poll. Jmorrison230582 (talk) 09:32, 11 July 2022 (UTC)
- (unindent) It doesn't matter that it is a sub-sample and has less respondents than a Scotland-only poll. The point is whether the sample used is representative of the Scottish population. My understanding is that it is. Jmorrison230582 (talk) 18:19, 11 July 2022 (UTC)
- We seem to have reached agreement that this is a subset and has less respondents than a Scotland only poll, that in itself is enough for this entry not to appear.
- Again we are playing with words "sample used is representative" any sized sample can be Representative the issue here is the smaller the sample the greater the margin of error. This is why BPC expect 1000 and why this norm is accepted across the Industry, this has established a consistency where it is broadly reasonable to compare Poll to Poll this no longer applies if you introduce a different and substantially lowered standard by using subsets and smaller samples. Soosider3 (talk) 06:48, 12 July 2022 (UTC)
- I notice that Professor John Curtice on his website Whatscotlandthinks.org has not used the Techne data on his Independence Polling tracker, given that Curtice is Chair of BPC I would suggest that is a fairly good indication of how this data is used and viewed by teh professionals.
- https://whatscotlandthinks.org/questions/how-would-you-vote-in-the-in-a-scottish-independence-referendum-if-held-now-ask/
- If the data i snot good enough to be used by Curtice then it should not be good enough for us to use Soosider3 (talk) 06:55, 12 July 2022 (UTC)
- That is interesting. His site include the poll as a UK result, but not as a Scottish result. Looking at the poll tables, they give a detailed breakdown of the UK result, but not the Scottish component. They note that they "oversampled" (interviewed more) people in Scotland, presumably to reduce the margin of error, but it doesn't give any detail as to how the sample was made representative of the Scottish population or electorate (as it did with the UK data). I'm happy to accept his view as a third party opinion. Jmorrison230582 (talk) 07:35, 12 July 2022 (UTC)
- I will say this tentatively but I think we have established common ground here.
- Are we in agreement that Techne is okay to have on as a UK poll but not as a Scotland poll? If so I will not remove entry ( in case I have misunderstood) If we are in agreement can I ask that you remove from Scotland table and by all means add to UK one.
- As someone new to editing Wikipedia this has been a steep learning curve for me, I have learned lots and hopefully will be a little less clumsy in the future Soosider3 (talk) 08:15, 12 July 2022 (UTC)
- That is interesting. His site include the poll as a UK result, but not as a Scottish result. Looking at the poll tables, they give a detailed breakdown of the UK result, but not the Scottish component. They note that they "oversampled" (interviewed more) people in Scotland, presumably to reduce the margin of error, but it doesn't give any detail as to how the sample was made representative of the Scottish population or electorate (as it did with the UK data). I'm happy to accept his view as a third party opinion. Jmorrison230582 (talk) 07:35, 12 July 2022 (UTC)
@Jmorrison230582: The 'oversample' is explained in the poll's methodology notes:
"The planned sample is represented by 1500 interviews, plus 124 over-sampling. The choice of over-sampling was based on the need to have a consistent number of cases in each weighting cell represented by the stratification between region or country and voting behavior in 2019 general elections.
"With regards to the question "Should Scotland be an independent country?", a 374 interviews' oversample has been done in Scotland. The Scottish overall sample is 501 cases."
This confirms the sample was weighted.
Techne UK also stated on their Twitter account that the poll is "Weighted and representative of the population of Scotland."
A number of polls have been missed out from WhatScotlandThinks tracker, for example Ipsos Mori's 9 February poll. I don't see any reason to exclude this poll purely on this basis, unless we equally remove all polls not published on John Curtice's website. AlloDoon (talk) 14:42, 12 July 2022 (UTC)
- For reference, all polls not included on John Curtice's site and likely reason:
- TechneUK 1 July 2022 poll - non-standard margin of error
- Ipsos Mori 9 February 2022 poll - non-standard referendum question
- Find Out Now/Daily Express 30 March 2021 poll - unknown reason
- - Survation 5 October 2020 poll - non-standard referendum question
- Survation 9 October 2019 poll - non-standard referendum question
- Survation 21 March 2019 poll - non-standard referendum question
- Deltapoll 29 August 2018 poll - non-standard referendum question
- YouGov 5 June 2018 poll - non-standard referendum question
- YouGov 16 December 2016 poll - unknown reason
- Missing from wikipedia: 23 April 2021 Poll by Survation & 4 October 2016 BMG poll
- I think there should be consistency, so either keep as is, or have a separate section for non-standard referendum questions and, in this case, margins of error, or remove them entirely. My preference is to keep the current system using notes for any non-standard questions/sample sizes. AlloDoon (talk) 15:08, 12 July 2022 (UTC)
- That is a classic strawman argument, no one suggested for a moment that we emulate Whatscotlandthinks.org it is a shame you seem to have expended so much time and effort constructing this strawman. I don't appreciate you misrepresenting me in that manner. What actually happened was that discussion between 2 editors were beginning to circle and we looked at a third party view to try to give us some perspective to break the circle.
- Interestingly you miss out the key reasons for not including the report in Scottish polling
- 1. It is a subset of a UK wide poll (the UK poll is fine and should be included in that section of this article)
- 2. Every other poll on the Wikipedia article has a sample size of 1000
- 3. I am not aware of any subset included in this Wikipedia article, so why introduce that change now
- 3. Smaller sample inevitably leads to a higher margin of error, that is why BPC has an industry standard across the UK that a 1000 is the norm for polling at a national level. This also allows for a higher confidence level when comparing polling results. Every other poll in this article has a margin of error of less than 3%
- 4. The scale and magnitude of this proposed change in practice cant be covered adequately in Notes, that's fine for comparable small matters but not for such a radically different record.
- 5. The risk of unforeseen consequences in my opinion is high, we would be allowing a precedent that allows subsets and sample sizes of only 500, once you let that genie out the bottle you will not be able to get it back in.
- The correct place for this poll is in the UK section and there should be no place in main article for subsets or sample sizes less than 1000, just a sit has been doing for several years 81.96.97.29 (talk) 07:44, 14 July 2022 (UTC)
There seems reasonable evidence that the Techne poll is appropriately weighted for Scotland, and designed to poll the independence question. The sample size is small, but for those reading this article it is stated clearly. I see no need to set an arbitrary limit to exclude polls which are otherwise in the public domain. Even polls by Non-BPC members are included in the table. The low sample size will appropriately reduce the weight of the poll in the moving averages in the graphs.
We shouldn't complicate things with a separate section for non-standard questions or small polls, the system of notes for anything people want to draw attention to is fine.
I don't think we can just do the same as What Scotland Thinks if they are in the habit of omitting polls.
We should just put the poll back. RERTwiki (talk) 15:19, 12 July 2022 (UTC)
Yougov/These Islands poll 29-31 March (again)
Hi. Just noticed the Yougov/TheseIslands polls from the end of March have been excluded from the table: too much else going on!
Last I was aware, the poll was being carried in both the 'independent country' and 'Remain/leave' tables as two distinct polls. I think that's correct, and much better than excluding the polls.
Reading the tables, their methodology was to ask half of respondents at random one or the other question. Hence the odd (519-510) split between the two questions.
Seems to me that a randomly selected half of a representative sample is a representative sample. In any case, these folks are professional statisticians and chose that method, and there is no reason to doubt that they intended the polls to be publishable. We shouldn't be excluding polls unless there is a very good reason to do so.
What are people's thoughts? RERTwiki (talk) 15:51, 26 July 2022 (UTC)
I think it sits better where they are at the moment Soosider3 (talk) 18:12, 26 July 2022 (UTC)
Sorry, I can't see it. If it's there could you point me at it? If your comment is a way to say that the polls should be excluded, could you outline why you think so? (RERTwiki (talk) 09:25, 28 July 2022 (UTC))
- My apologise hadn't realised polls had been removed completely, like you I thought they sat best where they had previously been as 2 smaller polls Soosider3 (talk) 15:28, 28 July 2022 (UTC)
OK. So at least 2 (of 4 active) seem to think the polls should be included. I'll give a few days for comments from JMorrison, who I think was the one who took them out. Absent further discussion I will put them back. (RERTwiki (talk) 11:17, 31 July 2022 (UTC))
- It is a poll by a reputable company, what seems to throw it into query is the manner of asking both Independent Qs in the same poll. The Leave/Remain section I believe fits within the Leave/remain table, however that leaves the section on the Standard Qs rather homeless, is there merit in looking at how we record these rather odd polls so we don't lose the data but not place it in the main Table, we maintain the long established precedent in the main table and create sections to capture this odd ones.
- Perhaps more technically minded folk could answer this, is it possible to have filters on the table that would allow users to select things like "over 1000 sampled" "BPC Members" "Standard Question" etc etc Allowing users to decide what data they want to see, rather than us wrestling with inclusion/exclusion and what appears to be an ever burgeoning number of wee tables. If technically possible I would suggest this is where we need to concentrate our discussions and hopefully provide users a more useful article.
- In some ways the Techne poll has recently thrown up issues that make it difficult to see where it fits, perhaps above approach (if possible) might be way to go as well as giving us a more robust and at same time more flexible and responsive article. Soosider3 (talk) 15:17, 31 July 2022 (UTC)
- I asked at the Maths Reference desk about these polls and got this answer -
- How these poll numbers are best presented in the article is not a maths question. A sample taken from a population is considered "representative" if the sampling procedure is such that the probability of a member of the population being included in the sample is uniform across the population: everyone has the same probability of being selected as anyone else. (In the practice of polling this is an ideal that is never fully attained.) If a representative sample is drawn from an existing representative sample, the new, smaller sample is also representative, since every member of the original population has the same probability of being included. --Lambiam 17:39, 2 August 2022 (UTC)
- I think this adds support to the view that the polls are indeed representative. The questions match those in the main table and the remain/leave table, respectively. I'd like to put these polls back in. As regards your question on filtering, I can't think of a good way to do that. I have to say I have long desired a field stating Y/N on 'Standard Question', as that would save me 5 minutes on every update of the charts. RERTwiki (talk) 09:29, 3 August 2022 (UTC)
- I support putting the polls back in. No convincing argument has been advanced to justify their removal as far as I am concerned. I have been quietly watching the removal of polls with concern and dismay.--Literaturegeek | T@1k? 23:06, 3 August 2022 (UTC)
- Agree, happy to reinstate if needed. AlloDoon (talk) 22:26, 4 August 2022 (UTC)
- I have put the polls back in both the main table and in the table of remain/leave polls. RERTwiki (talk) 10:57, 5 August 2022 (UTC)
- Can I suggest that you have a wee look at this site, very helpful in understanding Confidence Level and Confidence Interval. https://www.surveysystem.com/sscalc.htm
- The article until very recently only recorded polls of 1000+ providing a consistent table, I have not heard a single coherent explanation of why this should change. As stated elsewhere and poll can be 'representative' if carried out correctly, 1000 is the norm for BPC and for a reason, going below that Lowers Confidence Level and Increases Confidence Interval, which IMHO lowers the standard, my question remains Why? for what advantage to users. The only reason I can think of in changing this methodology is to benefit the clients as smaller polls cost less, that is not something that should be of interest to us, or that it is being proposed for partisan reasons. By all means record it on another table but lets not lower the standard. Soosider3 (talk) 13:57, 5 August 2022 (UTC)
- Regarding the confidence interval calculator, the page explicitly states that a sample size of 196 (95% confidence interval) is acceptable to most researchers.
- On the point regarding "The article until very recently only recorded polls of 1000+ providing a consistent table", this is untrue. Take 26 June 2016 poll with sample of 626, or the YouGov/These Islands poll 3 months ago. A large number of polls also link to organisations which commissioned them rather than data tables, which is perfectly acceptable in my opinion however it is preferable where possible to link directly to the data.
- Finally, please stop making accusations about people's partisanship and stick to the argument at hand. These accusations are against wikipedia guidelines for discussion pages (please see WP:TALK). Nobody is making accusations on your own political preferences, because it does not move the argument any further forward and would be purely speculative. AlloDoon (talk) 09:19, 6 August 2022 (UTC)
- Agree, happy to reinstate if needed. AlloDoon (talk) 22:26, 4 August 2022 (UTC)
Protocol for Inclusion/Exclusion of Polls, Techne
Folks - I believe that that the only way this Wikipedia Entry can be insulated from accusations of political bias is to be as inclusive as possible.
In other words, we should include any sensible poll unless we have unanimous agreement that a poll is fatally flawed. The only inevitably fatal flaw is that a poll is not known to be representative of Scotland. From the above discussion, there is room to argue that a poll of under 500 respondents is fatally flawed.
But if we don't have unanimity for exclusion, and someone is arguing coherently and rationally that a poll merits inclusion, then it should be included.
This being Wikipedia, there must be a caveat that we seek unanimity among the sane, but that is a longer story.
I think I can make a very strong case that Ipsos MORI polls are systematically biased to a higher showing pro-independence (and lower "don't know"). Is the right thing for me to do to just remove the polls and refuse to be convinced by arguments for inclusion? Should I just keep removing them in the face of a large majority in favour of their inclusion?
Obviously not. But at some level that's what is happening with the Techne poll. I think the poll should be included, I think there is a good case for it, and I think that should be enough for any poll. That's true even if some people don't like the poll.
If we go the other way, and allow committed individuals to censor the list despite the wishes of (in this case) the majority of editors, we are wide open to abuse and to accusations of being 'fake news' adapted to a 'narrative'.
My 2 cents. RERTwiki (talk) 11:17, 5 August 2022 (UTC)
- Interesting use of language, it is not conducive to constructive dialogue and the wee petty barbs do you no credit
- You bring in points that are not relevant to the ongoing discussion it is fairly well understood that IpsosMori methodology may produce a slightly higher recording of the Yes side just as Yougov does the same for the No side, however this is not the issue here and it is disingenuous of you to try to introduce this as a coherent point. In fact it rather shows where your own thinking is coming from.
- The matter that has been under consideration is whether to change long established practice, about sample size and the use of subsets, that for the main table the BPC standard has been applied for many years and has produced a sensible and easily understood table for users.
- I am still waiting to hear a coherent rationale for why lowering the standard somehow enhances the Article, be very clear that the proposal will lower the standard.
- Any poll carried out can be "representative" if weighted however the further it gets below 1000 the lower the Confidence Level and the Higher the Confidence Interval, that is why 1000 is established by BPC and why this article until very recently only listed polls of 1000.
- Suggest having a wee look at this it may help to clear your thinking https://www.surveysystem.com/sscalc.htm Soosider3 (talk) 13:47, 5 August 2022 (UTC)
- Soosider there has been a poll on the article with a subsample of 626 for 6 years and there clear consensus in this talk page is that these polls are acceptable on the article.
- Polls that should be included are British Polling Council polls which are weighted to representative for the population of Scotland. AlloDoon (talk) 08:26, 6 August 2022 (UTC)
- 1 poll out of 224 six years ago is not a sound basis to set a protocol on, it happened 184 polls ago, every Poll recorded here since has been consistent and compliant with the BPC norm of 1000 as does every other poll going back to 2014.
- The more logical option would be, rather than using an anomaly as a template, remove it as not being consistent with standard that has evolved and been used for 8 years.
- A simpler protocol would be any poll that is over 1000 and shows appropriate methodology. That maintains the standard and consistency of this particular table, by all means record it on another table but lets not mess up 8 years of good consistent work.
- I am well aware that I seem to be the lone voice advocating for this point of view but that does not make what is happening right, appropriate, consistent nor logical. Rather it feels like some people trying very hard to squeeze this poll in for reasons that are neither clear or sensible. Soosider3 (talk) 09:01, 6 August 2022 (UTC)
- I had not realised the YouGov/These Islands poll was also only removed very recently, and therefore was on the article with a sample size of 519 for 3 months. Both of these polls were also added to the article by other users and not removed despite a number of poll additions to the page... On your latter point about "trying very hard to squeeze this poll in for reasons that are neither clear or sensible", I would advise you to avoid personal attacks and avoid misrepresenting other peoples views and focus on the central argument. See also - WP:TALK. AlloDoon (talk) 09:11, 6 August 2022 (UTC)
- My expression - inclusion are neither clear or sensible is exactly that, my views and I believe I evidence that above, it is not reasonable to see them as personal attacks
- Lets try, as you say to stick to the central argument.
- I think I rather concisely summarise my point of view above and would welcome any thoughts you may have on the subject in hand.
- The idea of a protocol is an interesting one, given the very clear evidence in this main table it seems clear that a protocol had been operating for consistently for several years, polls over 1000 etc etc. My question remains why change it? and why now? By all means record these 'odd' polls but elsewhere in article. Soosider3 (talk) 10:09, 6 August 2022 (UTC)
- I had not realised the YouGov/These Islands poll was also only removed very recently, and therefore was on the article with a sample size of 519 for 3 months. Both of these polls were also added to the article by other users and not removed despite a number of poll additions to the page... On your latter point about "trying very hard to squeeze this poll in for reasons that are neither clear or sensible", I would advise you to avoid personal attacks and avoid misrepresenting other peoples views and focus on the central argument. See also - WP:TALK. AlloDoon (talk) 09:11, 6 August 2022 (UTC)
Looking over the rim of this particular teacup for a moment, if you look across Wikipedia, there are numerous pages which regularly include polls with a sample smaller than 1,000 in whatever the 'main' table is. Even if that includes regular outliers in some cases, for instance Tecnè polls for Italian GEs regularly put FI much higher than other pollsters (very obvious in the line of dots in the graph hovering around 10% whereas the median is more like 8%. I haven't asked them but given how many people participate in that page, I can only assume that they are a) by credible pollsters and b) that their take on these smaller/outlier polls is that it averages out overall without misleading the reader. If that's acceptable for that and other GE pages, I don't see why we need to be holier than thou on this page, especially if the 'holier' approach seems to be advocated by a single editor. Akerbeltz (talk) 10:21, 7 August 2022 (UTC)
- That gave me a Rye smile, I would remind you we are talking about standards as apply in the UK, most UK political poling record on wikipeadia complies to BPC standard as has this article for many years. I would suggest that Confidence Interval you refer to actually reinforces the reason for maintaining the BPC standard. Your sincerely - A lone Voice. Soosider3 (talk) 12:04, 7 August 2022 (UTC)
- A wry smile perhaps, rye has nothing to do with it :b And I disagree. The BPC may apply to UK polls, sure but with the amount of bickering that goes on on Wikipedia, I have little doubt that at least for bigger countries, some sort of standard is applied that says which polls can be included and which can't. In any case it always comes back to the point that apart from the general guidelines around sticking to BPC polls for UK polls, there is no apparent rule in force that limits inclusion to polls bigger than 1,000. I suggest if you're that passionate about it, you start a general debate on the appropriate page about sample sizes of BPC polls, get consensus and if you succeed, then come back here. But trying to somehow come up with a new standard on a sub-page as it were seems altogether the wrong place, cause even if you managed to get your point accepted here, there's a LOT of other pages with UK polls on it, that's a lot of endless debating and edit warring, energey better spent on a general policy page, no? Akerbeltz (talk) 18:12, 7 August 2022 (UTC)
- The rest of wikipedia is not my concern, there is clear precedent in this article that 1000 is the expected sample size, practice that has been established and maintained over many years, echoing the BPC standard. Why change that now? So far you have not given a coherent rationale other than a wee snide remark about 'holier' If you have a rationale please share it. Soosider3 (talk) 06:43, 8 August 2022 (UTC)
- Perhaps you need to spend a little bit more time on other Wiki pages, rather than just the one, before you come out with statements like that. Changes in editorial policy affect more than one page and you can't debate such issues in isolation. But this is getting repetitive. It's you who wants to change the status quo, so it's on you to convice others, not the other way round. Until then, I suspect you'll end up finding yourself reverted ad nauseum. Akerbeltz (talk) 08:59, 8 August 2022 (UTC)
- My point is that the status quo is samples of 1000 as evidenced by the article itself, if you wish to change the status quo it is for you to win that particular argument. Soosider3 (talk) 13:32, 8 August 2022 (UTC)
- Perhaps you need to spend a little bit more time on other Wiki pages, rather than just the one, before you come out with statements like that. Changes in editorial policy affect more than one page and you can't debate such issues in isolation. But this is getting repetitive. It's you who wants to change the status quo, so it's on you to convice others, not the other way round. Until then, I suspect you'll end up finding yourself reverted ad nauseum. Akerbeltz (talk) 08:59, 8 August 2022 (UTC)
- The rest of wikipedia is not my concern, there is clear precedent in this article that 1000 is the expected sample size, practice that has been established and maintained over many years, echoing the BPC standard. Why change that now? So far you have not given a coherent rationale other than a wee snide remark about 'holier' If you have a rationale please share it. Soosider3 (talk) 06:43, 8 August 2022 (UTC)
- A wry smile perhaps, rye has nothing to do with it :b And I disagree. The BPC may apply to UK polls, sure but with the amount of bickering that goes on on Wikipedia, I have little doubt that at least for bigger countries, some sort of standard is applied that says which polls can be included and which can't. In any case it always comes back to the point that apart from the general guidelines around sticking to BPC polls for UK polls, there is no apparent rule in force that limits inclusion to polls bigger than 1,000. I suggest if you're that passionate about it, you start a general debate on the appropriate page about sample sizes of BPC polls, get consensus and if you succeed, then come back here. But trying to somehow come up with a new standard on a sub-page as it were seems altogether the wrong place, cause even if you managed to get your point accepted here, there's a LOT of other pages with UK polls on it, that's a lot of endless debating and edit warring, energey better spent on a general policy page, no? Akerbeltz (talk) 18:12, 7 August 2022 (UTC)
Once again: Techne have stated that their poll is representative of Scotland. They are the professional pollsters/statisticians, not us. I believe we have no business setting an arbitrary limit of 1000 on sample sizes, and we haven't before. I'm well aware of the arithmetic on error bars, and we show sample sizes so people can view the results in context.
This situation is not symmetrical. Repressing a poll is a serious act of censorship, and is wide open to accusations of bias. Putting it in lets readers make up their own mind.
I'd be happy to exclude a poll if we had unanimity among the (sane) editors that a poll was seriously flawed. I think that is a simple statement of what the general protocol should be. I can't think of another way of preventing manipulation of the page.
Right now there is actually a strong majority in favour of including the Techne poll, and so I intend to do whatever I can to keep it in.
Yes, I have a settled view on these matters. So does the vast majority of Scotland. I can't imagine that the editors of this page self-select for people who don't care.RERTwiki (talk) 10:17, 8 August 2022 (UTC)
- And once again any poll can be 'representative' the question is how representative do we want the polls to be to allow easy comparison for the reader. The issue here is the standard that has been established and maintained on this article for many many years is in fact the status quo. It is for you to put forward the case as to why this change is required and now.
- There certainly does appear to be a majority for including this poll although the reasons for doing so are not clear, nor has any coherent rationale been made for including it or other small sample/subsets.
- Your wee rant about censorship is just that, maintaining a long established standard is not the same as censorship, not by a long chalk. Soosider3 (talk) 13:41, 8 August 2022 (UTC)
- Again, the status quo prior to your revert was that a poll with a sample of 626 was acceptable for 6 years, and a poll of 519 was acceptable for 3 months, and as others have stated this is standard across the industry and across wikipedia. AlloDoon (talk) 23:09, 8 August 2022 (UTC)
- I need to correct you on that interpretation, the 'status quo' was for 8 years and over 240 polls only those of 1000 were published on the main table, the one exception could more easily be viewed as an 'anomaly' that slipped through in which case the correct solution is to remove the anomaly not use it as a template to introduce other anomalies.
- The standard across the industry in UK political polling is 1000 as demonstrated by the other Wikipeadia pages that adhere to that standard, there is not one single sub 1000 poll included anywhere on these, not one. So why lower the standard for Polling on Scottish Independence
- https://en.wikipedia.org/wiki/Opinion_polling_for_the_2021_Senedd_election
- https://en.wikipedia.org/wiki/Opinion_polling_for_the_2019_United_Kingdom_general_election
- https://en.wikipedia.org/wiki/Opinion_polling_for_the_2021_Scottish_Parliament_election
- https://en.wikipedia.org/wiki/Opinion_polling_for_the_next_Scottish_Parliament_election Soosider3 (talk) 10:06, 9 August 2022 (UTC)
- I agree with Alldoon. The consensus here is arguing for no change. RERTwiki (talk) 08:00, 9 August 2022 (UTC)
- Again, the status quo prior to your revert was that a poll with a sample of 626 was acceptable for 6 years, and a poll of 519 was acceptable for 3 months, and as others have stated this is standard across the industry and across wikipedia. AlloDoon (talk) 23:09, 8 August 2022 (UTC)
I note there is a lot of talk about subsets not being valid as per the BPC FAQs which states one should be ‘very suspicious’ over these types of interpretations. However, clearly Techne, by over sampling, have enough respondents to make nationally representative sample of Scotland and it’s not just over interpretation of a ‘sub sample’. Reading through the BPC FAQs I also note that the same page also states that ‘care must be taken with polls conducted by pressure groups’ and ‘even if the poll is conducted by a reputable organisation it may be that it only touches on one aspect of the issue and is thus at best partial, if not actually biased’. As far as I can tell no one is arguing to remove Wings over Scotland or Scotland in Union sponsored polls so again unless there is evidence to the contrary from a reliable third party the Techne poll should be included. Dunk the Lunk (talk) 06:13, 9 August 2022 (UTC)
- To my mind the issue is nothing to do with partiality or bias, the issue is maintaining the status quo which was 1000 sample as being the norm until recently. There are over 240 polls on the main table only one of which has significantly less than 1000 as its sample. I would argue that what appears as an anomaly should be treated as such removed and status quo maintained. Soosider3 (talk) 10:10, 9 August 2022 (UTC)
- ‘Normal’ yes. A ‘requirement’ no. As the BPC say, quite clearly, there is no ‘minimum’ requirement. Just because a particular standard has become typical does not mean other methodologies are more or less valid. Many polling companies frequently conduct political polls using significantly less than 1000 people (eg first Google search here: https://yougov.co.uk/topics/politics/articles-reports/2022/07/21/liz-truss-holds-24-point-lead-over-rishi-sunak-amo ) which are widely reported in the press. The table in the article is clearly a ‘data’ table and should include all valid data, whether or not we like it, or think the poll is accurate. Dunk the Lunk (talk) 16:27, 9 August 2022 (UTC)
- A standard can be achieved in many ways, BPC has over the years established a standard for National UK polling of 1000, it has been established following poor performances from some polling companies especially those that used significantly less than 1000 sample. this has been the standard used on this article for over 8 years. So status quo has been well established and maintained. Its not about 'requirements' 'rules' its is about standards. To try to make a direct link between an internal party election and national polling makes no sense. 81.96.97.29 (talk) 14:23, 10 August 2022 (UTC)
- Standard, normal, typical. You can apply all those adjectives. However, as is explicitly stated on the BPC it is not a requirement as the YouGov/These Islands poll from March clearly illustrates. Techne (whose reputation is on the line) are confident enough to have published it, thereby opening themselves to critical assessment by the BPC and others. If there is third party evidence that the poll has been heavily criticised as inaccurate I am more than happy that it is excluded. But at the moment no one has provided this and we simply have a situation of one editor claiming that polls under 1000 are somehow ineligible for inclusion. As far as I can see every other editor thinks they should be included. I also note that the same editor has variously claimed the poll should be excluded because Techne aren’t member of the BPC (disproven by a simple check of the BPC website!) and because the BPC advises against reading to much into subsamples of polls (showing a fundamental misunderstanding of what this poll actually is). Finally, given that the poll shows very similar results to the YouGov/The Times poll of 1100 a month earlier and the March YouGov/The Island poll of 520 it’s not actually showing anything significantly different from these earlier polls (also note the two YouGov polls have very different sample sizes but have both basically produced the same result-status quo). Dunk the Lunk (talk) 15:59, 10 August 2022 (UTC)
- Well you fair drew in many aspects that are not directly concerning the issue in hand, neither is trying to pull in matters already resolved, I am pleased that Techne have updated their webpage (at last) to reflect their BPC membership.
- It is not for us to be questioning the reputation of any polling company, rather our task is surely to maintain a standard, as best we can.
- We do seem to be broadly in agreement that we stick to BPC pollsters.
- That is the key to the issue a long established standard across BPC political polling across the UK has been long reflected in this article and especially in the main table, that is a fact that is evidenced by simply looking at the table. Their is a single exception, one poll out of over 200 in an 8 year period, again that can not be disputed, it is there on screen. The move to want to include smaller polls is a very recent phenomenon. It basically boils down to seeing that one poll as either a precedent or an anomaly, I choose to see it as the later. As I believe doing so maintains an easier and more consistent table, by all means record these odd polls elsewhere in the article and if over time we see this as a new norm emerging then we should reconsider, but lets wait and build a body of evidence before acting. What we don't want to get into is interminable discussions about weightings, confidence levels or confidence Intervals etc 81.96.97.29 (talk) 15:41, 12 August 2022 (UTC)
- Absolutely not. Excluding any valid polls makes the job of editing this page one of exercising discretion, which is extremely onerous in context of a very active political controversy. The only tractable thing to do is include all valid polls. I've said above that if there is unanimity to exclude a poll, I would (implicitly, I guess!) go along with that, as a way to deal with unforseen circumstances. But just deciding to exclude valid polls makes the page highly susceptible to accusations of bias. RERTwiki (talk) 16:42, 14 August 2022 (UTC)
- Standard, normal, typical. You can apply all those adjectives. However, as is explicitly stated on the BPC it is not a requirement as the YouGov/These Islands poll from March clearly illustrates. Techne (whose reputation is on the line) are confident enough to have published it, thereby opening themselves to critical assessment by the BPC and others. If there is third party evidence that the poll has been heavily criticised as inaccurate I am more than happy that it is excluded. But at the moment no one has provided this and we simply have a situation of one editor claiming that polls under 1000 are somehow ineligible for inclusion. As far as I can see every other editor thinks they should be included. I also note that the same editor has variously claimed the poll should be excluded because Techne aren’t member of the BPC (disproven by a simple check of the BPC website!) and because the BPC advises against reading to much into subsamples of polls (showing a fundamental misunderstanding of what this poll actually is). Finally, given that the poll shows very similar results to the YouGov/The Times poll of 1100 a month earlier and the March YouGov/The Island poll of 520 it’s not actually showing anything significantly different from these earlier polls (also note the two YouGov polls have very different sample sizes but have both basically produced the same result-status quo). Dunk the Lunk (talk) 15:59, 10 August 2022 (UTC)
- A standard can be achieved in many ways, BPC has over the years established a standard for National UK polling of 1000, it has been established following poor performances from some polling companies especially those that used significantly less than 1000 sample. this has been the standard used on this article for over 8 years. So status quo has been well established and maintained. Its not about 'requirements' 'rules' its is about standards. To try to make a direct link between an internal party election and national polling makes no sense. 81.96.97.29 (talk) 14:23, 10 August 2022 (UTC)
- ‘Normal’ yes. A ‘requirement’ no. As the BPC say, quite clearly, there is no ‘minimum’ requirement. Just because a particular standard has become typical does not mean other methodologies are more or less valid. Many polling companies frequently conduct political polls using significantly less than 1000 people (eg first Google search here: https://yougov.co.uk/topics/politics/articles-reports/2022/07/21/liz-truss-holds-24-point-lead-over-rishi-sunak-amo ) which are widely reported in the press. The table in the article is clearly a ‘data’ table and should include all valid data, whether or not we like it, or think the poll is accurate. Dunk the Lunk (talk) 16:27, 9 August 2022 (UTC)
Request for Comment: Inclusion of Techne UK Poll 29/30 June 2022
Following on from the above discussion, I am requesting comment on whether this poll should be included in the article. AlloDoon (talk) 15:34, 12 July 2022 (UTC)
- That is a classic strawman argument, no one suggested for a moment that we emulate Whatscotlandthinks.org it is a shame you seem to have expended so much time and effort constructing this strawman. I don't appreciate you misrepresenting me in that manner. What actually happened was that discussion between 2 editors were beginning to circle and we looked at a third party view to try to give us some perspective to break the circle.
- Interestingly you miss out the key reasons for not including the report in Scottish polling
- 1. It is a subset of a UK wide poll (the UK poll is fine and should be included in that section of this article)
- 2. Every other poll on the Wikipedia article has a sample size of 1000
- 3. I am not aware of any subset included in this Wikipedia article, so why introduce that change now
- 3. Smaller sample inevitably leads to a higher margin of error, that is why BPC has an industry standard across the UK that a 1000 is the norm for polling at a national level. This also allows for a higher confidence level when comparing polling results. Every other poll in this article has a margin of error of less than 3%
- 4. The scale and magnitude of this proposed change in practice cant be covered adequately in Notes, that's fine for comparable small matters but not for such a radically different record.
- 5. The risk of unforeseen consequences in my opinion is high, we would be allowing a precedent that allows subsets and sample sizes of only 500, once you let that genie out the bottle you will not be able to get it back in.
- The correct place for this poll is in the UK section and there should be no place in main article for subsets or sample sizes less than 1000, just a sit has been doing for several years Soosider3 (talk) 21:06, 12 July 2022 (UTC)
I'm a strong supporter of the idea that we should not include data in this table which is a Scottish subset of a wider poll. The reason for this is that there is no gaurauntee that the Scottish subsample is structured to be representative of Scotland, rather than contributing to an overall structure which is representative of, for example, the UK: we can't get regular Scottish party preferences from the slices of UK data.
However, that's not the case for the Techne poll. Reading the notes on Methodology above in this talk, supplied by JMorrison, it is very clear that the poll is sampled for Scotland and is intended to poll the independence question. Indeed, it would be unbelievably dumb to even ask the independence question if that was not being done.
As regards sample size, I can't find with a quick scan any indication on the BPC website that there is a lower limit on sample size. Indeed there is a statement that uncertainty must be described, which seems to indicate that variable sample sizes are expected. Could you provide any reference supporting the idea that a sample size of 1000 is required? There are many polls published elsewhere, see for example Real Clear Politics, with lower sample sizes.
There is a good reason why the Wikipedia table reports sample sizes: to enable the readers to understand the error bars and determine the significance of a poll result. I am very much in favour of transparency, and allowing the public to see all the data. The alternative is essentially editorialisation by the Wikipedia Editors. In this of all topics, where passions are very high on all sides, any editorialisation, particularly exclusion of basic data, is almost impossible without violating the NPOV policy.
Please can we include this poll, which was intended and structured to answer the independence question, provide links to all the relevant data and methods, and leave the public to judge its significance.RERTwiki (talk) 08:53, 15 July 2022 (UTC)
- FYI there are two Panelbase polls in the table with less than 1000 in the sample, which have been there for some time.RERTwiki (talk) 09:02, 15 July 2022 (UTC)
- Thanks for that, hadn't noticed them before as my trawl of hyperlinks to datasets hasn't got back so far yet.
- My view would be that the Panelbase one of 25/26 June 2016 should be removed as sample size only 626, the other one dated 30/10 to 5/11 2014 should remain as sample size is 982. I also note that out of all this entries on this table there are only 2 under 1000, and one of those only just, also that it is over 6 years old. So perhaps not such a big issue but think we should try to maintain as reasonable and consistent approach as possible. Suggest "rule of thumb" no subsets unless over 1000 and margin of error is consistent with a full poll, and no polls significantly under 1000. I think all the other wee variations re 16/17 years polled, BPC members etc can be highlighted in Notes. Soosider3 (talk) 12:38, 15 July 2022 (UTC)
- Disagree with that, we should reinstate the Techne poll. This is starting to seem like cherry picking. Again, no issue in over 5 years since that poll was released. The general concensus seems to be to have the Techne poll, at least based on @Jmorrison230582:'s earlier comments, myself, RERTwiki and the two users who also reverted your edits. AlloDoon (talk) 00:03, 16 July 2022 (UTC)
- and again with the language. Again this is not a poll its a SUBSET of a poll, again perhaps teh reason for no issue in 5 years is because people have held the line and established a standard. Again why this poll, why now. As for consensus we will see Soosider3 (talk) 05:39, 16 July 2022 (UTC)
- I will point out that seems to be the consensus at the moment which is also based on the convention for using polls on this page. Still open to further discussion in the meantime, however I have reinstated the poll. As I said, we could explore having separate sections for polls which have margins of error over 3%/non-standard referendum questions or changing what polls should be included on the page. I am in favour of the status quo at the moment, that is any valid British Polling Council poll within a 95% confidence interval weighted for the population of Scotland. AlloDoon (talk) 00:07, 16 July 2022 (UTC)
- Adding to the above point, perhaps it is worth adding in a sentence to explain qualifying polls for the article? AlloDoon (talk) 00:08, 16 July 2022 (UTC)
- Not sure about consensus however as you raise convention on this page 1000+ sample has been teh convention for several years and you seem happy enough to ignore that to include a subset, why? Soosider3 (talk) 05:41, 16 July 2022 (UTC)
- For the third time, the poll's methodology notes explicitly state the poll is a representative and weighted sample of the population of Scotland, ie. not a subsample. AlloDoon (talk) 10:30, 16 July 2022 (UTC)
- Jeezo, for the umpteenth time its a subset, the weighting etc does not matter, it remains a subset. Full poll is 1652 of a sample this is 501. The tables clearly show that, there is no other explanation, none, zilch, zero. Its a subset that's pining for the fjords. Additionally at 501 it is about half the size of the polls in this table
- Both these are factual and easily checked
- Just because you want this does not make it sit with established convention on this article or even established practice across the polling industry. So I have several years of convention and the BPC polling standards to support my view. Soosider3 (talk) 11:01, 17 July 2022 (UTC)
- For the third time, the poll's methodology notes explicitly state the poll is a representative and weighted sample of the population of Scotland, ie. not a subsample. AlloDoon (talk) 10:30, 16 July 2022 (UTC)
- Not sure about consensus however as you raise convention on this page 1000+ sample has been teh convention for several years and you seem happy enough to ignore that to include a subset, why? Soosider3 (talk) 05:41, 16 July 2022 (UTC)
- Disagree with that, we should reinstate the Techne poll. This is starting to seem like cherry picking. Again, no issue in over 5 years since that poll was released. The general concensus seems to be to have the Techne poll, at least based on @Jmorrison230582:'s earlier comments, myself, RERTwiki and the two users who also reverted your edits. AlloDoon (talk) 00:03, 16 July 2022 (UTC)
Hi Folks. Sorry to have been away from the fun, been away from keyboard, as they say.
There is wording above to state that Techne Oversampled by 374 in Scotland, raising the sample above that of a subset poll. They have also said that the poll is representative of Scotland. It is intended to answer the Independence question for Scotland. They actually asked the question in two different ways, both 'independent country' and 'remain-leave'. That seems to be why the separate poll sizes are low. It also indicates that independence was very much the focus of the poll.
I think it's completely wrong to set an arbitrary cut-off for sample sizes to be included in the table. Apart from raising objections of cherry-picking, we have no authority or expertise to do that. People come here for data, and can make their own minds up about sample size.
This is a highly sensitive subject, and any hint of poorly motivated exclusion of data is wide open to accusations of violating NPOV.
In all honesty I don't see much support for Soosider's position. I think there is consensus that the poll should be reinstated (though the sample size is small...).
Anyone know the best way to proceed in this sort of circumstance?(RERTwiki (talk) 12:59, 25 July 2022 (UTC))
- @RERTwiki: the next step would be posting on either Wikipedia:Reliable sources/Noticeboard or Wikipedia:WikiProject Elections and Referendums asking for outside opinions. Happy to do so if dispute continues. AlloDoon (talk) 13:09, 25 July 2022 (UTC)
- Thanks: seems like a plan. I'm a pair of hands if you need them. RERTwiki (talk) 15:56, 26 July 2022 (UTC)
I would agree that asking for input and view from reliable sources and projects on elections and referendums appears sensible way to progress. Am happy to do anything I can to support this endeavour
Soosider3 (talk) 18:09, 26 July 2022 (UTC)
I'm responding to the RSN RfC here, because it appears the question was not whether the poll is an RS or the agency was reliable, but whether this poll can be used in a particular way. Reading over the entire range of discussions I am going to give an honest recommendation. All of you, some especially more than others, need to brush up on how opinion polling actually works, when done properly (as a reliable agency does). Let me address some particular points brought up repeatedly:
- "It is a subset" – and as has been pointed out, it was weighted properly for that population in extracting that data.
- "a sample size of 1000" – per the article, and per every poll ever, sample sizes in the 500-1500 range are about standard. Gallup is highly unusual in going to 2000.
- "higher margin of error" – The dramatizing of MoE above (it was a 43% increase in MoE!) was entertaining. Doubling the sample size to reduce the MoE by one point is silly if for the same price you could do two regular polls on different dates. If you want a more accurate opinion survey, look for rigorous pollsters and polling aggregators, not higher sample sizes.
- "cant be covered adequately in Notes" – It's an article about polling, so why do details about polling have to only be covered in the notes?
In general to keep in mind: 1) use poll aggregates when possible; 2) use reputable pollsters or those who have had their methodologies reviewed, and don't worry about sample sizes unless the pollster's report says to worry about it; 3) never report table subgroup results unless you know exactly what you are doing and what the pollster was doing... actually, it's SYNTH to do so anyway. SamuelRiv (talk) 04:53, 28 July 2022 (UTC)
- Thanks for the input, I believe the issue is around 2 polls both of which in their own way have used an unusual approach, ending up with sample sizes that are well below the established practice on this article, practice that has been consistent for many years.
- You mention that a poll can be between 500 and 1500, yet across political polling at a national level 1000 has been established as being the norm, this link to British Polling Council may be of use https://www.britishpollingcouncil.org/faqs-by-members-of-the-public/ "There is no, “minimum”, sample size for a poll which is acceptable, but around one thousand has become the established norm for a nationwide opinion poll in Great Britain."
- I would welcome your thoughts Soosider3 (talk) 14:05, 29 July 2022 (UTC)
- If after reading that FAQ your honest takeaway is that polls should have a sample size >=1000 to be considered reliable for a WP article, then I don't know what to tell you. Scientific American's explanation includes an example where Gallup re-weighted a subsample of 400 respondents (sound familiar???). Gallup's typical national poll sample size is 1000 btw. StackX primer if you know any math. AAPOR's best practices don't even give a suggested sample size range or minimum. It is a pretty basic idea that you'd rather have a representative (or weight-able, a "probability sample") small sample size than a large sample of unknown representation ("non-probability sample"), and that two small surveys are better than one big survey. What else in the methodology do you find "unusual" besides sample size? SamuelRiv (talk) 14:46, 29 July 2022 (UTC)
- That would be the bit where it stated "...but around one thousand has become the established norm for a nationwide opinion poll in Great Britain" seems very clear to me, established norm etc etc
- If the 1000 sample size stands then the others fall away. After all its only the BPCs standard Soosider3 (talk) 15:05, 29 July 2022 (UTC)
- "Around 1000" is in agreement with 500-1500 quoted elsewhere. If you think the range is too wide, then on what mathematical basis is that the case? Specific examples of undersampling in national-level polls are given in some of the sources I link, and they are of <100 people, which typically can happen when you look at cross-tabs (which is part of why you should never quote them). This particular poll surveyed about 1000 people and also published a re-weighted subsample poll of 500 people, which is entirely along the lines of what Gallup has done (cited above), along with every other polling company, and is within a reasonable sample size for a large population. You have cited nothing so far other than apparently your vague interpretation of the word "around". To illustrate how words mean different things in different contexts: if I say to "aim around the target area" that has a very different quantification than if I said "aim around the bullseye". SamuelRiv (talk) 15:56, 29 July 2022 (UTC)
- I have only 'cited' the BPC who are the organisation that sets the benchmark for Political Polling in the UK, I have cited precedent and established practice across a wide range of Political Polling in the UK. Have just checked the BPC website and Gallup do not appear to be members, if that is so perhaps that explains their different approach. Soosider3 (talk) 17:12, 29 July 2022 (UTC)
- Please define what you would consider adequately within the bounds of "around 1000" for polls, for both a sample and a subsample. SamuelRiv (talk) 17:33, 29 July 2022 (UTC)
- Suggest you take that up with BPC after all it is their standard. Eyeballing sites such as wikipedia and other political polling sites across UK would suggest that standard is well maintained and it rarely drops below 1000 and then only by a handful. Soosider3 (talk) 18:22, 29 July 2022 (UTC)
- So you have no idea other than "I know it when I see it", and that 500 is too low, even though I have cited examples of prestigious national polls doing exactly that, and even though you have in cases above demonstrated a clear lack of understanding of basic concepts of how polling actually works (concepts such as MoE -- do you even understand why your calculation of 43% was so silly?). SamuelRiv (talk) 19:30, 29 July 2022 (UTC)
- That is rather a condescending tone to take and adds little to the discussion.I also note your failure to respond to BPC standard instead you choose to deflect away from it.
- The only data in your 'cited' is to an article from 2004 and the small poll you alluded to provided a 5% MoE, hardly a shining example.
- I come back to the key issue, with BPC saying sample 1000, with the Polling Industry in the UK complying to that standard for National Political Polling and with WP (for UK National Polling) following that standard, the Question is where do these unusual polls belong, Yes I recognise they are polls however not of the standard that the main table has come to use, by all means record them on a different table but lets not lower the standard. Soosider3 (talk) 05:50, 30 July 2022 (UTC)
- So you have no idea other than "I know it when I see it", and that 500 is too low, even though I have cited examples of prestigious national polls doing exactly that, and even though you have in cases above demonstrated a clear lack of understanding of basic concepts of how polling actually works (concepts such as MoE -- do you even understand why your calculation of 43% was so silly?). SamuelRiv (talk) 19:30, 29 July 2022 (UTC)
- Suggest you take that up with BPC after all it is their standard. Eyeballing sites such as wikipedia and other political polling sites across UK would suggest that standard is well maintained and it rarely drops below 1000 and then only by a handful. Soosider3 (talk) 18:22, 29 July 2022 (UTC)
- Please define what you would consider adequately within the bounds of "around 1000" for polls, for both a sample and a subsample. SamuelRiv (talk) 17:33, 29 July 2022 (UTC)
- I have only 'cited' the BPC who are the organisation that sets the benchmark for Political Polling in the UK, I have cited precedent and established practice across a wide range of Political Polling in the UK. Have just checked the BPC website and Gallup do not appear to be members, if that is so perhaps that explains their different approach. Soosider3 (talk) 17:12, 29 July 2022 (UTC)
- "Around 1000" is in agreement with 500-1500 quoted elsewhere. If you think the range is too wide, then on what mathematical basis is that the case? Specific examples of undersampling in national-level polls are given in some of the sources I link, and they are of <100 people, which typically can happen when you look at cross-tabs (which is part of why you should never quote them). This particular poll surveyed about 1000 people and also published a re-weighted subsample poll of 500 people, which is entirely along the lines of what Gallup has done (cited above), along with every other polling company, and is within a reasonable sample size for a large population. You have cited nothing so far other than apparently your vague interpretation of the word "around". To illustrate how words mean different things in different contexts: if I say to "aim around the target area" that has a very different quantification than if I said "aim around the bullseye". SamuelRiv (talk) 15:56, 29 July 2022 (UTC)
- If after reading that FAQ your honest takeaway is that polls should have a sample size >=1000 to be considered reliable for a WP article, then I don't know what to tell you. Scientific American's explanation includes an example where Gallup re-weighted a subsample of 400 respondents (sound familiar???). Gallup's typical national poll sample size is 1000 btw. StackX primer if you know any math. AAPOR's best practices don't even give a suggested sample size range or minimum. It is a pretty basic idea that you'd rather have a representative (or weight-able, a "probability sample") small sample size than a large sample of unknown representation ("non-probability sample"), and that two small surveys are better than one big survey. What else in the methodology do you find "unusual" besides sample size? SamuelRiv (talk) 14:46, 29 July 2022 (UTC)
Having read through the discussion I think the poll should be included in the tables.
- 1) Techne UK are members of the BPC.
- 2) No one has identified that the BPC (or another reliable source) state sample sizes of this size should be ignored.
- 3) The small poll size is clearly identified in the sample size column.
- 4) The larger margin of error is clearly identified in the notes column.
- 5) Given 1-4 the poll is a reliable source, albeit with a larger margin of error.
- 6) The page is called Opinion polling on Scottish independence. It is not Opinion polling on Scottish Independence that meet X and Y criteria. While it is appropriate to only include reliable sources this is a reliable source.
- However
- 1) it may be worth adding a margin of error column to the table as well.
- 2) or it may be appropriate to move polls with non-standard questions and/or higher margins of error to separate table.
- 3) the graphical chart in this page says it only includes standard questions. It may be appropriate to exclude this poll and state it also only includes polls with sample sizes above X value. Dunk the Lunk (talk) 15:52, 1 August 2022 (UTC)
Protocol for Inclusion/Exclusion of Polls, Techne
Folks - I believe that that the only way this Wikipedia Entry can be insulated from accusations of political bias is to be as inclusive as possible.
In other words, we should include any sensible poll unless we have unanimous agreement that a poll is fatally flawed. The only inevitably fatal flaw is that a poll is not known to be representative of Scotland. From the above discussion, there is room to argue that a poll of under 500 respondents is fatally flawed.
But if we don't have unanimity for exclusion, and someone is arguing coherently and rationally that a poll merits inclusion, then it should be included.
This being Wikipedia, there must be a caveat that we seek unanimity among the sane, but that is a longer story.
I think I can make a very strong case that Ipsos MORI polls are systematically biased to a higher showing pro-independence (and lower "don't know"). Is the right thing for me to do to just remove the polls and refuse to be convinced by arguments for inclusion? Should I just keep removing them in the face of a large majority in favour of their inclusion?
Obviously not. But at some level that's what is happening with the Techne poll. I think the poll should be included, I think there is a good case for it, and I think that should be enough for any poll. That's true even if some people don't like the poll.
If we go the other way, and allow committed individuals to censor the list despite the wishes of (in this case) the majority of editors, we are wide open to abuse and to accusations of being 'fake news' adapted to a 'narrative'.
My 2 cents. RERTwiki (talk) 11:17, 5 August 2022 (UTC)
- Interesting use of language, it is not conducive to constructive dialogue and the wee petty barbs do you no credit
- You bring in points that are not relevant to the ongoing discussion it is fairly well understood that IpsosMori methodology may produce a slightly higher recording of the Yes side just as Yougov does the same for the No side, however this is not the issue here and it is disingenuous of you to try to introduce this as a coherent point. In fact it rather shows where your own thinking is coming from.
- The matter that has been under consideration is whether to change long established practice, about sample size and the use of subsets, that for the main table the BPC standard has been applied for many years and has produced a sensible and easily understood table for users.
- I am still waiting to hear a coherent rationale for why lowering the standard somehow enhances the Article, be very clear that the proposal will lower the standard.
- Any poll carried out can be "representative" if weighted however the further it gets below 1000 the lower the Confidence Level and the Higher the Confidence Interval, that is why 1000 is established by BPC and why this article until very recently only listed polls of 1000.
- Suggest having a wee look at this it may help to clear your thinking https://www.surveysystem.com/sscalc.htm Soosider3 (talk) 13:47, 5 August 2022 (UTC)
- Soosider there has been a poll on the article with a subsample of 626 for 6 years and there clear consensus in this talk page is that these polls are acceptable on the article.
- Polls that should be included are British Polling Council polls which are weighted to representative for the population of Scotland. AlloDoon (talk) 08:26, 6 August 2022 (UTC)
- 1 poll out of 224 six years ago is not a sound basis to set a protocol on, it happened 184 polls ago, every Poll recorded here since has been consistent and compliant with the BPC norm of 1000 as does every other poll going back to 2014.
- The more logical option would be, rather than using an anomaly as a template, remove it as not being consistent with standard that has evolved and been used for 8 years.
- A simpler protocol would be any poll that is over 1000 and shows appropriate methodology. That maintains the standard and consistency of this particular table, by all means record it on another table but lets not mess up 8 years of good consistent work.
- I am well aware that I seem to be the lone voice advocating for this point of view but that does not make what is happening right, appropriate, consistent nor logical. Rather it feels like some people trying very hard to squeeze this poll in for reasons that are neither clear or sensible. Soosider3 (talk) 09:01, 6 August 2022 (UTC)
- I had not realised the YouGov/These Islands poll was also only removed very recently, and therefore was on the article with a sample size of 519 for 3 months. Both of these polls were also added to the article by other users and not removed despite a number of poll additions to the page... On your latter point about "trying very hard to squeeze this poll in for reasons that are neither clear or sensible", I would advise you to avoid personal attacks and avoid misrepresenting other peoples views and focus on the central argument. See also - WP:TALK. AlloDoon (talk) 09:11, 6 August 2022 (UTC)
- My expression - inclusion are neither clear or sensible is exactly that, my views and I believe I evidence that above, it is not reasonable to see them as personal attacks
- Lets try, as you say to stick to the central argument.
- I think I rather concisely summarise my point of view above and would welcome any thoughts you may have on the subject in hand.
- The idea of a protocol is an interesting one, given the very clear evidence in this main table it seems clear that a protocol had been operating for consistently for several years, polls over 1000 etc etc. My question remains why change it? and why now? By all means record these 'odd' polls but elsewhere in article. Soosider3 (talk) 10:09, 6 August 2022 (UTC)
- I had not realised the YouGov/These Islands poll was also only removed very recently, and therefore was on the article with a sample size of 519 for 3 months. Both of these polls were also added to the article by other users and not removed despite a number of poll additions to the page... On your latter point about "trying very hard to squeeze this poll in for reasons that are neither clear or sensible", I would advise you to avoid personal attacks and avoid misrepresenting other peoples views and focus on the central argument. See also - WP:TALK. AlloDoon (talk) 09:11, 6 August 2022 (UTC)
Looking over the rim of this particular teacup for a moment, if you look across Wikipedia, there are numerous pages which regularly include polls with a sample smaller than 1,000 in whatever the 'main' table is. Even if that includes regular outliers in some cases, for instance Tecnè polls for Italian GEs regularly put FI much higher than other pollsters (very obvious in the line of dots in the graph hovering around 10% whereas the median is more like 8%. I haven't asked them but given how many people participate in that page, I can only assume that they are a) by credible pollsters and b) that their take on these smaller/outlier polls is that it averages out overall without misleading the reader. If that's acceptable for that and other GE pages, I don't see why we need to be holier than thou on this page, especially if the 'holier' approach seems to be advocated by a single editor. Akerbeltz (talk) 10:21, 7 August 2022 (UTC)
- That gave me a Rye smile, I would remind you we are talking about standards as apply in the UK, most UK political poling record on wikipeadia complies to BPC standard as has this article for many years. I would suggest that Confidence Interval you refer to actually reinforces the reason for maintaining the BPC standard. Your sincerely - A lone Voice. Soosider3 (talk) 12:04, 7 August 2022 (UTC)
- A wry smile perhaps, rye has nothing to do with it :b And I disagree. The BPC may apply to UK polls, sure but with the amount of bickering that goes on on Wikipedia, I have little doubt that at least for bigger countries, some sort of standard is applied that says which polls can be included and which can't. In any case it always comes back to the point that apart from the general guidelines around sticking to BPC polls for UK polls, there is no apparent rule in force that limits inclusion to polls bigger than 1,000. I suggest if you're that passionate about it, you start a general debate on the appropriate page about sample sizes of BPC polls, get consensus and if you succeed, then come back here. But trying to somehow come up with a new standard on a sub-page as it were seems altogether the wrong place, cause even if you managed to get your point accepted here, there's a LOT of other pages with UK polls on it, that's a lot of endless debating and edit warring, energey better spent on a general policy page, no? Akerbeltz (talk) 18:12, 7 August 2022 (UTC)
- The rest of wikipedia is not my concern, there is clear precedent in this article that 1000 is the expected sample size, practice that has been established and maintained over many years, echoing the BPC standard. Why change that now? So far you have not given a coherent rationale other than a wee snide remark about 'holier' If you have a rationale please share it. Soosider3 (talk) 06:43, 8 August 2022 (UTC)
- Perhaps you need to spend a little bit more time on other Wiki pages, rather than just the one, before you come out with statements like that. Changes in editorial policy affect more than one page and you can't debate such issues in isolation. But this is getting repetitive. It's you who wants to change the status quo, so it's on you to convice others, not the other way round. Until then, I suspect you'll end up finding yourself reverted ad nauseum. Akerbeltz (talk) 08:59, 8 August 2022 (UTC)
- My point is that the status quo is samples of 1000 as evidenced by the article itself, if you wish to change the status quo it is for you to win that particular argument. Soosider3 (talk) 13:32, 8 August 2022 (UTC)
- Perhaps you need to spend a little bit more time on other Wiki pages, rather than just the one, before you come out with statements like that. Changes in editorial policy affect more than one page and you can't debate such issues in isolation. But this is getting repetitive. It's you who wants to change the status quo, so it's on you to convice others, not the other way round. Until then, I suspect you'll end up finding yourself reverted ad nauseum. Akerbeltz (talk) 08:59, 8 August 2022 (UTC)
- The rest of wikipedia is not my concern, there is clear precedent in this article that 1000 is the expected sample size, practice that has been established and maintained over many years, echoing the BPC standard. Why change that now? So far you have not given a coherent rationale other than a wee snide remark about 'holier' If you have a rationale please share it. Soosider3 (talk) 06:43, 8 August 2022 (UTC)
- A wry smile perhaps, rye has nothing to do with it :b And I disagree. The BPC may apply to UK polls, sure but with the amount of bickering that goes on on Wikipedia, I have little doubt that at least for bigger countries, some sort of standard is applied that says which polls can be included and which can't. In any case it always comes back to the point that apart from the general guidelines around sticking to BPC polls for UK polls, there is no apparent rule in force that limits inclusion to polls bigger than 1,000. I suggest if you're that passionate about it, you start a general debate on the appropriate page about sample sizes of BPC polls, get consensus and if you succeed, then come back here. But trying to somehow come up with a new standard on a sub-page as it were seems altogether the wrong place, cause even if you managed to get your point accepted here, there's a LOT of other pages with UK polls on it, that's a lot of endless debating and edit warring, energey better spent on a general policy page, no? Akerbeltz (talk) 18:12, 7 August 2022 (UTC)
Once again: Techne have stated that their poll is representative of Scotland. They are the professional pollsters/statisticians, not us. I believe we have no business setting an arbitrary limit of 1000 on sample sizes, and we haven't before. I'm well aware of the arithmetic on error bars, and we show sample sizes so people can view the results in context.
This situation is not symmetrical. Repressing a poll is a serious act of censorship, and is wide open to accusations of bias. Putting it in lets readers make up their own mind.
I'd be happy to exclude a poll if we had unanimity among the (sane) editors that a poll was seriously flawed. I think that is a simple statement of what the general protocol should be. I can't think of another way of preventing manipulation of the page.
Right now there is actually a strong majority in favour of including the Techne poll, and so I intend to do whatever I can to keep it in.
Yes, I have a settled view on these matters. So does the vast majority of Scotland. I can't imagine that the editors of this page self-select for people who don't care.RERTwiki (talk) 10:17, 8 August 2022 (UTC)
- And once again any poll can be 'representative' the question is how representative do we want the polls to be to allow easy comparison for the reader. The issue here is the standard that has been established and maintained on this article for many many years is in fact the status quo. It is for you to put forward the case as to why this change is required and now.
- There certainly does appear to be a majority for including this poll although the reasons for doing so are not clear, nor has any coherent rationale been made for including it or other small sample/subsets.
- Your wee rant about censorship is just that, maintaining a long established standard is not the same as censorship, not by a long chalk. Soosider3 (talk) 13:41, 8 August 2022 (UTC)
- Again, the status quo prior to your revert was that a poll with a sample of 626 was acceptable for 6 years, and a poll of 519 was acceptable for 3 months, and as others have stated this is standard across the industry and across wikipedia. AlloDoon (talk) 23:09, 8 August 2022 (UTC)
- I need to correct you on that interpretation, the 'status quo' was for 8 years and over 240 polls only those of 1000 were published on the main table, the one exception could more easily be viewed as an 'anomaly' that slipped through in which case the correct solution is to remove the anomaly not use it as a template to introduce other anomalies.
- The standard across the industry in UK political polling is 1000 as demonstrated by the other Wikipeadia pages that adhere to that standard, there is not one single sub 1000 poll included anywhere on these, not one. So why lower the standard for Polling on Scottish Independence
- https://en.wikipedia.org/wiki/Opinion_polling_for_the_2021_Senedd_election
- https://en.wikipedia.org/wiki/Opinion_polling_for_the_2019_United_Kingdom_general_election
- https://en.wikipedia.org/wiki/Opinion_polling_for_the_2021_Scottish_Parliament_election
- https://en.wikipedia.org/wiki/Opinion_polling_for_the_next_Scottish_Parliament_election Soosider3 (talk) 10:06, 9 August 2022 (UTC)
- I agree with Alldoon. The consensus here is arguing for no change. RERTwiki (talk) 08:00, 9 August 2022 (UTC)
- Again, the status quo prior to your revert was that a poll with a sample of 626 was acceptable for 6 years, and a poll of 519 was acceptable for 3 months, and as others have stated this is standard across the industry and across wikipedia. AlloDoon (talk) 23:09, 8 August 2022 (UTC)
I note there is a lot of talk about subsets not being valid as per the BPC FAQs which states one should be ‘very suspicious’ over these types of interpretations. However, clearly Techne, by over sampling, have enough respondents to make nationally representative sample of Scotland and it’s not just over interpretation of a ‘sub sample’. Reading through the BPC FAQs I also note that the same page also states that ‘care must be taken with polls conducted by pressure groups’ and ‘even if the poll is conducted by a reputable organisation it may be that it only touches on one aspect of the issue and is thus at best partial, if not actually biased’. As far as I can tell no one is arguing to remove Wings over Scotland or Scotland in Union sponsored polls so again unless there is evidence to the contrary from a reliable third party the Techne poll should be included. Dunk the Lunk (talk) 06:13, 9 August 2022 (UTC)
- To my mind the issue is nothing to do with partiality or bias, the issue is maintaining the status quo which was 1000 sample as being the norm until recently. There are over 240 polls on the main table only one of which has significantly less than 1000 as its sample. I would argue that what appears as an anomaly should be treated as such removed and status quo maintained. Soosider3 (talk) 10:10, 9 August 2022 (UTC)
- ‘Normal’ yes. A ‘requirement’ no. As the BPC say, quite clearly, there is no ‘minimum’ requirement. Just because a particular standard has become typical does not mean other methodologies are more or less valid. Many polling companies frequently conduct political polls using significantly less than 1000 people (eg first Google search here: https://yougov.co.uk/topics/politics/articles-reports/2022/07/21/liz-truss-holds-24-point-lead-over-rishi-sunak-amo ) which are widely reported in the press. The table in the article is clearly a ‘data’ table and should include all valid data, whether or not we like it, or think the poll is accurate. Dunk the Lunk (talk) 16:27, 9 August 2022 (UTC)
- A standard can be achieved in many ways, BPC has over the years established a standard for National UK polling of 1000, it has been established following poor performances from some polling companies especially those that used significantly less than 1000 sample. this has been the standard used on this article for over 8 years. So status quo has been well established and maintained. Its not about 'requirements' 'rules' its is about standards. To try to make a direct link between an internal party election and national polling makes no sense. 81.96.97.29 (talk) 14:23, 10 August 2022 (UTC)
- Standard, normal, typical. You can apply all those adjectives. However, as is explicitly stated on the BPC it is not a requirement as the YouGov/These Islands poll from March clearly illustrates. Techne (whose reputation is on the line) are confident enough to have published it, thereby opening themselves to critical assessment by the BPC and others. If there is third party evidence that the poll has been heavily criticised as inaccurate I am more than happy that it is excluded. But at the moment no one has provided this and we simply have a situation of one editor claiming that polls under 1000 are somehow ineligible for inclusion. As far as I can see every other editor thinks they should be included. I also note that the same editor has variously claimed the poll should be excluded because Techne aren’t member of the BPC (disproven by a simple check of the BPC website!) and because the BPC advises against reading to much into subsamples of polls (showing a fundamental misunderstanding of what this poll actually is). Finally, given that the poll shows very similar results to the YouGov/The Times poll of 1100 a month earlier and the March YouGov/The Island poll of 520 it’s not actually showing anything significantly different from these earlier polls (also note the two YouGov polls have very different sample sizes but have both basically produced the same result-status quo). Dunk the Lunk (talk) 15:59, 10 August 2022 (UTC)
- Well you fair drew in many aspects that are not directly concerning the issue in hand, neither is trying to pull in matters already resolved, I am pleased that Techne have updated their webpage (at last) to reflect their BPC membership.
- It is not for us to be questioning the reputation of any polling company, rather our task is surely to maintain a standard, as best we can.
- We do seem to be broadly in agreement that we stick to BPC pollsters.
- That is the key to the issue a long established standard across BPC political polling across the UK has been long reflected in this article and especially in the main table, that is a fact that is evidenced by simply looking at the table. Their is a single exception, one poll out of over 200 in an 8 year period, again that can not be disputed, it is there on screen. The move to want to include smaller polls is a very recent phenomenon. It basically boils down to seeing that one poll as either a precedent or an anomaly, I choose to see it as the later. As I believe doing so maintains an easier and more consistent table, by all means record these odd polls elsewhere in the article and if over time we see this as a new norm emerging then we should reconsider, but lets wait and build a body of evidence before acting. What we don't want to get into is interminable discussions about weightings, confidence levels or confidence Intervals etc 81.96.97.29 (talk) 15:41, 12 August 2022 (UTC)
- Absolutely not. Excluding any valid polls makes the job of editing this page one of exercising discretion, which is extremely onerous in context of a very active political controversy. The only tractable thing to do is include all valid polls. I've said above that if there is unanimity to exclude a poll, I would (implicitly, I guess!) go along with that, as a way to deal with unforseen circumstances. But just deciding to exclude valid polls makes the page highly susceptible to accusations of bias. RERTwiki (talk) 16:42, 14 August 2022 (UTC)
- Standard, normal, typical. You can apply all those adjectives. However, as is explicitly stated on the BPC it is not a requirement as the YouGov/These Islands poll from March clearly illustrates. Techne (whose reputation is on the line) are confident enough to have published it, thereby opening themselves to critical assessment by the BPC and others. If there is third party evidence that the poll has been heavily criticised as inaccurate I am more than happy that it is excluded. But at the moment no one has provided this and we simply have a situation of one editor claiming that polls under 1000 are somehow ineligible for inclusion. As far as I can see every other editor thinks they should be included. I also note that the same editor has variously claimed the poll should be excluded because Techne aren’t member of the BPC (disproven by a simple check of the BPC website!) and because the BPC advises against reading to much into subsamples of polls (showing a fundamental misunderstanding of what this poll actually is). Finally, given that the poll shows very similar results to the YouGov/The Times poll of 1100 a month earlier and the March YouGov/The Island poll of 520 it’s not actually showing anything significantly different from these earlier polls (also note the two YouGov polls have very different sample sizes but have both basically produced the same result-status quo). Dunk the Lunk (talk) 15:59, 10 August 2022 (UTC)
- A standard can be achieved in many ways, BPC has over the years established a standard for National UK polling of 1000, it has been established following poor performances from some polling companies especially those that used significantly less than 1000 sample. this has been the standard used on this article for over 8 years. So status quo has been well established and maintained. Its not about 'requirements' 'rules' its is about standards. To try to make a direct link between an internal party election and national polling makes no sense. 81.96.97.29 (talk) 14:23, 10 August 2022 (UTC)
- ‘Normal’ yes. A ‘requirement’ no. As the BPC say, quite clearly, there is no ‘minimum’ requirement. Just because a particular standard has become typical does not mean other methodologies are more or less valid. Many polling companies frequently conduct political polls using significantly less than 1000 people (eg first Google search here: https://yougov.co.uk/topics/politics/articles-reports/2022/07/21/liz-truss-holds-24-point-lead-over-rishi-sunak-amo ) which are widely reported in the press. The table in the article is clearly a ‘data’ table and should include all valid data, whether or not we like it, or think the poll is accurate. Dunk the Lunk (talk) 16:27, 9 August 2022 (UTC)