User talk:DBSand/LR parser/Rewrite

Article entirely rewritten on May 8 2012

The goal of this major revision was to make LR parsers better understood by programmers who are users of LR parser generators, and give an intuitive understanding of the generator's work and parser's work without going into the mathematical details and formalisms of a parsing theory textbook. Where possible, the parser's actions are explained in terms of the basic shift-reduce framework, or by showing a concrete realistic (but small) example. The progression from concrete actions, to tables, to state generation, to theory, will likely feel backwards to some. I hope this ordering is helpful to beginners. The example input and example grammar are bigger than before. None of the existing commentary based on the old examples was retained as is. The prolog implementation was similarly dropped.

I welcome feedback from previous editors. I wonder if the old and new articles should both remain visible for awhile?

The above talk notes refer to the preceding versions of the article. DBSand (talk) 06:00, 9 May 2012 (UTC)

DBSand, thank you for your contribution. It looks like you've spent a lot of time researching this topic and have presented an in-depth example for the Wikipedia community and the general public. I have not read the article in its entirety, but it is clear that the primary example is substantially more intricate than the original "1+1" grammar.

Having said that, I am not sure that the Wikipedia article "LR Parser" is the correct resting place for this fine work. Please consider the possibility that there may be other areas of Wikipedia itself and the Wikipedia support structure including Wikibooks that may be a more appropriate recipient of your work.

The reason I bring this up is that it seems that the new article is rather more in-depth than the previous one and attempts to educate the reader. This is a purpose more suited to Wikibooks than Wikipedia. I personally have used this very Wikipedia page to implement a SLR parser on two occasions, and for this the "1+1" grammar works well. The sample grammar on the new page seems more appropriate for a researcher or student who wants to gain a deeper understanding of the theory of LR parsers. In my opinion, both "cookbook engineering" and "in-depth study" are topics outside the scope of Wikipedia, but it is more or less the nature of Wikipedia to attract and retain information that is well outside the scope of Wikipedia...

Let me come out and offer my critical remarks on the new version: it is of less use to an implementer such as myself than the original one due to the increased size and complexity of the example. However, I am entirely confident that there are some out there who found the original example to be so simple as to be useless for understanding the theory of LR parsers, and the new example substantially more informative.

My first concern is to retain the original content of the article somehow. There are a variety of possibilities such as reverting the article, merging the old article with the new one, and using a system of sub-pages to let the reader decide.

Awaiting remarks. 173.239.78.54 (talk) 01:17, 12 May 2012 (UTC)

I looked briefly at Wikibooks, but that seems to be for 100+-page textbooks rather than for 10-page introductions to single topics.

Wikipedia is for education at several levels. Wikipedia itself needs, IMO, (1) a quick gentle introduction to the major kinds of parsers now used in practice (mostly, recursive descent, and table-driven LALR(1), and operator precedence) without scaring away the reader with the formalisms of automata theory; (2) A gentle introduction to LALR(1) parsing sufficient for new users of existing generators; (3) An introduction to LR generators for the curious, and to motivate how to deal with conflicts on new grammars; and finally (4) a Wikipedia equivalent to a week of theoretical college course material on how to re-implement existing parsers & generators. I aimed to help 1,2, and 3.

The tutorial section and the generator-magic section could be factored out to other articles. But there is some gain in using a single example and terminology throughout both.

A good thing about the small scale of the 1+1 example, was that it was possible that the very curious would read and confirm every detail of the charts. That's unfortunately not likely with the new larger example. But the larger example does allow seeing by eyeball, how information flows into the parse stack and gets chunked there. That's essential for understanding the big picture. And the larger example shows interesting things happening, that never happened for 1+1. I initially had an even larger example, and also looked at something in between 1+1 and the current A=B+C*2 example.

I don't know the Wikipedia etiquette for handling large changes. But the advice pages said "go bold", not to hide the new stuff where it wouldn't be seen. My regret is in making the former article invisible and inaccessible to non-editors.

In the two SLR parsers you implemented, was that two grammars with an existing generator and existing parser skeleton? That's exactly the target audience I wanted to help.

A fun use of Wikibooks, would be the easily-executed python source code for an actual LALR(1) generator such as Ply, instrumented to explain what it is doing.

DBSand (talk) 06:14, 12 May 2012 (UTC)

(173.239.78.54 here, I just created the account with the memorable name Throwaway8493) Ok, so it sounds like you want to keep the content of the new version in Wikipedia, which is fine, but basically you have deleted a bunch of content that a lot of people have spent a lot of time on, content that was still evolving. We aren't really in a position where we can say that one version is better than another because there are many different people, and some will prefer the new while others will prefer the old. This is because the old one has some content the new one lacks. I would suggest merging the content (without cluttering it or sacrificing good organization) in a way that allows the article to do the greatest service to the greatest number of readers.

Fortunately we have some tools to handle the situation. I have two ideas, the slow process-oriented one and the quick-and-dirty one. Here is the process-oriented version:

1. Try to maintain continuity with the original article and avoid confusion. This is a process that seeks to incorporate feedback from editors before replacing the main article with the rewrite. This can be done by temporarily moving the new version to LR_parser/Rewrite, reverting the main article to the last version prior to your edits, and sticking a "This article is being rewritten at LR_parser/Rewrite" template at the top of the article. When the new article has stabilized and there is general agreement that the new article is better, the main article replaced with the rewrite. This gives editors warning that the rewrite is underway.

2. Find some way to stick the content of the old version in the new version. This could involve the creation of, say, LR_parser/Sample_code for the prolog code, or other sub-pages such as LR_parser/Examples. Or it may be simpler to keep everything on the main page. Basically we don't want to delete content. Moving the content around is fine, we just want to retain the 1+1 example somehow, which accounted for most of the content of the original article. Simple is good, so appending the original article to the new one and removing the redundant "Theory" section so there aren't two identical ones would be perfectly acceptable for a first draft.

3. This does not have to be perfect, and the article will surely go through many more iterations in the future. Many people will have ideas about how to deal with the examples, how to serve readers who don't care about the examples or source code, etc... I would say keep the rewrite in /Rewrite for at least a few days. If it is getting a lot of edits, it can stay there, but I wouldn't keep /Rewrite around for a long time, a month is way too long I think.

Anyway, that is the idea. Alternative quick-and-dirty approach is to just append the original article to the main article, delete the duplicate "Theory" section, forget about the process, and start hacking away.Throwaway8493 (talk) 08:32, 12 May 2012 (UTC)

I've added a link to the former article. I prefer to keep the new article visible & up front awhile, to get immediate feedback, especially about errors and major omissions. This weekend, I plan to reduce the size of the examples by removing the Assign stuff. If former editors are around now, I'd like to get work with them on creating a joint article that best serves the readers' needs.DBSand (talk) 18:39, 12 May 2012 (UTC)

I removed Assign from the example. This shrank the over-wide LR parse steps chart and made the other charts and diagrams a little smaller. I'm now looking at whether the grammar can be further flattened without losing its need for LR(1) lookahead.

Revising the parse tree diagram and the various charts is tediously hard. Whatever examples are used, they are likely to lock-down the explanatory content of the article for years, in the same way the original article did.DBSand (talk) 16:01, 13 May 2012 (UTC)

Forks Discouraged

Within minutes of my creating a separate article LR parser/Rewrite in the main Wiki namespace, two Wikipedia admins moved it into a personal namespace User:DBSand. And no links from articles in the main namespace to personal namespaces will persist longer than a day or two. W. has defense mechanisms against what it perceives as permanent forking of articles with similar topic coverage. Don't know how to proceed. I don't want to lock up the original article and prevent it from getting incremental changes as part of a slow merge. DBSand (talk) 19:49, 13 May 2012 (UTC)

I've left you a note on your talk page: maybe moving it to a subpage of the article's talk page is the best solution. Writ Keeper ⚇♔ 19:57, 13 May 2012 (UTC)

One option suggested by admins, is to stick the new article underneath talk:LR parse in main space. But the new article would remain invisible to readers and search engine.

A second option, suggested twice by Throwaway8493, is to append the old article to the end of the new article for awhile, until all merge issues are resolved. Is slightly ugly in short run, but gives equal treatment for readers and search engine, to the original and new content, and shows the need for further merge work in a very clear way. This method involves less Wikipedia magic than the first option. Preserves all history, all collaborations. Avoids untimely interference from random admins unfamiliar with this topic. Sounds good!

DBSand (talk) 01:30, 14 May 2012 (UTC) — Preceding unsigned comment added by DBSand (talk • contribs)