User:Hhhippo/Flow/TOC and filters

The following is a concept concerning the handling of archiving and tables of contents in the context of Flow. Note that these are just ideas of a single editor and by no means official plans of the WMF or wishes the community consents on.

The main features of this concept are:

A way to view flow boards as if one of our current archiving bots was active on them, showing only active topics by default, with older ones accessible in an archive.
A table of contents which can be used to easily browse the currently shown topics, as well as the archived ones.
An optional alternative to infinite scrolling: each viewer can decide to view Flow topics as a long list with infinite scrolling, or to browse through shorter lists that can be loaded as a whole.

To achieve these goals, two main components are suggested: Topic filters and Dynamic TOCs. The implementation can be made highly flexible and configurable, allowing to explore the new possibilities that come with Flow far beyond the original goals.

Topic filters

Topic filters can be applied to Flow boards to put restrictions on which topics are shown. There will be a default filter, but others can be selected by each viewer, and new ones defined on several levels within a project without changes to the underlying software.

Possible examples for topic filters are:

Active topics: Basically what we have now on talk pages with automated archiving: Show only topics that had their last contribution less than n days ago, but show at least m topics. Additonally, each topic could have an archive and a sticky bit, which can be set to manually override these rules. This could be the general default filter.

Archived topics: All non-active topics. (Or more generally: "Other topics" – all topics not shown with the currently applied filter.)

Open topics: Show the topics that are not excplicitly marked as closed, no matter how old they are. This is the same as active topics, but with n=infinite and m=0, and using the archive bit for closing. This could be a useful default for pages like XfD or ANI. Setting the archive bit could be combined with adding a closing comment.

Updated topics: Show only topics that had some kind of contribution after my last visit. This obviously works only if the board or the topic is on my watchlist.

Watched topics: Show the topics that are on my watchlist (themselves, not via the board), no matter if there is something unread.

New topics: Show only topics that have been created after my last visit (requires watching the board).

All topics: What it says on the box.

Combine filters: For example "show watched and new topics". The filters to be combined might refer to different boards, or even to different projects.

Excplicit topic list: Manually list a number of topics. A use case could be: a user creates a filter listing some topics and posts a link that others can follow to see these topics. Again, the topics don't have to live on the same board or on the same wiki. (This is probably not needed since it duplicates the functionality of a feed.)

Filter definition

Topic filters are defined by specifying a number of parameters, e.g.:

Filter name
Maximum age of a topic (wrt. creation time or last change)
Minimum number of topics to show
Respect archive bit (don't show topics that have it set)?
Respect sticky bit (show all topics that have it set)?
Show only watched topics?
Show only updated topics (new/edited comments since last visit)?
Show only new topics (created since last visit)?
List of other filters to include
List of topics to include explicitly
How to subdivide, if needed (see below): chronologically, alphabetically, ...
Sort options: newest or oldest first, measure age from topic creation or from last change, sort alphabetically (for processes like ANI, RFC, AfD,...)

Filters can be defined per project (using a page in the Mediawiki namespace), per user preference (using a page in user space) and per board (stored with the board). At each of these stages, the list of available filters can be appended or replaced. One filter can be declared the default (applied when loading the board without specifying a filter). Filters can be declared hidden (not shown on each board, but can still be applied using parameters in the URL). Archive/sticky bits are stored with each topic and can be set by anyone (at least technically, policies might say otherwise).

Dynamic TOC

The dynamic TOC serves as a table of contents for the topics currently shown, and at the same time as a tool for navigating all topics subscribed by the board.

Example structure (with a layout similar to the TOCs we have now):

+ Active topics
    Bla
    Foo
    Bar
+ Archived topics
    + 2014
        abc
        xyz
    + 2013
        gnus
        gnats

Features

Each filter name is a link, clicking it applies the filter and shows the corresponding topics in the main part of the page. The name of the currently applied filter is not a link, but boldfaced, just as with wikilinks to the current page.
Each topic title is a link. Clicking it applies the corresponding filter (or subdivision) and scrolls to the chosen topic.
The little "+" or triangle (same as for enhanced RC/watchlist) serves for expanding/collapsing the list of topics for a filter or subdivision in the TOC, without applying the filter. Expanding one list collapses all others.
All filters that are defined for this board as viewed by this user, and are not marked as hidden, will be shown in the TOC. However, no matter the filter configuration, there must always be a filter "all topics" or "other topics" to ensure that all topics on the board are accessible.

Subdivisions

If topic lists get very long (in terms of number of topics, comments or total amount of data), subdivisions will appear automagically (per year, month, other time spans, whatever results in reasonable list sizes). A filter can define if subdivisions should be chronologic or alphabetic. If needed, several levels of subdivisions will be used.
Subdivison titles can be clicked and act like their parent filters, but with additional restrictions applied e.g. regarding the age of the topics shown. Filters which are subdivided can still be applied as a whole, resulting in a large amount of topics shown using infinite scrolling.
If the TOC itself is already too large to be loaded in one shot, apply lazy loading: first load only what's to be shown immetiately. While the user is looking at that, preload anything that can be reached with one click (the next level of expansion). Update each time a subdivision or filter is expanded or collapsed, including dropping no longer needed data from memory (maybe depending on available memory).
If a board is too large to run a database query building the TOC on each page view, use a cached TOC. The TOC would then be rebuilt on each edit to the board rather than on each view. Existing aggregated data could be re-used where possible rather than building the TOC from scratch. Cached TOCs wouldn't work for user-defined filters, those could be disabled on large boards, or only built on excplicit request, with long loading time to be expected.

Infinite scrolling

If the topics shown by the applied filter exceed a certain amount of data, infinite scrolling is used for the main view (as is done with the unfiltered boards we have now). However, it is the reader's choice to use this view: each filter will get enough subdivisions to allow browsing it without need for infinite scrolling.
If a long list of topics is shown in the main view (by e.g. applying a whole filter, ignoring existing subdivisions), the TOC entries for the applied broad filter will still be subdivided. The TOC should then follow the scrolling in the main view, giving some indication which topics are currently shown on the screen and/or loaded into the browser.
The progress in scrolling could also be reflected in an updated URL, see WT:Flow/Design FAQ#More thoughts on infinite scrolling. To make this work, there should be separate URL schemes for "show all results from filter x, scroll down to subdivision y" and for "show only results from subdivision y of filter x". Both could be combined with "scoll to topic z".
The subdivisions of the TOC could also be shown in the main view as section titles (but this makes only sense if normal text isn't already shown in a title-like font size).

Positioning

The TOC should be positioned in the great white open to the right of the main part of the board. Ideally it should be floating (staying on the screen when scolling the main part), but that will need some mechanism to scroll within the TOC in case it doesn't fit on the screen. Alternatively, the TOC could be shown to the left of the main topic view, similar to the pattern used by file explorers or e-mail clients. This will however move the main conversations outside of center stage.

Bells and whistles

Some additional features one could dream of:

Mark updated topics by bold face in the TOC. The same for filters or subdivisions that contain updated topics. (This is somewhat colliding with "show current filter in bold" though.)
Mark topics currently shown in the main part by italic titles in the TOC.
Add buttons for jumping between updated topics.
Add a search box at the bottom of the TOC.
Making this accessible via screen-readers might be a nightmare. One could introduce a user preference that switches to an optically more clumsy version of the TOC that's easy to use with screen readers (similar for absence of javascript).
Instead of 'since my last visit', allow the viewer to set a threshold time that defines what's an 'updated topic'.
Show the number of topics with TOC entries for filters and subdivisions. Maybe also other info like number of new/updated/open topics, last edit etc. Take care not to overload the TOC, use e.g. tooltips, show/hide buttons, configurable details.