Flow/Functional Specifications/Search and Filter

This document describes a set of functional requirements regarding local and site search for Flow.

This document should not be taken as a final descriptor for any one specific release of the software, though recommendations regarding inclusion in the "Minimum Viable Product".

Basics
Searching in Flow can be broken down into two basic types:


 * Board Search - these searches happen only on the contextual Board (or Feed). They do not search outside of that area.
 * Site Search - these searches happen across all Boards (but not Feeds, as Feeds are already conglomerations from multiple Boards).

Board Search and Filter
It is perhaps best to think of a Board search as applying a set of "filters" to the content. Board searching shall operate on the following:


 * Topic title
 * Topic tags
 * Post authors
 * Post content
 * Scratchpad content

Control Behavior
The search control bar shall dock itself at the top of the user's viewport (unless Javascript is disabled). It shall always be fully open.

Filter Keywords
By default, Board searches will run against all areas. However, the results can be filtered with finer grains by applying specific keywords:


 * Title - text searches title only
 * Author - only Post author user names
 * Tag - only Topic tags
 * Content - only the content of Posts
 * Scratchpad - only contents of scratchpads (term may change)

Filter keywords are applied cumulatively and are to be tokenized on split characters (",", ";") or new Filter keywords.

Token Behavior
By default, search tokens are to be split by whitespace except when:


 * Two or more tokens are surrounded by single or double quote characters
 * Two or more tokens are prefixed with a filter keyword (e.g., "Author:"), until the next lexical token break (, or ;) or the next filter keyword

Tokens are case-insensitive ("Foo" is the same as "foo" and "fOo").

Intersection
There are two general types of token intersection:


 * Cumulative intersection (equivalent to "OR" tokens) - all tokens are treated uniquely
 * Composite intersection (equivalent to "AND" tokens) - both tokens must have hits

By default, search intersection will be cumulative. That is, tokens will be treated as "OR" rather than "AND". Nested intersection (e.g., "This AND that OR (Foo AND Bar)") will not be supported.

Including composition intersection ("AND") will likely be beyond the scope of the MVP (in fact, writing a high-performance search system that allows nested intersections is likely beyond the scope of the Foundation's resources or desires).

Accordingly, we'll stay with cumulative intersection.

Stop Words
Each language will have to supply their own set of "stop words". Stop words will be removed from any search before it is performed (to reduce complexity). Common stop words (in English) include:


 * Articles (a, the)
 * Non-Specific pronouns (He, she, it, we, they)
 * Prepositions (in, on)
 * Non-used control tokens (or, and)

Search Examples
These are non-exhaustive examples, meant as illustration only.


 * Foo: returns all Topics where the text "Foo" is in the Topic title, a tag, an author name, or is included within a Post or scratchpad.
 * Foo Bar: returns all Topics where the text "Foo" or the text "Bar" is in the Topic title, a tag, an author name, or is included within a Post or scratchpad.
 * "Foo Bar": returns all Topics where the exact text "Foo Bar" is in the Topic title, a tag, an author name, or is included within a Post or scratchpad.
 * Author:Jorm : returns all Topics in which User:Jorm has posted. Does not return Topics where Jorm has only been mentioned.
 * Author:Jorm Author:Werdna : returns all Topics in which either User:Jorm or User:Werdna has posted. Does not return Topics where either Jorm or Werdna have been mentioned.

Results Display
The display of search and filter results should be intelligent and its behavior should modify itself accordingly.

When the returned results are large (say, greater than 5 total Topics, or 20 total Posts), Topics should always be displayed as collapsed.

Search result entries should, when collapsed, indicate in some way what element was triggered in the filter.

TODO: mockup

Search Highlighting
In results, search terms should be highlighted. Typically this will be a yellow background field for the term, though the color may change depending upon the background color of the element.

Keyboard Navigation
When search filters are active, certain keyboard controls should become relevant:


 * N - jumps to next instance of a highlighted term
 * P - jumps to previous instance of a highlighted term

Site Search
Site searching within the Flow space will occur from the "master" search control. This search should operate upon:


 * Topic title
 * Topic tags
 * Post content
 * Post authors

Advanced Filters
Site search within Flow should allow for the following advanced options:


 * Return results from all wikis (true global search)
 * Return results in specific languages (Flow knows the language of the wiki a Topic is housed on)

Limitations
It is unlikely that the site search system will allow for complicated or natural language filtering unless we can configure the site search to "hook out" to the native Flow search.

Site search results will also likely be restricted to the local wiki (mostly for sanity's sake), though it is emminently possible to remove this constraint (indeed, it is likely more difficult to constrain the search to the local wiki only).

Site search results will likely have to display as closed links that lead to a "single Topic" Flow Board, since the site search "bed" is not Flow-enabled (something for the future?).

General Complexities
Since content is returned in a lazy-loaded, infinite-scroll format, the browser's built-in "Ctrl" search function will be broken or severely limited. See "Keyboard Navigation," above.