Manual:Searchindex table/Redesign notes

Current problems

 * Separate "title" and "text" search sucks.
 * Ordering of results poor
 * No clear spot for expanded metadata:
 * Category membership (as in MWSearch/Lucene)
 * Media file metadata?
 * LiquidThreads threading data

More or less resolved issues

 * MySQL's inconsistent Unicode support
 * worked around with horrid expansion to ASCII
 * MySQL's short-words limit (4+ chars required by default)
 * worked around by expanding short words when we index :P
 * MySQL's stopword list
 * partially worked around with the short-words expansion, but longer words on stopword list not yet treated

Solution for ordering
MySQL's boolean search doesn't offer ranking info, but the *non*-boolean does. o_O It's possible to order by the non-boolean ranking on your boolean results... which might or might not be useful.

Solution for separate fields and metadata

 * Merge title and text info into a single field
 * and metadata...?
 * Use a special prefix on words, tacked into the index?
 * index content: "vincent vanu00 gogh wasu00 au00 painter. category:painters"
 * search query: "+category:painters +vanu00 +gogh"
 * index content: "myu00 deletion proposal isu00 that weu00 should delete more things! thread:7483"
 * search query:"+thread:7483 +deletion +proposal"