MediaWiki r52338 - Code Review

Jump to: navigation, search
Repository:MediaWiki
Revision:r52337‎ | r52338 (on ViewVC)‎ | r52339 >
Date:02:27, 24 June 2009
Author:brion
Status:ok
Tags:
Comment:
* (bug 8445) Multiple-character search terms are now handled properly for Chinese

Big fixup for Chinese word breaks and variant conversions in the MySQL search backend...
- removed redunant variant terms for Chinese, which forces all search indexing to canonical zh-hans
- added parens to properly group variants for languages such as Serbian which do need them at search time
- added quotes to properly group multi-word terms coming out of stripForSearch, as for Chinese where we segment up the characters. This is based on Language::hasWordBreaks() check.
- also cleaned up LanguageZh_hans::stripForSearch() to just do segmentation and pass on the Unicode stripping to the base Language implementation, avoiding scary code duplication. Segmentation was already pulled up to LanguageZh, but was being run again at the second level. :P
- made a fix to Chinese word segmentation to handle the case where a Han character is followed by a Latin char or numeral; a space is now added after as well. Spaces are then normalized for prettiness.
Modified paths:

Diff [purge]

Loading diff…

Status & tagging log