User:JDrewniak (WMF)/notes/Search widgets at Wikimedia

This page contains technical research on the possible improvements to the search on desktop completed by JDrewniak (WMF)

Search widgets at Wikimedia
There are currently (As of October 2019) 6 different search typeahead implementations at the Wikimedia Foundation.


 * 1) searchSuggest
 * 2) OOUI TitleWidget & TitleOptionWidget
 * 3) MobileFrontend SearchOverlay
 * 4) www.wikipedia.org search widget
 * 5) Wikidata typeahead for item statements
 * 6) Wikidata typeahead for site search

How do they differ?
All widgets contain the same basic functionality: Producing a list of search suggestions when typing a query into a search input. On top of that:


 * MobileFrontend search widget adds a watchstar beside each suggestion, but it's the only one that lacks support for keyboard navigation.
 * SearchSuggest is the oldest and most bare-bones, as it doesn’t include thumbnails in the suggestions.
 * The OOUI-based TitleOptionWidget is versatile in that it has the option to render with or without the thumbnail.
 * The portal implementation is the smallest and most self-contained, requiring only 200 lines of JS (including comments).

The major differences between the implementations are in the underlying frameworks used to build them. All widgets except the portal depend on jQuery and mw.Api. The mobile widget depends on the Mustache templating library, and the OOUI widget depends on OOUI.

Performance
These search widgets, on both mobile and desktop, are loaded on every page. The idea of replacing the old SearchSuggest widget has been around for a while (see https://phabricator.wikimedia.org/T125725) and a new specialized OOUI widget was even created https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/255436/, but the major blocker for deploying the OOUI based widget was the need to load OOUI on every page (~100kb).

On mobile, the biggest dependency outside of jQuery is the Mustache templating library. However, Mustache is only an 8kb library and is used for many other widgets on mobile, such as the main menu.

Design
Having 4 different implementations of the same features inevitably leads to design inconsistency. Those concerns have been expressed in this task: https://phabricator.wikimedia.org/T153417.

Instrumentation
There exists 3 search widget related eventLogging schemas: Schema:Search and Schema:MobileWebSearch (which is based on Schema:Search) and Schema:SearchSatisfaction. Schema:Search is the oldest of the three and has been marked deprecated. Instrumentation code for Schema:MobileWebSearch still exists in the MobileFrontend code base in MobileWebSearchLogger.js and is still functional (though buggy). Schema:MobileWebSearch is therefore tied to the mobile search widget.

Schema:SearchSatisfaction is used by the Search Platform team measure the efficacy of search algorithms. Its methodology is described in this research document. It is currently active and uses data collected from the SearchSuggest widget on desktop. The searchSuggest widget emits events with  that are registered with the SearchSatisfaction schema. Since this schema is active, any new or revised desktop search widget should remain compatible with the searchSatisfaction schema and emit the same events.

The feasibility of porting the mobile implementation to desktop
Because of the underlying framework dependencies, neither implementation can be ported from Vector to Minerva & vice versa without incurring a performance penalty. Porting the OOUI widget to mobile would mean loading the OOUI library on every page (~100kb) and porting the mobile implementation to desktop would mean loading Mustache on every page (~8kb). If the Mustache dependency could be dropped from the mobile implementation, porting it over to desktop could be feasible. We would however, still have to add keyboard navigation support to the mobile widget as well as compatibility with the searchSatisfaction schema.

The feasibility of creating a new widget
The search suggestion feature is well suited for newer Javascript frameworks that rely on unidirectional data-flow and explicit state management. A search widget is fairly self-contained in that it doesn't interact or depend on other UI elements on page. With a React-like framework, the core behaviour of a search typeahead can be modelled as 1.) responding to input events 2.) Making a network request 3.) Updating a template automatically based on the network response.

This behaviour can now be achieved much easily than in the past. Using a Codepen with existing HTML/CSS from the design task above, I was able to create a typeahead widget in a few hours using the Preact framework: https://codepen.io/j4n/pen/PooqWQLq