Wikimedia Search Platform/Decision Records/Recommendation Flags in Search

What is the problem or opportunity?
For the needs of a Structured Tasks project, we need to be able to filter out articles that have recommendations assigned to them. Currently we allow for two types of recommendations, that are calculated beforehand and assigned to articles - link and image. In both cases, results are calculated periodically.

What does the future look like if this is achieved?
Ability to filter the results will allow a flexible display of articles that can be combined with other criteria - like full-text search or category.

What happens if we do nothing?
Static list of articles can still be displayed without the help - but that will greatly reduce the usability of the solution.

Options
Only one option written down - decision was made during a meeting

Current solution
Currently, we support article topics and drafts, both coming from ORES. Current structure example: "ores_articletopic": [ "Culture.Media.RadioIn the first tag, "Culture.Media.Radio" is a value of a tag, 475 is a term frequency value used as a weigh to sort the values in search. We currently put also draft data there, which isn't ideal. This is a technical debt, we need to resolve before streamlining platform solution

Desired solution
We want to have a field that behaves the same way, but is designed for more general data. Proposed format is "/|". Here's a more detailed example: "weighted_tags": [ "classification.ores.articletopic/Culture.Media.RadioThis structure allows us to reuse preexisting features

Existing tags from ORES will be migrated to this field.

Migration
Recommendation features are currently under development and can leverage the new structure immediately, but ORES classifications need to be migrated. Steps required:


 * 1) Implement recommendation features with new structure in mind
 * 2) Develop handling of the new structure alongside old one with BC code in CirrusSearch (search both old and new fields).
 * 3) Reindex articles to add the new field in the elasticsearch mapping
 * 4) Repopulate ores articletopics and drafttopics for all the articles (see  )
 * 5) Remove BC code from CirrusSearch
 * 6) Reindex the elasticsearch indices to remove the old fields (using the   options)