Discovery/Status updates/2019-04-08

This is the weekly update for the week starting 2019-04-08

Discussions

Search

• New WikibaseCirrusSearch code is deployed and enabled on all wikis, including Wikidata & Commons. [1]
• We re-indexed all Greek, Turkish, and Irish-language wikis with better handling of Greek accents [2], Turkish İ/ı [3], and Irish initial mutation [4] when lowercasing searches in quotes. [5]
• We noticed that indices can move to readonly state when disk space is low and needs to be reset manually once disk space is available again, so we made it a cookbook [6]
• David wrote a patch to help with an exception from CirrusSearch/Sanity/Checker (cannot fetch ids from index) [7]
• We fixed/finalized more things with the ES6 upgrade:
  • Create checks that alerts on cirrussearch update lags [8]
  • Elasticsearch 6: the classic similarity is deprecated [9]
  • Setup elasticsearch on cloudelastic100[1-4] [10]
  • Rack/setup/install cloudelastic100[1-4].eqiad.wmnet systems [11]
• Erik converted mjolnir from KafkaRDD to direct kafka-python usage, because KafkaRDD in pyspark only supports the 0.8 api [12]

Wikidata Query Service

• Deployed patch to fix an issue in Blazegraph workbench [13]
• Working on improving cache behavior of Updater [14]

--

• View all open tickets related to Discovery.
• Looking to get involved? See tasks marked as Easy or volunteer needed