Wikimedia Discovery/FAQ

=Work of the Discovery department=

What are the problems you are trying to solve?
People often use external search services to find Wikimedia and Wikipedia content.

The problem is that people tend to go back to to the external service for additional searches, even though they're already on Wikipedia.

The Discovery Department wants to create a better experience for the users of the sites by creating more accessible searching and discovery mechanisms than are presently available on our sites.

There is also an opportunity to explore surfacing information from sister projects to enhance the discovery of that knowledge for projects that have less visibility.

Lastly, the team is providing a foundation for product development that is data driven as well as user driven to iterate to the useful services and features our users desire and need.

What are you trying to understand?
The Discovery Department tracks four core metrics (also known as key performance indicators) for search:


 * 1) Zero results rate for search. If users receive no results, it means we've not been able to help find what they're looking for, so we measure the zero results rate.


 * 1) User engagement with search results. If users do not click on results, then we haven't given them the results they wanted.


 * 1) Search latency. The faster our search works, the better.


 * 1) API use. It's important that apps and third parties can search our site too.

You can see the full range of metrics that we track on the [http://discovery.wmflabs.org/metrics/ Discovery Department's search dashboard].

What happens afterwards?
The team and users will post ideas for deliberation, and will collectively come up with proposals.


 * rfc>Wikimedia Discovery/RFC|Wikimedia Discovery/RFC

How will this affect other products that the Wikimedia Foundation is developing?
The research carried out will help bring more understanding to search and discovery mechanisms across all platforms, and user flows from readers to editors and will inform decisions made on how to improve those mechanisms on desktop, mobile web, and mobile apps, as well as in specific products like VisualEditor.

We also are exploring API usage, best practices, mix of content from inter-wiki projects like Wiktionary, Wikivoyage, Wikimedia Commons and more, and utilization of open data sources like OpenStreetMap to expand contextual knowledge discovery.

We will, of course, be publishing our research, so that it may be read and taken into account by the broader movement and other interested parties.

What does your overall strategy look like ?



 * Year 0 - Look inward and improve the search experience across our projects
 * Year 1 - Look outward and see if we can incorporate new data streams and public curation models for relevance

What does year 0 include ?
We call year 0 Discovery because we are focused on learning and understanding user pathways and appreciation for other knowledge sources.
 * 1516-goals>Wikimedia Engineering/2015-16 Goals|2015-16 Goals

What does year 1 include ?
Potential ideas that we need your feedback on:


 * Identify pathways for the community to improve relevance via Wikidata
 * Actively highlight difficult to find knowledge and empower the ability to surface it in search, reading and editing flows
 * Research open sources of knowledge to continually strengthen the legitimacy of our content through curation by humans and machines

This feels like a huge long term project. Is it?
Our users interested in search request a lot of improvements: phab1>phab:T87632|inter wiki, phab2>phab:T104984|multi-lingual, media search phab3>phab:T95223|[1]phab2>phab:T104565|[2], improving UX, improving search relevance, and others. The Discovery Department aims to improve search in these areas, and that will take a lot of time! During this process, we will continually re-evaluate our plans on a quarterly and annual level to assess our impact and hold ourselves to the same standards as any other team at the Wikimedia Foundation.

How does this align with strategy?

 * Relevancy, accuracy and trustworthy ratings on index entities
 * Extended context to geospatial, temporal, multimedia and relational paths of knowledge
 * Display Inter-wiki projects (internal) and potentially open data sources
 * Mobile, voice, and modern consistent interface opportunity
 * Multiple-lingual and global respective experiences and results

See also: 

Is all your work open source?
Yes. All of our code is contained in public repositories, and falls under the copyright>Copyright|same licensing as MediaWiki. See this list of all the code repositories the department supports.

How do you know if we are succeeding for our users?

 * Qualitative
 * Surveys, mailing-list>mail:discovery|discovery@ mailing list, talk>Talk:Wikimedia Discovery|talk pages
 * Quantitative
 * http://discovery.wmflabs.org/

Will there be any element of human curation?

 * We'd like to explore this and need your help on our rfc>Wikimedia_Discovery/RFC#Public_Curation_of_Relevance|RFC to think through how to do it right.

I'd like to see a list of search results. Can the department provide this information?
This is a common question by editors and researchers alike. The idea of providing a list of queries where a page does not exist was researched and determined to be to difficult to accomplish with our resources. The biggest concerns were ensuring privacy, the difficulty in creating a usable list out of many junk queries, and the time it would take to create such a list would be costly.

I want to help! How can I get involved?
We'd love the input of anyone who wants to join us in building and improving search. Here's how you can do that:


 * Join our mailing-list>mail:discovery|mailing list
 * Post on our talk>Talk:Wikimedia Discovery|talk page
 * Review what we're discovery>Discovery</>|working on and phab>phab:tag/discovery</>|create tasks for us

= Knowledge Engine=

Note that there was a discussion with the community and previous Executive Director of the Wikimedia Foundation on this topic: ke-faq>M:Knowledge_Engine/FAQ</>|Knowledge Engine/FAQ

What is the Knowledge Engine?
"Knowledge Engine" was an early term used to describe a number of initiatives that related to search and discovery of content across Wikimedia projects. It was referenced by the Knight Foundation under ([<tvar|knight>http://www.knightfoundation.org/grants/201551260/</> "What we fund / Journalism / Knowledge Engine By Wikipedia"]), and stated: "To advance new models for finding information by supporting stage one development of the Knowledge Engine by Wikipedia, a system for discovering reliable and trustworthy public information on the Internet."

There were requests to publish the details of the grant and Lila Tretikov [<tvar|lila-talk>https://meta.wikimedia.org/w/index.php?title=User_talk:LilaTretikov_(WMF)&oldid=15294374#Knowledge_Engine_grant</> shared her thoughts and the grants activities and outcomes]. The Knight Foundation agreed to the publication of [<tvar|agreement>https://upload.wikimedia.org/wikipedia/foundation/a/a7/Knowledge_engine_grant_agreement.pdf</> the grant application] in February 2016.

As the concept evolved, it included a wide variety of ideas, many of which ended up being discarded. Although the original grant referred to "Knowledge Engine by Wikipedia" as "the Internet's first transparent search engine," the term "Knowledge Engine" was also used to easily refer to what the Discovery team was focusing on rather than a new project. The Wikimedia Foundation has stopped using the term as of Q3 FY15-16 as it caused confusion.

Regardless of what "Knowledge Engine" may have meant to different people at different times in the past, this page reflects the current thinking and plans, as understood by the Discovery department.

Are you building a search engine to compete with Google?
No, as we said when this FAQ was created in early November 2015, we are not creating a general internet search engine. The team has been tasked with improving a search function for Wikimedia sites.

In the sense that every site on the Internet competes for attention, and especially as a "go-to" resource for users, Wikipedia and its sister sites compete with Google and other search engines for certain kinds of inquiries. Namely, people who want to learn. Our visitors are people who want information and knowledge, and our mission is to serve them well. Our current search function does not serve our users well, so we are improving the existing wt-search>wikitech:Search</>|CirrusSearch infrastructure with better relevance, multi language, multi project search and incorporating new data sources (like maps>Maps</>|Maps) for our projects. We want a relevant and consistent experience for users across searches for both <tvar|wp-home>wikipedia.org</> and our project sites. Looking farther forward, we will explore including other sources of open knowledge. We remain fully committed to the movement's vision and values.

In the past some within the WMF talked about building a noncommercial search engine. The WMF thought about that direction but that did not go anywhere as an actual deliverable. By the the time this page was created the focus was on the search function of Wikimedia projects and [<tvar|knight-announcement>http://www.knightfoundation.org/press-room/press-release/wikimedia-foundation-explore-new-ways-search-and-d/</> the joint announcement with Knight] noted that the focus of initial research was improving the in-house function. That announcement also leaves open the possibility of broadening the reach of the program as it develops to include other publicly available sources of information. In February 2016 the management of the WMF [<tvar|blog-url>http://blog.wikimedia.org/2016/02/16/wikimedia-search-future/</> clarified] that the WMF is "not building a global crawler search engine."

Nobody at the WMF intended for the search function at WMF sites to do things to help you find things to buy, as Google does, and there is no evidence that the WMF wanted to do all the many, many things that Google/Alphabet does.

Could search results in Wikipedia include more information from its sister projects?
The Wikimedia movement's vision is to make the sum of all human knowledge freely available to everyone. Wikipedia is our largest and most well-known project, but there are many other projects like Wikimedia Commons and Wikidata that move us towards making our vision happen. These projects have millions of users every month! So, can we make a search system that's good and meets the needs of our users and show content from around our movement? We think people would use it. As a result it would bring more attention to the great work in projects across the movement by using one of the larger, more well-known places people visit.

What licenses will those new data sources be under?
This will need more discussion as we want to be able to conform to the standards and policies of the Wikimedia projects they would need to serve. Our first exploration was with phab-ticket>phab:T105090</>|OSM licensing and legal and we'll want to learn from that in any further work.

Does that mean we are looking to shift search traffic away from third parties?
We love all the [<tvar|ext-traffic>http://discovery.wmflabs.org/external/</> third party traffic] that we get and we hope that it increases over time.

What we want is to improve people's ability to find what they want, once they are here. Too many times, users experience this:


 * 1) Search on Google, Bing, etc
 * 2) Follow Wikipedia Link
 * 3) Read
 * 4) Leave and search Google, Bing, etc again because you are specifically looking for more information but couldn't find it using the on-wiki search (CirrusSearch).

We want people who come to the Wikipedia and WIkimedia projects to find the information they want, and the more effective we are at serving them, the more they will come and the more they will stay. This means we are better serving our mission.

In addition, the WMF survives on donations, and the more people come directly to us, and the more they find what they need here, the more likely they are to donate.

=Knight Grant=

What is this grant for?
The Knight Foundation has awarded the Wikimedia Foundation an exploratory grant to research and evaluate ways to measure and improve search results on Wikimedia projects.

This is a restricted grant, and the funds may only be used by the Discovery team for the deliverables specified in the grant.

This grant does not increase the team's budget for this fiscal year.

Links:
 * [<tvar|knight-grant>https://upload.wikimedia.org/wikipedia/foundation/a/a7/Knowledge_engine_grant_agreement.pdf</> Grant agreement, with scope-of-work at the end] - awarded 2015-09-18, accepted by WMF 2015-11-20


 * [<tvar|knight-pr>http://www.knightfoundation.org/press-room/press-release/wikimedia-foundation-explore-new-ways-search-and-d/</> Joint press release] - Knight Foundation web site, 2016-01-06


 * "[<tvar|knight-blog>http://knightfoundation.org/blogs/knightblog/2016/1/6/exploring-how-people-discover-knowledge-wikipedia-and-its-sister-projects/</> Exploring how people discover knowledge on Wikipedia and its sister projects]" - Knight Foundation blog, 2016-01-06


 * "wmf-pr>wmf:Press_releases/Wikimedia_Foundation_to_explore_new_ways_to_search_and_discover_reliable,_relevant,_free_information_with_$250,000_from_Knight_Foundation</>|Wikimedia Foundation to explore new ways to search and discover reliable, relevant, free information with $250,000 from Knight Foundation" - Wikimedia Foundation blog, 2016-01-06


 * [<tvar|grant-announce>http://www.knightfoundation.org/grants/201551260/</> Grant announcement] - Knight Foundation web site, date unknown, identifying the grant period as running from 2015-09-01 to 2016-08-31


 * lila-faq>meta:Knowledge_Engine/FAQ</>|Knowledge Engine FAQ with Lila

Who is the Knight Foundation?
The Knight Foundation is a philanthropic organization dedicated to supporting media, journalism and fostering communities and the arts.

As part of their efforts, they have previously given grants to the Wikimedia Foundation to support Wikipedia Zero.

What are the activities the WMF said they would conduct under the grant?

 * Answer key questions:
 * Would users go to Wikipedia if it were an open channel beyond an encyclopedia?
 * Can the Wikimedia Foundation get Wikipedia embedded via carriers and Original Equipment Manufacturers?
 * Use Key Performance Indicators (KPIs) to inform product iteration, and establish key understanding and feature development for the prototypes
 * Conduct tests with potential users
 * Create a public-facing dashboard of key KPis


 * Measure:
 * User satisfaction (by analyzing rate at which queries surface relevant content)
 * User-perceived load time
 * No results rate
 * Application Programming Interface (API) usage

What are the deliverables?

 * Test results exploring relevancy of content surfaced
 * Test results from research and user testing
 * An improved search engine and API for Wikipedia searches
 * A public-facing dashboard of core metrics used in product development
 * A sample prototype on a small dataset to showcase possibilities
 * User testing and research on current user flows to understand the search and discovery experience