User:OrenBochman/Search/Test Plan

=Overview=

Functional Test

 * Testing Indexing
 * Indexing of MediaWiki source
 * Indexing of hypertext source
 * Indexing of external links
 * Testing Update Mechanism
 * Requires a robot that uploads content into the test Wikipedia installation.
 * Test new pages added
 * Test existing pages are updated
 * Test Indexing propagated from indexer to searcher
 * Test Indexing propagated from searcher to replication.
 * Integration tests of component.
 * Tests OI Repository is online and working.
 * Tests Search UI is online and working.
 * Offer search results from Sister Projects
 * Tests Search suggestions are online and working.
 * Tests Search extension is healthy.
 * last update in special pages
 * Test Index is not corrupted.

Relevance Test

 * Checks to what degree ranking is altered overtime (by various configuration).
 * Select N=100 queries.
 * Store to 10 results for 100 queries.
 * Associate these with a ranking profile. (Query cooking + Analyzer + Linguistics etc).
 * Order of results is consistent across multiple hosts

Benchmarks

 * Benchmark indexing.
 * Benchmark testing.

Regression Tests
tests that bugs from the past do not return

Testing Q&A

 * Converting Did you know to triavia question for a open relevence format question
 * User engagement trivia game.
 * why - today the full questions return nothing - this is very bad.


 * Stacey Porter
 * 1) Which Olympic silver bronze medalist was the first Aboriginal member of Australian women's national team to represent softball Olympics.
 * 2) Which aboriginal Olympic silver bronze medalist Australian national team softball


 * Tinirau
 * 1) Which Devonian stem tetrapod, a transitional species between fish and land vertebrates, was named after the half-human half-fish character in Polynesian legend?
 * 2) Devonian stem tetrapod Polynesian


 * Soft & Cuddly
 * 1) In which game must the protagonist must reassemble his mother's dismembered body
 * 2) game mother's dismembered.


 * World Naked Gardening Day
 * 1) What gardening day in May was first observed in 2005?


 * Umberto Barulli
 * 1) Who was the Communist Party leader appointed head of state in San Marino in 1988?
 * 2) Communist Party leader head of state in San Marino in 1988


 * Dynasphere
 * 1) It was criticized for its poor braking and steering capabilities, as well as its gerbiling tendencies?
 * 2) w:Special:Search/criticism OR braking OR steering OR gerbiling OR tendencies OR capabilities OR poor OR well)
 * 3) w:Special:Search/~criticism OR ~braking OR ~steering OR ~gerbiling OR ~tendencies OR ~capabilities OR ~poor OR ~well)
 * 4) w:Special:Search/braking OR gerbiling
 * 5) w:Special:Search/(criticism OR capabilities) AND (braking OR steering) AND (gerbiling OR tendencies) works
 * 6) return monowheel not Dynasphere

antitest
crowdsource user feedback directly into results instead of findinf bugs.

Moving Target

 * lots of arros - lots of test.
 * precise hits - degree of closensss.
 * Cps pf missing - special tests.
 * how does it move - monitoring.
 * unexpectred movement - spike.
 * precision + accuracy - non determinism.


 * oracles.
 * dictionary.