Thread:Extension talk:Lucene-search/Speed performance tuning for Lucene-search extension

Hi, we're trying to build a highly scalable Wikipedia search mirror, which is required to handle 10,000 requests per second for searches. I tried using Lucene-search extension, but just couldn't get the average fulltext search time go up, our current average search time is around 500ms.

It seems that the deviation of search time is large too, with some searches being 4ms and others being 2,000ms. These performances tests were carried out using JMeter with 10 user request thread and 500 loops. We've tried tuning the JVM memory, tried storing the whole index (index built on the 20111007 wikipedia snapshot XML dump) in RAM, tried increasing/decreasing the SearcherPool.size parameter. So our question is that whether this extension is designed to perform better speedwise? Thanks very much