Extension talk:SphinxSearch/LQT Archive 1

Did you mean?
I have SphinxSearch working on my site and it is fast! Now I am attempting to implement the "Did you mean?" function and get the following error on the search: " Uninitialized string offset: 0 in C:\path\to\SphinxSearch_spell.php on line 96 " line 96 is this: if ($value[0] == "&") { $correction = explode(" ",$value); $word = $correction[1]; $suggstart = strpos($value, ":") + 2; $suggestions = substr($value, $suggstart); $suggestionarray = explode(", ", $suggestions); $guess = $this->bestguess($word, $suggestionarray); if (strtolower($word) != strtolower($guess)) { $word_suggestions[$word] = $guess; $this->suggestion_needed = true; }           }        } Can anyone help with this? Steve Goble14:06, 30 May 2008 (UTC)


 * You seem to have your PHP error reporting level set too high. If you have to have it that way, change the line to:

if (substr($value, 0, 1) == "&") {
 * P. S. There will probably be other places where you get a similar warning... Svemir Brkic 14:51, 30 May 2008 (UTC)

You are the man! That fixed the error but "Did you mean" does not work. The search is conducted but returns 0 matches for a mispelling. Crap! One thing after another. Steve Goble15:16, 30 May 2008 (UTC)

I got it working. I have ASPELL on my FCKEditor and trying to use that with CLI. I just enabled the php-pspell and that works good enough for me!! Thanks again, Svemir! BTW I am on a Windows 2008 box. Works great! Steve Goble

More Windows Install Issues
Please excuse my ignorance, how do we poor windows users perform steps 5 and 6?

Steve Goble18:34, 14 May 2008 (UTC)

Anyone?????

Answer is:
 * Step 5 = C:\path\to\searchd.exe --install --config C:\path\to\sphinx.conf
 * I tried using this option by adding to startup group, made a service out of it but it didn't work for me.
 * I recommend using the scheduler, create a batch file and setup to run each time windows boots up. instead of install( given above) use path/to/sphinx/installation/searchd --config /path/to/sphinx.conf &
 * Step 6 = Windows Task Scheduler

installation issues
Problem 1:

WARNING: key 'sql_group_column' is deprecated in /var/sphinx/sphinx.conf line 35; use 'sql_attr_uint' instead. WARNING: key 'sql_group_column' is deprecated in /var/sphinx/sphinx.conf line 36; use 'sql_attr_uint' instead.

solution: replace sql_group_column by sql_attr_uint


 * This depends on the version of Sphinx you are using. CVS version of the extension is already using sql_attr_uint, as it requires the latest version of Sphinx.

ERROR: unknown key name 'FROM' in /var/sphinx/sphinx.conf line 48 col 10. FATAL: failed to parse config file '/var/sphinx/sphinx.conf'.

solution: merge line 48 with 47 making sure to remove the change of line / (not sure why this error)


 * Weird. Maybe some invisible extra character sneaked in somehow. Anyway, I merged the lines in CVS, just in case. Svemir Brkic 02:05, 6 April 2008 (UTC)

Problem 2:

Query failed: connection to localhost:3312 failed I can search from the command line but it does not seem to work from the site. Any idea what i am missing??

Solution: used 3306, i think it's ubuntu's default make sure to run /usr/local/bin/searchd and it works! also 0.0.0.0 is used as should be by default (but is not)

Problem 3:

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 775173422 bytes) in .../extensions/SphinxSearch/sphinxapi.php on line 311

Solution: This problem was cause by the solution to problem 2, that is using port 3306 which is used by mysql already... see http://sphinxsearch.com/forum/view.html?id=1178

@page_title DOES NOT WORK
for some reason @old_text works but @page_title never returns any results... maybe it has something to do with the new releases of sphinx. --207.96.208.130 22:22, 1 April 2008 (UTC)


 * This syntax is only mentioned in a TODO list on the main page of this extension. We do not pass on everything you enter to sphinx. We generally try to create web interface to specify advanced options. Svemir Brkic 02:07, 6 April 2008 (UTC)


 * Until we provide user-friendly interface for @page_title searches, make sure to select "match all words" when submitting your search. You can make that the default if you download version 0.6beta3 and uncomment the line in SphinxSearxh.php that sets $wgSphinxMatchAll to '1'. Svemir Brkic 03:10, 10 May 2008 (UTC)

match all words as default
not sure if this is something that is controlled by Sphinx or not, but it would be better for me if "match all words" was the default. thx! matching mode set to: SPH_MATCH_ALL by default it is SPH_MATCH_EXTENDED, but does not work properly as http://www.sphinxsearch.com/doc.html#extended-syntax explains


 * The point of having the SphinxSearch.php file separately is so that you can set things such as $wgSphinxSearch_mode for yourself. The reason we use SPH_MATCH_EXTENDED is to be able to modify the query internally and do "match all" vs. "match any" with a radio box instead of teaching users the Sphinx query syntax. Svemir Brkic 03:08, 6 April 2008 (UTC)

Making title match appear first
Is there a way to do this?
 * Greater weight is already given to title matches. Do you need any title match (even a partial one) to appear before any content match (even an exact one?) Svemir Brkic 02:19, 26 March 2008 (UTC)


 * YES. All direct title match should appear first. This is not the case. --207.96.208.130 22:12, 1 April 2008 (UTC)

Sphinx API has changed recently, and that probably caused title matches to show so far in the search results. SetWeights method has been deprecated and the method to use now is SetFieldWeights. The new method expects an associative array of index names and weights. Until we release the next version, here is what you should try:


 * In SphinxSearch.php, change the line that sets $wgSphinxSearch_weights to:

$wgSphinxSearch_weights = array('old_text'=>1, 'page_title'=>100);


 * In SphinxSearch_body.php, change the $cl->SetWeights call in wfSphinxSearch method to:

$cl->SetFieldWeights($wgSphinxSearch_weights);

This will not make all the title matches appear before all the text matches, but it will probably give much better results. Svemir Brkic 21:13, 5 April 2008 (UTC)


 * Note: Above has been committed to CVS and released in 0.6beta3 package. Svemir Brkic 03:13, 10 May 2008 (UTC)

Create this page for search result not present
how do we do this
 * You should not need to do anything special. If your wiki allows edits by the current user, and the user searches for a term without an exact title match, a line like this will show above the search results:

There is no page titled "foo". You can create this page.


 * I do get this, but if the search is made from the Sphinx Search page itself, i do not get the link. --207.96.208.130 21:23, 26 March 2008 (UTC)
 * SOLVED: it turns out you need to press the "GO" button and not the "SEARCH" button. Added that button as default and solved the problem. cheers! --207.96.208.130 14:35, 27 March 2008 (UTC)

Namespaces for weight
Is it possible to set a weight to different namespace that way the checkboxes can be removed all together...
 * This is possible and I will look into it for the next release. It will not replace the need for checkboxes, as in some cases you want the users to specify which namespaces they are interested in. Svemir Brkic 02:17, 26 March 2008 (UTC)
 * Amazing! --207.96.208.130 21:24, 26 March 2008 (UTC)

Showing X of Y documents in search results, but document links not showing
I've got everything working and it is apparently using sphinxsearch as my default search engine, but the results simply say "myword found 8 times in 8 documents", and then nothing is listed underneath it for those documents where the word is in the title only. However, if the word is contained in the body of the entry, those entries show up.

I also had to comment out a php error, it was in sphinxsearch_body.php, which stated something about a variable that was undeclared called 'time', and I had to comment it out. It was in this function

$preamble = sprintf(wfMsg('sphinxSearchPreamble'),               (($page-1)*$wgSphinxSearch_matches+1 > $res['total']) ? $res['total'] : ($page-1)*$wgSphinxSearch_matches+1,                ($page*$wgSphinxSearch_matches > $res['total']) ? $res['total'] :  $page*$wgSphinxSearch_matches,                $res['total'],                $term,'' //              $res[time]


 * Proper fix for this is to change it to $res['time'] which I just did in the CVS version of the code. Please do so locally and let me know if there are other problems. Svemir Brkic 19:05, 2 February 2008 (UTC)

You can see where I commented $res[time] out and replaced with empty single quotes. This didn't seem to break anything so I'm not sure what it was supposed to do. I'm on a windows machine btw. All I really care about is that the document listing appear when the word appears in the document title.
 * "Displaying 0-0 of 0 matches for query specifi* retrieved in 0.000 sec with following stats: specifi* found 8 times in 8 documents".  This is all I'm getting when searching for a particular wildcard.  It appears the main 0-0 of 0 matches is wrong, as the next line shows that it is indeed finding the word in 8 of 8 documents.  When I use the console, here are the results:
 * C:\Sphinx>search --config sphinx.conf speci*
 * Sphinx 0.9.8-dev (r985)
 * Copyright (c) 2001-2007, Andrew Aksyonoff
 * using config file 'sphinx.conf'...
 * index 'wiki_main': query 'speci* ': returned 5 matches of 5 total in 0.000 sec
 * displaying matches:
 * 1. document=23, weight=1, page_namespace=8, :::old_id=47 page_title=Sidebar page_namespace=8
 * 5. document=58, weight=1, page_namespace=6, :::old_id=116 page_title=SIF_Implementation_Specification_1.5r1.pdf page_namespace=6
 * words: 1. 'speci*': 5 documents, 5 hits
 * words: 1. 'speci*': 5 documents, 5 hits


 * Your query indicates that you are using a "star" search, which is not enabled by default in sphinx (it is a new, almost undocumented feature.) Our default sphinx.conf does not enable it either. Maybe you are using a different config file for command line search? To enable it in the config file used by the extension, you need to add this to the main index section:

min_infix_len = 1 enable_star = 1


 * After this, you need to do a full reindex and restart the searchd (--rotate alone is not enough when you change the config file.) Svemir Brkic 20:57, 2 February 2008 (UTC)

Everything is OK it seems, but no search-results
I've installed the Sphinx-extension on a 1.11 mediawiki with the most recent version of Sphinx. Everything seems to work OK, I installed the service (windows-server), tested the search on the commandprompt, which gives results. When I go to special:searchSphinx, it displays OK.

The only thing is that nothing happens when I try to search something, it reloads and displays nothing. Do you have any idea what might be causing the problem? It also seems it cannot create a searchd.pid file & logfiles, although the search-indexes are created without a problem.


 * Please clarify "reloads and displays nothing". Do you get a blank screen? In that case, you would need to check your php error log for clues. Or you get the same thing as Mark describes below? Svemir Brkic 15:44, 5 January 2008 (UTC)


 * Hi Svemir, Thanks for your response. I have the same problem as Mark. I discovered that if I disable the internal search, the sphinx special pages suddenly is gone (it also isn't visible in Special:Specialpages). If internalsearch isn't disabled, the sphinxsearch special page is visible, but it doesn't return results. (i've got the same config as mark, only the mediawikiversion is 1.11) 213.132.179.227 10:03, 7 January 2008 (UTC)


 * We have the same problem. We're on Sphinx 0.9.7 (Win32) and MediaWiki 1.9.7 installed on Win2003 server.  After starting search daemon, I can run test.php from command line and get back results there, but searches entered from Sphinx Special Page in the wiki just bounce you back to the main page.  The URL looks like the search took place, however.  For example, a search on the term "SAM" bounces you back to the wiki's main page, but now this query string appears on the URL: "sphinxsearch=SAM&fulltext=Search&match_all=0&ns0=1" --Mark price 01:26, 14 December 2007 (UTC)


 * Did you make any changes to you sphinx search configuration file - SphinxSearch.php? You could also try making Sphinx the default search, just to verify if the problem is in the search itself or the way your wiki handles the paths in the "special page only" case. Svemir Brkic 15:44, 5 January 2008 (UTC)


 * We have the same problem. SphinxSearch 0.5.3 with sphinx 0.9.8-svn-r1112 on MediaWiki 1.11 on Windows2003 Server.Rroblem is MediaWiki 1.11 needs "title" argument in URL.Like "/mediawiki/index.php?title=XXXX:SphinxSearch&search=foo&....". I change SphinxSearch_body.php line 351 like this,and it goes well. --219.121.144.178 15:26, 13 March 2008 (UTC)

$wgOut->addHTML("                                 


 * Interesting. This made me think I did not notice the problem because I am using "short" links. However, even after changing to the default index.php?title=... configuration, the search still worked. Can anyone else confirm that this fixes the search in their configuration? Svemir Brkic 11:26, 15 March 2008 (UTC)


 * This did not fix for me and I started getting parse errors. I changed the end like this:

$wgOut->addHTML("                                 ");


 * which cleared the parse errors but still just a blank page. 18:56, 12 May 2008 (UTC)
 * We had the same when we tried using Sphinx-0.9.8-svn-r1112 (Jan 28, 2008 snapshot). Getting the previous version (Sphinx 0.9.7) solved the problem for us.130.234.189.190 11:52, 30 January 2008 (UTC)


 * I just tested it with 0.9.8-svn-r1112 on MW 1.11 on Linux and it works correctly. I will try it on Windows at some point as well. Would you please make sure you were using the correct version of sphinxapi.php? It needs to be copied from your sphinx download/api folder into the SphinxSearch extension folder each time you change the version of sphinx on your system. If you still have issues with 0.9.8, please post your sphinx.conf somewhere so we can take a look. Svemir Brkic 19:55, 2 February 2008 (UTC)


 * I also tested it with 0.9.8-svn-r1112 on MW 1.11 on Linux, but mine is giving me the same error as the above users are experiencing. I am doing this on a shared hosting plan. I installed Sphinx to /home/myusername/local/sphinx. I can get everything to work on the command line via SSH, but not on the actual site over http -- it just returns me to the main page, despite the URL looking as though it should have performed the query. I have double-checked the version of sphinxapi.php and gone through all the installation instructions twice but I am dumbfounded. Also, the search daemon seems to stop running after about 5 or 10 minutes. The process will not stay alive any longer than that. I am not that great with Linux yet, but I am slowly becoming more familiar. I'd rather not bother installing an earlier version, so I would appreciate any suggestions. My sphinx.conf file appears below [UPDATE: removed to save space]. --Wikitonic 18:40, 13 February 2008 (UTC)


 * There seem to be two separate issues here. One is related to how your wiki is setup in general regarding the URLs etc. - please send me a link if you do not mind. Another issue is with the search deamon being stopped. Perhaps your shared host does not allow user processes to run longer than certain amount of time, or use more than certain amount of memory, etc. You should probably ask - maybe they are willing to make an exception if they know exactly what are you doing. Svemir Brkic 00:59, 14 February 2008 (UTC)


 * Ok, I will check with my webhost about user processes. In the meantime, you can find my wiki here. Unfortunately, it is locked to the general public right now since we haven't officially launched. Even viewing is disabled for anonymous users. I'll get in touch with you about letting you in, after I take care of the search daemon disruption issue. --Wikitonic 16:03, 14 February 2008 (UTC)


 * Well, I meant to give an update sooner but I guess now is better than never. My shared hosting plan doesn't allow user processes, so until it does, I guess everything is a moot point. I have removed my sphinx conf file so as not to take up unnecessary space. Thanks for your help anyway, Svemir Brkic. -Wikitonic 21:18, 3 March 2008 (UTC)

Major failures -- help!
Line #s 304 and 305 error out on SphinxSearch_body.php in version 1.8 of Mediawiki. Either i need to comment it out or upgrade to v1.11

Even after upgrading line # 171 errors with Fatal error: Call to undefined method SphinxClient::SetFilter in C:\wamp\www\wiki\extensions\SphinxSearch_body.php on line 171

Commenting doesn't help since it gives me a "Fatal error on the DB" or something like that. I'm thinking i need to install some package, but don't know which one (something in PEAR?)

Help :(


 * It looks like you are running this extension on Windows. To the best of my knowledge, this extension has not been tried as such yet. But, at least in theory, there is nothing that should prevent it from working. Of course, the big differences are all path related. So, let's first start by making sure your setup is correct. Were you successfully able to perform step 3, step 4, and step 5? Can you also please verify that step 7 was done correctly and you have the sphinxapi.php file in your C:\wamp\www\wiki\extensions\ directory? --Gri6507 12:20, 17 October 2007 (UTC)


 * I kinda solved this; upgrading to v1.11 off course takes care of the lines 304 & 305. As for the setFilter method, apparently the rc1 of sphinxapi doesn't have this method. I'm trying to copy-paste the method into my API as first option and then will try and build the current non-production API. Will keep you informed


 * Thanks for looking into this. I am running my installation with MW 1.9.3 and I don't know which version Svemir (the other developer) is running. I will start a new section on the main page with information about known supported MW versions. As for sphinxapi.php being incorrect, I am assuming you are using v0.9.8rc1? Both Svemir and I based this extension on 0.9.7 (the latest stable release). We'll keep a keen eye on the Sphinx project to make sure that our extension will be completely compatible with future version of Sphinx.


 * On a side note, I was wondering if you have implemented the windows equivalent of setting up the cron jobs to keep the indexes up to date. If you have, can you please add that information to the documentation? We would much appreciate it! --Gri6507 12:36, 18 October 2007 (UTC)


 * I'm still stuck and couldn't get much progress. Apparently the line
 * $sql = "SELECT old_text FROM ".$db->tableName('text')." WHERE old_id=".$docinfo['attrs']['old_id']; ends up with the value of $sql being Select old_text from 'text' where old_id=


 * I'm not sure why in the first place text is in single quotes (looks like some bug to me) and why the old_id is not getting picked up. Searchd does show the hit coming to it, but it could failing because i'm using a hacked version of the API


 * Also to answer Gri6507's question, i'm using 0.9.6 rc1 because that's the one that has the windows binaries. I don't have Visual Studio or VC++ to compile from the source code, so even my step #2 (using latest version and compiling) is at hold.


 * Adding the Windows cron job shouldn't be too tough (my guess); but i'll try it and let you know -- ALl the above posts bought to you by the guy who had the so useful signature Help :


 * According to Sphinx's website, http://www.sphinxsearch.com/downloads/sphinx-0.9.7-win32-release.zip is a windows release of 0.9.7. Is there any reason you are not using it? --Gri6507 11:42, 19 October 2007 (UTC)


 * Doesn't seem to contain the sphinxapi.php -- that's the reason why i had to choose an older version; this should probably be posted on that developer's website saying the API is missing from the 0.9.7 windows release, but i'm too lazy...any helpers? :)


 * I have updated Step #1 and Step #7 with details of how to obtain the sphinxapi.php for Windows. It seems that the intent of the Win32 release binaries package is to only contain the binary EXEs. The PHP files are in either the source code or the API packages. --Gri6507 11:34, 25 October 2007 (UTC)


 * Does it even work on Windows? Installation instruction (step 1) of the Sphinx site has this to say "At the moment, Windows version of Sphinx's searchd daemon is not intended to be used in production because it can only handle one client at a time."

Directory Structure
How about making it so that Sphinx is in its own subdirectory in the extension directory so that things will be cleaner? (I have a lot of extensions and each has their own directory) Also could the sphinxapi.php be also included in the SphinxSearch tar so that its one stop shopping? Although this could just be implemented in a script if you guys go that way. --SellFone 21:22, 30 October 2007 (UTC)


 * This is an idea I have toyed around with for some time now. As SphinxSearch has grown to include more and more files, the appeal of having it reside in its own directory has increased. I am currently working on the automatic installation & configuration script for the extension. Perhaps I will make it install the extension in its own directory. Thanks for the suggestion! --Gri6507 23:15, 30 October 2007 (UTC)

Windows --rotate workaround?
I want Sphinx to update our index very often. If I could, I would love for the index to be incrementally updated every time the db changes. Barring that, I'd install a task to run every 15 minutes or so. However, no matter how often I update the index, I need to take down the Sphinx daemon to do it (limitation on Windows). Can anyone suggest a workaround or modification to the code such that a search request, when the daemon isn't running, waits for it to respond and re-searches? I don't so much care about restarting the daemon. I do care about search appearing broken while the daemon is down.
 * I am not sure if it is going to work, but here's what I'd try. Open the sphinxapi.php file. In function _Connect , around line 136, there is a call to


 * change that to


 * where the 30 is the timeout in seconds for establishing the connection. The basic idea is that if searchd is not running, no one will be listening on the other end of the socket until searchd comes back to life. This change should block MW from dieing during that brief period of time. Of course, it would be up to you to make sure that
 * before running the indexer, you must stop searchd
 * after running the indexer, you must restart searchd
 * Let me know if that works :-) --Gri6507 22:22, 3 November 2007 (UTC)


 * That looks promising. I'll give it a try.  Since my post, I installed the search on a separate machine (Linux) and that works pretty good.  But, this procedure may be what I need to reduce the number of servers in the equation.  I'll come back with results. --Cedarrapidsboy 14:25, 5 November 2007 (UTC)
 * UPDATE - the above code change didn't appear to have an effect. The search still timed-out to a blank page.
 * Stop searchd
 * Issue search request
 * Start searchd (within 30 sec)
 * --205.175.225.24 16:30, 5 November 2007 (UTC)


 * Ok. I think I found the issue. According to PHP documentation, the fsockopen function may not honor the timeout ("Note: Depending on the environment, the Unix domain or the optional connect timeout may not be available."). So, to work around that, change the following code in sphinxapi.php


 * to


 * This way, you can set the waiting period yourself via the use of $connect_timeout variable. I tested this on my machine and it seems to work as expected. Please post your results when you try it out. --Gri6507 23:05, 5 November 2007 (UTC)
 * Unfortunately, same result. Blank page.  I did the following:
 * Kill searchd
 * Issue search request
 * Start searchd
 * In this case, searchd was still running on a separate machine.
 * --Cedarrapidsboy 20:20, 6 November 2007 (UTC)
 * UPDATE!
 * Here's a change to the code that works:


 * I added an additional timeout. Without it, a single connection was waiting for 30 seconds, just as long as the entire loop.  The previous code never tried the connection again.  This code *did* work for me using the testing steps above.  --Cedarrapidsboy 20:30, 6 November 2007 (UTC)


 * Glad to see that it's working for you! I will submit this as an improvement suggestion to the developers of Sphinx. --Gri6507 20:41, 6 November 2007 (UTC)

Running on separate machines
The installation directions seem to assume that MediaWiki and sphinxd are on the same machine. How should I configure Sphinx and the extension if MediaWiki (and its database) are hosted separately from where Sphinx is installed? --Emufarmers 04:43, 5 November 2007 (UTC)
 * You are correct. I should update the main page with these instructions. To make this extension work in your case you will need to configure sphinx.conf. In that file, modify the src_wiki_main section to specify the correct sql_host = hostname, where hostname is the name of the machine running the MySQL database for your wiki (default is localhost). This way, you can install Sphinx on the same machine as the web server (as opposed to the MySQL server), and all instructions as listed on the main page are still valid.
 * Please let me know if you have any more questions, or, for that matter, if this worked for you. --Gri6507 13:05, 5 November 2007 (UTC)
 * Hi, sorry for the delay. I should have been more clear: My wiki is on shared hosting, so I can't install Sphinx on its server.  I run the search backend (presently Lucene, but Sphinx sounds promising) on a machine in my home.  With Lucene, my backend machine SSHs into the webserver, grabs a dump of the wiki, indexes it, and then runs search queries from the webserver through the index it generates and sends the results back to the webserver.  It's a rather convoluted setup (and it's even messier when it comes to updating the index), but I'm wondering if I can do anything along the same lines here. --Emufarmers 20:08, 11 November 2007 (UTC)
 * Ok, I understand your setup. Sphinx.conf file can be configured to make sphinxd run on a different machine (let's call it Machine S, for sphinx) from the machine running MySQL (let's call it Machine M, for mysql). However, in that case Machine S has to have netword access to Machine M. My guess is that something similar to your present setup with SSH tunnels could be done here as well. If you are interested in trying this out, please let me know via email (see extension credits) and we could work through these questions then. --Gri6507 22:17, 11 November 2007 (UTC)

Sphinx Search Terms Limit
It seems that the Sphinx search only accepts 10 search terms. Perhaps it is the same story for the built-in MW search? Any way to change that? Perhaps make it unlimited? Cedarrapidsboy 13:50, 5 November 2007 (UTC)


 * I did not look at the code yet, but the limit seems to happen only in the sense of number of separate words and counts displayed on top of the search results. That lists only up to 10 words, but the eleventh word I used was also used to filter (and rank) the results. Svemir Brkic 14:31, 5 November 2007 (UTC)


 * Ah... I can confirm that.  I tested it, but the 11th and above search terms were not highlighted red, so didn't think they were included in the results.  I'd still be interested in getting all search terms highlighted.  Thanks for the reply!  --Cedarrapidsboy 14:56, 5 November 2007 (UTC)


 * This seems to be fixed in the latest version of sphinx (0.9.8-rc2) - according to change log: fixed highlighting (uses 256 words by default now instead of former 10) Svemir Brkic 16:05, 28 March 2008 (UTC)

Wildcard Search
Is there a way to enable wildcard search when using Sphinx?
 * According to http://www.sphinxsearch.com/forum/view.html?id=843, the latest 0.9.8 SVN snapshots available at http://www.sphinxsearch.com/downloads.html should have wildcard support. This feature has not yet been tested with this extension, so I can't guarantee that it works. Svemir and I will be taking a look at it as time permits (probably after the 1st of the year). You are very welcome to try this out on your own and post your results here. --Gri6507 13:25, 10 December 2007 (UTC)


 * Please see a note above about min_infix_len and enable_star config options. That makes * searches work for me. Svemir Brkic

Warnings on MW 1.11
Warning: Call-time pass-by-reference has been deprecated - argument passed by value; If you would like to pass it by reference, modify the declaration of [runtime function name]. If you would like to enable call-time pass-by-reference, you can set allow_call_time_pass_reference to true in your INI file. However, future versions may not support this any longer. in extensions/SphinxSearch_PersonalDict.php on line 75 (also lines 142 and 190)

-71.217.0.96 08:18, 5 January 2008 (UTC)


 * Thanks for the notice. I have fixed this in the CVS already. If you are using a public release, all you need to do is remove the ampersands from all calls to readPersonalDictionary. The method is already declared correctly (the ampersands should stay there.) Svemir Brkic 13:15, 2 February 2008 (UTC)

Problems with search within page title / SOLVED
Hi, I installed the sphinxsearch on ubuntu and it is working very good. Thx for this extension! I had some problems with title search. Originally, the suggested query (default value in the installation package) was:

#sql_query	= SELECT page_id, page_title, page_namespace, old_id, old_text \ #                 FROM mw_page, mw_revision, mw_text \ #                  WHERE rev_id=page_latest AND old_id=rev_text_id

The page_title content in the database looks like 'HOW_TO_edit_homepage'. There are underlines between each character sequence (mediawiki replaces blanks through underlines when new page is created). I changed the query (for initial and incremental index):

initial index: sql_query	= SELECT page_id, replace(page_title,'_',' ') as page_title, page_namespace, old_id, old_text \ FROM mw_page, mw_revision, mw_text \ WHERE rev_id=page_latest AND old_id=rev_text_id

incremental updates: SELECT page_id, replace(page_title, '_',' ') as page_title, page_namespace, old_id, old_text FROM mw_page, mw_revision, mw_text WHERE rev_id=page_latest AND old_id=rev_text_id AND page_touched>=DATE_FORMAT(CURDATE, '%Y%m%d070000')

Now it works. --Wikigeil 17:15, 21 January 2008 (UTC)


 * I do not think this is necessary. Perhaps there was another issue with your setup and it got resolved while you were changing the queries. If you look at this line in the suggested spihinx.conf file:

charset_table  = 0..9, A..Z->a..z, _->, a..z, \


 * Among other things, this instructs Sphinx to consider an underscore the same as a space. Perhaps you should try again with the original queries, as they would probably work faster. On the other hand, I might be misunderstanding what were you trying to fix, so please let me know if that is the case. Svemir Brkic 13:05, 2 February 2008 (UTC)


 * Hi Svemir, thx for response. The charset_table property for main index is:

# charset definition and case folding rules "table" charset_table	= 0..9, A..Z->a..z, _->, a..z, \ U+C0->a, U+C1->a, U+C2->a, U+C3->a, U+C4->a, U+C5->a, U+C6->a, \ U+C7->c,U+E7->c, U+C8->e, U+C9->e, U+CA->e, U+CB->e, U+CC->i, \ U+CD->i, U+CE->i, U+CF->i, U+D0->d, U+D1->n, U+D2->o, U+D3->o, \ U+D4->o, U+D5->o, U+D6->o, U+D8->o, U+D9->u, U+DA->u, U+DB->u, \ U+DC->u, U+DD->y, U+DE->t, U+DF->s, \ U+E0->a, U+E1->a, U+E2->a, U+E3->a, U+E4->a, U+E5->a, U+E6->a, \ U+E7->c,U+E7->c, U+E8->e, U+E9->e, U+EA->e, U+EB->e, U+EC->i, \ U+ED->i, U+EE->i, U+EF->i, U+F0->d, U+F1->n, U+F2->o, U+F3->o, \ U+F4->o, U+F5->o, U+F6->o, U+F8->o, U+F9->u, U+FA->u, U+FB->u, \ U+FC->u, U+FD->y, U+FE->t, U+FF->s,
 * It contains underscore too. --Wikigeil 17:34, 4 February 2008 (UTC)


 * Yes, that is why I think the replace function in the query is not necessary. Everything works fine without it, with the original suggested queries. Svemir Brkic 04:21, 6 February 2008 (UTC)


 * Ugh, I guess I was too convinced that searches worked correctly in our case - or that they could not be much better. Having the underscore in the charset table makes it a regular character, so titles do not get indexed properly. If "_-> ," is removed, titles get indexed correctly and replace function is not needed anymore. Of course, if you do want to index words with underscores (maybe if your wiki contains lots of code examples with underscores in function names?) you should replace "_-> ," with just "_," and still use the replace function in the query. Svemir Brkic 15:19, 8 May 2008 (UTC)

Undefined index: matches in SphinxSearch_body.php
When: error_reporting(E_ALL); ini_set("display_errors", 1); is set in LocalSettings.php and I search for a word that is not found, such as "wouuld" I get the Undefined index: matches in SphinxSearch_body.php error. I have checked my installation carefully and everything seems to be working. Perhaps this "error" should not be raised.


 * Running:
 * MediWiki:      1.12alpha (r30598)
 * PHP:           5.1.6
 * MySQL:         5.0.22
 * SphinxSearch:  0.5.3

--KevinS 23:22, 5 February 2008 (UTC)


 * Thanks for the report. I fixed the cause of the error message in the CVS version of the code. We should have a new release soon - including a basic category filter :-) Good to know that the extension otherwise works in MW 1.12. Svemir Brkic 05:06, 6 February 2008 (UTC)

Searching multiple wikis
I currently have 3 wikis indexed with Sphinx. The search works well, but it is returning results for all of them. I've set the "$wgSphinxSearch_index = X"; line in the SphinxSearch.php. Am I missing something? --N0ctrnl 20:28, 24 March 2008 (UTC)


 * The default setup assumes there will be one main index and one incremental index. Searches are performed with an "*" to indicate all available indexes ($wgSphinxSearch_index indicates which is the main one.) With the latest versions on spinx API it is possible to do this in a better way, but for now perhaps you can work around it by running separate sphinx client for each wiki - each on its own port. Svemir Brkic 02:14, 26 March 2008 (UTC)


 * I kinda suspected that'd be the answer. Not what I'd hoped, but I suppose it's not all that bad.  Thanks very much for the reply. --N0ctrnl 13:02, 26 March 2008 (UTC)

i got this messege error
ERROR: index 'guitarpedia': sql_query: Table 'guitarpediadb.documents' doesn't exist (DSN=mysql://****:***@mysql.guitarpedia.org:3306/guitarpediadb). > and i dont know what to do


 * "documents" is never used as a table (or anything else) anywhere in the default SphinxSearch configuration. Did you maybe change SphinxSearch.php or sphinx.conf to use that table for something? MediaWiki does not have such a table by default. Perhaps if you describe your changes to SphinxSearch and MediaWiki someone may be able to help you. Svemir Brkic 11:59, 3 May 2008 (UTC)

can someone show how to edit sphinx.conf correctly
i tried too many times and always fails at some other problem. can someone who runs the sphinx on his mediawiki show his sphinx.conf file? and what we need to change?

—

Yes this would be very helpful. I am also having issues with my sphinx.conf. I have it mostly working, however when I attempt to index my wiki I get an error regarding my sql_query_pre = wiki_ entry in the sphinx.conf: "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'wiki_' at line 1". wiki_ is a standard MediaWiki table prefix so I am unsure why it throws an error. I found the other presets here.


 * Now, I don't know what's in your sphinx.conf, but based on mine, sql_query_pre is not the place to put your database prefix. Take another look at step 2; you have to go through sphinx.conf and tack your prefix onto the table names (but not the row names!).  It's annoying, but I eventually figured out that there are annoyances all over the place with MediaWiki if you use a prefix.  My advice is to bite the bullet now and get rid of the prefix now if it's an option. —Emufarmers(T 03:51, 6 August 2008 (UTC)
 * Now, I don't know what's in your sphinx.conf, but based on mine, sql_query_pre is not the place to put your database prefix. Take another look at step 2; you have to go through sphinx.conf and tack your prefix onto the table names (but not the row names!).  It's annoying, but I eventually figured out that there are annoyances all over the place with MediaWiki if you use a prefix.  My advice is to bite the bullet now and get rid of the prefix now if it's an option. —Emufarmers(T 03:51, 6 August 2008 (UTC)

Is there any way to prevent Sphinx from indexing particular pages?
I realize this runs counter to what most people would want, but some pages don't need to be indexed. I've made some reasonable searches here and on the Sphinx site, and believe this is more relevant to a MediaWiki discussion than Sphinx in general. Jon Doran, 9 May 2008


 * You could modify the query in sphinx.conf to filter out any pages you do not want. It could be done based on namespace, a join with some other table (e.g. categorylinks,) or some new field or table you would create yourself. Svemir Brkic 01:23, 10 May 2008 (UTC)


 * Thanks for the suggestions. I did not consider the query, but now that you mention it, there is a lot I can do with it.  Jon Doran, 10 May 2008

How to search Chinese
I'm chinese wiki's admin, and I have configed sphinx.conf follow all the steps. Now I can search English words correctly but if I just search only chinese words like "注册"， I can get nothing? That means i must search mixed words like "sbc 注册"，so it can give my right results. Who can help me？ --Fzy 163 01:21, 15 May 2008 (UTC)


 * Initial term check was too strict and it would not let Unicode-only strings through. I have changed it in the CVS, but you can see below how to fix it in the version you are currently using. Svemir Brkic 13:14, 16 June 2008 (UTC)


 * thanks a lot, it get works ^_^

Having trouble with the Sort By command
I'm trying to use the Sort By command so that I can see the latest postings on our wiki in the search, but I'm not sure of the syntax - what should it be? $wgSphinxSearch_sortby = "SPH_SORT_TIME_SEGMENTS, 'rev_timestamp'"; doesn't seem to work - I added rev_timestamp to the SQL query for the indexing, but no luck - can someone help me?


 * The way the code currently works, it always uses SPH_SORT_EXTENDED as the sort mode, and only uses $wgSphinxSearch_sortby as a second argument in the SetSortMode call. I will make this more flexible, but until then you can edit SphinxSearch_body.php and find this line:

$cl->SetSortMode(SPH_SORT_EXTENDED, $wgSphinxSearch_sortby);


 * Set your $wgSphinxSearch_sortby to 'rev_timestamp' and change above line to:

$cl->SetSortMode(SPH_SORT_TIME_SEGMENTS, $wgSphinxSearch_sortby);

Svemir Brkic 02:52, 13 August 2008 (UTC)

Blank search check
In function «wfSphinxSearch», please, replace line if (!preg_match('/[\w\d]/', $term)) { to line if (!preg_match('/[\w\pL\d]/u', $term)) {

Because first variant ignore non-latin unicode quieries (for example russian search terms).


 * I could not get this to work on my installation for some reason - \pL would not match any random Russian word I copy-pasted. Internally, PHP saw those words as \x... sequences, but they just would not match - maybe they were not fully valid UTF-8. However, I am not sure why we would go to such great lengths here anyway. I have changed it to:

if (trim($term) === '' ) {


 * and it works fine now. I committed it to CVS and will make it a part of the next release. Thanks for pointing this out. Svemir Brkic 13:04, 16 June 2008 (UTC)

Special:Search not recognized
I'm running SphinxSearch 0.6 with MediaWiki 1.12, and Special:SphinxSearch works great. However, when I set Special:Search yields a No such special page error. —Emufarmers(T 11:22, 12 June 2008 (UTC)


 * Are your above lines followed by this:


 * Svemir Brkic 12:24, 16 June 2008 (UTC)


 * Er, no, I hadn't seen that code before; I had ExtensionFunctions installed, which I assumed was sufficient. Adding those lines (and adjusting the paths) does seem to make things work, but I'm a bit confused about why they're necessary. —Emufarmers(T 22:29, 16 June 2008 (UTC)


 * require line is necessary to make sure extAddSpecialPage function is available. It is a function inside ExtensionFunctions.php file. Basically, it provides a backwards-compatible way of adding a sepcial page. The conditional in extAddSpecialPage call specifies whether Sphinx replaces the default Search special page or is used as a stand-alone search page. Svemir Brkic 01:05, 17 June 2008 (UTC)

Problem with SphinxSearch_body?
I have installed Sphinx Search on a wiki (MW1.10), running on Ubuntu Hardy. When I go to the Special:SphinxSearch page, I get the following error...

Catchable fatal error: Object of class Title could not be converted to string in /var/www/wiki/extensions/SphinxSearch_body.php on line 456

Please advise. Eric --87.86.35.34 15:10, 11 July 2008 (UTC)


 * What version of PHP are you using? Title class has a __toString method, which in PHP 5 makes conversion to string possible. If you are using PHP 4, try changing any place that does  (or something like that) to   Svemir Brkic 15:29, 11 July 2008 (UTC)

root@wiki:/var/www/wiki/extensions# php --version PHP 5.2.4-2ubuntu5.1 with Suhosin-Patch 0.9.6.2 (cli) (built: May 9 2008 16:34:16) Copyright (c) 1997-2007 The PHP Group Zend Engine v2.2.0, Copyright (c) 1998-2007 Zend Technologies

Eric --87.86.35.34 15:37, 11 July 2008 (UTC)


 * In that case maybe your version of MW does not have the __toString method in Title.class? If so, above fix would still work. I think I will update extension code to use that anyway. Svemir Brkic 16:50, 11 July 2008 (UTC)

Thanks for your continued help Svemir. I have tried to implement your fix: As far as I can tell I changed any line that dereferences $title to a call to getPrefixedText. I still get the error above though. The offending line 456 has no reference to class Title, and so far as I can tell neither do it's backlinks.

454             $wgOut->addHTML(" 455               465               457              ");

Do you have any suggestions on what I can change further? Thanks again, Eric --87.86.35.34 16:04, 16 July 2008 (UTC)


 * It seems to be the  part. Try   216.52.121.66 16:58, 16 July 2008 (UTC)

Problem solved! Good job mystery user! Thanks again Svemir - great job. Eric --87.86.35.34 08:55, 17 July 2008 (UTC)

Install directions confusing
I just put everything in a directory called "SphinxSearch" in my extensions directory, and included that from my LocalConfig file, and it worked. The code actually references the "SphinxSearch" directory, so I don't understand the flat layout proposed by the installation guide. Djbclark 21:56, 14 July 2008 (UTC)

Question when searching for IP's
We use the Wiki here in an IT setting so many of our articles refer IP addresses. The default search does not find any variation of IPs when searched (for example 102., 102.160.2.2, 106..etc.) Can anyone tell me if this search does a better job with this? Thanks.
 * It would certainly do a better job than MySQL full-text index - even in default configuration. You could also tweak it further, but I am not sure I fully understand what exactly you need. If you provide a some specific examples of data and search strings that should match it, I can test it. Svemir Brkic 22:45, 15 July 2008 (UTC)

Sure. Say that I have a few articles that have the line of text 192.165.1.0 in them. So, if searching for 192.165.1.0, would it return any results? Or variations of it, such as "192.165"?
 * Yes, both searches will match that article. It will consider 192, 165, 1, and 0 as separate "words". You can tell it whether to search for all those words or any of them (it is an option on the search page, but you can also change the default.) Since proximity of the matched words is an important factor, you will get the articles that have entire IP in them first. Svemir Brkic 16:46, 18 July 2008 (UTC)

Installing issues
I am trying to install SphinxSearch 0.9.8 on Linux RHEL with mySQL. I did the ./configure and everything seemed fine. Then when build the binaries with make, I get the follwing:

[root@BUILD sphinx-0.9.8]# make Making all in src make[1]: Entering directory `/root/sphinx-0.9.8/src' if test -d ../.svn; then svn info .. --xml | perl svnxrev.pl; fi; make all-am make[2]: Entering directory `/root/sphinx-0.9.8/src' if g++ -DHAVE_CONFIG_H -I. -I. -I../config -DSYSCONFDIR="\"/usr/local/etc\"" -I/usr/local/include -I/usr/include/mysql -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic -fasynchronous-unwind-tables -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -fno-strict-aliasing -fwrapv   -Wall -g -D_FILE_OFFSET_BITS=64 -O3 -DNDEBUG -MT sphinx.o -MD -MP -MF ".deps/sphinx.Tpo" -c -o sphinx.o sphinx.cpp; \ then mv -f ".deps/sphinx.Tpo" ".deps/sphinx.Po"; else rm -f ".deps/sphinx.Tpo"; exit 1; fi In file included from sphinx.cpp:14: sphinx.h:54:19: error: mysql.h: No such file or directory sphinx.h:1102: error: ISO C++ forbids declaration of âMYSQL_RESâ with no type sphinx.h:1102: error: expected â;â before â*â token sphinx.h:1103: error: ISO C++ forbids declaration of âMYSQL_FIELDâ with no type sphinx.h:1103: error: expected â;â before â*â token sphinx.h:1104: error: âMYSQL_ROWâ does not name a type sphinx.h:1105: error: âMYSQLâ does not name a type sphinx.cpp: In constructor âCSphSource_MySQL::CSphSource_MySQL(const char*)â: sphinx.cpp:16697: error: class âCSphSource_MySQLâ does not have any field named âm_pMysqlResultâ sphinx.cpp:16698: error: class âCSphSource_MySQLâ does not have any field named âm_pMysqlFieldsâ sphinx.cpp:16699: error: class âCSphSource_MySQLâ does not have any field named âm_tMysqlRowâ sphinx.cpp: In member function âvirtual void CSphSource_MySQL::SqlDismissResultâ: sphinx.cpp:16706: error: âm_pMysqlResultâ was not declared in this scope sphinx.cpp:16709: error: âm_pMysqlResultâ was not declared in this scope sphinx.cpp:16709: error: âmysql_free_resultâ was not declared in this scope sphinx.cpp:16711: error: âm_pMysqlFieldsâ was not declared in this scope sphinx.cpp: In member function âvirtual bool CSphSource_MySQL::SqlQuery(const char*)â: sphinx.cpp:16717: error: âm_tMysqlDriverâ was not declared in this scope sphinx.cpp:16717: error: âmysql_queryâ was not declared in this scope sphinx.cpp:16720: error: âm_pMysqlResultâ was not declared in this scope sphinx.cpp:16720: error: âm_tMysqlDriverâ was not declared in this scope sphinx.cpp:16720: error: âmysql_use_resultâ was not declared in this scope sphinx.cpp:16721: error: âm_pMysqlFieldsâ was not declared in this scope sphinx.cpp: In member function âvirtual bool CSphSource_MySQL::SqlIsErrorâ: sphinx.cpp:16728: error: âm_tMysqlDriverâ was not declared in this scope sphinx.cpp:16728: error: âmysql_errnoâ was not declared in this scope sphinx.cpp: In member function âvirtual const char* CSphSource_MySQL::SqlErrorâ: sphinx.cpp:16734: error: âm_tMysqlDriverâ was not declared in this scope sphinx.cpp:16734: error: âmysql_errorâ was not declared in this scope sphinx.cpp: In member function âvirtual bool CSphSource_MySQL::SqlConnectâ: sphinx.cpp:16740: error: âm_tMysqlDriverâ was not declared in this scope sphinx.cpp:16740: error: âmysql_initâ was not declared in this scope sphinx.cpp:16743: error: âmysql_real_connectâ was not declared in this scope sphinx.cpp: In member function âvirtual void CSphSource_MySQL::SqlDisconnectâ: sphinx.cpp:16749: error: âm_tMysqlDriverâ was not declared in this scope sphinx.cpp:16749: error: âmysql_closeâ was not declared in this scope sphinx.cpp: In member function âvirtual int CSphSource_MySQL::SqlNumFieldsâ: sphinx.cpp:16755: error: âm_pMysqlResultâ was not declared in this scope sphinx.cpp:16758: error: âm_pMysqlResultâ was not declared in this scope sphinx.cpp:16758: error: âmysql_num_fieldsâ was not declared in this scope sphinx.cpp: In member function âvirtual bool CSphSource_MySQL::SqlFetchRowâ: sphinx.cpp:16764: error: âm_pMysqlResultâ was not declared in this scope sphinx.cpp:16767: error: âm_tMysqlRowâ was not declared in this scope sphinx.cpp:16767: error: âm_pMysqlResultâ was not declared in this scope sphinx.cpp:16767: error: âmysql_fetch_rowâ was not declared in this scope sphinx.cpp: In member function âvirtual const char* CSphSource_MySQL::SqlColumn(int)â: sphinx.cpp:16774: error: âm_pMysqlResultâ was not declared in this scope sphinx.cpp:16777: error: âm_tMysqlRowâ was not declared in this scope sphinx.cpp: In member function âvirtual const char* CSphSource_MySQL::SqlFieldName(int)â: sphinx.cpp:16783: error: âm_pMysqlResultâ was not declared in this scope sphinx.cpp:16786: error: âm_pMysqlFieldsâ was not declared in this scope sphinx.cpp:16787: error: âm_pMysqlResultâ was not declared in this scope sphinx.cpp:16787: error: âmysql_fetch_fieldsâ was not declared in this scope sphinx.cpp:16789: error: âm_pMysqlFieldsâ was not declared in this scope make[2]: *** [sphinx.o] Error 1 make[2]: Leaving directory `/root/sphinx-0.9.8/src' make[1]: *** [all] Error 2 make[1]: Leaving directory `/root/sphinx-0.9.8/src' make: *** [all-recursive] Error 1

SOLUTION I had a similar issue on FC9. I did "yum install mysql-devel" and that fixed it. Try installing the mysql-devel version for your mysql install and then building sphinx.