Extension talk:SphinxSearch

About this board

Archives 
The talk page now uses liquid threads, all open messages prior to liquid threads have been converted into a talk subpage and can be found here.
It is recommended to use SphinxSearch 0.8.5 and a recent stable release of Sphinx 2.1. Please bear in mind that this extension only handles the communication between MediaWiki and Sphinx and any specific issues related to the search feature (character sets, ability to search with *, search categories, minimum length on search terms etc.) are handled in Sphinx (see sphinx.conf file) and those questions should be redirected to the Sphinx forum.
The development on this extension is done on a voluntary basis and while this forum provides a platform to share experiences and solutions, it is up to its community members to fill in suggestions.

Help

When seeking help and/or support, you might want to consider to mention your system environment (SphinxSearch Extension version, Sphinx version, MW version etc.) otherwise it might be difficult for people to make appropriate recommendations.

For Windows users and related issues see here, for Linux users and related issues see here, and some advice on how to configure a SQLite setup see here.


Issues with SphinxSearch in 1.39

2
46.31.222.226 (talkcontribs)

We upgrade from the version 1.35.8 LTS to 1.39.1 LTS and we get the following Issues:

1- "connection to localhost:9312 failed (errno=111, msg=Connection refused)"

2- "Interner Fehler: received zero-sized searchd response"


Hier are the fix for this issues:

  • SphinxMWSearchResult:

@@ -20,9 +20,10 @@ class SphinxMWSearchResult extends RevisionSearchResult {

    * @param stdClass $row

    * @param SphinxClient|null $sphinx_client

    */

   public function __construct( $row, $sphinx_client ) {

   public function __construct( $row, $sphinx_client, $terms ) {

       $this->sphinx_client = $sphinx_client;

       $this->initFromTitle( Title::makeTitle( $row->page_namespace, $row->page_title ) );

               $this->terms = $terms;

   }

   /**

@@ -40,9 +41,9 @@ public function getTextSnippet( $terms = [] ) {

       if ( $wgSphinxSearchMWHighlighter ) {

           $h = new SearchHighlighter();

           if ( $wgAdvancedSearchHighlighting ) {

               return $h->highlightText( $this->mText, $terms, $contextlines, $contextchars );

               return $h->highlightText( $this->mText, $this->terms, $contextlines, $contextchars );

           } else {

               return $h->highlightSimple( $this->mText, $terms, $contextlines, $contextchars );

               return $h->highlightSimple( $this->mText, $this->terms, $contextlines, $contextchars );

           }

       }

@@ -57,7 +58,7 @@ public function getTextSnippet( $terms = [] ) {

       $excerpts = $this->sphinx_client->BuildExcerpts(

           [ $this->mText ],

           $wgSphinxSearch_index,

           implode( ' ', $terms ),

           implode( ' ', $this->terms ),

           $excerpts_opt

       );


  • SphinxMWSearchResultSet:

@@ -245,7 +245,7 @@ public function next() {

       if ( isset( $this->mResultSet[$this->mNdx] ) ) {

           $row = $this->mResultSet[$this->mNdx];

           ++$this->mNdx;

           return new SphinxMWSearchResult( $row, $this->sphinx_client );

           return new SphinxMWSearchResult( $row, $this->sphinx_client, $this->mTerms );

       } else {

           return false;

       }

Rrosenfeld (talkcontribs)

Many thanks for this patch! I observed the same issue as you and this patch solved it for me, too. Since something broke the formatting of your patch, here's what I used:

--- SphinxMWSearchResult.php.org	2023-03-14 09:32:43.000000000 +0100
+++ SphinxMWSearchResult.php	2023-06-29 15:47:04.311199198 +0200
@@ -20,9 +20,10 @@
 	 * @param stdClass $row
 	 * @param SphinxClient|null $sphinx_client
 	 */
-	public function __construct( $row, $sphinx_client ) {
+	public function __construct( $row, $sphinx_client, $terms ) {
 		$this->sphinx_client = $sphinx_client;
 		$this->initFromTitle( Title::makeTitle( $row->page_namespace, $row->page_title ) );
+		$this->terms = $terms;
 	}
 
 	/**
@@ -40,9 +41,9 @@
 		if ( $wgSphinxSearchMWHighlighter ) {
 			$h = new SearchHighlighter();
 			if ( $wgAdvancedSearchHighlighting ) {
-				return $h->highlightText( $this->mText, $terms, $contextlines, $contextchars );
+				return $h->highlightText( $this->mText, $this->terms, $contextlines, $contextchars );
 			} else {
-				return $h->highlightSimple( $this->mText, $terms, $contextlines, $contextchars );
+				return $h->highlightSimple( $this->mText, $this->terms, $contextlines, $contextchars );
 			}
 		}
 
@@ -57,7 +58,7 @@
 		$excerpts = $this->sphinx_client->BuildExcerpts(
 			[ $this->mText ],
 			$wgSphinxSearch_index,
-			implode( ' ', $terms ),
+			implode( ' ', $this->terms ),
 			$excerpts_opt
 		);
 
--- SphinxMWSearchResultSet.php.org	2023-03-14 09:32:43.000000000 +0100
+++ SphinxMWSearchResultSet.php	2023-06-29 15:48:16.289051002 +0200
@@ -245,7 +245,7 @@
 		if ( isset( $this->mResultSet[$this->mNdx] ) ) {
 			$row = $this->mResultSet[$this->mNdx];
 			++$this->mNdx;
-			return new SphinxMWSearchResult( $row, $this->sphinx_client );
+			return new SphinxMWSearchResult( $row, $this->sphinx_client, $this->mTerms );
 		} else {
 			return false;
 		}
Reply to "Issues with SphinxSearch in 1.39"

Issues with SphinxSearch in 1.34

5
206.130.173.59 (talkcontribs)

I upgraded mediawiki to 1.34 and SphinxSearch to the version that matches version 1.34. When I run a search I get this error:

[895083e3cfe4edcfab999700] /testwiki/index.php?search=Celia&title=Spezial%3ASuche&go=Seite ArgumentCountError from line 28 of /srv/www/htdocs/testwiki/extensions/SphinxSearch/SphinxMWSearchResult.php: Too few arguments to function SphinxMWSearchResult::getTextSnippet(), 0 passed in /srv/www/htdocs/testwiki/includes/widget/search/FullSearchResultWidget.php on line 65 and exactly 1 expected

When I took a look at SphinxMWSearchResult.php found that getTextSnippet is expecting the parameter terms

       public function getTextSnippet( $terms ) {


However I found this in the 1.34 release notes.

* SearchResult::getTextSnippet( $terms ) the $terms param is being deprecated

  and should no longer be passed. Search engine implemenations should be

  responsible for carrying relevant information needed for highlighting with

  their own SearchResultSet/SearchResult sub-classes.


Is the error caused by a bug or by a configuration issue?

Lavamind (talkcontribs)
2601:646:C900:1338:D1E3:28DE:1BAE:1145 (talkcontribs)

Thank you, Lavamind!

195.234.58.40 (talkcontribs)

for 1.35 add the if

       public function getTextSnippet( $terms = [] ) {
               global $wgAdvancedSearchHighlighting, $wgSphinxSearchMWHighlighter, $wgSphinxSearch_index;
               if (empty($terms)) {
                       $terms=$this->mterms ;
               }

195.234.58.40 (talkcontribs)

sorry, I didn't read all your comments:

the solution from Lavamind is the solution

Reply to "Issues with SphinxSearch in 1.34"

Sphinx sql_query parameter broken in MW 1.34+

2
Lavamind (talkcontribs)

If anyone, after upgrading to MW 1.34, is finding out that any recently edited pages disappear completely from the index, fear not. There's a fix.

MediaWiki is changing the manner in which content is referenced in the database. To configure Sphinx to correctly index all pages using the new schema, change the following sphinx.conf parameter:

   sql_query  = SELECT page_id, page_title, page_namespace, page_is_redirect, old_id, old_text FROM page, revision, text WHERE rev_id=page_latest AND old_id=rev_text_id

to:

   sql_query = SELECT page_id, page_title, page_namespace, page_is_redirect, old_id, old_text FROM page, slots, content, text WHERE slots.slot_revision_id=page.page_latest AND content.content_id=slots.slot_content_id AND text.old_id=REPLACE(content.content_address, 'tt:', '')
2601:646:C900:1338:7D7:4344:E56D:609F (talkcontribs)

Thank you!

Reply to "Sphinx sql_query parameter broken in MW 1.34+"

1.35.1 Mediawiki with SphinxSearch functional

1
64.148.137.131 (talkcontribs)

diff -rupN SphinxSearch/SphinxMWSearch.php /opt/SphinxSearch/SphinxMWSearch.php

--- SphinxSearch/SphinxMWSearch.php     2020-07-14 05:23:08.000000000 -0500

+++ /opt/SphinxSearch/SphinxMWSearch.php        2021-03-05 12:51:28.199277717 -0600

@@ -73,7 +73,7 @@ class SphinxMWSearch extends SearchDatab

        }

        public static function infixSearch( $namespaces, $term, $limit, &$results, $offset = 0 ) {

-               $search_engine = new SphinxMWSearch( wfGetDB( DB_REPLICA ) );

+               $search_engine = new SphinxMWSearch( wfGetLB( DB_REPLICA ) );

                $search_engine->namespaces = $namespaces;

                $search_engine->setLimitOffset( $limit, $offset );

                $result_set = $search_engine->searchText( '@page_title: ' . $term . '*' );

diff -rupN SphinxSearch/SphinxMWSearchResult.php /opt/SphinxSearch/SphinxMWSearchResult.php

--- SphinxSearch/SphinxMWSearchResult.php       2020-07-14 05:23:08.000000000 -0500

+++ /opt/SphinxSearch/SphinxMWSearchResult.php  2021-03-05 12:46:22.049271402 -0600

@@ -15,9 +15,10 @@ class SphinxMWSearchResult extends Revis

        public $sphinx_client = null;

-       public function __construct( $row, $sphinx_client ) {

+       public function __construct( $row, $sphinx_client, $terms ) {

                $this->sphinx_client = $sphinx_client;

-               $this->initFromTitle( Title::makeTitle( $row->page_namespace, $row->page_title ) );

+        $this->initFromTitle( Title::makeTitle( $row->page_namespace, $row->page_title ) );

+        $this->terms = $terms;

        }

        /**

@@ -25,7 +26,7 @@ class SphinxMWSearchResult extends Revis

         *

         * @return string highlighted text snippet

         */

-       public function getTextSnippet( $terms ) {

+       public function getTextSnippet( $terms = [] ) {

                global $wgAdvancedSearchHighlighting, $wgSphinxSearchMWHighlighter, $wgSphinxSearch_index;

                $this->initText();

@@ -34,9 +35,9 @@ class SphinxMWSearchResult extends Revis

                if ( $wgSphinxSearchMWHighlighter ) {

                        $h = new SearchHighlighter();

                        if ( $wgAdvancedSearchHighlighting ) {

-                               return $h->highlightText( $this->mText, $terms, $contextlines, $contextchars );

+                               return $h->highlightText( $this->mText, $this->terms, $contextlines, $contextchars );

                        } else {

-                               return $h->highlightSimple( $this->mText, $terms, $contextlines, $contextchars );

+                               return $h->highlightSimple( $this->mText, $this->terms, $contextlines, $contextchars );

                        }

                }

@@ -51,7 +52,7 @@ class SphinxMWSearchResult extends Revis

                $excerpts = $this->sphinx_client->BuildExcerpts(

                        [ $this->mText ],

                        $wgSphinxSearch_index,

-                       implode( ' ', $terms ),

+                       implode( ' ', $this->terms ),

                        $excerpts_opt

                );

diff -rupN SphinxSearch/SphinxMWSearchResultSet.php /opt/SphinxSearch/SphinxMWSearchResultSet.php

--- SphinxSearch/SphinxMWSearchResultSet.php    2020-07-14 05:23:08.000000000 -0500

+++ /opt/SphinxSearch/SphinxMWSearchResultSet.php       2021-03-05 12:47:39.304272996 -0600

@@ -233,7 +233,7 @@ class SphinxMWSearchResultSet extends Se

                if ( isset( $this->mResultSet[$this->mNdx] ) ) {

                        $row = $this->mResultSet[$this->mNdx];

                        ++$this->mNdx;

-                       return new SphinxMWSearchResult( $row, $this->sphinx_client );

+                       return new SphinxMWSearchResult( $row, $this->sphinx_client, $this->mTerms );

                } else {

                        return false;

                }

Reply to "1.35.1 Mediawiki with SphinxSearch functional"
Henryfunk (talkcontribs)
Reply to "Indexing HTML"

1.34 (possibly newer)

1
64.148.137.131 (talkcontribs)

Line 76 of SphinxMWSearch.php is incorrectly coded.


SearchDatabase class requires an ILoadBalance construct

Without you get error:

SearchDatabase::__construct() must implement interface Wikimedia\\Rdbms\\ILoadBalancer,

        instance of Wikimedia\\Rdbms\\MaintainableDBConnRef given,

        called in /var/www/html/mediawiki/extensions/SphinxSearch/SphinxMWSearch.php on line 76


To fix, change line 76 from wfGetDB( DB_REPLICA ) to wfGetLB( DB_REPLICA )

Reply to "1.34 (possibly newer)"

How to customize advanced search namespaces?

1
Shmaro38 (talkcontribs)

I am using SphinxSearch 0.9.1 for MediaWiki 1.31 LTS. I am trying to customize the advanced search (searching using namespaces) that is included in my Special:Search page. I have found the following manual for altering the default namespace: Manual:$wgNamespacesToBeSearchedDefault. However, the changes I make on my LocalSettings.php does not seem to be reflecting on the Special:Search page.


Specifically, I have included the following code:


$wgNamespacesToBeSearchedDefault = [NS_HELP => false, ];


in LocalSettings.php to see if it would disable the "Help" namespace from the advanced search, but there was no change to the page.

Is there something I am doing wrong? Please help. I am new to MediaWiki and php in general, hence I understand that I may be missing a big point. Thank you!

Reply to "How to customize advanced search namespaces?"

When might we expect compatibility with 1.34?

1
WhitWye (talkcontribs)

Although the download page does offer a version for 1.34, at least in my experience that throws errors after installation, for example "[728f57963e6ba6317d0fee1a] 2020-01-30 15:54:50: Fatal exception of type "ArgumentCountError". To be fair, the page we're discussing here only shows compatibility up through 1.33. Is an update something we should expect soon?

Reply to "When might we expect compatibility with 1.34?"

Sphinx Extension Doesnt work for me

2
Bsashidh (talkcontribs)

I am new to media wiki and for that matter to any web application development. I was exploring Mediawiki and have installed "MediaWiki 1.18.2" and have attempted to integrate Sphinx search. I have used "Sphinx 2.0.4-release".

The problem I am facing is that I am unable to search through the contents of the pages or the documents embedded in the MediWiki. It is only able to search the Title of the Pages. This is the same irrespective of the "GO" button or the "SEARCH" button. Whereas Through the command in the Sphinx Search, I am able to find all the text. Can someone please help me in debugging this.

MWJames (talkcontribs)

The documentation is quite extensive (compared with other extensions), and it might be that you need to do the setup more than ones to fully understand the implication in how searchd, indexer and SphinxSearch is working together. If you follow the steps outlined in the documentation you should be able get the extension working but it will be hard for anyone to give any guidance as long their are no particular problems to solve.

For example did you do "Step 4 - Test Out Sphinx" to ensure that the infrastructure is setup correctly otherwise if this step fails something is wrong in your configuration and searching within MediaWiki will definitely not work as advertised.

In order to use Sphinx in MediaWiki you have register the extension, see "Step 9 - Extension Installation - Local Settings"

You should also consult the Extension:SphinxSearch/Troubleshooting page.

SphinxSearch and MediaWiki 1.19.0 -- fixed

4
212.23.156.114 (talkcontribs)

After migraging to MediaWiki 1.19.0 SphinxSearch 0.85 does not work on the Spechial:Search page anymore (no results). The sidebar Search works.

After logging the searchd with --console, you found a message "query error: no such filter attribute 'page_is_redirect'" in the console. I searched this text and found it in 'SphinxMWSearch.php'. I change the line

$cl->SetFilter( 'page_is_redirect', array( 0 ) );

to

// $cl->SetFilter( 'page_is_redirect', array( 0 ) );

and everything works (for me).

I also change the method 'userHighlightPrefs' to

	public static function userHighlightPrefs() {return array( 2, 75 ); }

because the the definition changed (and I got warning when I activated "$wgShowExceptionDetails" in LocalSettings.php).

I hope this helps others fixing the same problems.

Svemir Brkic (talkcontribs)

Perhaps page_is_redirect is missing in the sphinx.conf? It did not use to be there by default, but in more recent versions you should see it in at least three places in your sphinx.conf, in both sql_query settings, and in one sql_attr_uint setting. After you change the config file, you need to reindex the wiki.

212.23.156.114 (talkcontribs)

Hello Svemir,

thanx, that was the solution.

MWJames (talkcontribs)

During our tests with MW 1.19 / SphinxSearch 0.8.5 we haven't seen any problems and therefore are able to run SphinxSearch 0.8.5 without any changes on MW 1.19.