Extension talk:CirrusSearch/2021

Discussion related to the CirrusSearch MediaWiki extension.

See also the open tasks for CirrusSearch on phabricator.

Does CirrusSearch use search index in 'searchindex'?

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

It seems like CirrusSearch only use these from Elasticsearch but not MySQL tables. Key to Sky-{遠い空へ}-(讨论|贡献) 21:38, 10 January 2021 (UTC)

Is it safe to delete searchindex since I'm using CirrusSearch as default search backend? Key to Sky-{遠い空へ}-(讨论|贡献) 21:43, 10 January 2021 (UTC)

It doesn't use the searchindex table. However, to prevent errors in an event that you disable this extension, you can truncate the table (delete the contents of the table) instead of dropping it Ciencia Al Poder (talk) 23:19, 10 January 2021 (UTC)

The discussion above is closed. Please do not modify it. No further edits should be made to this discussion.

Search index cannot be recreated after upgrade to 1.35.1

After upgrading from 1.34.4 to 1.35.1 (CirrusSearch 6.5.4 (ca9d1b7) 01:59, 1. Jan. 2021) I tried to recreate the search index:

php /var/www/html/extensions/CirrusSearch/maintenance/UpdateSearchIndexConfig.php --startOver
php /var/www/html/extensions/CirrusSearch/maintenance/ForceSearchIndex.php

Second command leads to the following error:

JobQueueError from line 889 of /var/www/html/includes/jobqueue/JobQueueDB.php: Wikimedia\Rdbms\DBQueryError: Error 1062: Duplicate entry 'cirrusSearchElasticaWrite--1-' for key 'job_cmd' (database)
Function: JobQueueDB::doBatchPush
Query: INSERT INTO `job` (job_cmd,job_namespace,job_title,job_params,job_timestamp,job_sha1,job_random) VALUES ('cirrusSearchElasticaWrite',-1,'',…

#0 /var/www/html/includes/jobqueue/JobQueueDB.php(280): JobQueueDB->getDBException(Object(Wikimedia\Rdbms\DBQueryError))
#1 /var/www/html/includes/jobqueue/JobQueueDB.php(217): JobQueueDB->doBatchPushInternal(Object(Wikimedia\Rdbms\DatabaseMysqli), Array, 0, 'JobQueueDB::doB...')
#2 /var/www/html/includes/libs/rdbms/database/Database.php(3929): JobQueueDB->{closure}(Object(Wikimedia\Rdbms\DatabaseMysqli))
#3 /var/www/html/includes/libs/rdbms/database/DBConnRef.php(68): Wikimedia\Rdbms\Database->onTransactionPreCommitOrIdle(Object(Closure), 'JobQueueDB::doB...')
#4 /var/www/html/includes/libs/rdbms/database/DBConnRef.php(609): Wikimedia\Rdbms\DBConnRef->__call('onTransactionPr...', Array)
#5 /var/www/html/includes/jobqueue/JobQueueDB.php(216): Wikimedia\Rdbms\DBConnRef->onTransactionPreCommitOrIdle(Object(Closure), 'JobQueueDB::doB...')
#6 /var/www/html/includes/jobqueue/JobQueue.php(374): JobQueueDB->doBatchPush(Array, 0)
#7 /var/www/html/includes/jobqueue/JobQueue.php(344): JobQueue->batchPush(Array, 0)
#8 /var/www/html/includes/jobqueue/JobQueueGroup.php(171): JobQueue->push(Array)
#9 /var/www/html/extensions/CirrusSearch/includes/Updater.php(380): JobQueueGroup->push(Array)
#10 /var/www/html/extensions/CirrusSearch/includes/Updater.php(216): CirrusSearch\Updater->pushElasticaWriteJobs(Array, Object(Closure))
#11 /var/www/html/extensions/CirrusSearch/maintenance/ForceSearchIndex.php(225): CirrusSearch\Updater->updatePages(Array, 0)
#12 /var/www/html/maintenance/doMaintenance.php(107): CirrusSearch\Maintenance\ForceSearchIndex->execute()
#13 /var/www/html/extensions/CirrusSearch/maintenance/ForceSearchIndex.php(678): require_once('/var/www/html/m...')
#14 {main}

I emptied the job table and tried it again => same error.

Any suggestions? A2yp (talk) 08:51, 22 January 2021 (UTC)

Since some versions, CirrusSearch recommends to set jobs in Redis. As the extension page says:

• Due to the actual handling of jobs by the CirrusSearch extension, it is advisable to set up jobs in Redis to prevent messages like Notice: unserialize(): Error at offset 64870 of 65535 bytes in JobQueueDB.php and subsequent errors like Unsupported operand types. See task T157759.

Maybe there's some bug that went unnoticed because of this recommendation. You can bugreport it. Ciencia Al Poder (talk) 12:12, 22 January 2021 (UTC)

Works with Redis. Thank You! A2yp (talk) 17:15, 22 January 2021 (UTC)

The problem of breaking index of elasticsearch

I am running Personal MediaWiki (version 1.35.1) with CirrusSearch and elasticsearch (6.5.4). Two weeks ago, the elasticsearch index suddenly disappeared. Running UpdateSearchIndexConfig.php in CirrusSearch will recreate the index.

What could cause elasticsearch indexes to suddenly disappear? Naramoksu (talk) 05:15, 30 January 2021 (UTC)

See my post above yours on here. I was having the same problem, my indexes disappeared every two or three weeks. I think my server was being hit with a “meow” attack because port 9200 was accidentally open to the outside world.

It could be that you have port 9200 open. Or like me, you could have ES running in docker and think you’ve got it locked down since you’re using a firewall to block port 9200. The thing is, docker apparently bypasses the firewall rules by default. But there’s a solution for that in the link I provided.

I just implemented the fix today, so I don’t know if this is actually the fix, but reading about the meow attack, it seems to check out. Lucasjkr (talk) 01:25, 11 March 2021 (UTC)

When an index is recreated (the suggestions index at least), a new index is created, the alias is updated to point to the new one, and the old index removed. I haven't seen Cirrus Search removing indices by itself without creating a replacement one. Ciencia Al Poder (talk) 21:21, 30 January 2021 (UTC)

This problem seems to be due to my wiki side. At that time, my wiki was sometimes overloaded while running runJobs.php and the page did not open, and in the process, the ElasticSearch index was broken. Currently, it is not recurring after adjusting the frequency. 125.186.13.171 (talk) 10:07, 9 May 2021 (UTC)

Elasticsearch Dies every week or two weeks, no idea why!

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

I'm running mediawiki 1.35 with Elasticsearch 6.5.4, and every so often, could be every week or two weeks, Elasticsearch deletes its indexes and doesn't bring them back, bringing searching our wiki to a halt. Looking at the logs doesn't really yield anything else, except that this is occurring during maintenance of some sort.

I do see lots of warnings about the use of deprecated fields, so I was thinking/hoping that perhaps I needed to upgrade ES, but it appears that MW doesn't support ES 6.8 or later, so that's not the solution.

Has anyone else encountered this?

Other relevant info includes:

The wiki is tiny: 220 pages. ES isn't required, but it's search is a lot better than what's built into MW, so we'd LIKE to keep using it. I have two ES nodes running, each with 2GB RAM allocated to them. Again, its a tiny wiki, so I can't imagine that the issue is memory related.

But ever few weeks, searching the wiki yields no results and an error (it's working properly at the moment, so I can't tell you the exact words, unfortunately). ES is still running, the logs say the indexes were deleted. To restore operations, I have to bring down ES, delete all of its data, start it up again, and run the following commands:

Cirrussearch/maintenance/UpdateSearchConfig.php

Cirrussearch/maintenance/ForceSearchIndex.php --skipLinks --indexOnSkip

Cirrussearch/maintenance/ForceSearchIndex.php --skipParse

MW/maintenance/runjobs.php --maxtime = 3600

I'm really at wits end here. Short of writing a script that brings the wiki down for maintenance, blows away the indexes and repopulates them all at a time when its not being used, I'm not sure what else to do.

Does this sound familiar to anyone else here?

Thanks! Lucasjkr (talk) 22:31, 9 March 2021 (UTC)

> this is occurring during maintenance of some sort

Can you elaborate on that? What kind of maintenance is happening when indexes are deleted? Ciencia Al Poder (talk) 22:59, 9 March 2021 (UTC)

This instance started up on 2/17/2021, worked fine right up until last weekend, then Monday was not returning search results. Here are the last log entries from it. I'm posting a link to an image, because I'm afraid they'll just post as a long blob here.

https://imgur.com/a/24e2UWr

You can see that the maintenance deletes the indexes and then appears to repopulate (when its warning about deprecated fields), but after a couple weeks, it deletes the index templates all together and that's when I think the issues arise. Lucasjkr (talk) 14:35, 10 March 2021 (UTC)

This sounds like some external process is doing maintenance on your cluster. Do you perhaps have elasticsearch curator installed and running? Curator is typically used with logstash+elasticsearch to delete indices as they age out. Seems like a long shot, but something along these lines seems plausible.

CirrusSearch itself should only be able to delete indices when using UpdateSearchConfig, and even then only after a new index has been succesfully created and promoted to live usage. EBernhardson (WMF) (talk) 18:52, 10 March 2021 (UTC)

The server has Apache, PHP, Docker (which runs the ES containers).

Could that be because Docker? Anything else I've run in docker containers just runs until I tell it to stop, this maintenance seems like something new. Lucasjkr (talk) 21:52, 10 March 2021 (UTC)

Well, it appears I've been victim to a "meow" attack! The keyword to search for was "MetaDataDeleteIndexService", which brought me to:

https://stackoverflow.com/questions/63067062/elastic-search-indexes-gets-deleted-frequently

On my Ubuntu Server, which is on a public IP, I'm using UFW to deny all traffic, and then allowing access to port 22, 80, 443 from the network range my org uses. Since I can't SSH to the server unless I'm either on site or connected to the VPN, I assumed that access to port 9200 was blocked as well. The thing is, Docker essentially usurps iptables and UFW by default. So, I was pretty surprised to discover that despite not having access to port 22 from home, I could issue a GET request to port 9200 and get a full response from ES!!!

It turns out, that is a VERY wrong assumption, and I found this article that explains why ES was serving connections from public IP's , and what config changes need to be made to remedy the situation. I've already implemented and can now get a response from 9200, while the wiki continues to work fine. So hopefully this is the end of that!

https://www.techrepublic.com/article/how-to-fix-the-docker-and-ufw-security-flaw/

Let this be a warning to others that are running ES inside of Docker (whether for their Mediawiki installations or something else), that they need to bolt down their servers a little more than they thought!

I'm also very thankful that while our Team has been using the wiki for reference materials, we made a conscious decision not to store any non-public info in that wiki, but instead just posted references to where that info could be located. Lucasjkr (talk) 22:48, 10 March 2021 (UTC)

oops, even that Tech Republic article wasn't stopping 9200 from being accessible from the outside world.

Final solution, inside docker-compose, bind the port of the container only to localhost:

ports:

- 127.0.0.1:9200:9200 24.91.31.26 (talk) 15:14, 11 March 2021 (UTC)

The discussion above is closed. Please do not modify it. No further edits should be made to this discussion.

Which port should I use for Elasticsearch working with Mediawiki?

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

As I guess, following the last topics, port 9200 can cause some problems but my server refuses this port so I can allow it for elasticsearch or change it in /etc/elasticsearch/elasticsearch.yml

Any suggestions, please? Tim1mike (talk) 10:56, 11 March 2021 (UTC)

The discussion above is closed. Please do not modify it. No further edits should be made to this discussion.

Exotic Elasticsearch port

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

I am working with AWS, and AWS Elasticsearch servers are allowed to communicate on port 80 only.

I have setup the server in LocalSettings.php this way :

$wgCirrusSearchServers = [ [ 'host' => "elasticsearchserver.amazonaws.com", 'port' => 80 ] ];

It works well with Mediawiki 1.31.

When upgrading to 1.35, the ElasticSearch server is shown in Special:Version

MediaWiki	1.35.1
PHP	7.3.23 (apache2handler)
MySQL	5.7.26-log
Elasticsearch	6.5.4

I use

CirrusSearch

6.5.4 (f835850) 13 juillet 2020 à 11:28

Elastica

6.1.3 (3e3b76f) 13 juillet 2020 à 11:44

then when creating the index with UpdateSearchIndexConfig.php, I get the following error message :

PHP Fatal error:  Declaration of Elasticsearch\Endpoints\Indices\Exists::getParamWhitelist() must be compatible with Elasticsearch\Endpoints\AbstractEndpoint::getParamWhitelist(): array in ....../extensions/Elastica/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Endpoints/Indices/Exists.php on line 62

Do you think it is related to the exotic port ? Any clue ? Emmanuel T. (talk) 13:45, 24 March 2021 (UTC)

This doesn't look related to the port. This looks like a problem with mismatched versions between the Elastica library (ruflin/elastica) and the low level Elasticsearch client library (elasticsearch/elasticsearch). Perhaps composer is giving some errors when it is run? EBernhardson (WMF) (talk) 19:14, 24 March 2021 (UTC)

I am running into the same problem and I am not sure I understand exactly what is happening.

When composer is run I see this:

- Installing elasticsearch/elasticsearch (v6.7.2): Loading from cache

I suspect this may be something to do with the version of Elastic Search. I will try updating that.

... Okay, Elasticsearch 6.7.2 doesn't help me either. same issue. Xdaveyx (talk) 09:52, 25 March 2021 (UTC)

I had this same exact same issue and error.

Like @EBernhardson (WMF) mentioned, it appears to be an issue in the version of ruflin/elastica being used. A recent update to the elastica extension set composer to pull "ruflin/elastica": "6.1.3" instead of the 6.1.1 that was previously being pulled.

For me, I reset the commit locally to the 3e3b76f3b7208167342fee843c401f2587dacde3 commit of REL1_35, then ran composer, which pulled the working version of ruflin/elastica. AnSiOLAtra (talk) 12:18, 25 March 2021 (UTC)

Thankyou @EBernhardson (WMF) that was the clue I needed. It ran for me! Xdaveyx (talk) 13:30, 25 March 2021 (UTC)

If you look at https://github.com/ruflin/Elastica/compare/6.1.1...6.1.3...

There's no change in the constraint of elasticsearch/elasticsearch. There's also very minimum changes to the actual code either.

https://github.com/ruflin/Elastica/blob/6.1.1/composer.json#L17 - "^6.0

https://github.com/ruflin/Elastica/blob/6.1.3/composer.json#L17 - "^6.0"

I'm guessing what caused the issue is maybe https://github.com/wikimedia/mediawiki-extensions-Elastica/commit/c101a4c17fff7e8711b0199cc9f7c342699e1221 which allowed 6.7.2 aswell.

If we look at PHP Fatal error:  Declaration of Elasticsearch\Endpoints\Indices\Exists::getParamWhitelist() must be compatible with Elasticsearch\Endpoints\AbstractEndpoint::getParamWhitelist(): array in ....../extensions/Elastica/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Endpoints/Indices/Exists.php on line 62

https://github.com/elastic/elasticsearch-php/blob/v6.7.2/src/Elasticsearch/Endpoints/Indices/Exists.php#L45

https://github.com/elastic/elasticsearch-php/blob/v6.7.2/src/Elasticsearch/Endpoints/AbstractEndpoint.php#L51

The return type casting isn't there. So the error doesn't seem to match the version that is apparently being installed Reedy (talk) 14:34, 25 March 2021 (UTC)

The 6.8 versions do have that return type... https://github.com/elastic/elasticsearch-php/blob/6.8.x/src/Elasticsearch/Endpoints/Indices/ExistsAlias.php and https://github.com/elastic/elasticsearch-php/blob/6.8.x/src/Elasticsearch/Endpoints/AbstractEndpoint.php

But the 6.8 release has been repeatedly buggy, hence part of the reason for https://github.com/wikimedia/mediawiki-extensions-Elastica/commit/c101a4c17fff7e8711b0199cc9f7c342699e1221 Reedy (talk) 14:45, 25 March 2021 (UTC)

Thank you @EBernhardson (WMF) and @AnSiOLAtra, I reset the commit as explained and it works now. Emmanuel T. (talk) 15:17, 30 March 2021 (UTC)

The discussion above is closed. Please do not modify it. No further edits should be made to this discussion.

give more weight to precise search

so, a user on hewiki searched for a phrase, without adding quotes.

they expected the articles containing the phrase to appear before other articles (with both individual words in title and/or content), but this is not the current order, and one of these articles (the one they were actually looking for) landed in place 83 in the search results.

SUGGESTION:

try a first search pass for the precise phrase (pretending it's quoted), and if this round finds anything, give those articles significant weight, so they will appear in the first 20 at least - methinks at the very top.

we should serve better readers who do not want to deal with "syntax", like quotes, and still expect articles with precise phrase to be on top.

peace קיפודנחש (talk) 21:29, 28 March 2021 (UTC)

Parse Error from MetaNameSpace store when running on PHP 8

PHP 8 isn't supported yet.

We're working on building out a wiki family. We're bringing our first wiki online, with wikiid "fr". When I try to update the Search Index config, I get the following error, and I'm not sure where it's coming from:

extensions/CirrusSearch/maintenance# php updateSearchIndexConfig.php --wiki=fr
indexing namespaces...
[1412852ef0291ba91e117343] [no req]   ParseError from line 79 of /var/www/html/aaen/extensions/CirrusSearch/includes/MetaStore/MetaNamespaceStore.php: syntax error, unexpected token "match"

Backtrace:
#0 /var/www/html/aa-en/extensions/CirrusSearch/includes/MetaStore/MetaStoreIndex.php(119): AutoLoader::autoload()
#1 /var/www/html/aa-en/extensions/CirrusSearch/maintenance/IndexNamespaces.php(35): CirrusSearch\MetaStore\MetaStoreIndex->namespaceStore()
#2 /var/www/html/aa-en/extensions/CirrusSearch/maintenance/updateSearchIndexConfig.php(54): CirrusSearch\Maintenance\IndexNamespaces->execute()
#3 /var/www/html/aa-en/maintenance/doMaintenance.php(107): CirrusSearch\Maintenance\UpdateSearchIndexConfig->execute()
#4 /var/www/html/aa-en/extensions/CirrusSearch/maintenance/updateSearchIndexConfig.php(70): require_once(string)
#5 {main}

Any help would be appreciated! Blinkingline (talk) 21:43, 5 April 2021 (UTC)

Hi,

you most probably use PHP 8 which is not supported yet. DCausse (WMF) (talk) 10:25, 6 April 2021 (UTC)

Hrm...we're running 7.4 from nginx. Let me double check from the CLI. Thanks!

Edit: Yup! PHP8 is installed and is what the CLI is using. Thanks again. Blinkingline (talk) 13:00, 6 April 2021 (UTC)

If you (like I did) really have to use php8, you can do a quick-and-dirty fix. The issue here is, that "match" is a restricted php keyword sinc php 8, so it is no longer available as class name. To fix it you can just search-and-replace it by something else.

cd wherever-your-mediawiki-installation-lives

grep -rlw extensions/CirrusSearch -e 'Match' | xargs -i@ sed -i 's/\([^a-zA-Z]\)Match\([^a-zA-Z]\)/\1ElasticMatch\2/g' @

grep -rlw extensions/Elastica -e 'Match' | xargs -i@ sed -i 's/\([^a-zA-Z]\)Match\([^a-zA-Z]\)/\1ElasticMatch\2/g' @

mv extensions/Elastica/vendor/ruflin/elastica/lib/Elastica/Query/Match.php extensions/Elastica/vendor/ruflin/elastica/lib/Elastica/Query/ElasticMatch.php

Again, I recommend using php7 while php8 is not officially supported. Take a look e.g. into how to set up php-fpm if you need to run different services on the same webserver. Inlimity (talk) 13:43, 14 April 2021 (UTC)

labels

When I do a search, the labels appear in the results and it doesn't look good at all, how do I remove them.

example, searching stones Results:

Stones stones the stones 699 bytes (94 words) - 14:00 23 Apr 2021

and it would have to appear like this

Stones stones the stones 699 bytes (94 words) - 14:00 23 Apr 2021 2806:106E:1E:17A7:EDE:1AC1:6547:8B0B (talk) 00:11, 27 April 2021 (UTC)

I don't see any difference in the examples you posted. Ciencia Al Poder (talk) 07:38, 27 April 2021 (UTC)

excuse me. example result, looks like this: :

<nowiki>

Stones

[[Stones]], ''the stones''...

</nowiki>

I want them to appear without the markup:

[[Stones]], ''the stones''... 2806:106E:1E:17A7:EDE:1AC1:6547:8B0B (talk) 22:12, 27 April 2021 (UTC)

Are you sure you have Extension:CirrusSearch installed? I don't get markup on search results, and I don't have set any special setting that would control that behavior... Ciencia Al Poder (talk) 07:23, 28 April 2021 (UTC)

yes, and elastic too. although thinking about it, I think I only activated them and did not make any configuration... 2806:106E:1E:17A7:EDE:1AC1:6547:8B0B (talk) 00:49, 1 May 2021 (UTC)

elasticsearch

my wiki is in godaddy, how can I use elasticsearch in goddady, do I have to contract the service in elasticsearch ?. when executing UpdateSearchIndexConfig.php it tells me "Couldn't resolve host". I already went up and activated Elastica .Thanks 2806:106E:1E:E35:DEC1:FC4F:3B8A:5D6D (talk) 03:20, 22 May 2021 (UTC)

You need to install ElasticSearch. This requires full server access, which means you need a VPS or bare metal/physical server. Ciencia Al Poder (talk) 21:14, 23 May 2021 (UTC)

CirrusSearch breaks with PHP 8.0

I've updated my MediaWiki install to 1.36.0 with PHP 8.0.4, updated all the extensions to the 1.36 versions, and I'm getting the error below. Any thoughts as to what's going on?

I apologize for replying to my own post, but when I tried to post the version information I was getting Abusefilter-warning-linkspam errors. Breaking it up was the only way I found to get around it.

[05fe5f82e1a0b673c67b307a] /mediawiki/index.php?title=Special%3ASearch&search=test&fulltext=Search ParseError: syntax error, unexpected token "match", expecting ":"
        Backtrace:
        from /var/www/localhost/htdocs/mediawiki/extensions/CirrusSearch/includes/Query/SubPageOfFeature.php(124)
        #0 /var/www/localhost/htdocs/mediawiki/extensions/CirrusSearch/includes/Parser/FullTextKeywordRegistry.php(89): AutoLoader::autoload()
        #1 /var/www/localhost/htdocs/mediawiki/extensions/CirrusSearch/includes/Parser/QueryParserFactory.php(34): CirrusSearch\Parser\FullTextKeywordRegistry->__construct()
        #2 /var/www/localhost/htdocs/mediawiki/extensions/CirrusSearch/includes/Search/SearchQueryBuilder.php(129): CirrusSearch\Parser\QueryParserFactory::newFullTextQueryParser()
        #3 /var/www/localhost/htdocs/mediawiki/extensions/CirrusSearch/includes/CirrusSearch.php(240): CirrusSearch\Search\SearchQueryBuilder::newFTSearchQueryBuilder()
        #4 /var/www/localhost/htdocs/mediawiki/includes/search/SearchEngine.php(95): CirrusSearch\CirrusSearch->doSearchText()
        #5 /var/www/localhost/htdocs/mediawiki/includes/search/SearchEngine.php(187): SearchEngine->{closure}()
        #6 /var/www/localhost/htdocs/mediawiki/includes/search/SearchEngine.php(96): SearchEngine->maybePaginate()
        #7 /var/www/localhost/htdocs/mediawiki/includes/specials/SpecialSearch.php(446): SearchEngine->searchText()
        #8 /var/www/localhost/htdocs/mediawiki/includes/specials/SpecialSearch.php(228): SpecialSearch->showResults()
        #9 /var/www/localhost/htdocs/mediawiki/includes/specialpage/SpecialPage.php(646): SpecialSearch->execute()
        #10 /var/www/localhost/htdocs/mediawiki/includes/specialpage/SpecialPageFactory.php(1386): SpecialPage->run()
        #11 /var/www/localhost/htdocs/mediawiki/includes/MediaWiki.php(309): MediaWiki\SpecialPage\SpecialPageFactory->executePath()
        #12 /var/www/localhost/htdocs/mediawiki/includes/MediaWiki.php(913): MediaWiki->performRequest()
        #13 /var/www/localhost/htdocs/mediawiki/includes/MediaWiki.php(546): MediaWiki->main()
        #14 /var/www/localhost/htdocs/mediawiki/index.php(53): MediaWiki->run()
        #15 /var/www/localhost/htdocs/mediawiki/index.php(46): wfIndexMain()
        #16 {main} Jcwren (talk) 02:07, 1 June 2021 (UTC)

|MediaWiki

|1.36.0

|-

|PHP

|8.0.5 (apache2handler)

|-

|MariaDB

|10.3.27-MariaDB-0+deb10u1

|-

|ICU

|65.1

|} Jcwren (talk) 02:08, 1 June 2021 (UTC)

! colspan="5" |Other

|-

!Extension

!Version

!License

!Description

!Authors

|-

|CirrusSearch

|6.5.4 (264629b) 20:53, 26 May 2021

|GPL-2.0-or-later

|Elasticsearch-powered search for MediaWiki

|Nik Everett, Chad Horohoe, Erik Bernhardson and others

|-

|EditSubpages

|3.5.0 (3fbabba) 23:34, 26 May 2021

|GPL-2.0-only

|Allows sysops to unlock a page and all subpages of that page for anonymous editing via MediaWiki:Unlockedpages

|Ryan Schmidt and Prod

|-

|Elastica

|6.1.3 (9f6e66a) 23:34, 26 May 2021

|GPL-2.0-or-later

|Base Elasticsearch functionality for other extensions by providing Elastica library

|Nik Everett and Chad Horohoe

|-

|Lockdown

|– (2409546) 03:05, 27 May 2021

|GPL-2.0-or-later

|Per namespace group permissions

|Daniel Kinzler, Platonides, Mark A. Hershberger and others

|} Jcwren (talk) 02:08, 1 June 2021 (UTC)

Hi,

please see Compatibility#PHP, sadly 1.36 is not yet compatible with PHP8.

All this work is being tracked in T248925. DCausse (WMF) (talk) 07:11, 1 June 2021 (UTC)

David is right. Note that almost all work on wider PHP 8.0 compatibility testing is stalled on the CirrusSearch update to ElasticSearch 6.7+ to allow us to make progress. Jdforrester (WMF) (talk) 22:34, 1 June 2021 (UTC)

Thank you. The system got updated to PHP 8.0 (although I'm not sure at what point), and MW was running except for the search box. I finally figured out how to get it running under PHP 7.4 after enabling some missing modules (mbstring and a few others). I have to mess with MW/PHP/Apache so infrequently that it's basically a relearning experience every time I get into it. And, of course, some of the distro tools change, I have to figure out if any of the PHP config options are relevant, blah blah blah. I generally try to leave it alone, but recent security updates forced me to lay hands on it. Jcwren (talk) 11:25, 5 June 2021 (UTC)

"Index is unknown retrying..." error on index generation script

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

When attempting to run php /var/www/mediawiki-1.35.2/w/extensions/CirrusSearch/maintenance/UpdateSearchIndexConfig.php, I'm getting the following output:

indexing namespaces...

mw_cirrus_metastore missing, creating new metastore index.

Creating metastore index... mw_cirrus_metastore_first Scanning available plugins...

none

ok

Index is unknown retrying...

I'm not sure how the index could be unknown when this is the script that's supposed to generate it. Here's my settings:

$wgCirrusSearchIndexBaseName = $wgDBname;

$wgDisableSearchUpdate = true;

$wgCirrusSearchClusters = [

    'default' => [

        [

            'host' => 'search-cirrus-randomcharsid.us-east-1.es.amazonaws.com',

            'port' => 443,

            'scheme' => 'https'

        ]

    ]

];

Any ideas on why this might be the case? Thanks in advance! Jeffrey Wang 05:46, 10 June 2021 (UTC)

The message `Index is unknown retrying...` comes from the code that is waiting for the new metastore index to report it is fully created and healthy. While not particularly clear, it seems this means the request to create the index was submitted, but then the later requests to ask about the status of this new index are reporting no such index currently exists. Critically it appears no check is being performed against the index creation request response, it seems likely the index creation request is failing and then Cirrus bails when checking that index.

Cirrus will need to be adjusted to do a better job at error reporting here, but that will only improve reporting it wouldn't fix your actual problem. My first guess would be, which version of elasticsearch are you using? I'm pretty sure the existing metastore index would be rejected by elastic >= 7.0. EBernhardson (WMF) (talk) 19:37, 10 June 2021 (UTC)

Hi @EBernhardson (WMF), thanks for your reply! I am using Elasticsearch 6.5.4 on AWS's managed version of Elasticsearch, and using MediaWiki 1.35 (and CirrusSearch and Elastica are both on the REL1_35 branch of Git).

If it helps, I have "elasticsearch/elasticsearch": "6.7.2" in my composer.local.json because of T276854. So I guess I'm already laughing a bit at myself for picking two different versions. I'm using 6.5.4 because of the recommendation on the CirrusSearch extension page, but I'm wondering if it's safe to go up to 6.7 or 6.8. Jeffrey Wang 03:05, 11 June 2021 (UTC)

MediaWiki 1.33.x to 1.36.x require Elasticsearch 6.5.x (6.5.4 recommended).

Any other version (even newer ones than the requirement) will report ES is not compatible and fail instantly. (or at least that was happening some versions ago) Ciencia Al Poder (talk) 08:12, 11 June 2021 (UTC)

@Ciencia Al Poder The problem is that when I don't add that line into my composer.local.json I get the issue reported at T276854. Jeffrey Wang 08:23, 11 June 2021 (UTC)

I stand corrected, I get the error reported at https://phabricator.wikimedia.org/T267106 Jeffrey Wang 18:42, 11 June 2021 (UTC)

The discussion above is closed. Please do not modify it. No further edits should be made to this discussion.

Cirrus does not work on namespace

Hello,

On our wiki, it seems that cirrus systematically returns an error when searching for files, models, modules, etc.

For exemple : fallout-wiki.com /index.php?search=Template%3Aico&title= Sp%C3%A9cial%3ARecherche&profile=advanced&fulltext=1&ns0=1

Do you know where this problem could come from?

Sincerely, Kims79 (talk) 17:53, 16 June 2021 (UTC)

Could you provide more details about the errors? Errors might be found in the mediawiki logs and in the elasticsearch logs.

If the problem is only when searching non-default namespaces it is possible that the general index was not properly created. You can check this by listing all the indices from elasticsearch: curl elastic_host:9200/_cat/indices. If you don't see an index named mywiki_general but only mywiki_content then it's probably the reason of your problem.

To solve this you must create and populate the general index running:

• php $MW_INSTALL_PATH/extensions/CirrusSearch/maintenance/UpdateOneSearchIndexConfig.php --type general
• Rerun the ForceSearchIndex scripts like you did when setting up CirrusSearch (see the README file) DCausse (WMF) (talk) 07:20, 18 June 2021 (UTC)

Search results empty, nothing seems to have been indexed

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

MediaWiki: 1.35.2

CirrusSearch: 6.5.4 (203237e) 03:44, 9 May 2021

Elastica: 6.1.3 (8af6b45) 21:04, 9 May 2021

Elasticsearch: 6.5.4

I've built the search index, and it shows that it ran without any errors:

root@test-snipped-wiki-1:/var/www/wikis/a# php /var/www/mediawiki_shared/extensions/CirrusSearch/maintenance/ForceSearchIndex.php --skipLinks --indexOnSkip --conf ./LocalSettings.php 

[snipped] Indexed 9 pages ending at 10 at 53/second

[...]

Indexed a total of 1616 pages at 103/second

root@test-snipped-wiki-1:/var/www/wikis/a# php /var/www/mediawiki_shared/extensions/CirrusSearch/maintenance/ForceSearchIndex.php --skipParse --conf LocalSettings.php 

[snipped] Indexed 48 pages ending at 50 at 49/second

[...]
Indexed a total of 1616 pages at 72/second




However, all my indices are empty


```

# curl -XGET localhost:9200/_cat/indices

green open wiki_a_general_first     ZYIVvSdQS4C6czO9MajjqQ 4 0  0 0  1kb  1kb

green open mw_cirrus_metastore_first 33OLZIV4TfCiv4JmqcIDYA 1 0 27 6 16kb 16kb

green open wiki_a_content_first     uJkfN6K4S_ytm1cv__9QKw 4 0  0 0  1kb  1kb

green open wiki_a_archive_first     F_ZvMuQBR6SxTtvXtT232A 4 0  0 0  1kb  1kb

```


Elasticsearch is running on the same server as MediaWiki, and I have the following on my LocalSettings.php:


```

wfLoadExtension( 'Elastica' );

wfLoadExtension( 'CirrusSearch' );

$wgSearchType = 'CirrusSearch';

$wgCirrusSearchIndexBaseName =  'wiki_a';

```


Not sure what's going on, if I go to a page and add ?action=cirrusdump I get an empty JSON array `[]`, and all my search results are empty. [[User:AudentioJakeB|AudentioJakeB]] ([[User talk:AudentioJakeB|talk]]) 20:35, 22 June 2021 (UTC)
:Hi,
:CirrusSearch index updates rely on the MediaWiki jobqueue and it's possible that the index requests are kept there or even failed.
:If the documents are kept in the jobqueue they should appear with <code>mwscript showJobs.php --group</code>.
:If this is all empty then perhaps there were some failures and you should check the mediawiki and elasticsearch logs. [[User:DCausse (WMF)|DCausse (WMF)]] ([[User talk:DCausse (WMF)|talk]]) 14:18, 24 June 2021 (UTC)
:Ah yes, seems you're correct! There was an issue in our deploy script that was preventing the job runner from booting up. Have only upgraded our test environment so didn't notice anything with lack of emails or anything else fired off by that system. Thanks! [[Special:Contributions/172.220.104.121|172.220.104.121]] ([[User talk:172.220.104.121|talk]]) 19:04, 24 June 2021 (UTC)
{{Archive bottom}}

== UpdateSearchIndexConfig.php finishes running almost immediately ==
{{Archive top|result=Turn on error reporting|status=resolved}}

I seem to continue having trouble with UpdateSearchIndexConfig.php, so I tried running it with the --startOver flag once. After this happened, I tried running UpdateSearchIndexConfig.php with and without this flag, and all I am getting now is this:

<code>$ php /var/www/mediawiki/w/extensions/CirrusSearch/maintenance/UpdateSearchIndexConfig.php</code>

<code>indexing namespaces...</code>

And then the script finishes executing 0.1 seconds after this is printed out. What's the reason behind this early termination? I have errors turned on, so there isn't anything coming from.

Per what other people have done, I'm using Elasticsearch 6.8. (Please don't tell me to use 6.5.4, I've seen people using 6.8 and it works, I just don't know why it's so difficult to set it up on AWS Elasticsearch as opposed to a self-hosted solution.) [[User:MyWikis-JeffreyWang|Jeffrey Wang]] 19:46, 29 June 2021 (UTC)
:Enable PHP error reporting. It looks like the script is finishing with an error, but error reporting is disabled and no error message is printed at all. [[User:Ciencia Al Poder|Ciencia Al Poder]] ([[User talk:Ciencia Al Poder|talk]]) 07:53, 30 June 2021 (UTC)
:Hmm, I already said "I have errors turned on", but I guess you knew I didn't. Appreciate the reminder. There are so many moving parts to this, it's hard to even set up a simple MVP of it. [[User:MyWikis-JeffreyWang|Jeffrey Wang]] 13:29, 30 June 2021 (UTC)
{{Archive bottom}}

== Elasticsearch server version can't be found ==
{{Flow summary|AWS Elasticsearch won't work with CirrusSearch and shouldn't be used.}}

For some reason, the Elasticsearch server version isn't displaying.
{| class="wikitable"
!Product
!Version
|-
|[https://www.mediawiki.org/ MediaWiki]
|1.35.3
|-
|[https://php.net/ PHP]
|7.4.16 (fpm-fcgi)
|-
|[https://mariadb.org/ MariaDB]
|10.3.27-MariaDB-0+deb10u1
|-
|[http://site.icu-project.org/ ICU]
|65.1
|-
|[http://www.lua.org/ Lua]
|5.1.5
|-
|[https://www.elastic.co/products/elasticsearch Elasticsearch]
|}
Running UpdateOneSearchIndexConfig.php causes:

<code>php maintenance/UpdateOneSearchIndexConfig.php --indexType general --wiki internal</code>

<code>Fetching Elasticsearch version...unable to determine, aborting.</code>

cURL returns:

<code>{
"name" : "OJSpnVJ",
"cluster_name" : "626799937287:mywikis-cirrus-dev",
"cluster_uuid" : "E_kkBLrERAu0e_3k7okq1w",
"version" : {
"number" : "6.8.0",
"build_flavor" : "oss",
"build_type" : "zip",
"build_hash" : "8169a24",
"build_date" : "2021-04-21T19:26:55.782637Z",
"build_snapshot" : false,
"lucene_version" : "7.7.0",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}</code>

Quite odd, any ideas? [[User:MyWikis-JeffreyWang|Jeffrey Wang]] 14:13, 30 June 2021 (UTC)
:Looks like AWS Elasticsearch 6.8 sucks! Don't use it. [[User:MyWikis-JeffreyWang|Jeffrey Wang]] 14:39, 30 June 2021 (UTC)

== Cirrus search not showing auto suggest/auto complete in search engine ==

Hi Using these,
{| class="wikitable"
|[https://www.mediawiki.org/ MediaWiki]
|1.35.1
|-
|[https://php.net/ PHP]
|7.4.15 (apache2handler)
|-
|[https://www.mysql.com/ MySQL]
|8.0.25
|-
|[https://www.elastic.co/products/elasticsearch Elasticsearch]
|6.5.4
|}
i have integarted cirrus search with elastic search , Now ima getting results if i hit enter on the search engine.


Search engine not giving me auto suggestion when typing into search engine,

showing error in console

'''{"error":{"code":"toomanyvalues","info":"Too many values supplied for parameter \"namespace\".'''

'''The limit is 50.","limit":50,"lowlimit":50,"highlimit":500,'''

Pls suggest.. [[User:Pooja2425|Pooja2425]] ([[User talk:Pooja2425|talk]]) 12:39, 1 July 2021 (UTC)
:This seems a reply to an API query. Can you post here the api request URL, or at least the parameters? [[User:Ciencia Al Poder|Ciencia Al Poder]] ([[User talk:Ciencia Al Poder|talk]]) 13:07, 1 July 2021 (UTC)
:Request URL: http://wiki/api.php?action=opensearch&format=json&formatversion=2&search=test&namespace=0%7C2%7C4%7C12%7C202%7C208%7C300%7C302%7C304%7C306%7C308%7C310%7C312%7C314%7C316%7C318%7C320%7C322%7C324%7C326%7C328%7C330%7C332%7C334%7C336%7C338%7C340%7C342%7C346%7C348%7C350%7C352%7C354%7C356%7C358%7C360%7C364%7C368%7C370%7C372%7C374%7C378%7C380%7C382%7C384%7C386%7C388%7C390%7C392%7C396%7C400%7C402%7C404%7C406%7C408%7C410%7C418%7C422%7C424%7C428%7C434%7C436%7C440%7C442%7C444%7C448%7C450%7C452%7C454%7C456%7C458%7C460%7C462%7C464%7C466%7C468%7C470%7C472%7C474%7C476&limit=10 [[User:Pooja2425|Pooja2425]] ([[User talk:Pooja2425|talk]]) 13:15, 1 July 2021 (UTC)
:Actually iam getting desired results after hitting enter into search box, no auto suggestion are there.
:pls help [[User:Pooja2425|Pooja2425]] ([[User talk:Pooja2425|talk]]) 13:15, 1 July 2021 (UTC)
:The list of namespaces that display that query is: 0|2|4|12|202|208|300|302|304|306|308|310|312|314|316|318|320|322|324|326|328|330|332|334|336|338|340|342|346|348|350|352|354|356|358|360|364|368|370|372|374|378|380|382|384|386|388|390|392|396|400|402|404|406|408|410|418|422|424|428|434|436|440|442|444|448|450|452|454|456|458|460|462|464|466|468|470|472|474|476
:You have a list of 80 namespaces to search :O That's not supported. [[User:Ciencia Al Poder|Ciencia Al Poder]] ([[User talk:Ciencia Al Poder|talk]]) 22:03, 1 July 2021 (UTC)
:Hi {{FlowMention|Ciencia Al Poder}} 
:How do I limit these namespaces of Search :O
:When i try to type anything in my search box. Automatically iam getting below in request. How to limit these..ie. 10  0|2|4|12|202|208|300|302|304|306|308|310|312|314|316|318|320|322|324|326|328|330|332|334|336|338|340|342|346|348|350|352|354|356|358|360|364|368|370|372|374|378|380|382|384|386|388|390|392|396|400|402|404|406|408|410|418|422|424|428|434|436|440|442|444|448|450|452|454|456|458|460|462|464|466|468|470|472|474|476 [[User:Pooja2425|Pooja2425]] ([[User talk:Pooja2425|talk]]) 10:01, 13 July 2021 (UTC)

== Cirrus Search not giving auto suggestion/complete when typing into search bar ==
{{Archive top|result=Duplicate of [[Extension talk:CirrusSearch/2021#h-Cirrus_search_not_showing_auto_suggest/auto_complete_in_search_engine-2021-07-01T12:39:00.000Z]]|status=resolved}}

Hi I have integrated the elastic search server with Cirrus search. Its giving me results but auto suggest is not working, Result came after hitting enter .

Request URL of '''Cirrus search''': http://wiki/api.php?action=opensearch&format=json&formatversion=2&search=test&namespace=0%7C2%7C4%7C12%7C202%7C208%7C300%7C302%7C304%7C306%7C308%7C310%7C312%7C314%7C316%7C318%7C320%7C322%7C324%7C326%7C328%7C330%7C332%7C334%7C336%7C338%7C340%7C342%7C346%7C348%7C350%7C352%7C354%7C356%7C358%7C360%7C364%7C368%7C370%7C372%7C374%7C378%7C380%7C382%7C384%7C386%7C388%7C390%7C392%7C396%7C400%7C402%7C404%7C406%7C408%7C410%7C418%7C422%7C424%7C428%7C434%7C436%7C440%7C442%7C444%7C448%7C450%7C452%7C454%7C456%7C458%7C460%7C462%7C464%7C466%7C468%7C470%7C472%7C474%7C476&limit=10


Request URL of '''Sphinx Search :''' 

https://wikidev.equinor.com/wiki/api.php?action=opensearch&format=json&formatversion=2&search=rrr&namespace=0%7C2%7C4%7C12%7C202%7C208%7C300%7C302%7C304%7C306%7C308%7C310%7C312%7C314%7C316%7C318%7C320%7C322%7C324%7C326%7C328%7C330%7C332%7C334%7C336%7C338%7C340%7C342%7C346%7C348%7C350%7C352%7C354%7C356%7C358%7C360%7C364%7C368%7C370%7C372%7C374%7C378%7C380%7C382%7C384%7C386%7C388%7C390%7C392%7C396%7C400%7C402%7C404%7C406%7C408%7C410%7C418%7C422%7C424%7C428%7C434%7C436%7C440%7C442%7C444%7C448%7C450%7C452%7C454%7C456%7C458%7C460%7C462%7C464%7C466%7C468%7C470%7C472%7C474%7C476&limit=10'''&suggest=true'''

SInce iam not getting '''suggest=true into Cirrus Search.'''


Pls suggest, '''I need auto suggestion/complete help when typing into Cirrus Search Bar.''' [[User:Pooja2425|Pooja2425]] ([[User talk:Pooja2425|talk]]) 06:01, 5 July 2021 (UTC)
:What do you get on error console? [[User:Ciencia Al Poder|Ciencia Al Poder]] ([[User talk:Ciencia Al Poder|talk]]) 16:06, 5 July 2021 (UTC)
:{"error":{"code":"'''toomanyvalues'''","info":"Too many values supplied for parameter \"namespace\". The limit is 50.","limit":50,"lowlimit":50,"highlimit":500,"docref":"See http://wikipoc.equinor.com/wiki135/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce&gt; for notice of API deprecations and breaking changes."}}
:I am getting response, but not getting auto complete /auto suggest when typing into search engine as we are getting in default wiki search engine and sphinex search integrated search engine.
:pls suggest , if i have to reduce the limit then how to reduce limit of my request. [[User:Pooja2425|Pooja2425]] ([[User talk:Pooja2425|talk]]) 05:21, 6 July 2021 (UTC)
:Please don't open more than one thread for the same problem [[User:Ciencia Al Poder|Ciencia Al Poder]] ([[User talk:Ciencia Al Poder|talk]]) 14:04, 6 July 2021 (UTC)
{{Archive bottom}}

== Only minimal indexing, most pages are not indexed, almost as ForceSearchIndex.php isn't populating ==
{{Archive top|status=resolved}}

Hi, I have same problem that was already posted in talks and I have tried every possible solution that was presented here:

*[[Extension talk:CirrusSearch/2018#h-CirrusSearch_Only_Partially_Indexing-2018-06-11T15:59:00.000Z|CirrusSearch Only Partially Indexing]]
*[[Extension talk:CirrusSearch/2020#h-ForceSearchIndex.php_isn't_populating_ES-2020-10-24T13:36:00.000Z|ForceSearchIndex.php isn't populating ES]]
*[[Project:Support desk/Flow/2019/08#h-Some_pages_index_and_other_do_not_in_CirrusSearch-2019-08-16T16:28:00.000Z|Some pages index and other do not in CirrusSearch]]

{| class="wikitable"
|Product
|Version
|-
|MediaWiki
|1.36.0
|-
|PHP
|7.3.19 (cgi-fcgi)
|-
|MariaDB
|10.5.11-MariaDB
|-
|ICU
|64.2
|-
|Elasticsearch
|6.5.4
|-
|CirrusSearch
|6.5.4 (264629b)
|-
|Elastica
|6.1.3 (9f6e66a)
|}


I have tried Kibana and eventualy used Elasticsearch Head Chrome extension to find out state of indexes in elasticsearch. I have used '''php Saneitize.php''' to find that pages are not indexed but it was obvious because of 150 pages only 15 pages are indexed.

I did noticed something interesting for this problem analysis but in the end I wasn't able to catch what is happening. After standard set od statements for populating elastic with mediawiki pages

Step 0. -> $wgDisableSearchUpdate = true

Step 1. -> '''php UpdateSearchIndexConfig.php'''

Step 2. -> '''#'''$wgDisableSearchUpdate = true

Step 3. -> '''php ForceSearchIndex.php --skipLinks --indexOnSkip''' 

Step 4. -> '''php ForceSearchIndex.php --skipParse''' 

and restarting elasticsearch service, after some time, few (2-4, usually less than 10) pages would become additionaly indexed.

So, initialy zero, than after some steps with restarting 4 pages, than after some steps 6 pages, than after some steps 15, etc. and than I would not be able to repeat. Strange.

In the end I was not able to find pattern or if it is some elastic memory cache sharding problem or some error in cirrus sending pages for indexing. I was not able to catch any php error and pages were randomly choosen for indexing.


Any suggestion? Tnx [[User:Matija.pu|Matija.pu]] ([[User talk:Matija.pu|talk]]) 11:11, 13 July 2021 (UTC)
:I would suggest to check the elasticsearch logs but restarting elastic should not have any impact on the number of indexed pages (elasticsearch has no cache that could explain what you see, only <code>refresh_interval</code> of the index set to a high value could explain this but it's set to a low value by default by CirrusSearch).
:The behavior you describe suggests that it is a JobQueue issue. Please see [[Manual:Job_queue]] and check that it is properly setup. [[User:DCausse (WMF)|DCausse (WMF)]] ([[User talk:DCausse (WMF)|talk]]) 18:25, 14 July 2021 (UTC)
:Yes! This was JobQueue issue. With '''php runJobs.php''' I did get for every page and file to become indexed in elasticsearch.
:Tnx! [[User:Matija.pu|Matija.pu]] ([[User talk:Matija.pu|talk]]) 11:54, 16 July 2021 (UTC)
{{Archive bottom}}

== php UpdateSuggesterIndex.php quitting... ==
{{Archive top|result=config $wgCirrusSearchUseCompletionSuggester must be set to 'yes' first.|status=resolved}}

Hi ,

When I try to run  php UpdateSuggesterIndex.php  getting '''Completion suggester disabled.''' Hence unable to get auto suggest in search engine.

Iam using these Elastica 6.3.1 and Cirrus Search 6.5.4
{| class="wikitable"
!Product
!Version
|-
|[https://www.mediawiki.org/ MediaWiki]
|1.35.1
|-
|[https://php.net/ PHP]
|7.4.15 (apache2handler)
|-
|[https://www.mysql.com/ MySQL]
|8.0.25
|-
|[https://www.elastic.co/products/elasticsearch Elasticsearch]
|6.5.4
|-
|[http://www.lua.org/ Lua]
|5.1.5
|}
pls suggest [[User:Pooja2425|Pooja2425]] ([[User talk:Pooja2425|talk]]) 12:20, 14 July 2021 (UTC)
:Please see docs/settings.txt, the completion suggester must be enabled with the config variable:
:<code>$wgCirrusSearchUseCompletionSuggester = 'yes'</code> [[User:DCausse (WMF)|DCausse (WMF)]] ([[User talk:DCausse (WMF)|talk]]) 13:12, 14 July 2021 (UTC)
{{Archive bottom}}

== Guidelines for sizing an Elasticsearch cluster ==

I've just gotten AWS Elasticsearch working with my dev wikis (one cluster for 5 fairly large wikis) and I'm troubleshooting various operational issues. One was happening to run out of disk space, so I had to increase the amount of disk space available. It would be nice to add to the CirrusSearch documentation guidelines on how to size an Elasticsearch cluster based on the size of the wiki, including allowing plenty of room for growth, re-indexing during upgrades, etc. [[User:Justin C Lloyd|Justin C Lloyd]] ([[User talk:Justin C Lloyd|talk]]) 17:46, 2 August 2021 (UTC)
:Thanks for the suggestion,
:I agree that the documentation is lacking but there are no easy ways to provision the size of the elasticsearch machines as the criteria influencing the index sizes are too numerous to comprehensively be taken into account.
:On the other hand it should be clearly noted that the reindexing does indeed require at least the size of the index + some extra room to not reach disk usage watermarks during this maintenance operation.
:Another solution could also to provide the store size of the the wikis hosted by the WMF, this could help future users to judge based on the WMF wikis how much space they take. This data is somehow already shared via https://grafana.wikimedia.org/d/fomV6x0Gz/elasticsearch-index-stats but could perhaps be made more comprehensive and linked from the documentation. [[User:DCausse (WMF)|DCausse (WMF)]] ([[User talk:DCausse (WMF)|talk]]) 07:26, 3 August 2021 (UTC)
:I appreciate the feedback.
:Even general guidelines could be helpful, especially for those who are new to both Redis and Elasticache (and I'm using AWS Elasticache, so it's managed meaning I have fewer knobs I can turn). I have five wikis whose MySQL databases, according to the information_schema data and combining data and indexes, are about 54, 26, 9, 8, and 4 GB respectively, though they all share a single EC cluster, so about 101 GB total, which grows maybe 10 GB per year at most. From what I can see in Kibana via _stats, the total index size is about 12.8 GB. So based on that I'd guess a bare minimum of 25 GB per node, so just to be completely safe, say 50 GB.
:So I think these kind of general guidelines could be useful in the docs, with the appropriate caveats, and even then listing some common criteria that influence index sizes could help a helpful addition.
:Just my two cents, of course.
:EDIT: Actually [https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/sizing-domains.html#aes-bp-storage Sizing AWS Elasticsearch domains] looks like a good resource. [[User:Justin C Lloyd|Justin C Lloyd]] ([[User talk:Justin C Lloyd|talk]]) 13:57, 3 August 2021 (UTC)

== updateSearchIndexConfig.php Error (solved) ==
{{Archive top|status=resolved}}

Hello, when I run updateSearchIndexConfig.php I get the following Error:


indexing namespaces...

Cannot spawn child: CirrusSearch\Maintenance\IndexNamespaces

[f67ce5b55238d49774ab9a4b] [no req]   Error from line 722 of maintenance/includes/Maintenance.php: Class 'CirrusSearch\Maintenance\IndexNamespaces' not found

Backtrace:

#0 extensions/CirrusSearch/includes/Maintenance/Maintenance.php(88): Maintenance->runChild(string, NULL)

#1 extensions/CirrusSearch/maintenance/updateSearchIndexConfig.php(53): CirrusSearch\Maintenance\Maintenance->runChild(string)

#2 maintenance/doMaintenance.php(107): CirrusSearch\Maintenance\UpdateSearchIndexConfig->execute()

#3 extensions/CirrusSearch/maintenance/updateSearchIndexConfig.php(70): require_once(string)

#4 {main}


  I am using:
{| class="wikitable"
!Product
!Version
|-
|MediaWiki
|1.35.3 (7c02aae)
17:45, 14 August 2021
|-
|PHP
|7.3.29 (fpm-fcgi)
|-
|MariaDB
|10.5.8-MariaDB
|-
|ICU
|65.1
|-
|Elasticsearch
|6.5.4
|}
{| class="wikitable"
|CirrusSearch
|6.5.4 (ad4210f) 06:13, 22 July 2021
|GPL-2.0-or-later
|Elasticsearch-powered search for MediaWiki
|Nik Everett, Chad Horohoe, Erik Bernhardson and others
|-
|Elastica
|6.1.3
|GPL-2.0-or-later
|Base Elasticsearch functionality for other extensions by providing Elastica library
|Nik Everett and Chad Horohoe
|}
Can someone help me figure out what is causing this? [[User:Xionous|Xionous]] ([[User talk:Xionous|talk]]) 18:32, 15 August 2021 (UTC)
:Did you install the extension (add the required wfLoadExtension( 'CirrusSearch' );)? [[User:Ciencia Al Poder|Ciencia Al Poder]] ([[User talk:Ciencia Al Poder|talk]]) 20:26, 15 August 2021 (UTC)
:Yes
:wfLoadExtension( 'Elastica' );
:wfLoadExtension( 'CirrusSearch' );
:$wgDisableSearchUpdate = true;
:$wgCirrusSearchServers = [ 'localhost' ];
:$wgSearchType = 'CirrusSearch'; [[User:Xionous|Xionous]] ([[User talk:Xionous|talk]]) 20:31, 15 August 2021 (UTC)
:Well, I can't say what's the problem here. extension.json includes the CirrusSearch\Maintenance namespace autoload, and the maintenance/IndexNamespaces.php file exists on REL1_35 branch... be sure the file exists too in your filesystem (on the CirrusSearch extension folder) [[User:Ciencia Al Poder|Ciencia Al Poder]] ([[User talk:Ciencia Al Poder|talk]]) 08:44, 16 August 2021 (UTC)
:Reinstalled mediawiki and it works now, as in it is not giving the error but searching returns no results. Looks like i forgot to mention this was an upgrade from an old version of mediawiki. Is there something i need to do in order for it to create the indexes again? [[User:Xionous|Xionous]] ([[User talk:Xionous|talk]]) 21:55, 16 August 2021 (UTC)
:Figured it out i had to run:
:php updateSearchIndexConfig.php --startOver
:php forceSearchIndex.php
:php maintenance/runJobs.php [[User:Xionous|Xionous]] ([[User talk:Xionous|talk]]) 00:43, 18 August 2021 (UTC)
{{Archive bottom}}

== Mediawikia wiki, php 8.010 error ==
{{Archive top|status=resolved}}

Hello, 

After switching to mediawikia 1.37wmf22 it seems that elastic is now working correctly.

Nevertheless, an error persists when searching with a ":" character.


[ec3539461ed077b3d9b1303c] /index.php?search=%C3%A9v%C3%A8nement+%3A+&title=Sp%C3%A9cial%3ARecherche&go=Lire&ns0=1&ns1=1&ns2=1&ns3=1&ns4=1&ns5=1&ns6=1&ns7=1&ns8=1&ns9=1&ns10=1&ns11=1&ns12=1&ns13=1&ns14=1&ns15=1&ns200=1&ns201=1&ns202=1&ns203=1&ns274=1&ns275=1&ns710=1&ns711=1&ns828=1&ns829=1&ns2300=1&ns2301=1&ns2302=1&ns2303=1&ns3000=1&ns3001=1 ParseError: syntax error, unexpected token "match"

Backtrace:

from /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/Elastica/vendor/ruflin/elastica/lib/Elastica/Query/MatchQuery.php(10)

#0 /var/www/vhosts/fallout-wiki.com/httpdocs/vendor/composer/ClassLoader.php(322): Composer\Autoload\includeFile()

#1 /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/CirrusSearch/includes/MetaStore/MetaNamespaceStore.php(94): Composer\Autoload\ClassLoader->loadClass()

#2 /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/CirrusSearch/includes/MetaStore/MetaNamespaceStore.php(78): CirrusSearch\MetaStore\MetaNamespaceStore->queryFilter()

#3 /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/CirrusSearch/includes/Searcher.php(468): CirrusSearch\MetaStore\MetaNamespaceStore->find()

#4 /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/CirrusSearch/includes/Util.php(124): CirrusSearch\Searcher->CirrusSearch\{closure}()

#5 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/poolcounter/PoolCounterWorkViaCallback.php(74): CirrusSearch\Util::CirrusSearch\{closure}()

#6 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/poolcounter/PoolCounterWork.php(162): PoolCounterWorkViaCallback->doWork()

#7 /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/CirrusSearch/includes/Util.php(182): PoolCounterWork->execute()

#8 /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/CirrusSearch/includes/Searcher.php(474): CirrusSearch\Util::doPoolCounterWork()

#9 /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/CirrusSearch/includes/Searcher.php(700): CirrusSearch\Searcher->findNamespace()

#10 /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/CirrusSearch/includes/Hooks.php(530): CirrusSearch\Searcher->updateNamespacesFromQuery()

#11 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/HookContainer/HookContainer.php(338): CirrusSearch\Hooks::onSearchGetNearMatch()

#12 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/HookContainer/HookContainer.php(137): MediaWiki\HookContainer\HookContainer->callLegacyHook()

#13 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/HookContainer/HookRunner.php(3192): MediaWiki\HookContainer\HookContainer->run()

#14 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/search/SearchNearMatcher.php(168): MediaWiki\HookContainer\HookRunner->onSearchGetNearMatch()

#15 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/search/SearchNearMatcher.php(70): SearchNearMatcher->getNearMatchInternal()

#16 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/specials/SpecialSearch.php(341): SearchNearMatcher->getNearMatch()

#17 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/specials/SpecialSearch.php(200): SpecialSearch->goResult()

#18 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/specialpage/SpecialPage.php(646): SpecialSearch->execute()

#19 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/specialpage/SpecialPageFactory.php(1366): SpecialPage->run()

#20 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/MediaWiki.php(314): MediaWiki\SpecialPage\SpecialPageFactory->executePath()

#21 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/MediaWiki.php(925): MediaWiki->performRequest()

#22 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/MediaWiki.php(559): MediaWiki->main()

#23 /var/www/vhosts/fallout-wiki.com/httpdocs/index.php(53): MediaWiki->run()

#24 /var/www/vhosts/fallout-wiki.com/httpdocs/index.php(46): wfIndexMain()

#25 {main}


Cordialy [[Special:Contributions/176.135.141.72|176.135.141.72]] ([[User talk:176.135.141.72|talk]]) 07:31, 10 September 2021 (UTC)
:This seems to be a compatibility issue with PHP 8. Apparently it's resolved already, looking at [[phab:T268861]]. Did you upgrade Elastica as well? [[User:Ciencia Al Poder|Ciencia Al Poder]] ([[User talk:Ciencia Al Poder|talk]]) 16:43, 12 September 2021 (UTC)
:It doesn't work, I'll try a 1.37 upgrade once it's officially released and I'll let you know :) [[Special:Contributions/176.172.11.143|176.172.11.143]] ([[User talk:176.172.11.143|talk]]) 06:34, 15 September 2021 (UTC)
:If you download current master of CirrusSearch it should be fixed [[User:Ciencia Al Poder|Ciencia Al Poder]] ([[User talk:Ciencia Al Poder|talk]]) 09:55, 15 September 2021 (UTC)
:Hello,
:I confirm that the bug is still present (under the alpha mediawiki version):
:[99a80ba4319b973cca1c54a7] /index.php?search=%C3%A9v%C3%A9nement+%3A+c&title=Sp%C3%A9cial%3ARecherche&go=Lire&ns0=1 ParseError: syntax error, unexpected token "match"
:Backtrace:
:from /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/Elastica/vendor/ruflin/elastica/lib/Elastica/Query/MatchQuery.php(10)
:#0 /var/www/vhosts/fallout-wiki.com/httpdocs/vendor/composer/ClassLoader.php(322): Composer\Autoload\includeFile()
:#1 /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/CirrusSearch/includes/MetaStore/MetaNamespaceStore.php(95): Composer\Autoload\ClassLoader->loadClass()
:#2 /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/CirrusSearch/includes/MetaStore/MetaNamespaceStore.php(79): CirrusSearch\MetaStore\MetaNamespaceStore->queryFilter()
:#3 /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/CirrusSearch/includes/Searcher.php(469): CirrusSearch\MetaStore\MetaNamespaceStore->find()
:#4 /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/CirrusSearch/includes/Util.php(125): CirrusSearch\Searcher->CirrusSearch\{closure}()
:#5 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/poolcounter/PoolCounterWorkViaCallback.php(74): CirrusSearch\Util::CirrusSearch\{closure}()
:#6 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/poolcounter/PoolCounterWork.php(162): PoolCounterWorkViaCallback->doWork()
:#7 /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/CirrusSearch/includes/Util.php(183): PoolCounterWork->execute()
:#8 /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/CirrusSearch/includes/Searcher.php(475): CirrusSearch\Util::doPoolCounterWork()
:#9 /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/CirrusSearch/includes/Searcher.php(701): CirrusSearch\Searcher->findNamespace()
:#10 /var/www/vhosts/fallout-wiki.com/httpdocs/extensions/CirrusSearch/includes/Hooks.php(531): CirrusSearch\Searcher->updateNamespacesFromQuery()
:#11 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/HookContainer/HookContainer.php(338): CirrusSearch\Hooks::onSearchGetNearMatch()
:#12 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/HookContainer/HookContainer.php(137): MediaWiki\HookContainer\HookContainer->callLegacyHook()
:#13 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/HookContainer/HookRunner.php(3192): MediaWiki\HookContainer\HookContainer->run()
:#14 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/search/SearchNearMatcher.php(168): MediaWiki\HookContainer\HookRunner->onSearchGetNearMatch()
:#15 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/search/SearchNearMatcher.php(70): SearchNearMatcher->getNearMatchInternal()
:#16 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/specials/SpecialSearch.php(341): SearchNearMatcher->getNearMatch()
:#17 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/specials/SpecialSearch.php(200): SpecialSearch->goResult()
:#18 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/specialpage/SpecialPage.php(647): SpecialSearch->execute()
:#19 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/specialpage/SpecialPageFactory.php(1366): SpecialPage->run()
:#20 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/MediaWiki.php(314): MediaWiki\SpecialPage\SpecialPageFactory->executePath()
:#21 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/MediaWiki.php(925): MediaWiki->performRequest()
:#22 /var/www/vhosts/fallout-wiki.com/httpdocs/includes/MediaWiki.php(559): MediaWiki->main()
:#23 /var/www/vhosts/fallout-wiki.com/httpdocs/index.php(53): MediaWiki->run()
:#24 /var/www/vhosts/fallout-wiki.com/httpdocs/index.php(46): wfIndexMain()
:#25 {main} [[Special:Contributions/176.134.81.16|176.134.81.16]] ([[User talk:176.134.81.16|talk]]) 04:33, 16 September 2021 (UTC)
:Post your problem to [[phab:T268861]] [[User:Ciencia Al Poder|Ciencia Al Poder]] ([[User talk:Ciencia Al Poder|talk]]) 13:54, 16 September 2021 (UTC)
{{Archive bottom}}

== CirrusSearch not showing autocomplete on Main Search Bar ==

Hello,


CirrusSearch will autocomplete when I am creating a page link and typing in the page name, but it will not autocomplete on the main search page. Here's the <code>LocalSettings.php</code> I have set up.

<pre>
#CirrusSearch
wfLoadExtension( 'Elastica' );
wfLoadExtension( 'CirrusSearch' );
$wgSearchType = 'CirrusSearch';
$wgCirrusSearchUseCompletionSuggester = 'yes';
$wgJobRunRate = 2;
$wgCirrusSearchCompletionSettings = 'fuzzy-subphrases';
$wgCirrusSearchCompletionSuggesterSubphrases = [
  'build' => true,
  'use' => true,
  'type' => 'anywords',
  'limit' => 10,
];

The indexing appears to be complete, and the search results when I manually press enter seem to be good. But it simply doesn't autocomplete when doing a proper search, despite working in the visualeditor when linking a page. 2601:98A:4102:2530:CDCD:45BD:50B2:2B99 (talk) 20:05, 15 October 2021 (UTC)

To make things more confusing, if I search, and then run a search in the secondary search bar that pops up above the results, autocomplete works there too. It's only on the wiki's main search bar at the top of the page. 2601:98A:4102:2530:CDCD:45BD:50B2:2B99 (talk) 20:28, 15 October 2021 (UTC)

Hit F12 to open the browser's console and look for any JavaScript error that may break other scripts of the page. Also, type some characters and see if it performs a request to the api to fetch search suggestions, and see if it returns anything in the response. Ciencia Al Poder (talk) 09:34, 16 October 2021 (UTC)

"Did you mean" not working

I installed CirrusSearch and I enabled the completion suggester. I get suggestions when I enter text into the search field.

What I haven't been able to figure out is how to enable "Did you mean" suggestions like WP has them. Also, I'd like to have the "Showing results for X. No results found for Y." feature. How do I do this? Hamburg0815 (talk) 17:13, 25 October 2021 (UTC)

Hi,

this feature should be enabled by default but will use the title of the indexed pages as "language model". It might be that on your wiki the titles themselves do not bear enough information to generate a useful "language model". I'd suggest increasing the information given to this feature by providing the opening_text or if you can afford it the whole text. The configuration should look like this:

$wgCirrusSearchPhraseSuggestUseOpeningText = true;
// or
// $wgCirrusSearchPhraseSuggestUseText = true;
// to use the whole text (will obviously require more space and memory on your elasticsearch cluster)

You will need to reindex your wiki using UpdateSearchIndexConfig after changing these configuration variables.

The Showing results for X. No results found for Y should be working once you have CirrusSearch able to detect typos properly. DCausse (WMF) (talk) 08:48, 26 October 2021 (UTC)

Hi DCausse,

thank you for your reply. I think it was "kind of" working all along because I realized that for some titles, I get a suggestion when I misspell it but for most titles I don't. For example, when I search for "parasymapthikus", it suggests the correctly spelled title "parasympathikus". When I search for "vagotnoie", it just says "no results" but it does not suggest "vagotonie" which is an existing title. Note that I switched two consecutive letters each time, but one of them was recognized as a misspelling while the other wasn't.

Is there a setting to make the search algorithm more fuzzy?

I get the same results with or without the wgCirrusSearchPhraseSuggestUseOpeningText parameter set to true. I did run UpdateSearchIndexConfig.php after I changed the param and in addition to that, I also ran ForceSearchIndex.php, runJobs.php, and UpdateSuggesterIndex.php. Hamburg0815 (talk) 17:37, 26 October 2021 (UTC)

Hi,

there are many parameters to tune this algorithm (see profiles/PhraseSuggesterProfiles.config.php).

You could create your own profile tuning based on the default one by adding:

// copied from profiles/PhraseSuggesterProfiles.config.php (doc removed here)
$wgCirrusSearchPhraseSuggestProfiles = [
    "my_profile" => [
                'total_hits_threshold' => 15000,
                'mode' => 'always',
                'confidence' => 2.0,
                'max_errors' => 2,
                'real_word_error_likelihood' => 0.95,
                'max_term_freq' => 0.5,
                'min_doc_freq' => 0.0,
                'prefix_length' => 2,
                'collate' => false,
                'collate_minimum_should_match' => '3<66%',
                'smoothing_model' => [
                        'stupid_backoff' => [
                                'discount' => 0.4
                        ]
                ],
    ]
];
// add a new fallback profile using this new settings
$wgCirrusSearchFallbackProfiles = [
    "my_phrasesuggest_profile" => [
            'methods' => [
                    'phrase-default' => [
                            'class' => \CirrusSearch\Fallbacks\PhraseSuggestFallbackMethod::class,
                            'params' => [
                                    'profile' => 'my_profile',
                            ]
                    ],
            ]
    ]
];
// Tell cirrus to use this fallback profile
$wgCirrusSearchFallbackProfile = "my_phrasesuggest_profile";

But again the algorithm highly depends on the data it has been fed with and feeding it with more text ($wgCirrusSearchPhraseSuggestUseText = true) is probably to more helpful than fine-tuning all this. DCausse (WMF) (talk) 08:46, 27 October 2021 (UTC)

Hi,

I tried the fallback profile as you suggested but unfortunately, it didn't make a difference. I also tried setting $wgCirrusSearchPhraseSuggestUseText = true but it didn't change anything, either.

$wgCirrusSearchPhraseSuggestProfiles = 'alternative' didn't improve things, either. I set $wgDebugLogFile and looked at the log, but there weren't any errors or warnings that caught my eye.

I noticed when I misspell a word in the search box, it always suggests the correct spelling in the as-you-type suggestions. But when I submit a search, it only works for some words but not for others. Hamburg0815 (talk) 13:14, 27 October 2021 (UTC)

Hi,

The profile I provided is just a copy of the default profile and is meant to be tuned but I have doubts you could greatly increase recall this way.

I agree that it is frustrating to see the typo being properly corrected during as-you-type but not after hitting the search button. This was brought up multiple times on WMF wikis as well T135920 (albeit for slightly different reasons).

Unfortunately I don't see other ways to improve the behavior other than:

• adding more text
• implement a completely new DYM component that works better on wikis without enough content DCausse (WMF) (talk) 12:10, 28 October 2021 (UTC)

Hi DCausse,

thanks, that makes sense. So as our wiki grows, this feature should work better. Hamburg0815 (talk) 19:03, 28 October 2021 (UTC)

elastic search using log4j 2.11.1.jar

Hi Team,

we are using below,

MediaWiki	1.35.3
PHP	7.4.23 (apache2handler)
MySQL	8.0.26
Lua	5.1.5
Elasticsearch	6.5.4

/usr/share/elasticsearch/lib/log4j-1.2-api-2.11.1.jar

log4j-api-2.11.1.jar

log4j-core-2.11.1.jar

x-pack-security/log4j-slf4j-impl-2.11.1.jar

please provide us any patch which is higher then log4j>2.15.0 Pooja2425 (talk) 12:09, 14 December 2021 (UTC)

"We" don't provide ElasticSearch. ElasticSearch was installed by yourself from an external source and you should ask them Ciencia Al Poder (talk) 12:23, 14 December 2021 (UTC)

May want to check this site out: Mitigate Log4j2 / Log4Shell in Elasticsearch (xeraa.net) DHillBCA (talk) 16:50, 14 December 2021 (UTC)

Thanks alot @DHillBCA for help,

I checked this, it seems i need to add -Dlog4j2.formatMsgNoLookups=true into etc/elasticsearch/jvm.options

because we are using elastic search 6.5.4 version.

pls let me know where i can ask questions for this. Pooja2425 (talk) 06:25, 15 December 2021 (UTC)

Also should i remove the JndiLookup class ??

pls suggest. Pooja2425 (talk) 08:30, 15 December 2021 (UTC)

Removing JndiLookup is not recommended, per the article.

If my read of the article is correct, the step you took is a good patch in the absence of updating log4j to 2.16 (2.15 was found to have related issues, so a new version was released). 2.16 does this by default.

Updating to log4j 2.16 and ensuring you're using an up-to-date version of the Java SDK appears to be the best defense against this problem. DHillBCA (talk) 13:54, 15 December 2021 (UTC)

6.8.21 should be good. From linked article: "[this version sets]Dlog4j2.formatMsgNoLookups=true in the JVM options and remove the JndiLookup class for you "

From elastic itself: "As of December 13, 2021, we have released Elasticsearch 6.8.21 and 7.16.1 which set the JVM option identified below and remove the vulnerable JndiLookup class from Log4j out of an abundance of caution" Realsalt (talk) 15:57, 15 December 2021 (UTC)

I guess that question I have is should this be flagged as a critical version on this main page. Something like {{Warning }} template? Current text says "MediaWiki 1.33.x - 1.38.x require Elasticsearch 6.5.x - 6.8.x (6.8.21+ recommended)". Is that sufficient? Realsalt (talk) 16:00, 15 December 2021 (UTC)