Topic on Help talk:CirrusSearch

problems to reindex by elasticsearch

2
JanTappenbeck (talkcontribs)

current my it-admin moved the virtuell machine with our mediawiki runs (win7 64bit).

first the elastic search process Needs 100% cpu - today the cpu is better and in an other log-file i found the Information the index will be demage.

so i open cmd-window with admin-rights and send following command:

C:\PHP> C:\PHP>php c:/mediawiki/extensions/CirrusSearch/maintenance/updateSearchIndexCon fig.php

and this is the report....

´╗┐content index...
        Fetching Elasticsearch version...1.4.4...ok
        Scanning available plugins...none
        Infering index identifier...eblwiki-eblw__content_first
        Picking analyzer...german
        Index exists so validating...
                Validating number of shards...ok
                Validating replica range...ok
                Validating shard allocation settings...done
                Validating max shards per node...ok
        Validating analyzers...ok
        Validating mappings...
                Validating mapping...ok
        Validating cache warmers...
        Validating aliases...
                Validating content alias...ok
                Validating all alias...ok
                Updating tracking indexes...done
general index...
        Fetching Elasticsearch version...1.4.4...ok
        Scanning available plugins...none
        Infering index identifier...eblwiki-eblw__general_first
        Picking analyzer...german
        Index exists so validating...
                Validating number of shards...ok
                Validating replica range...ok
                Validating shard allocation settings...done
                Validating max shards per node...ok
        Validating analyzers...ok
        Validating mappings...
                Validating mapping...ok
        Validating cache warmers...
        Validating aliases...
                Validating general alias...ok
                Validating all alias...ok
                Updating tracking indexes...done
                Deleting namespaces...done
                Indexing namespaces...
Unexpected Elasticsearch failure.
Elasticsearch failed in an unexpected way.  This is always a bug in CirrusSearch
.
Error type: Elastica\Exception\Bulk\ResponseException
Message: Error in one or more bulk request actions:

index: /eblwiki-eblw__general_first/namespace/-2 caused UnavailableShardsException[[eblwiki-eblw__general_first][0] Primary shard is not active or isn't assigne
d is a known node. Timeout: [1m], request: org.elasticsearch.action.bulk.BulkShardRequest@2f8d8f73]
index: /eblwiki-eblw__general_first/namespace/3 caused UnavailableShardsException[[eblwiki-eblw__general_first][0] Primary shard is not active or isn't assigned
 is a known node. Timeout: [1m], request: org.elasticsearch.action.bulk.BulkShardRequest@2f8d8f73]
index: /eblwiki-eblw__general_first/namespace/4 caused UnavailableShardsException[[eblwiki-eblw__general_first][1] Primary shard is not active or isn't assigned
 is a known node. Timeout: [1m], request: org.elasticsearch.action.bulk.BulkShardRequest@7a9b6074]
index: /eblwiki-eblw__general_first/namespace/7 caused UnavailableShardsException[[eblwiki-eblw__general_first][0] Primary shard is not active or isn't assigned
 is a known node. Timeout: [1m], request: org.elasticsearch.action.bulk.BulkShardRequest@2f8d8f73]
index: /eblwiki-eblw__general_first/namespace/8 caused UnavailableShardsException[[eblwiki-eblw__general_first][1] Primary shard is not active or isn't assigned
 is a known node. Timeout: [1m], request: org.elasticsearch.action.bulk.BulkShardRequest@7a9b6074]
index: /eblwiki-eblw__general_first/namespace/12 caused UnavailableShardsException[[eblwiki-eblw__general_first][0] Primary shard is not active or isn't assigne
d is a known node. Timeout: [1m], request: org.elasticsearch.action.bulk.BulkShardRequest@2f8d8f73]
index: /eblwiki-eblw__general_first/namespace/13 caused UnavailableShardsException[[eblwiki-eblw__general_first][1] Primary shard is not active or isn't assigne
d is a known node. Timeout: [1m], request: org.elasticsearch.action.bulk.BulkShardRequest@7a9b6074]
index: /eblwiki-eblw__general_first/namespace/102 caused UnavailableShardsException[[eblwiki-eblw__general_first][0] Primary shard is not active or isn't assign
ed is a known node. Timeout: [1m], request: org.elasticsearch.action.bulk.BulkShardRequest@2f8d8f73]
index: /eblwiki-eblw__general_first/namespace/103 caused UnavailableShardsException[[eblwiki-eblw__general_first][1] Primary shard is not active or isn't assign
ed is a known node. Timeout: [1m], request: org.elasticsearch.action.bulk.BulkShardRequest@7a9b6074]

Trace:
# 0 C:\mediawiki\extensions\Elastica\Elastica\lib\Elastica\Bulk.php(345): Elastica\Bulk->_processResponse(Object(Elastica\Response))
# 1 C:\mediawiki\extensions\Elastica\Elastica\lib\Elastica\Client.php(284): Elastica\Bulk->send()
# 2 C:\mediawiki\extensions\Elastica\Elastica\lib\Elastica\Index.php(147): Elastica\Client->addDocuments(Array)
# 3 C:\mediawiki\extensions\Elastica\Elastica\lib\Elastica\Type.php(187): Elastica\Index->addDocuments(Array)
# 4 C:\mediawiki\extensions\CirrusSearch\maintenance\indexNamespaces.php(56): Elastica\Type->addDocuments(Array)
# 5 C:\mediawiki\extensions\CirrusSearch\maintenance\updateOneSearchIndexConfig.php(290): CirrusSearch\Maintenance\IndexNamespaces->execute()
# 6 C:\mediawiki\extensions\CirrusSearch\maintenance\updateOneSearchIndexConfig.php(213): CirrusSearch\Maintenance\UpdateOneSearchIndexConfig->indexNamespaces()
# 7 C:\mediawiki\extensions\CirrusSearch\maintenance\updateSearchIndexConfig.php(51): CirrusSearch\Maintenance\UpdateOneSearchIndexConfig->execute()
# 8 C:\mediawiki\maintenance\doMaintenance.php(97): CirrusSearch\Maintenance\UpdateSearchIndexConfig->execute()
# 9 C:\mediawiki\extensions\CirrusSearch\maintenance\updateSearchIndexConfig.php(58): require_once('C:\\mediawiki\\ma...')
# 10 {main}


very interesting is the statement "Elasticsearch failed in an unexpected way. This is always a bug in CirrusSearch"

is there someone to help me ???

here a part of current elasticsearch log:

[2015-10-09 11:14:10,356][WARN ][indices.cluster          ] [Living Hulk] [eblwiki-eblw__content_first][2] failed to start shard
org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: [eblwiki-eblw__content_first][2] failed to fetch index version after copying it over
	at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:158)
	at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:132)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)
Caused by: org.apache.lucene.index.CorruptIndexException: [eblwiki-eblw__content_first][2] Preexisting corrupted index [corrupted_6KkIhZscQ92OMn2XKGuIYw] caused by: CorruptIndexException[codec header mismatch: actual header=0 vs expected header=1071082519 (resource: BufferedChecksumIndexInput(MMapIndexInput(path="C:\elasticsearch\data\elasticsearch\nodes\0\indices\eblwiki-eblw__content_first\2\index\_ky.fnm")))]
org.apache.lucene.index.CorruptIndexException: codec header mismatch: actual header=0 vs expected header=1071082519 (resource: BufferedChecksumIndexInput(MMapIndexInput(path="C:\elasticsearch\data\elasticsearch\nodes\0\indices\eblwiki-eblw__content_first\2\index\_ky.fnm")))
	at org.apache.lucene.codecs.CodecUtil.checkHeader(CodecUtil.java:136)
	at org.apache.lucene.codecs.lucene46.Lucene46FieldInfosReader.read(Lucene46FieldInfosReader.java:57)
	at org.apache.lucene.index.SegmentReader.readFieldInfos(SegmentReader.java:289)
	at org.apache.lucene.index.IndexWriter.getFieldNumberMap(IndexWriter.java:864)
	at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:816)
	at org.elasticsearch.index.engine.internal.InternalEngine.createWriter(InternalEngine.java:1487)
	at org.elasticsearch.index.engine.internal.InternalEngine.start(InternalEngine.java:277)
	at org.elasticsearch.index.shard.service.InternalIndexShard.performRecoveryPrepareForTranslog(InternalIndexShard.java:732)
	at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:231)
	at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:132)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)

	at org.elasticsearch.index.store.Store.failIfCorrupted(Store.java:480)
	at org.elasticsearch.index.store.Store.failIfCorrupted(Store.java:461)
	at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:120)
	... 4 more

BY 100% CPU !!!!!

regards Jan

Florianschmidtwelzow (talkcontribs)
Reply to "problems to reindex by elasticsearch"