Extension talk:Replace Text/Archive 2020

"Extension:ReplaceText just reloads" issue (+ solution!)

I administrate a MediaWiki (version 1.31.1) on a Third Party Plesk Server host.

The Replace Text Extension I have installed (version 1.4.1) used to work, and it stopped working after some Domain issues. I had to reset my Plesk Server settings as best from memory as I could, and likely what I think occurred were modified PHP/Apache settings.

The Issue: When I went to the Special:ReplaceText page on my Wiki, filled it out and clicked the Continue button ... the page just reloaded. I was no longer redirected to a page that showed all the pages that would be replaced like I had been before. And in fact, no text replacing was done at all!

The Solution: I got ahold of User:Yaron Koren, one of the authors of this Extension, and he was able to assist me and resolved the issue! Here is the solution for those who may be having the same problem as I had:

1. Open the SpecialReplaceText.php file in your wiki's extension folder in a Text Editor.
2. Search for getFullURL() (was on Line 411 for me) and change it to getLocalURL().
3. On the next line, change 'post' to 'get'.
4. Search for post (this second instance of post was on Line 576 for me) and change it to get.
5. On the next line, change getFullUrl() to getLocalUrl().
6. Save the file.

And viola! My Special:ReplaceText page works like it did before the Domain Issues I had before! No PHP/Apache Settings to update/modify!

I really believe Extension:Replace Text#Known issues ought to include this information. I would add it myself, but I don't really feel like I've done a good job explaining my problem (and I really don't know what triggered it; only a guess on Yaron's part). Naupe (talk) 18:41, 5 February 2020 (UTC)

This is probably a problem with your domain setup. You may have a different $wgServer than your actual server, and there's a redirect under the hood, causing POST data being lost.

Typical failures: Your $wgServer is set to http:// but your server is set to automatically redirect all traffic to https://. A similar one is your $wgServer being http://example.com and your webserver is automatically redirecting all traffic to http://www.example.com

You can see this if you open the developer tools (hit F12) and open the network tab. If you make this tab persistent (to not reload on every page load), hitting the submit button will give you a GET or POST request that's replied with a HTTP 301 or HTTP 302 status code, causing the server to reissue the same request again on the new location Ciencia Al Poder (talk) 20:42, 5 February 2020 (UTC)

can't replace HTML-like strings

I'm trying to replace HTML-like tags spread across hundreds of pages in my wiki. I'm using regular expressions, e.g.

<repeat>(.*?)</repeat>

but keep getting Wikimedia\Rdbms\DBQueryError error suppoedly indicating a software bug. Is there a way around this?

My version is MW 1.30.1, PHP 5.5.9, MySQL 5.7.25 (with ReplaceText for MW 1.30 of course) 217.132.232.233 (talk) 06:45, 22 February 2020 (UTC)

Can you temporarily set $wgShowSQLErrors = true; in LocalSettings.php and repeat the action, and see what's the error message? Ciencia Al Poder (talk) 14:43, 22 February 2020 (UTC)

It was already set, nothing shows except what I already mentioned (error occured when querying DB, which might indicate a bug in software "Wikimedia\Rdbms\DBQueryError") ~ 217.132.232.233 (talk) 19:40, 22 February 2020 (UTC)

Trying to reproduce this issue, the error that shows is this:

Function: ReplaceTextSearch::doSearchQuery

Error: 1139 Got error 'repetition-operator operand invalid' from regex

It looks like this is a limitation in MySQL's implementation - you cannot use the non-greedy wildcard, '*?'.

It looks like MySQL 8 might have a better implementation of Regex, but I cannot test at the moment. FFS ^Talk 00:29, 23 February 2020 (UTC)

So...is there a way around this limitation? ~ 217.132.232.233 (talk) 08:49, 23 February 2020 (UTC)

That's actually a regex question, more than a mediawiki question... If you don't have any other tags in between those, you can simply do this:

<repeat>([^<]*)</repeat>

This obviously won't work if there's any '<' in the text between the "repeat" tags. In that case, my regex knowledge is exhausted, and I would suggest you ask that question on StackOverflow or the like. FFS ^Talk 16:03, 25 February 2020 (UTC)

ok, thanks! 217.132.232.233 (talk) 19:00, 25 February 2020 (UTC)

can you use backslash escape character? Johnywhy (talk) 11:19, 17 February 2021 (UTC)

replaceAll.php

I tried this with a regular expression replacement. It correctly showed the pages to be changed, but then just froze (I needed to hit control-C). Running runJobs.php didn't have any effect. No pages were changed.

It may have been a problem with my regex. I ended up using the web interface and just using normal text replacement. So unfortunately I can't assist much further (though happy to try things out that won't break my wiki). It may be worth looking into how replaceAll.php operates in this sort of case though. Thanks. Jonathan3 (talk) 17:24, 23 February 2020 (UTC)

Here's what I typed. Maybe the mistake will be obvious!

php replaceAll.php "{{aaa(.*)=(.*)bbb=(.*)ccc=(.*)ddd=(.*)" "{{eee$1fff=$2ggg=$3hhh=$4iii=$5" --regex --debug --summary "jjj" --user "kkk"

I also tried it without "--debug". Jonathan3 (talk) 18:06, 23 February 2020 (UTC)

Using regexp notation in command line parameters is tricky, because some characters like { and $ have a special meaning for the shell.

By {{ you probably want to mean template syntax. However, in regexp, the { character is used to define how many times the previous character/group/expression must be present, and you need to escape it. You probably need to replace it by \\{\\{, and the two \ are to tell the shell to interpret it as a literal \ and not a escape character for the shell itself. This should be needed only in the search string and not in the replacement string.

The $1, $2 etc in the replacement are also problematic because $ is used by the shell to indicate it's a variable name. Since it's not defined, it will replace it with blanks! You should put a single \ before all of them: \$1, \$2, etc Ciencia Al Poder (talk) 10:23, 24 February 2020 (UTC)

Thank you for the detailed response.

I hadn't escaped the { or } because the extension page states: The special characters that one can use in the search string are "( ) . * + ? [ ] |"

The other stuff is new to me - it's lucky the script made no changes were made or it would have caused me more bother! Jonathan3 (talk) 15:21, 24 February 2020 (UTC)

P.S. Despite this being my fault, it would have been nice if the script had given a error message (even an unhelpful one) rather than frozen :-) Jonathan3 (talk) 15:22, 24 February 2020 (UTC)

Multiline replace

I am trying to delete some templates with their content. I have gone as far as using something like:

 {{WoodhouseENELnames\n(.*?)\n}}

Which only captures text in the template if there is one line break after the template name and another before the end of template. How can I edit this so that it captures any text and any number of lines until it reaches the first "}}" bit (non-greedy)? Spiros71 (talk) 15:58, 23 April 2020 (UTC)

Just get rid of \n in both places - try

{{WoodhouseENELnames(.*?)\}\}

Jonathan3 (talk) 19:43, 23 April 2020 (UTC)

"No pages were found containing the string "\{\{WoodhouseENELnames(.*?)\}\}" or "{{WoodhouseENELnames(.*?)}}" I guess multiline is not supported by simply using ".*?"

So far the only workaround I found is using multiple expressions like:

{{WoodhouseENELnames\n(.*?)\n+(.*?)\n+}}

{{WoodhouseENELnames\n(.*?)\n+(.*?)\n+(.*?)\n+}}

{{WoodhouseENELnames\n(.*?)\n+(.*?)\n+(.*?)\n+(.*?)\n+}} Spiros71 (talk) 19:50, 23 April 2020 (UTC)

It seems that you need to escape } but must not escape { in ReplaceText. Try it again :-) Jonathan3 (talk) 20:06, 23 April 2020 (UTC)

That did not work either:

No pages were found containing the string

{{WoodhouseENELnames(.*?)\}\} Spiros71 (talk) 07:30, 24 April 2020 (UTC)

I was sure I'd tried it, but clearly not. Sorry. Jonathan3 (talk) 21:16, 24 April 2020 (UTC)

Needs a few clarification

1) How is this changing bulk replace text? Only one page at a web request? so do I need to refresh wiki that it continues the job to change? Or once I click submit, it does the job with no worries about web request?

2) Pages will be done replacing in which order? By date of creation order?

3) If using command line how to tell the command to add an End Of Line at the end of new text? For example

$ replaceText.php "foo" "bar\n"

is something valid? Farvardyn (talk) 17:52, 8 May 2020 (UTC)

1 - When using the web interface it uses the job queue. I imagine the command line script does too. I always run runJobs.php afterwards. Jonathan3 (talk) 23:16, 11 May 2020 (UTC)

Using ReplaceText with Translate extension needs marking pages again

Hi,

I am using Replace text for a few lines I have to replace in some of my namespaces.

But everytime I use it on the Translate namespaces, it works and replaces them, but I always have to mark pages for translation, which is very annoying for a wiki with hundreds of pages.

Is there any means to avoid this by any means ?

Best regards, Tuxxic (talk) 07:18, 15 May 2020 (UTC)

I can feel your pain. I think this is a question for the Translate extension which picks up on edits whether they are done by bots or humans. [[kgh]] (talk) 07:37, 15 May 2020 (UTC)

Thanks, I didn't know if I chose the right board. I contact them :) Tuxxic (talk) 13:00, 15 May 2020 (UTC)

Minor misstatement

The following statement in the "Known issues" section appears to be partially untrue since administrators are given the "suppressredirect" right by default in MW 1.31-1.36:

* In order for redirects not to be created on page moves (i.e. if the "Save the old titles as redirects to the new titles" checkbox is unchecked), you will have to give the relevant user the 'suppressredirect' permission. Assuming you've given the 'replacetext' permission only to sysops, you would need to add the following to LocalSettings.php:

:<syntaxhighlight lang=php>$wgGroupPermissions['sysop']['suppressredirect'] = true;</syntaxhighlight> Tlgonline (talk) 16:48, 5 September 2020 (UTC)

Ok, changed Ciencia Al Poder (talk) 16:28, 6 September 2020 (UTC)

"Replace text in page titles" dependency on "Replace text in page contents" result. (?)

If "Replace text in page contents" is disabled, OR it could not find a match.

Than (if used) "Replace text in page titles, when possible" will always fail to find any matches too.

v1.4.1 (also seen with earlier versions)

Bug, or intentional ?

If intentional, I don't mind understanding why.

---

ReplaceText: 1.4.1
MediaWiki: 1.31.0
PHP: 7.0.30-0+deb9u1 (fpm-fcgi)
MariaDB: 10.1.26-MariaDB-0+deb9u1 MvGulik (talk) 18:15, 27 September 2020 (UTC)

As a person who has used this plugin many times and am satisfied, I say that this plugin is very sensitive.

This means that if you enter a space after the text and the original text does not have the distance or line, you will be faced with the problem of finding the text.

So you have to insert exactly the text you want to find, without any lots, even a lot of distance.

Hopefully it will be functional my tips. best regards Sokote zaman (talk) 19:40, 27 September 2020 (UTC)

> Hopefully it will be functional my tips.

Nope.

I have no problems with "Replace Text"'s input, or with RE for that matter.

It is as I stated, on this side.

Now if you have contrary data to add, I'm all ears. As it might be that there is some local(MW-site) part that plays a role in this. But I doubt that at this moment. MvGulik (talk) 09:35, 28 September 2020 (UTC)

Odd. Had a case where I first did the all the target page-text replacing. And after that I did a leftover page-title change (same change as with text part). Which worked without issues. Seems there is something else also playing a role here ...

nts:

- (both actions where done with all name-spaces selected)

- (both actions where done within a short time period. (a few minutes))
MvGulik (talk) 09:18, 16 November 2020 (UTC)

ReplaceText: 1.4.1

MediaWiki: 1.31.0

PHP: 7.0.30-0+deb9u1 (fpm-fcgi)

MariaDB: 10.1.26-MariaDB-0+deb9u1

Target:"File:Test_File.ext"

Settings:

- "Use regular expressions" (on)

- "Replace text in page titles, when possible" (on)

-- Others: Off.

Search: "(?i)test file"

Replace: ""

Result: Target found.

Search: "(?i)test file"

Replace: "Test File"

Result: Target not found.

(Ok. Could be because there would be no effective change ...)

Search: "(?i)test file"

Replace: "foo/bar"

Result: Target not found.

(... not so in this case though)

Settings change:

- "Use regular expressions" (off)

Search: "Test File"

Replace: "foo bar"

Result: Target found.

Search: "Test File"

Replace: "foo/bar"

Result: Target not found.

(Guess that rules this particular behaviour out as a RE-only related undocumented-feature/issue) MvGulik (talk) 03:09, 17 December 2020 (UTC)

Extension doesn’t work after upgrading to MW 1.35

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

Hello,

I have updated my Wiki from 1.29 to 1.35 and installed the extension. When I try to use it I see the following error:

Can you please help me to resolve this problem?

Translation of the error: [X3Wfcvven95FpnAv0QBJhQAAAQo] 2020-10-01 09:20:50: fatal exception of the type "Wikimedia\RDBMS\Dbqueryerror» Fokebox (talk) 09:11, 1 October 2020 (UTC)

Had a similar issue. My teammate tracked it down to commit bcd6b2f732824d26a55ff3892442e143df7994a6 causing the issue. We use MySQL 5.6 and there's no evidence that it actually supports CAST AS INTEGER. Reversing that commit can fix the issue till the developers fix it in the mainline versions. I can't link to the commit since the spam filter gets activated. HerrHulaHoop (talk) 07:54, 2 October 2020 (UTC)

Here it is what I have for my Wiki:

!Product

!Version

|-

|MediaWiki

|1.35.0

|-

|PHP

|7.4.10 (cgi-fcgi)

|-

|MySQL

|5.7.27-30

|-

|ICU

|50.1.2

|-

|Lua

|5.1.5

|}

If it helps to understand what causing the error. Fokebox (talk) 09:18, 2 October 2020 (UTC)

You need to physically edit a file in your replace text extension:

• Go to /src/ReplaceTextSearch.php
• Edit line 54 from 'CAST(SUBSTRING(content_address, 4) AS INTEGER) = old_id' to 'CAST(SUBSTRING(content_address, 4) AS SIGNED) = old_id'
• Save it and try again.

Hope this helps! HerrHulaHoop (talk) 09:44, 2 October 2020 (UTC)

Works perfectly! Thank you! Fokebox (talk) 10:13, 2 October 2020 (UTC)

The discussion above is closed. Please do not modify it. No further edits should be made to this discussion.

How do I replace non regular wikitext?

Hello, recently we've migrated to MediaWiki 1.35. We are using this nice exension for a long time. Yesterday I've pached it as it is decribed in the topic below.

Now I found when I'm tryng to replace text that is located within the MediaWiki NameSpace, let's say in Common.css, the web interface returns successiful replacement, but replacement is not actually done.

For example I've placed "Replace_Test" as comment into Common.css. Then I've used the web interface of Extension:ReplaceText in otder to replace "Replace_Test" with "Replace_Success". At this point the extension returns:

"Replace_Test" will be replaced with "Replace_Success" in one page.

But the replacement is not done, and when I inspected the outpot of the background job I found the following messageg:

2020-11-03 05:41:43 replaceText MediaWiki:Common.css user_id=4 target_str=Replace_Test replacement_str=Replace_Success use_regex= edit_summary=Text replacement - "Replace_Test" to "Replace_Success" create_redirect= watch_page= doAnnounce=1 namespace=8 title=Common.css requestId=X6Dtk94IEEHzBCOoVGYh3wAAAAA (id=2151,timestamp=20201103054139) STARTING

2020-11-03 05:41:43 replaceText MediaWiki:Common.css user_id=4 target_str=Replace_Test replacement_str=Replace_Success use_regex= edit_summary=Text replacement - "Replace_Test" to "Replace_Success" create_redirect= watch_page= doAnnounce=1 namespace=8 title=Common.css requestId=X6Dtk94IEEHzBCOoVGYh3wAAAAA (id=2151,timestamp=20201103054139) t=3 error=replaceText: Wiki page "MediaWiki:Common.css" does not hold regular wikitext.

Is there any workaround of this issue? Spas.Z.Spasov (talk) 05:58, 3 November 2020 (UTC)

Are you sure that this affects all pages in the MediaWiki: namespace, and not just specific pages like Common.css (and probably Common.js)? And did this work before? Yaron Koren (talk) 15:55, 3 November 2020 (UTC)

Hi, Yaron Koren, I found this behavior when I was taring to replace a file system path (like `resources-local/` to `images/resources-local/`) which appears in several css/js pages in our wiki. I made an additional test. For its purpose, I've placed the string Replace_Test in three pages as follow:

1. MediaWiki:Common.css -> with content model: CSS
2. MediaWiki:InternalWhitelist -> with content model: wikitext
3. Test (NS=0) -> with content model: Javascript

At this point, the web interface of Replace Test returns: "Replace_Test" will be replaced with "Replace_Success" in 3 pages.

Actually the string was replaced only in the page with with content model wikitext. Here is the full output of the job queue: https://pastebin.com/L1gpyrf4

I've found the code that causes this behavior and the possible reason of it, but can't propose an elegant solution:

src/ReplaceTextJob.php:84:      if ( $this->title->getContentModel() !== CONTENT_MODEL_WIKITEXT ) {
src/ReplaceTextJob.php:128:	$new_content = new WikitextContent( $new_text );

Probably the content model must be mater of choice within the Replace Text's interface, based on $wgContentHandlers.

Here is my current workaround:

/**
 * Around line 84 in src/ReplaceTextJob.php
 */
global $wgContentHandlers;
//if ( $this->title->getContentModel() !== CONTENT_MODEL_WIKITEXT ) {
if ( ! in_array( $this->title->getContentModel(), array_keys($wgContentHandlers) ) ) {
	$this->error = 'replaceText: Wiki page "' .
		//$this->title->getPrefixedDBkey() . '" does not hold regular wikitext.';
		$this->title->getPrefixedDBkey() . '" does not hold any known content model, see $wgContentHandlers.';
	return false;
}
/**
 * Around line 128 in src/ReplaceTextJob.php
 */
//$new_content = new WikitextContent( $new_text );
switch ($this->title->getContentModel()) {
	case CONTENT_MODEL_WIKITEXT:
		$new_content = new WikitextContent( $new_text );
		break;
	case CONTENT_MODEL_JAVASCRIPT:
		$new_content = new JavaScriptContent( $new_text );
		break;
	case CONTENT_MODEL_JSON:
		$new_content = new JsonContent( $new_text );
		break;
	case CONTENT_MODEL_CSS:
		$new_content = new CssContent( $new_text );
		break;
	case CONTENT_MODEL_TEXT:
		$new_content = new TextContent( $new_text );
		break;
	default: // CONTENT_MODEL_UNKNOWN
		$new_content = new FallbackContent( $new_text );
}

Spas.Z.Spasov (talk) 21:06, 3 November 2020 (UTC)

Replace Text doesn't like commas

I'm trying to replace text in some templates with lists of parameters like in "country = Mexico, France, Turkey" to "country1 = Mexico | country2 = France | country3 = Turkey", but Replace Text says that there are no matches for the first text. If I try only with "country = Mexico" I get results but the moment I put the comma back it doesn't work. Is this a bug maybe? Ideas on how to solve it? I don't get any error messages. Cojoilustrado 23:31, 5 November 2020 (UTC)

search or replace new line

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

How do I search for something with a new line in between, and how to replace something with new line? Mshastchi (talk) 14:41, 15 November 2020 (UTC)

The discussion above is closed. Please do not modify it. No further edits should be made to this discussion.

add option for case insensitive

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

It would be great if there was an option to allow for case insensitive regex 2601:586:581:4C80:F9F0:E0F5:6425:4227 (talk) 07:50, 17 November 2020 (UTC)

also it would be great if there was an option to increase the amount of pages to effect. 2601:586:581:4C80:F9F0:E0F5:6425:4227 (talk) 07:52, 17 November 2020 (UTC)

In extension page an case-insensitive option is descriped. (?i) before the string.

Search string: (?i)iphone

Replacement string: iPhone

The above will unify the casing of all mentions of iPhone/iphone/IPHONE to iPhone. Gunnar.offel (talk) 18:48, 1 December 2020 (UTC)

The discussion above is closed. Please do not modify it. No further edits should be made to this discussion.

MW 1.35 Replace Text 1.4.1

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

Could you help me diagnose this error?

I have run the update.php script.

[5d452609228b03e4bc6380de] /index.php?title=Special:ReplaceText Wikimedia\Rdbms\DBQueryError from line 1699 of D:\inetpub\wwwroot\BDD\includes\libs\rdbms\database\Database.php: A database query error has occurred. Did you forget to run your application's database schema updater after upgrading?

Error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'INTEGER) = old_id) ORDER BY page_namespace, page_title LIMIT 250' at line 1 (localhost) Function: ReplaceTextSearch::doSearchQuery Query: SELECT page_id,page_namespace,page_title,old_text FROM `page`,`revision`,`text`,`slots`,`content` WHERE (old_text LIKE '%business%' ESCAPE '`' ) AND page_namespace = 0 AND (rev_id = page_latest) AND (rev_id = slot_revision_id) AND (slot_content_id = content_id) AND (CAST(SUBSTRING(content_address, 4) AS INTEGER) = old_id) ORDER BY page_namespace, page_title LIMIT 250 Backtrace:

1. 0 D:\inetpub\wwwroot\BDD\includes\libs\rdbms\database\Database.php(1683): Wikimedia\Rdbms\Database->getQueryException(string, integer, string, string)
2. 1 D:\inetpub\wwwroot\BDD\includes\libs\rdbms\database\Database.php(1658): Wikimedia\Rdbms\Database->getQueryExceptionAndLog(string, integer, string, string)
3. 2 D:\inetpub\wwwroot\BDD\includes\libs\rdbms\database\Database.php(1227): Wikimedia\Rdbms\Database->reportQueryError(string, integer, string, string, boolean)
4. 3 D:\inetpub\wwwroot\BDD\includes\libs\rdbms\database\Database.php(1907): Wikimedia\Rdbms\Database->query(string, string, integer)
5. 4 D:\inetpub\wwwroot\BDD\includes\libs\rdbms\database\DBConnRef.php(68): Wikimedia\Rdbms\Database->select(array, array, array, string, array)
6. 5 D:\inetpub\wwwroot\BDD\includes\libs\rdbms\database\DBConnRef.php(313): Wikimedia\Rdbms\DBConnRef->__call(string, array)
7. 6 D:\inetpub\wwwroot\BDD\extensions\ReplaceText\src\ReplaceTextSearch.php(64): Wikimedia\Rdbms\DBConnRef->select(array, array, array, string, array)
8. 7 D:\inetpub\wwwroot\BDD\extensions\ReplaceText\src\SpecialReplaceText.php(303): ReplaceTextSearch::doSearchQuery(string, array, string, string, boolean)
9. 8 D:\inetpub\wwwroot\BDD\extensions\ReplaceText\src\SpecialReplaceText.php(172): SpecialReplaceText->getTitlesForEditingWithContext()
10. 9 D:\inetpub\wwwroot\BDD\extensions\ReplaceText\src\SpecialReplaceText.php(73): SpecialReplaceText->doSpecialReplaceText()
11. 10 D:\inetpub\wwwroot\BDD\includes\specialpage\SpecialPage.php(600): SpecialReplaceText->execute(NULL)
12. 11 D:\inetpub\wwwroot\BDD\includes\specialpage\SpecialPageFactory.php(635): SpecialPage->run(NULL)
13. 12 D:\inetpub\wwwroot\BDD\includes\MediaWiki.php(307): MediaWiki\SpecialPage\SpecialPageFactory->executePath(Title, RequestContext)
14. 13 D:\inetpub\wwwroot\BDD\includes\MediaWiki.php(940): MediaWiki->performRequest()
15. 14 D:\inetpub\wwwroot\BDD\includes\MediaWiki.php(543): MediaWiki->main()
16. 15 D:\inetpub\wwwroot\BDD\index.php(53): MediaWiki->run()
17. 16 D:\inetpub\wwwroot\BDD\index.php(46): wfIndexMain()
18. 17 {main} Issiegainsley (talk) 01:12, 30 November 2020 (UTC)

I don't know.

Maybe just comment out the wfLoadExtension( 'ReplaceText' ); line while running update.php.

If it works all right, reintroduce the line and check again.

If it still doesn't work, get the latest extension code and see if that works.

I've not upgraded to MW1.35 yet as it seems that quite a few things don't work with it. Jonathan3 (talk) 01:34, 30 November 2020 (UTC)

Good to know about 1.35. I will abandon for now. 2601:446:4400:700:91F9:3433:1352:6C25 (talk) 02:18, 30 November 2020 (UTC)

Thank you!! 2601:446:4400:700:91F9:3433:1352:6C25 (talk) 02:18, 30 November 2020 (UTC)

P.S. When I moved my wiki (MW 1.34.1) to a new server (using MW 1.34.4) I had to comment out a few extensions to avoid errors in the update script. Can't remember which. It all worked again fine in the end. Jonathan3 (talk) 22:32, 30 November 2020 (UTC)

This error needs to be reported.

Apparently your MYSQL version is too old and requires the prefix "unsigned" before INTEGER

https://stackoverflow.com/questions/52380447/mysql-5-6-cast-string-to-int-returns-error-1064 Ciencia Al Poder (talk) 11:46, 1 December 2020 (UTC)

It must have been reported, although I cannot find the ticket number. It has already been fixed in Fix MySQL invalid syntax regression from bcd6b2f7 (Ia84f135a) · Gerrit Code Review (wikimedia.org) but it hasn't been merged into the REL1_35 as far as I can determine. Peculiar Investor (talk) 20:53, 15 January 2021 (UTC)

We are using MySQL 8 199.247.45.1 (talk) 13:16, 1 December 2020 (UTC)

8.0.17 to be exact. 199.247.45.1 (talk) 13:17, 1 December 2020 (UTC)

A fix for this was two month before posted:

https://www.mediawiki.org/w/index.php?title=Extension_talk:Replace Text/Archive_2020#c-HerrHulaHoop-2020-10-02T09:44:00.000Z-Fokebox-2020-10-02T09:18:00.000Z Gunnar.offel (talk) 18:38, 1 December 2020 (UTC)

The problem is, it won't be fixed if it's not reported properly... Ciencia Al Poder (talk) 22:43, 1 December 2020 (UTC)

For MW 1.35, on <yourwikipath> /extensions/ReplaceText/src/ReplaceTextSearch.php

replace the following:

                 $conds = [

                         $comparisonCond,

                         'page_namespace' => $namespaces,

                         'rev_id = page_latest',

                         'rev_id = slot_revision_id',

                         'slot_content_id = content_id',

                         'CAST(SUBSTRING(content_address, 4) AS INTEGER) = old_id'

                 ];

with

                 $conds = [

                         $comparisonCond,

                         'page_namespace' => $namespaces,

                         'rev_id = page_latest',

                         'rev_id = slot_revision_id',

                         'slot_content_id = content_id',

                         'CAST(SUBSTRING(content_address, 4) AS SIGNED) = old_id'

                 ]; Michele.Fella (talk) 19:09, 1 December 2020 (UTC)

yeah, this change is what i linked before. Gunnar.offel (talk) 21:22, 1 December 2020 (UTC)

@Gunnar.offel you are right, the change is the same! I have the bad habit to troubleshoot low level by myself. Credits to you as you pointed a link before my post!! thanks 151.21.194.165 (talk) 20:10, 17 January 2021 (UTC)

The patch works beautifully, thanks.

For what it is worth, this is not an issue with MW1.35 as such, for on my local test installation (MW 1.35 / PHP 7.4.12 / MariaDB 10.4.16), the bug doesn't occur. However, it did occur on my live wiki (MW 1.35 / PHP 7.4.11 / MySQL 5.7.31-0ubuntu0.18.04.1) before I patched it. Henryfunk (talk) 09:19, 11 December 2020 (UTC)

Looks like this is an issue with MySQL 8 then? Got the same issue on MW 1.35.0, PHP 7.4.3, MySQL 8. Jeffrey Wang 01:07, 13 December 2020 (UTC)

No, for it also occurs with MySQL 5.7.31. Henryfunk (talk) 19:27, 30 December 2020 (UTC)

The developers have fixed it, Gerrit change 631706 but haven't merged it into REL1_35 as far as I can tell. Peculiar Investor (talk) 21:20, 15 January 2021 (UTC)

There is no interest in back-porting a merged patch to the 1.35 release branch. If you want to use the extension with MW 1.35 you need to use master. [[kgh]] (talk) 10:46, 18 January 2021 (UTC)

I would have thought that since 1.35 is the long term support (LTS) release branch that it would be standard policy to maintain extensions on the REL1_35 branch as well. Given that Replace Text is a bundled extension with MediaWiki 1.35 this can be problematic for anyone who intends to use MediaWiki 1.35.

While Wiklpedia and all the related wikis run on the master branch, wiki's maintained by others tend to track Version lifecycle which makes NO mention of the master branch. The Special:ExtensionDistributor/ReplaceText by default offers 1.35 (latest stable MediaWiki). The dropdown does allow selection of master (latest development version). Why would you expect someone on the LTS version lifecycle to NOT use the matching extension version?

Please reconsider this decision for the benefit of all. Thank you. Peculiar Investor (talk) 13:09, 18 January 2021 (UTC)

Do not get me wrong. This comment was just summarizing reality. I am not the developer of this extension, actually not a developer at all. I just happen to be the first clown who came up with the idea of back-porting this to 1.35 might be a sensible idea. Au contraire, I am all in favour of back-porting to 1.35. [[kgh]] (talk) 15:16, 18 January 2021 (UTC)

That's a concerning decision for an extension bundled in the official MediaWiki tarball Ciencia Al Poder (talk) 12:35, 18 January 2021 (UTC)

See my comment above. It was not a decision. I was just the summary after being ignored with the very same question for about a month. [[kgh]] (talk) 15:18, 18 January 2021 (UTC)

@Kghbln There is hope after all. Gerrit change 656920 seems to be moving forward Peculiar Investor (talk) 04:22, 22 January 2021 (UTC)

I will keep fingers crossed. [[kgh]] (talk) 07:41, 22 January 2021 (UTC)

I reported this problem. 1_35 is useless to me without this extension. 2601:446:4400:700:5513:30C:9B67:52AC (talk) 15:23, 18 January 2021 (UTC)

The required patch was now merged into REL1_35. I suppose that this resolved the issue.

@Peculiar Investor: Thanks for bringing momentum back to this issue. I was disillusioned and did not have the power to make a difference. [[kgh]] (talk) 15:10, 2 February 2021 (UTC)

The discussion above is closed. Please do not modify it. No further edits should be made to this discussion.

Is there a better way to search wikitext than using ReplaceText?

I often use ReplaceText in order to search the wikitext throughout the site, but there are limitations to using this method, especially when there are thousands of search results. For example, if I want to find all pages with 'alt=' in the wikitext, using ReplaceText only shows me the number of results equal to $wgReplaceTextResultsLimit. Is there a better way to search the wikitext? I've tried to find an extension, but haven't had any luck. Ahancie (talk) 19:27, 2 December 2020 (UTC)

If you're using CirrusSearch, you can use the insource: option, see here: Help:CirrusSearch#Insource FFS ^Talk 13:22, 3 December 2020 (UTC)

Thank you, that helps. Prior to our current upgrade to 1.34.4, ReplaceText would list ALL pages found in a search, but only allow changes to the value of $wgReplaceTextResultsLimit. For example, if the ReplaceText showed 10,000 pages containing the text I wish to change, then I know I will need to use the ReplaceText 4 times to get all pages changed. Now, I can only see 3,000 pages (which is what we set $wgReplaceTextResultsLimit to), and I have no idea how many pages I am dealing with. Is there a way to get ReplaceText back the way it used to be, so it search ALL pages and lists them, even if only a certain number of them can be changed at a time? Ahancie (talk) 17:31, 8 December 2020 (UTC)

ReplaceText's specific regex behaviour/limitation ?

I'm having a bit of a problem understanding ReplaceText's specific regex behaviour/limitations.

I'm trying to target the following two cases with a single regex.

" abc1def = 234 "
" abc5def = "

Which should not really be a problem for regex in general.
Using a regex like " *abc[0-9]def *= *[0-9]* *" on https://regexr.com/ (default setup) works fine in this respect.

But ReplaceText's regex implementation seems to not want to do this.
ReplaceText's goes for the " abc1def = " part, but skips/ignores the number part in this case.

I'm I missing something about ReplaceText's regex implementation?

ReplaceText: 1.4.1
MediaWiki: 1.31.0
PHP: 7.0.30-0+deb9u1 (fpm-fcgi)
MariaDB: 10.1.26-MariaDB-0+deb9u1
MvGulik (talk) 22:27, 5 December 2020 (UTC)

The leading part of your expression is full of optional matches (asterisks). Since it doesn't need to match, it may be skipped altogether. Ideally you should add some other character at the end that is not optional (maybe a newline, a pipe, etc) Ciencia Al Poder (talk) 22:43, 5 December 2020 (UTC)

> "Ideally you should add some other character at the end that is not optional"

I spotted something similar happening while trying to work this out, but had not followed up on that yet.

And your right, that actually works. (oddly enough)

Seems a bit of an odd constrain to me.

Although using multiple optional matches can be a general source of RE hiccups, this RE is relative simple (I think).

Anyway.

Thanks for the tip. Saves me some unnecessary replace runs. :) MvGulik (talk) 01:23, 6 December 2020 (UTC)

After some more ReplaceText's RE testing in this case it feels to me like ReplaceText's RE switches quantifiers in some particular cases from greedy to extreme none-greedy.

(Note: My knowledge on RE's 'greedy' vs 'none greedy' is a bit rusty at the moment)

Extreme none-greedy: as in 'not matching anything even if there is something to match'.

For example "[a-c]*[0-9]*" seems to work fine. Capturing anything from "a" to "abc123". (seems: as in based on replace preview (which can be a bit iffy at times).)

But "abc[0-9]*" will not match the number part in strings like "abc123". (while "abc[0-9]+" will only match the first digit)

It will match all digits after adding a quantifier modifier "abc[0-9]*?". (or "abc[0-9]+?" in second case)

Is ReplaceText's RE by default 'none greedy'?

(Or em I looking at it from the wrong angle?) MvGulik (talk) 09:07, 6 December 2020 (UTC)

doesn't work anymore with namespaces

In 1.34 you can replace namespaces by replace the text Bastel: to DIY: for example.

In 1.35 the namespaces seems to work in another way. If you try to replace it now, the text of the namespace isn't found anymore. it simply says "no page found" / "Es wurde keine Seite gefunden". i verified, that 91 pages exist with this schematic. Gunnar.offel (talk) 20:37, 8 December 2020 (UTC)

Suggestion: adding an option to the replaceAll.php script to add changes to job queue like the special page

While using Replace Text to make a massive amount of changes to a wiki (25,000+ pages), the database would die often because of the massive amount of stress it's being put under.

I tried using both the CLI script and Special:ReplaceText. The problem with the CLI script is if it dies, when it's being re-run, it has to restart and do a SELECT SQL query to search for all the pages to replace again. If the CLI script keeps dying, it becomes really, really inefficient to keep doing this SQL query, especially if you increase the limit from 250 to say 1000.

The benefit of the job queue is if it dies, it can pick right up from where it left off. Furthermore, the SELECT query only needs to run once. And usually the SELECT query won't die, but if it does, it's more probable it will succeed than all 250+ changes to the database.

Perhaps it would be better to add an option in the CLI to allow the option to add the pages to be replaced to the job queue, just like the special page?

Sorry if this is the wrong place to make the suggestion, I can move it to Phabricator or somewhere else if that's preferred. Jeffrey Wang 02:01, 13 December 2020 (UTC)

Sorry for the delay. This is certainly an option... one other option is to add some flag to replaceAll.php that would let you set a "wait" period between one replacement and the next - say, one second. (Which, for 25,000 pages, would add about 7 hours to the overall running time.) Or, the script could run replacements in batches of 100 and then wait some time between each batch and the next. Any thoughts on those different approaches? Yaron Koren (talk) 20:08, 18 December 2020 (UTC)

Thanks Yaron for the reply! I was able to get through 25,000+ changes in less than 7 hours. Here's my experience:

• Trying to do a SELECT for all 25,000+ changes to be made also crashed the DB. So maybe doing one big SELECT won't work either.
• SELECT for 1000-2000 at a time worked well. The limitation here is you'd need to run the job queue before doing another SELECT, but this still works decently well.
• The job queue would still often crash, but at least all the jobs ran beforehand did not have to be repeated.

I ended up just setting $wgReplaceTextResultsLimit to 1000 and entering the same query using the web form each time, then running through the job queue. So the most annoying part was entering the same query 10+ times into the form.

I think running replacements in smaller batches would be the most prudent approach. This is what Special:ReplaceText already does anyway, and it works really well.

Here's some pseudocode I thought of, which would've worked perfectly in my case:

• Add a flag to specify how many (let's call this amount n) to do at a time (essentially an equivalent to $wgReplaceTextResultsLimit for the replaceAll.php script)
• SELECTthe first n
• run php maintenance/runJobs.php
  • if stderr returned, sleep(5) and then call php maintenance/runJobs.php again
  • keep running until exit code 0 returned and job queue finished
• Loop back, SELECT the next n and repeat until SELECT query empty

The goal here is to minimize the number of times that php extensions/ReplaceText/replaceAll.php has to be called from the shell, or at the very least making sure it is able to handle sudden interruptions. Jeffrey Wang 22:19, 18 December 2020 (UTC)

That's good to know that even the SELECT query by itself can crash the server. Given that, maybe the best solution is just to add a "limit" flag to the script, to let you limit each run to some number of replacements? Would using the job queue still be necessary at that point? Yaron Koren (talk) 17:12, 20 December 2020 (UTC)

I would still prefer the job queue because let's say we SELECT 100, we run through 25 changes, and then the script crashes. If using the job queue, we can just finish the last 75 changes. Whereas if we didn't use the job queue, we would select another 100, presumably including the 75 changes, which kind of wastes the computing effort in from the first SELECT. TL;DR: Not using the job queue means more redundant SELECTing. Though I suppose it would not make too much of a difference if the limit is set at a reasonably low number, so it is probably fine to not use the job queue, since that'd be easier to code. In that case, my only hope is that the script can loop the select/replace n pages each time (n = limit flag), so there's no need to call the script 100 times. Jeffrey Wang 20:36, 21 December 2020 (UTC)

Oh, that's very interesting - it sounds like it's really just the SELECT that's causing the problem for you. If so, that might make the solution very easy - no need for a new "limit" parameter, just have the script do, say, 100 replacements at a time, and keep going until there are no more replacements to be made, rather than trying to do them all at once. Would that work? And if so, what's a reasonable number of replacements to do each time? Yaron Koren (talk) 21:04, 21 December 2020 (UTC)

How to remove lines starting with pipe symbol "|" on Wiki pages?

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

Dear all,

I have some redundant information in my Wiki which I would like to clean up and remove.

I'm talking about an SMW attribute named "Titel". Each page in my wiki has this attribute.

So my task is to remove a line as follows on each page:

Titel=Some individual text... So the search string should be something like:

Titel.* which I would then replace with an empty string. Unfortunately this search gives me a database error: ____________ [X@287-FSfWJq11TyEbd5ZgAADQA] /index.php?title=Spezial:Text_ersetzen Wikimedia\Rdbms\DBQueryError from line 1603 of /www/htdocs/w00997ce/hvg-wiki-2020-03/includes/libs/rdbms/database/Database.php: A database query error has occurred. Did you forget to run your application's database schema updater after upgrading? Query: SELECT page_id,page_namespace,page_title,old_text FROM `page`,`revision`,`text`,`slots`,`content` WHERE (old_text REGEXP '|Titel=.*') AND page_namespace = '0' AND (rev_id = page_latest) AND (rev_id = slot_revision_id) AND (slot_content_id = content_id) AND (SUBSTRING(content_address, 4) = old_id) ORDER BY page_namespace, page_title LIMIT 250 Function: ReplaceTextSearch::doSearchQuery Error: 1139 Got error 'empty (sub)expression' from regexp (localhost) ... ____________ Root cause of this issue seems to be the pipe symbol, if I leave this out it seems to work. However, I need to remove the entire line incl. the pipe symbol on each page. Does anybody have a hint how to overcome this problem? Many thanks! Sochin67 (talk) 12:10, 31 December 2020 (UTC) You need to "escape" the pipe. I think it's done as \| in regular expressions but have not tested it with ReplaceText right now. Tenbergen (talk) 14:31, 31 December 2020 (UTC) Dear Tina, thanks for your feedback! I tried it with a search for the following string: \|Titel=.* The DB error is gone. However, unfortunately only the string from the search field ("|Titel=") is removed but the rest of the line remains. I also tried this without the escape + pipe (just "Titel=.*") and the result is the same. So for some reason the wildcard functionality is not working correctly. Do you have any idea concerning this issue? Many thanks! Sochin67 (talk) 16:44, 31 December 2020 (UTC) You need some character or string which is guaranteed to be after the wildcard, to force the regex to match the whole line. Probably the simplest, if you're certain the parameter will have an entire line to itself, would be something like \|Titel=.*\n. Another suggestion would be if it's always followed by another parameter, in which case you could match a second pipe: \|Titel=.*\|, though note that this will require you to replace with a pipe instead of an empty string, and will also not work correctly if piped links can appear in the "Titel" parameter. 「ディノ奴千？！」☎ Dinoguy1000 18:13, 31 December 2020 (UTC) It worked with \|Titel=.*\n - thank you very much!! Sochin67 (talk) 11:29, 1 January 2021 (UTC) Cool. Did not know missing support for \n was added/fixed. Two other cents, to find any and all variations. (assuming one template parameter per line) Search: "(?i)(\n) *\| *tItEl *=.*\n" Replace: "$1" MvGulik (talk) 11:33, 4 January 2021 (UTC) The discussion above is closed. Please do not modify it. No further edits should be made to this discussion.