Extension talk:AbuseFilter

Jump to navigation Jump to search

About this board

19 previous topics. Previous discussion was archived at Extension talk:AbuseFilter/Archive 1 on 2016-10-24.

Can't create new filter

6
Summary by EscoBye

$wgServer was set to HTTP, should be HTTPS.

EscoBye (talkcontribs)

Hello, I installed the AbuseFilter extensions and have the following configurations:


wfLoadExtension( 'AbuseFilter' );

$wgGroupPermissions['sysop']['abusefilter-modify'] = true;

$wgGroupPermissions['*']['abusefilter-log-detail'] = true;

$wgGroupPermissions['*']['abusefilter-view'] = true;

$wgGroupPermissions['*']['abusefilter-log'] = true;

$wgGroupPermissions['sysop']['abusefilter-privatedetails'] = true;

$wgGroupPermissions['sysop']['abusefilter-modify-restricted'] = true;

$wgGroupPermissions['sysop']['abusefilter-revert'] = true;


I ran update.php without any issues and can access Special:AbuseFilter but when I try to create and save a new filte the page simply refreshes and the filter is not saved. I tried adding 3 different filters and the filter list is still empty. I am logged in with a bureaucrat/admin account. Does someone know why this is happening?

Daimona Eaytoy (talkcontribs)

I've never heard of this bug. What version of MW are you running, and what version of AbuseFilter? Knowing the PHP version and the DBMS may help, too.

As an aside, note that "abusefilter-privatedetails" allows viewing the IP address of registered people who triggered an AbuseFilter -- I'm unsure if it's intentional to allow sysops to do that.

EscoBye (talkcontribs)

I forgot to include the versions in my first post but here they are:

MW: 1.34.2

PHP: 7.4.7

MySQL: 8.0.20

AbuseFilter: 3c2035d, probably REL1_34?

Daimona Eaytoy (talkcontribs)

Yeah, they seem fine, and everything should be supported. To clarify, could you please provide detailed steps for your attempt? Also, is there anything logged either in the JS console, or in the MW logs (see Manual:How to debug)?

EscoBye (talkcontribs)

I just noticed that I get the following warning: Mixed Content: The page at 'https://domain/w/Special:AbuseFilter/new' was loaded over a secure connection, but contains a form that targets an insecure endpoint 'http://domain/w/Special:AbuseFilter/new'. This endpoint should be made available over a secure connection.

The subitted form probably never goes through since it's submitted over HTTP, any idea how to make the form submit over HTTPS?


Here are the steps:

  1. Starting at https://domain/w/Special:AbuseFilter/new
  2. Description: Prevent gibberish usernames
  3. Conditions: action == "createaccount" & accountname rlike "^[A-Za-z0-9]{10,}$" & accountname rlike "^[A-Z][a-z]{5,}(?:[A-Z].*[0-9]|[0-9].*[A-Z])"
  4. Hide details of this filter from public view checked
  5. Prevent the user from performing the action in question checked
EscoBye (talkcontribs)

I managed to solve the issue, the $wgServer was set to HTTP by mistake, I forgot to edit it when enabling SSL. Thanks for the help Daimona!

How to export/import all filters?

7
Sokote zaman (talkcontribs)

hello

How can I copy all the filters on my site?

How can I export the filters on my site?

thanks

Daimona Eaytoy (talkcontribs)

There's no way to do that, see T42191.

Sokote zaman (talkcontribs)

So what is this guide for?

php dumpBackup.php \

--plugin=AbstractFilter:extensions/ActiveAbstract/AbstractFilter.php \

--current \

--output=gzip:/dumps/abstract.xml.gz \

--filter=namespace:NS_MAIN \

--filter=noredirect \

--filter=abstract

Daimona Eaytoy (talkcontribs)

I'm unsure, but that seems to mention "AbstractFilter", which has nothing to do with AbuseFilter.

Sokote zaman (talkcontribs)

Thank you for taking the time to follow up

Ciencia Al Poder (talkcontribs)

If you're interested, there's a grabAbuseFilter.php in Manual:Grabbers to import all public filters from one wiki to other.

Daimona Eaytoy (talkcontribs)

Thanks for pointing this out, I'll mention it on phabricator. However, note that the script doesn't currently work, because it doesn't update the abuse_filter_action table (used to retrieve enabled actions for existing filters). I'm also unsure about the impact of not adding an entry in abuse_filter_history. I wouldn't recommend using it as-is on a public-facing wiki, because it would put the DB in an inconsistent state.

Reply to "How to export/import all filters?"
L2212 (talkcontribs)

I'm trying to learn how to use the AbuseFilter, to deal with a quite annoying vandalizer on sc.wiki, but the guides Extension:AbuseFilter/Rules_format and Extension:AbuseFilter/Conditions guides are not enough for me to understand it and configure it well. For example, watching the en.wiki filters I discovered that the category for articles is "page_namespace == 0" and for user talk pages is "page_namespace == 2", but where can I find the list for all of the other kinds of pages? And what does "irlike" mean, exactly? Because I used it in a filter, but then it gave me a few false positives and I had to change it. And how can I specify not words or characters, but changes like adding multiple empty paragraphs?


To make an example, I need a filter to block edits like this one, where that same vandal has been adding empty paragraphs to an article (sometimes adding text before it, like here) but I don't know how to do it. Is there any other guide where I could read which variables are available and what all of those variables do?

Dinoguy1000 (talkcontribs)

For example, watching the en.wiki filters I discovered that the category for articles is "page_namespace == 0" and for user talk pages is "page_namespace == 2", but where can I find the list for all of the other kinds of pages?

With apologies for singling this point out, these are namespace numbers, and a full list of them can be found at w:Wikipedia:Namespace (for the English Wikipedia; you can get the list of namespaces specific to the Sardinian Wikipedia via the API). (Also, the namespace with number 2 is User, not User talk, which is instead 3 - you can remember content versus talk namespaces because content namespaces always have an even number, while talk namespaces always have an odd number.)

Ciencia Al Poder (talkcontribs)

One start point is Special:AbuseFilter/examine, where you can examine past edits and test code against it.

For empty lines, you can use \n that represents a new line. added_lines can be used to test only for the lines added/modified during the change, and write a regular expression like added_lines rlike '\n{10,}' which should match the addition of 10 or more lines in a row

Daimona Eaytoy (talkcontribs)

To add to what others said, see also: m:Small wiki toolkits/Starter kit/AbuseFilter, and w:Regular expression.

Also, while we do try to make guides easy to read and understand, one should always keep in mind that AbuseFilter is a "technical" tool. No matter how guides are simplified, they will always assume some prerequisites; at minimum, this means the basis of boolean logic and value/type comparisons; ideally, also string manipulation and regular expressions.

These concepts aren't too difficult to learn, and I always encourage people to try. However, it's very important to prevent any collateral damage while experimenting with AF. For this reason, I highly recommend avoiding to enable any filter consequence if you're not sure of the result. It shouldn't even warn the user, let alone disallow or block them.

L2212 (talkcontribs)
Reply to "Abusefilter guide"

My AbuseFilter not working properly.

20
Summary by Farvardyn

Thanks a lot to all!

Farvardyn (talkcontribs)

I exported Special:AbuseFilter/5 (User self-renaming or moving user talk pages into article talk space) and Special:AbuseFilter/12 (Replacing a page with obscenities) then imported to my wiki. It seems everything is installed properly and I did set them as Disallow/Enabled/Hidden. Then with another non-admin ordinary account, I tried to add <nowiki>#REDIRECT [[Talk:Test]]</nowiki> in my non-admin talk page, and it did not catch it. The same with second functionality above: I added a bad word (the word starting with f), and it did not hit it niether. The filter was set as 'Disallow', so why why hitting/disallowing was not working fine?

Daimona Eaytoy (talkcontribs)

For filter 5, adding #REDIRECT [[Talk:Test]] to a page is not supposed to cause a match, because this text doesn't match the regular expression used by the filter. Likewise, filter 12 has nothing to do with bad words at all. Are you sure that the links above are correct?

Farvardyn (talkcontribs)
Dinoguy1000 (talkcontribs)

In your first comment, you linked to filters on both MediaWiki.org (this site) and English Wikipedia. Daimona probably just missed this. Wikipedia's filter 5 only triggers on page moves, so simply editing your user talk page and replacing the contents with a redirect won't trigger it. Filter 12 only triggers if the edit includes a reduction in the page size (i.e. replacing most or all of the page's content) along with the profanity, so simply adding a swear also won't trigger it.

Farvardyn (talkcontribs)

@Daimona Eaytoy @Dinoguy1000 I also installed both `New user blanking articles` and `Large deletion from article by new editors` with Warn,Disallow and default error message for both: `abusefilter-warning` and `abusefilter-disallowed`. Then as normal user I tried to large edit and blanking an article, and I got this message:

`[Xrqt36tVvzaBrdFz40ulNwAAA4I] 2020-05-12 14:08:32: Fatal exception of type "Error"`

I tried also by disabling the extension completely and I could large delete/blanking an article and I could do, so I assume one of these filters is triggered and is working fine to disallow/warn it. But why I don't see any static hit (still 0) with irrelevant error message as I did not customize error messages.

Dinoguy1000 (talkcontribs)

A fatal exception means that somewhere the code errored out. Since it didn't log any action, it probably happened before the extension would have logged it. But I can't help with this problem further than that; you'll need someone who's familiar with the code side of the extension probably.

Daimona Eaytoy (talkcontribs)

So you meant filter 5 and 12 on enwiki IIUC. The links above point to filters 5 and 12 on mediawikiwiki, aka the wiki we're on currently. Assuming that I got it right this time, here are the answers.

  • Filter 5: It catches page moves, not redirections. So, in order to trigger it, you'd have to move your own user page to another title in the User: namespace.
  • Filter 12: The filter has too many conditions for me to be able to guess what went wrong. You have to ensure that the account you're using has no more than 30 edits, that the size of the page before and after the edit matches the numbers there, and that the new content of the page matches the regexp used by the filter.
Daimona Eaytoy (talkcontribs)

Whoops, I see Dinoguy1000 replied while I was writing, and the answers are indeed correct.

As for the error: the first thing that comes to mind is, ensure you have installed composer dependencies, especially wikimedia/equivset, as that's needed for using ccnorm. Aside from that, I'd need to see the complete stack trace of the error to understand what's going on (see Manual:How to debug).

Farvardyn (talkcontribs)

@Daimona Eaytoy@Dinoguy1000 Thanks for trying to help. I did read the how to debug link. As about below in index.php:


    error_reporting( -1 );


    ini_set( 'display_startup_errors', 1 );


    ini_set( 'display_errors', 1 );


I got no error. As about:


$wgDebugDumpSql = true;


$wgShowSQLErrors = true;


$wgShowDBErrorBacktrace = true;


$wgDebugLogFile = "debug-{$wgDBname}.log";


I got this trace error: [EDITED by Farvardyn] so I did run composer under AbuseFilter/ folder then `utfnormal` and `equivset` got installed. I also make sure my test account has no more than 30 contribs. But this time, with my test account I can largely delete an article and blank it with no error. This time I get no error nor blanking/largely deletion is trigerred! What else should I do? I highly appreciate your help.

Daimona Eaytoy (talkcontribs)

Glad to see that the first issue is now resolved! As above, unfortunately, there's not much we can do. I can just copy below the conditions checked by the filter, and you could make sure that all of these are satisfied:

  • No more than 30 contribs
  • Page size before the edit greater than 300 bytes, and lower than 300 bytes after the edit, OR more than 5000 bytes removed with the edit
  • The page is not a redirect
  • The page title doesn't contain any of "Sockpuppet investigations", "Sandbox", or the username of the test account
  • The new text of the page contains one of the bad words
  • The old text of the page doesn't contain one of those bad words
Farvardyn (talkcontribs)

@Daimona Eaytoy May I invite you to see that on my wiki? The page I tried blanking is: [EDITED by Farvardyn] and I will delete this link from here after you got it. Please create an account and try to blank it. Let me know which permissions do you need that I grant them to your account in order to test it. After that I can delete your account if you wish to not stay in my wiki. I highly appreciate your help.


Daimona Eaytoy (talkcontribs)

Thank you for sharing the link! AFAICS, you did indeed blank the page, but did not add any bad word. Both conditions should hold. Your blankings are perfect, just make sure to add a bad word as well the next time you try blanking the page :-)

Farvardyn (talkcontribs)
Farvardyn (talkcontribs)

I googled how to translate abusefilter-warning and abusefilter-disallow messages, that it always shows the translated version rather than English and found nothing in google. Please advise. @Daimona Eaytoy @Dinoguy1000

Daimona Eaytoy (talkcontribs)

Since they're custom system messages (see this page for help, I don't know if there's a more appropriate one), I'm not sure if there's a built-in way to translate them. I have little knowledge of the matter.

Dinoguy1000 (talkcontribs)

The translation method for built-in messages (i.e. subpages of the message with the subpage name being the ISO 639 language code for the language being translated to, e.g. for a French translation of a built-in message MediaWiki:Foobar, you would go to MediaWiki:Foobar/fr) might also work for custom messages, though I've never tried it myself.

Farvardyn (talkcontribs)

How can I set a custom translation for built-in message like abusefilter-disallow? I avoid automatic translation like google translator, I want to set a custom translation. How can I do so? @Daimona Eaytoy @Dinoguy1000

Dinoguy1000 (talkcontribs)

Do you mean abusefilter-disallowed? abusefilter-disallow isn't a built-in message. In any case, for built-in messages, you would determine the language tag for the language you want to translate to (Wikipedia has lists of these, linked from the ISO 639 page), and add that as a subpage name for the message you want to translate (e.g. for a French version of abusefilter-disallowed, it would be abusefilter-disallowed/fr). Note that built-in messages of MediaWiki itself, and any extensions used by WMF wikis (such as AbuseFilter), are already likely to be translated into most popular languages, so any built-in messages you're interested in translating may already have been translated into the language(s) you're interested in. This should be self-evident when you view the appropriate subpages, since you'll see translated content there instead of the generic "page doesn't exist" message.

Farvardyn (talkcontribs)

Got it. Thanks. Just one more question: CommentStreams added `staff` user group. Any other user groups are already translated except this one.How can I translated a user group? @Dinoguy1000 @Daimona Eaytoy

Farvardyn (talkcontribs)

The main reason I want to customize Mediawiki:abusefilter-disallowed is that I want to use Fmbox template to add an icon beside the text. But if I use any of Mbox templates, it looks like https://i.stack.imgur.com/1swED.png because Mediawiki:abusefilter-disallowed already uses a red box by default itself too. How to fix it? @Daimona Eaytoy@Dinoguy1000

Reply to "My AbuseFilter not working properly."
Farvardyn (talkcontribs)

Is `$wgAbuseFilterCentralDB = 'metawiki'; just an example? or can we really set it and use metawiki as a central db for abuse filter?

Daimona Eaytoy (talkcontribs)

It depends on who "we" is. If it's for a WMF wikis, then yes, 'metawiki' can be used as a central DB for filters. Otherwise, no. The variable should contain the name of a database which is recognized and accessible by your wiki.

How to export/import private filters?

2
Farvardyn (talkcontribs)
Majavah (talkcontribs)

How to use this only allow some link?

6
Summary by Tunglinwu

use !(added_links irlike "^(?:(https?:)?\/\/(allowed1.com|allowed2.org|allowed3.net)\n?)*$") only allow some link.

Can use this block ad link.


But it in [https://allow.com link ] until be block.

Tunglinwu (talkcontribs)

If I only want to allow some links

e.g. i only allow http://www.abc.com/*

use all_links?

all_link !== "http://www.abc.com"

Daimona Eaytoy (talkcontribs)

I think you should use added_lines instead of all_links (i.e. ignore pre-existing links). AF lacks support for array intersection, so perhaps your best option could be something like !(added_links irlike "(https?:)?\/\/(allowed1.com|allowed2.org|allowed3.net)") [not tested].

Tunglinwu (talkcontribs)

No, it can't use.

All links trigger "added_links" function include your way.

Daimona Eaytoy (talkcontribs)

Ah, my example was incomplete. The actual check would be: !(added_links irlike "^(?:(https?:)?\/\/(allowed1.com|allowed2.org|allowed3.net)\n?)*$")

A quick test would seem to indicate that it works.

Tunglinwu (talkcontribs)

IF in [https://allow.com ALLOW] until be block.

Tunglinwu (talkcontribs)

it's worked, thank you.

Reply to "How to use this only allow some link?"

Special:AbuseLog not showing actions taken

11
Justin C Lloyd (talkcontribs)

I just upgraded our wikis from 1.30 to 1.34 and now the Special:AbuseLog page (e.g. when clicking a value in the Hit count column on Special:AbuseFilter) is not showing any results. There are no errors anywhere I can find, and examining the database tables (i.e. select afl_filter, count(afl_filter) from abuse_filter_log group by afl_filter) does show the results there and the row counts correspond to the filter ids. Is this possibly a bug or perhaps is there something I may have missed doing during the upgrade procedure?


Daimona Eaytoy (talkcontribs)

As long as you followed the usual update procedure (notably, update core, update AbuseFilter, run update.php), this could indicate some sort of bug. However, I'm not aware of similar bugs. Could you please check (e.g. by using the debugging toolbar) what query is being executed on Special:AbuseLog, and try executing it manually on the DB?

Justin C Lloyd (talkcontribs)

I enabled the variable, along with $wgDebugDumpSql and even $wgDebugLogFile for good measure. However, I'm not even seeing the toolbar that should be there (which I've never used, so this is good to know about in general).


Daimona Eaytoy (talkcontribs)

Hmmm, weird. In theory, $wgDebugToolbar should be enough to enable the toolbar. Without knowing what query is being executed, it's hard to determine where the problem lies. Perhaps you could try other methods to inspect what queries are reaching your DB, like with pure mysql logging, although that could be a bit noisy. Of note, what RDBMS are you using?

Justin C Lloyd (talkcontribs)

Ah it's because I'm using Varnish so $wgUseSquid is enabled.


Justin C Lloyd (talkcontribs)

I'm using AWS Aurora MySQL (5.6 compatible).

Justin C Lloyd (talkcontribs)

I think this may be the query in question:

SELECT  *  FROM `abuse_filter_log` LEFT JOIN `abuse_filter` ON ((af_id=afl_filter))   WHERE afl_filter = '28' AND afl_deleted = '0'  ORDER BY afl_timestamp DESC LIMIT 51

I noticed that in the debug log.


Apparently if I take out the afl_deleted check, it seems to work.

Seems that all of the afl_deleted values are NULL.

Justin C Lloyd (talkcontribs)

Ok it seems like I need to add abuse-filter-hidden-log group permissions, that seems to resolve it. Just curious why that would be needed now, if you might have any idea.


Daimona Eaytoy (talkcontribs)

Ahhh, this is T240895... It should really be fixed and backported. In the meanwhile, you can manually run the SQL commands here to unbreak the world.

Justin C Lloyd (talkcontribs)

Thanks! I've tested the SQL queries against one of my dev wikis and it looks to have worked. Appreciate the help!

Reply to "Special:AbuseLog not showing actions taken"
Sokote zaman (talkcontribs)

Hi

Please provide a code that the user can post only after logging in to the user's home page and user practice page and discussion page.

Thanks in advance for your help

Matěj Suchánek (talkcontribs)

Something like:

action == 'edit'
& (page_namespace == 2 | page_namespace == 3)
& user_age == 0

Please see Extension:AbuseFilter/Rules format for guidance on creating filters.

Sokote zaman (talkcontribs)
Reply to "access"

Event when an filter is triggered

4
JakePotrebic (talkcontribs)

Maybe I'm missing something but is there no event that is triggered when a filter is triggered? I see the AbuseFilterShouldFilterAction but that is supposed to return a value telling the extension whether to filter the action or not.

Daimona Eaytoy (talkcontribs)

Do you mean a hook that is called by AbuseFilter whenever a filter is triggered? If so, no, there's nothing like that at the moment. Here is the full list of hooks currently available. May I ask you what your use case would be?

JakePotrebic (talkcontribs)

Yeah thats exactly what I was hoping this extension had.


I am working on an extension to forward info from certain hooks (page edits, creations, deletes, etc. ) to a configurable URL for custom webhooks for various applications (discord, slack, etc.). Since Gamepedia uses this extension, I was hoping to include this.

Daimona Eaytoy (talkcontribs)

Yeah, I confirm that such a feature doesn't exist. However, we do send this information to other consumers: the $wgAbuseFilterNotifications global variable can be used to tell AF where this info should be sent; the options are RC, UDP, or both. Maybe it would be possible to use it for your use case?

Reply to "Event when an filter is triggered"