Talk:Quarry

Discussion area for discussion about Quarry itself and help about individual queries.

Some weeks ago I've seen a queryable way to check whether a wiki is closed or not. Unfortunately I don't find it anymore (should have written down the link due to my age...). Any hints?

If Wikimedia - https://noc.wikimedia.org/conf/highlight.php?file=dblists/closed.dblist

This configuration is stored in configuration file, not in database.

wikitech:Help:Toolforge/Database#Metadata_database is probably what you are looking for.

@Edgars2007 and others. Any idea how to find images like this https://commons.wikimedia.org/wiki/File:Arbol_y_cielo_sepia.jpg

For only hidden cat the quarry is like this: https://quarry.wmflabs.org/query/38499

Hi! is there any way to know if a revision changed the proofread status of a Wikisource page?

Is there someone who (has the authority) to execute update querries on four misnamed Wikisource volumes? It would entail the renaming of the 4 djvu files and rename about 820 pages in each volume and replace the the same in the main namespace pages.

Use bot to move these pages

Someone tried to help me do that but the move failed. I never used the bot. I made a list of the changes need to be done https://en.wikisource.org/wiki/User_talk:Ineuw#Changes_required_for_the_four_History_of_Mexico_(Bancroft)_volumes

The list of changes is incomplete and probably the suggested changes are incorrect.

1. I think subpages of file pages are unsuppoerted (as "/" is not a legal character in a filename AFAIK); I see you decided to change "/" to "-"
2. moving an index/file page requires to move the associated pages in Page namespace; and this is good idea to prepare the list of required moves in advance as after the move you lose the list of pages
3. There are sometimes problems with redirects in Index namespace (they may need manual fixing of page content model by admin and they seems to be not fully supported by proofreadpage). I would suggest avoiding such redirects, but this requires fixing all pages linking to them.

The (2) suggests that the move is performed by an admin who is a also a bot operator or has online support of a bot operator (to synchtonise the changes and make the mess as short in time as possible). (Admin rights should not be necessery, but they may be needed for fixing (3) or any accidental mistakes).

Making such a move on an index being worked on in always a mess for them.

I made some such moves in pl.ws, but not such massive ones. My bot has not the bot flag here, so I cannot help at the moment.

Ankry, thanks for the advice. I forgot about the "/"slash in the index file name and it was converted to a "-" hyphen automatically by move process. I will correct my list accordingly. Already posted a bot request on en.WS

Replying to my own comment. I began the move on the Mediawiki Commons, where the originals and images are kept - before renaming the .djvu file in Wikisource. The Wikisource move failed and I am stuck with a self-made mess. Would it be possible to restore EVERYTHING to their original state on the Commons and Wikisource? The reason being that a forward moving process of correcting my mistakes will leave behnid a big mess.

Maybe @ankry: knows?

Any example of quarry, which I can pass to pagepile to create new?

That is tracked as phab:T190242. One Day...

Yes, but pagepile says it can create pagepile from quarry. I need an example of such quarry.

Hi, can anybody help me create a query for a list of "Categories with exactly 1 category in them"? Thanks

Nvm, I got it quarry:query/37399

I've been trying to run https://quarry.wmflabs.org/query/28096 for the last few days, but it keeps timing out. It normally works (or works after timing out once or twice). Are things running slow on the back-end? If so, would it be possible to increase the time-out from 30 minutes to 60 minutes or so? Thanks!

It says it's executed in 894.39 seconds. Is this still an issue?

I just reran it again today, and it worked first time with an execution time of 735.61 - so maybe this is resolved. However, let's see how it goes over the next week...

Note that Quarry is "simply" an interface for Wiki replicas databases. A timeout certainly comes from these DB.

It's back to timing out again today. :-( It must somehow be linked to server loads, as I'm not changing the query between runs, and the amount of data to analyse and the number of results are roughly similar.

It's still timing out. :-(

... and today it completed in 484.18 seconds (~8 minutes). Very weird - the run time seems to be very random.

It's been back to timing out for most of this last week.

I have been having the same problem with query/36685 :(

The query has been timing out for the last fortnight now. :-(

The query has been timing out for the last fortnight now. :-( I've filed a bug report at https://phabricator.wikimedia.org/T226050

Same issue with quarry:quarry/36666 which had been taking about 15 minutes but is now timing out after 30.

I experience the same thing. It seems analytics cluster is overloaded or there may be some other issue. All my simple queries now take 20+ minutes or timeout, a year ago they run under 2 minutes.

For Toolforge users the easy solution is to try the web cluster (sql --cluster=web enwiki), but the issue should be solved anyway.

Per IRC thread everything should be back to normal soon

IRC thread?

I was able to run quarry:query/36685 about 20 minutes ago, and it didn't timeout this time!

I'm getting error 500 from https://quarry.wmflabs.org/query/runs/all - and https://quarry.wmflabs.org/query/37235 seems to be queued, which is new!

Error 500 should be fixed now based on what guys have said on IRC (#wikimedia-cloud)

It seems to be working now, thanks! :-)

The font size on https://quarry.wmflabs.org/ is now much smaller than before. Was this intentional?

Which part is smaller?

I can confirm that. The black input field, "Fork of...", "This query has been published...", query names under My profile. Firefox.

@Framawiki: I think this is related to https://github.com/wikimedia/analytics-quarry-web/commit/29e39d686992ca48d973fc03ed2e073bac591e2e#diff-84c0b0cbacf66dd21e5b6405ef115fa3R48

Buh. Patch: https://gerrit.wikimedia.org/r/c/analytics/quarry/web/+/518315

It should be better now, can you confirm?

Seem correct, thanks!

We can get number of people who put specific page into Watchlist through [https://en.wikipedia.org/w/api.php?action=query&prop=info&titles=Main%20Page&inprop=watchers API]. Could We become able to do this through Quarry?

I'd like to create page list most watchlisted. Thank you.

No, the relevant tables are not public for privacy reasons.

Due to the ongoing migration to the actor table, quite a number of queries need modifications. Currently I have 75 queries in my Quarry account, and it would be nice if there was a "Search in my queries" functionality that would allow me to look for e.g. "_user". Would it be possible to add such a search field to Quarry?

I currently simply use scraping for such purposes (also it acts like backup). But yeah, this would be very, very nice.

There should be thread or phab ticket for this. Something like "meta-quarry" (that is, query the quarry data).

phab:T90509 seems related, but it sounds like you might be hoping for searching of the SQL query body itself too.