@SSastry (WMF) I have flicked through the test pages and tried to identify if there were tests on the Wikisource Page: and Index: namespaces as align with mw:Extension:ProofreadPage . Browsing the list was too hard and tedious to try and determine and I could see no means to narrow tests to a wiki or a sister of wikis. As these two namespace are critical work areas for the WSes, are you able to point to such tests or indicate that there has been some level of testing and success? Thanks.
Topic on Talk:Parsing/Replacing Tidy
Yes, the UI needs an update to display results per-wiki. But, we've tested against enwikisource, itwikisource, frwikisource, and eswikisource ... about 200 pages each. However, all test pages are from the article namespace.
You can look at http://mw-expt-tests.wmflabs.org/visualdiff/pngs/enwikisource/ for the list of pngs for enwikisource. Look for pngs with ".diff.png" -- those show the rendering diffs for that title between Tidy and its replacement. Similarly for the other 3 wikis.
"Page:" and "Index:" namespaces are important content namespaces at Wikisource. I believe that there's also a Wikipedia with a "List:" namespace. Can the tests be expanded (within reason) to run a few other namespaces?
For a bunch of technical reasons, this is going to be somewhat non-trivial. It involves (a) generating a random list of titles (b) generating a full list of titles (including templates) that need to be exported to create a dump (c) generating a dump (d) importing the dumps into two VMs (e) updating the test database with the new titles from (f) rerunning tests
At this time, we have a document set of steps and scripts and process to get this all done. But, it involves a bunch of co-ordinating with different people to run scripts on different servers, etc. It can be done and we'll dicsuss this.
But, if we are going to be doing this, it is good to get recommendations for any other things that need coverage beyond what we've tried to do already so we don't have to repeat this process again. Can you add a section to the replacing_tidy page with a list of other namespaces / projects / languages / wikis that should be added to the test set?
Another constraint: It takes about 2 days to get each test run completed and to weed through failures and retry failed tests. So, we don't want to do a huge jump beyond the 64K titles we are already testing. But, we can take appropriate sized samples based on the size of the wikis (if we need to include any new ones).
At this time, both Tim and I are not aware of reasons why some of these namespaces will need specialized testing with respect to Tidy. So, we think it is best for the ParserMigration deployment this quarter when we can readily test any page anywhere. We can then evaluate if we need any additional automated testing.