If a page was reviewed, and then was updated, does it go back to unreviewed status? I'm currently using FlaggedRevs and was wondering if this could replace that.
No, it remains reviewed.
Please record a screencast of yourself doing New Page Patrol
(moved from Talk:Article_creation_workflow since this is a more appropriate place)
Hi, everyone. Thanks for all your ideas and feedback. I'm really excited about this project--I think we have a chance of making something really useful.
One of the things we've realized in talking with new page patrollers is that, in the absence of any real coherent UI, everyone has tailored their own custom interface from scripts, browser plugins, etc. This has its good and bad points, but it's also meant that we at the WMF can't actually examine the interface most of your are using, because we don't have it.
Any good software design effort starts with understanding what people are doing now. So, we're hoping that a few of you could send us screencasts of yourselves doing NPP. Here's specifically what we'd like you to do:
- Begin recording a screencast of your NPP session. I think 10-15 minutes would be about right, but feel free to take more or less time as you see fit.
- Clearly state your username, so we can easily link your comments with your video.
- Start doing NPP. Spend about the first half of the time moving slowly, talking through the process and explaining what you're doing, why you chose that specific tool, what you like/dislike about it, and any other options you tried in the past. If you're using gestures or other off-screen interface elements, describe them. Also tell us about what you're looking for in the article, the policy decisions you're making, and your thought process. If you suspect copyvio, tell us what feature triggered that suspicion; if something seems like spam, what's spammy about it? Show us your process.
- Spend the rest of the time running through NPP in your usual way. This gives us an idea of the pace at which things happen, how long specific steps take, etc. Don't try to go faster or slower than you normally would here. The idea is to understand where the bulk of the time and effort go, so we can streamline that process.
- Wrap up the video.
- If you're using the usertesting.com tool, upload the video per the instructions below. If you're using your own tool, send your video (the whole file, or a link if it's really big) to me at email@example.com.
- Send any additional comments you have in an email to firstname.lastname@example.org.
- Feel free to post a link to your video and your comments as a followup to this message. This is encouraged, but a video of one's personal computer is, well, personal, and I want to protect your privacy as well.
If you want to user the usertesting.org tools (this is the quickest/easiest route if you don't already have screencasting software), follow these handy instructions from Jorm:
- Go to our usertesting.com launch page.
- Click the "start" link (it's the only link available)
- Do not close this tab or window. It's okay to background it, but don't close it.
- The Java applet may require rights to run. Grant the rights as needed.
- A few moments later, a "box" interface will appear. Resize it to contain your entire work area (either your entire desktop or all the tools you use at once)
- Press the red button in the lower left of the interface to begin recording.
- A countdown will start. Once the countdown has finished, the recording has begun.
- Speak aloud your thoughts as you work. This is very important.
- When you are finished, click the "Done" button (down near where the start button was)
- The browser window (that you didn't close earlier) will then refresh with a screen cap from your session and ask you to upload the file.
- Click "Upload this file". The upload will begin. This may take a while.
- You'll be presented a list of screencasts. The filenames will be datestamps. Please note which one is yours and list it below, if possible.
If you have your own screencasting software, feel free to use it, and send your video and comments to email@example.com. If you can't get the usertesting.com stuff to work (it's still quite beta) or you just want to get some screencasting software anyway, here's a list of tools, and here's another one.
Once we've received what seems like a representative sample, I'll compile all videos, we'll look them over, and we'll let you all know what we've learned.
Thanks in advance for your help. This will be invaluable information for making a tool that's actually useful to existing new page patrollers.Raindrift 20:59, 20 September 2011 (UTC)
There's also a great deal of material WikiProject Screencast, though it focuses on creating how-to screencasts for general consumption, not screencasts for user experience research. Do we want to make a MediaWiki-focused general documentation page for this here, including materials on usertesting.com?
Would closed captioning be an acceptable replacement to using a voice?
In my very honest opinion, I believe requesting screencasts of new page patrolling is a demonstration of lack of WMF faith, recognition, and confidence in the stats based evidence that experienced users and admins have taken a year to research and produce through sheer dedication, determination, and long hours at the unappetising task of new page patrol. Add to that the WMF accusations that those researchers are a bunch of diehard deletionists. Adequate proof that NPP is a broken process manned mainly by a transient pool of raw beginners has been provided - now is the time to look for a solution that directly addresses the problem - nifty tools that streamline the work of those who already know what they are doing are most welcome, but they can be developed later.
Hey Kudpung: I'm sorry if you feel like the screencasting is about doubting your hard work on the quantitative statistics side. It's truly not. The request got put out because those working on engineering at the WMF, like Raindrift, are very interested in helping solve these problems, but they aren't New Page Patrollers. The point of asking for a screencast is to have you lend us a little of your hands-on experience in patrolling, which is what we lack. It's certainly not going to tell us the hard numbers which I think you're referring to, or refute anything of that kind.
Steve, I do understand, and I don't doubt for a moment that it is well intended. It may not be a demonstration of lack of confidence in the work that Snottywong, Blade, and I and others have already researched since October last year, but it certainly does appear that you are not satisfied with that feedback, and that this NPP Zoom project is still lacking focus on critical issues. I still feel therefore this request for screencasts will only serve to duplicate, rather than substantiate, the information that has already been made available to you and your team in the form of adequate tables and stats by mature and dedicated users - who are not , BTW, a bunch of dedicated exclusionists. Snottywong is an expert at providing data and has already provided plenty of scripts that were to be used to extract data for the ACTRIAL. If you require more tables and up-to-date data runs, I am sure that if he is asked nicely he would be only too pleased to oblige - he is after all, like me and Blade, deeply concerned with these NPP and new page creation issues.
As far as the request for screencasts is concerned, the wannabe Wikipedians who are patrolling new pages supercficially at the speed of light will not want to slow down in order to address such a request, while we, the experienced patrollers, are also content builders and admins, and won't necessarily have the time or inclination to engage on a mission so complex. It requires downloading special software - which I am not even aware of being available for my computer platfiorm - fiddling with it until it works, then uploading the files to some destination that you will no doubt create.
Because of the difference between the work of newbies (that will most probably never be made available as screencasts anyway) and screencasts of the more professional patrolling done by established editors and admins, I doubt that the screencasts will ever provide you with useful new information. I for example do most of my patrolling one one computer screen that has a dedicated Wikipedia window for NPP, while flitting back and forth from other Wikipedia tasks such as adding content, deleting pages, following up SPI, and answering new messages on my talk page.
Thus with all due respect, I would strongly suggest that for the time it would take for you to review the content of several three-hour screencasts, you may prefer to consider doing some new page patrolling yourself in that time, and gain some first-hand empirical experience. If other members of your WMF team, such as Raindrift (best if those who design cars have actually driven one), and perhaps those who heavily criticised our efforts at ACTRIAL without even reading the background to the trial, would also do the same, you would have some first-hand notes to compare.
You may also wish to review these slightly older, but equally important statistics prepared by BlanchardB: w:en:User:Blanchardb/CSD_statistics BlanchardB that provide a similar experience. The actual situation today is however far worse.
I've spent aa few hours over the weekend looking more closely into this screencast idea, and I'm beginning to see where it could indeed possibly help the developers understand more about the NPP process. Unfortunately, in spite of all my web design work, I've never needed to make any screencasts and I don't have the software to do it. None of Steve's suggestions work, and the only available solutions for my platform cost money. Perhaps the WMF could buy me a licence for one ;)
VLC Media Player can record stuff off the desktop (select File->Open Capture Device and select "desktop" for the capture mode), but you will need to edit the video afterward to add the audio track.
Unfortunately that didn't work either. FWIW I use Mac OS 10.6.8.
Hey, Kudpung. Thanks for taking the time to try recording a screencast for us. It'll be really helpful!
It just happens that I, also, run OSX 10.6.8. The usertesting.org screencast software works okay for me. It's rather beta, but I did manage to record a test with it. If it works for you, that's probably the most expedient way.
If you prefer to use a piece of desktop software, Telestream ScreenFlow is probably the best option. The demo is fully functional, but will leave a watermark on your exported video. That's fine, though. I'm sure we can ignore it. :)
If you use ScreenFlow: It's not obvious now to stop recording if you haven't set a key combo in the preferences. If you switch back to the ScreenFlow application and hit 'File->New Recording', it'll ask if you want to stop the current one instead. When exporting, I recommend using the "Web - Low (multi-pass)", and scaling to the smallest size that maintains readability. 75% worked for me, but it'll depend on the size of your fonts. That produced ~500kbit video, which is small enough that 15 minutes of it could still be emailed.
As you might have surmised, we don't need the screencasts to understand the nature of the issue. The data you and others have collected are quite clear in that regard. Rather, we'd like to use them in designing a solution. I'm imagining a toolset that makes newbie page patrolling easier and more accurate, and provides a way for experienced patrollers to quality-check and mentor the newbies in a way that encourages them to learn, but also to disable patrolling for the ones who just can't get it. A whole host of very useful features depend on getting (mostly) everyone together in one place, which means we have to try to meet everyone's needs.
We're not terribly concerned with the workflow of newbie NPPers—we need to understand how they work to some degree, but it seems education is a more important goal here than efficiency. Also, I can become a newbie NPPer in short order, so that research is easy to do locally. Serious experience, on the other hand, is hard to get. It seems like each experienced NPPer has built their own UI to some degree, or at least uses Mediawiki very differently. To capture some of that functionality in the code we write, we need to see how you all work.
Hi Raindrift, thanks for all those tips. I'll try some of them out when I get home this evening. Please search around this confusing talk page - you'll find a thread somewhere I posted today with more detail on what I've got lined up for a screencast if I can get the software to work.
I'm still struggling with challenge of royalty-free screencast software, but in the meantime, i'd like to expand on one or two points you made. The serious NPPer probably hasn't built him/herself a sophisticated interface - they are not all geeks - they mainly just limit themselves to Twinkle - which is perfectly adequate, is quick, and takes up no screen space. The less experienced users often don't even know what Twinkle is - which ironically is probably not a bad thing, except that users do not get notified of CSD tags. I first found out what Twinkle is when (what seems) years ago, when I stumbled on a user page that had a "I patrol pages with Twinkle" userbox on it and I wondered what Twinkle was. I probably have one of the best possible working environments for optimal Wikipedia workflow, with several Mac Minis, a MacBookPro, all with large screens in a row and all sharing, and all operated from one keyboard and one magic track pad.
You say that you can become a newbie NPPer in short order, so that research is easy to do locally. Yes, and if you already have a solid knowledge of policy, page creation, and standard editing tools already, and spent two hours reading the instructions listed at WP:NPP, WP:DP, and WP:CSD and consigned most of the notability acronyms and around 30 CSD criteria to memoire, in less than an hour you'll be as proficient at NPP as our best patrollers. BUT, you are an adult, have an analytical mind, and probably a PhD - any page patrolling you do will not put you in the shoes of a 12 - 13 year old. However, you might wish to count how many pages you treat, doing all the steps required at WP:NPP, in one hour's solid patrolling without getting distracted, but you will be if you also have en.Wiki admin tools. What slows me down is that I do physical deletions too, ans sometimes salt pages and block the users when I'm patrolling. All this can dramatically reduce the actual number of pages patrolled. A 15 minute screencast may show me getting through 15 new pages or as few as three. There is also the fact that I cherry pick: from the front of the queue the page titles that I know are going to be a challenge for the average patroller, and of course pages with any spammy sounding names, and bios. At the back of the queue, I usually go for the bios first, then the ages with Indian sounding names. To save time, I go through about 10 - 20 picked pages on the list and let them start loading in separate browser tabs while I work. I have monitors and widows open with my 'tool box' user sub pages.
I should point out that I do a lot of work in the lesser namespaces, and quite often find material that is legally problematic and therefore needs to be oversighted. As such, I'm not sure it'd be a good idea to record my discovery thereof.
This... is a very good point.
Would it be possible for you to do your recordings in a namespace that is less likely to run into this type of stuff?
Perhaps you could share your page patrolling session in real time through screen sharing on a video conference with maybe Steve and Ian? I think the major issues are more about establishing how many pages you are able to fully examine during your patrol session, what percentage of new pages you empirically find to be really in need of rapid deletion, what pages are passed as 'OK' patrolled that should'nt be , and the frequency with which you have to correct a patroller's tagging. Also as an admin, roughly the percentage of new pages you summarily delete immediately.
Yeah, we'd be happy to do a screenshare any time.
I think we need to set up a screenshare session. I had another go with test.com. this morning. It took 29 minutes for a 10 minute session to upload, and when I got the result back after waiting another 14 minutes, it started and then just hung.
Okay, email sent about scheduling a time...
I've done a test run with Blade about screensharing andit worked perfectly. I'm nevertheless still looking into solutions for making screencasts on Mac. it looks as if the free trial of SnapzPro might be worth looking at.
Thanks for busting this out! Some initial fly-by-the-seat-of-my-pants thoughts and interface nitpicking:
- I understand the idea of "zooming in" is the reason for the previous and next buttons being at the bottom or top, but the previous article button plus the large amount of space given to Article and User areas visually takes up a lot when I first scanned the page. I had a hard time parsing all that at first glance, and thus got "stuck" a bit before moving on to the detailed patrolling actions. Perhaps removing the metadata and just having the buttons would make it less distracting?
- Considering that the stats available suggest people mostly patrol from the front of the queue despite the fact that the documentation strongly suggests you should patrol from the back, I think making the oldest patrolled articles the default should be a priority, though we should include the option to see the most recent ones. The current interface encourages one to patrol articles that may be only minutes or hours old.
- Why are we showing edit section buttons but no main edit button the whole article? It should probably be all or nothing.
- The deletion section should actually say "Nominate for deletion". The language makes a difference especially if we're not restricting the tool to experienced users.
- I'd suggest "and..." instead of "But also". The caps on both lead-in instructions are a lot as well.
- Not that I think it's a bad choice, but what was the reasoning behind making the patrolling and deletion actions menu on the left? I feel like we're used to seeing article content start from the left after the sidebar, and I'm supposed to/naturally want to start reading from the left before my eye moves to the right to take action on the content I've just read.
- Would things like categorizing and page moves happen inside the tool? Or do the arrows indicate I would leave to do so?
Thanks again Brandon
Comments, in order:
- Having additional metadata was a strong request from everyone I talked to. Reading over all the NPP pages and how-tos, it seemed also that this was super-desirable. Investigating into the third party tools also showed that this was something that we wanted.
- Yes, I want to start from the back of the queue. The document itself should outline that process better; I'm working on it.
- A very good point. That screen there is really, and truly, a screen cap cut and paste. We'll have to massage it.
- Noted. Fixed in my copy.
- Noted. Fixed in my copy.
- We have most of our actionable "chrome" on the left. I'm not married either way. This interface is very clunky right now and not what I'm actually envisioning in the end.
- Yes, they would. It's not obvious (yet) but those actions will cause flyouts to happen where you add categories and/or rename the page. The arrows indicate that there are sub-menus.
One reason for focussing on the front of the queue is to quickly get rid of the attack pages. Currently this is one of the things NPP does very well, I'm more critical than most about the quality of tagging and the number of incorrect deletions, but I scarcely ever see an incorrect G10 tag. I occasionally see articles tagged as A7 or BLPprod that should be G10 and would have been deleted more quickly if they had been. But G10s at the back of the queue or in mainspace are thankfully rare.
Deleting G10s ASAP is important - some will be being used for cyber-bullying and if so we want them gone before everyone in the school has had the text linking them to the new Wikipedia article about one of the girls in the school being a pornstar or hooker. Shifting our focus from the front to the back of the queue can only be done if we first find an effective way to screen the G10s and ideally some of the G3s out.
A large proportion of the articles that get to the back of the queue are Goodfaith but borderline notability, somewhat spammy or on obscure subjects, the ones that are easy calls either to patrol or to delete are usually dealt with upfront. I'm pretty sure that most of the unpatrolled articles are actually looked at by a patroller at the front of the queue, otherwise we'd see more G10s at the back. So I'd be very uncomfortable encouraging some of our newest patrollers to shift to the back of the queue.
It's true that we do need to continue dealing with attack pages quickly, but considering that they're only about 5% of CSD (while A7 and G11 together are about 45%) that's a compelling reason for any new tool to obey the current instructions at WP:NPP and encourage people to start at the back of queue, though naturally there should be an option to switch to the front.
Personally I wouldn't lose sleep if we made a change that meant a thousand extra articles at any one time on Myspace bands who will have their first rehearsal next week if they can find a drummer, and bright new upcoming footballers who haven't actually played their first professional game. I really can't see that causing Jeremy Paxman to want to interview someone about that on Newsnight or the Information Commissioner summoning in the chair of Wikimedia UK for a meeting about the precise relationship between the chapter and the Foundation. But a change that meant up to a thousand extra attack pages on the site at any one time - I'd hate to be the person who had to justify that to the press. If we did it knowingly and deliberately I'd be embarrassed to be associated with the project.
The back of the queue often bumps up to the thirty day mark, though with most articles patrolled or deleted in their first few minutes the actual queue is rarely more than 10,000 or so unpatrolled articles. If you could shift the emphasis to the back of the queue and process as many articles a day then in theory you would have an extra 500 attack pages on the site as a result of this change. But the length of the queue oscillates widely as it depends on volunteer numbers - so sometimes it would be much more.
The 5% of new articles that are attack pages currently get deleted so quickly that most people take a while at NPP to realise how common attack pages are, and lots of people who haven't spent time at NPP or cat Speedy would be quite shocked to learn its as high as 5%.
There is a danger, or even an irresponsibility in say only 5%. Let's say that most attack pages get deleted fairly quickly. The solution requested by the community at WP:ACTRIAL was not conjured up by a group of exclusionists as was suggested by the WMF at Bugzilla, but from far deeper concerns such as for example, that Sod's Law is that the one attack page that doesn't get quickly deleted is the one that could involve the Foundation in a litigation for millions of dollars. Since the deal was struck with Google to reference Wikipedia pages at the speed of light, this is not good. It's a risk we should neither be taking with donated funds nor with the inconsitency of amature patrolling at such a crucial stage of page creation. It's my guess that most patrollers are fascinated by the live feed in the side bar, and when those ten entries are lost from view, the unpatrolled pages rely on people working directly from special:new pages which as far as I understand has to be constantly manually refreshed.
The immediate problem therefore is not the number of patrollers who are available for work, but those who are totally incompetent to be doing it. Unfortunately there are no bots that can count the number of times I have deleted attack pages that were labelled A7 because the patrollers were not interested in taking time to read past the first sentence of the often long and carefully crafted pieces of libel.
Let's also not forget that there are, or have been times, that the 30 day period has been greatly exceeded, hence the creation of the Snotbot last fall as a desperate measure to get at least something done to track the unpatrolled pages.
These are all reasons why it is most important to listen to what the the mature editors and admins have to say who have had long and involved experience at NPP specifically to research its weaknesses.
Working from the back is indeed good advice for actually clearing out the backlog, but it shouldn't be forced on everyone. The wiki benefits from the fact that the advice is not always followed. The front of the queue always contains things that clearly shouldn't be on the wiki. Even if someone intends to work from the back, taking a short look at the front first can be greatly beneficial in keeping out attack pages and other wholly inappropriate content.
Of course, it seems that the real underlying problem is that we put attack pages which should truly be speedily deleted into the same queue as notability problems or promotional articles at all. We should also be figuring out a way to put the egregious, obvious CSD stuff at the front-and-center of people's attention immediately, but leave time for quality patrolling and editing of articles for things that deserve.
Agree. Patrolling for new attack pages should almost fall under vandalism patrolling and recent changes patrolling. It's almost as if there should be two levels of patrolling, one which is done as soon as possible after the article is created just to check that it's not a "dangerous" article (i.e. not an attack page or a copyvio, or otherwise worth of speedy deletion), and then a second level of patrolling to identify issues with articles that actually have a chance of sticking around. Noob patrollers should be more than capable of the first level of patrolling, but we don't want them to hit the "mark this page as patrolled" button when they're done, because they've only checked maybe 10% of the things that should be checked.
Two-tier patrolling is not a bad idea for speed, but who is going to do this checking as soon as possible after the article is created? It still does not address the problem that noobs far too often can't tell the difference between G10 attack, G3 hoax, G1 nonsense, A1 no context, and A3 no content.
I was going to say, this is assuming most NPPers can actually tell the difference. I watch CAT:CSD and frequently check the articles tagged G1; almost every single time, they're either G3 or G10. I frequently have to tell users to slow down so they can use the right criterion, because many of them try to go faster than they can (I move very quickly, but that's because my reading speed is in the top quarter of the 99th percentile; most people cannot read like that, and make bad mistakes when they try to move too fast). In short, there are only a very few NPPers I'd trust to accurately make the distinctions, and they're a small enough percentage that far too many pages run the risk of being miscategorized.
Right, so that confirms that we're basically back to square one with this sugestion of two-tier patrolling, i.e. patrolling the patrollers.
For an example everyone can view, see the history of The Mad Russian; tagged G1 but a clear G3 (I happened to know a good place to redirect it, but the edit history is there).
Blade, I'm not sure io this message of yours is complete. Should there be some links to something?
Yeah, fixed; not sure what happened. Just type it in on en.wiki, as I can't seem to link things from here.
[17:25] SigmaWP Ironholds: Will CSDing a page with your proposed NPP interface automatically notify the author? [17:25] SigmaWP Oops tabfail [17:25] SigmaWP jorm: ^ <irrelevant material /> [17:25] jorm I don't see why it can't, SigmaWP. we have the information. [17:26] SigmaWP Excellent! [17:26] SigmaWP This will make TW obsolete! [17:26] jorm Once we're using native code, everything is doable. [17:26] jorm Hey, can you add that as a comment on the talk page? [17:26] SigmaWP Right
Modifying Page Info on disambiguation pages
It's usually a good thing that Page Info lists "No citations" under "Possible issues" – but not in the case of disambiguation pages, which are explicitly not supposed to have citations. Could the Page Curation toolbar be modified to prevent this notice from appearing on any page containing a disambiguation tag (of which there are several, including aliases)?
A button for returning the new page feed page.
It will be nice to have such button so curators can go back to the full list if they decided to skip some of the new pages.
Nomenclature and other stuff
Nomenclature and other stuff
- Anyone who thinks "Triage" is a non-military term should read about its history.
- This talk page interface is pretty unusable - I am using what I consider a massive screen, and I can only see the contents.
- Does this exist or is it just mock ups?
1) True, "triage" as a system came from military hospitals, but its name has grown to be more general since then and simply means "sorting by urgency". Certainly "patrol" sounds like it's an obligation to walk the perimeter fence of the camp to make sure nothing gets in (which is precisely the feeling that all this focus on NPP is trying to avoid). 2) It's unfinished software. The newer, all-singing all-dancing, version is in the works LiquidThreads 3.0 3) As it says in the first line: "This document is a work in progress. Comments are appreciated but this is not a final draft."
Are there any plans to deploy the page curation toolbar on other language versions (by default, or on request)?
In lieu of a plan to do this kind of deployment by default, any community that wants a new extension like this should request it on Bugzilla.
Will this be released as a Mediawiki extension?
Will this be released as a Mediawiki extension? This is really a neat idea. I could see my organization using this tool.
I'd like to try to write some patches for this to fix small bugs I've encountered. Any chance I could at least see the source code? Thanks!
I'm certain that it's visible at the git repository. Do you have git access?
...I don't know. (Bear with me, I'm new to git) Someone at enwiki did give me a link to this gerrit repository, though. Is that right?
Yes, it does appear to be that repository. This is not properly documented. I spend 10 mins looking for it, and I'm an experienced dev that knows where to look.
Nathan, I'm sorry that I did not run across this earlier. Yes, I believe https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/extensions/PageTriage.git;a=summary is right. And if you could point to the page on enwiki where someone mentioned this to you, I'd be much obliged -- that way I can communicate with them as well. Thanks!
This might help those of us unfamiliar with git:
I found Patrick's tutorial very good. If you do get to the point of submitting the change to gerrit, can you add "tychay" to the "reviewer" list? I'd like to see if I can expedite reviewing the change for submission. :-)