Talk:Article Creation Workflow/Design

Instructional text
This page looks good :-) You're absolutely right, of course, that we offer far too much instructional text. Users who are task-focused perceive the text as an impediment -- e.g., just this past weekend, I tried to test out using the Article Creation Wizard, and even though its text is relatively tight, I still found myself skipping past most of it. The trouble with our existing instructional text is that we try to accommodate all possible use cases [1], which makes the text really, really long. That backfires because users then skip all of it (like I did with the ACW). It'd be much more effective for us to keep instructional text super-succinct --covering just the most common use cases--, because that way, at least some people would get some information that could help them, rather than nobody getting any. Sue Gardner 01:59, 14 September 2011 (UTC)

[1] a laudable goal in software development, but maybe less of a laudable goal for instructional text :-)

MoodBar comments
Does anyone reading the MoodBar comments think that the articles created by these individuals would be kept on Wikipedia? I understand and appreciate the desire to make article creation less horrible, but it comes with a cost. Who's going to pay? --MZMcBride 04:11, 14 September 2011 (UTC)


 * Thanks for reading through the page and for the comments MZ. In my view, the two changes proposed here are:


 * 1) Adding clear instructional material and warnings through one or two interstitial pages that appear before the edit window when you're hitting a redlink
 * 2) Possibly providing a noindexed space outside the main namespace to create articles and strongly encouraging people to work there first
 * Neither of those changes seem like they will make anyone "pay". Rather, we'd like to tell newbies much more clearly upfront what the expectations are, and then ask them to start articles in a place other than the mainspace where both the article and their skills have time to develop. Even without the second step, what we want to do is reduce the costs placed on the community by making it clearer what we do and do not want in Wikipedia. Steven Walling (WMF) &bull; talk   06:50, 14 September 2011 (UTC)
 * Number two seems uncannily similar to AfC. Killiondude 07:38, 14 September 2011 (UTC)
 * Hey Killion. You're right, it is pretty similar, but if you get a chance to read through "Future Phases" section there's more detail there about the ways it's different (as currently proposed anyway). For starters, it would not necessarily require review in order to publish, because it's aimed at registered users rather than anonymous ones. If you want to pick apart that description and leave us some feedback, it'd be a big help. Steven Walling (WMF) &bull; talk   17:12, 14 September 2011 (UTC)
 * The only difference between the proposed workspace idea and AfC is that no review is required (or even encouraged, by the looks of it) by an experienced editor. This means that, while your warnings might occasionally convince a new editor to start their articles in the workspace, there is nothing stopping them from immediately moving it from the workspace to mainspace when they realize their article is not yet "live".  Therefore, we're just adding a superfluous step in the article creation process which won't stop anyone from creating an inappropriate article in mainspace, and won't provide for any type of review process by an editor who is familiar with WP policies.  A better process would be to require new users to go through AfC (or a similar AfC-like process) when they want to create a new article, until they have been properly educated on what is expected in a new article.  Thus, en:WP:ACTRIAL.  Snottywong 18:17, 14 September 2011 (UTC)
 * I think I might let Brandon reply about the particulars of the workspace design and how it might be adapted, but I do want to make a general point: when it comes to this or other WMF Projects, nothing has to be taken 100% as-is with an all-or-nothing decision. It's not substantially happening yet, but we'd like to live in a world where there is a lot more of this exact thing going on: experienced editors showing up very very early in the design process and giving us specific feedback about the potential process and policy implications of new features. Steven Walling (WMF) &bull; talk   18:32, 14 September 2011 (UTC)
 * The description of the future phases and how they work is deliberately vague in certain areas, especially regarding how policies work out. For example, it could easily be set up that Anonymous IP editors can be allowed to create articles in the Workspace but not publish to the Mainspace - this allows us to have articles created by IP users but not allow them to pollute the Main space.  That's only one policy change that could occur.  Another idea could include "not automatically deleting poorly-written articles but instead unpublishing and then tagging them" so that work isn't lost.--Jorm (WMF) 18:36, 14 September 2011 (UTC)
 * Yeah, where they can collect dust just like everything else that needs maintenance. As evidenced by the numerous backlogs on such essential problems as copyright and fair-use issues, we can't handle all of it at the rate it's coming in.  At the risk of being a dick, I'd urge you to click on the link Snottywong provided you above and give input there. The Blade of the Northern Lights  (話して下さい ) 22:01, 14 September 2011 (UTC)

'Specific feedback' has already been provided, but Brandon and Erik Moeler, as evidenced by gheir comments at Bugzilla, apparently do not wish to entertain it. It should not be a foregone conclusion that  WMF office bods are more competent  than the volunteers who  do  the grunt  work, and if Steve wants experienced editors to  show up very very early in the design process and provide  you (the WMF) with specific feedback about the potential process and policy implications of new features, then a more civil  and friendly  approach should be considered by those who dismiss the good proposals  for solutions  that have already been made by  groups of highly competent editors who have hands-on, first-hand experience of the problems. All these vague suggestions above for 'new' solutions are so similar in other guises  to  the en:WP:ACTRIAL that  was reached by  an overwhelming  consensus of the community  at  en.Wiki, that  the original  trial  might  just  as well  be allowed to  take place - it  has many  safeguards that  are not  even being  mentioned here. A solution is required now for the en.Wiki, not in  two  or three years time -  which  is what  will  happen  if this discussion on  vague alternatives is allowed to  continue. Note that the goal  of  Snottywong  and Blade with  their ACTRIAL was never  to  delete poorly  written  articles, and is not  the product  of a cabal of  exclusionists as was suggested at  Bugzilla; it  was simply intended to  prevent  the creation  of thousands of  uncontroversially  useless new  pages, and at  the same time offer three parallel avenues for fast-tracking  serious articles to  live mainspace -  al  with  relatively  uncomplicated and minor changes to  the site software and user interface. The current decline in   potentially  serious new editors is due to  their being  driven away  by  the daunting  text-walls of dozens of pages of policy, guidelines, and other instructions, while we are leaving  the door open  to a deluge of spam, copyvio, vandal, test, attack, and nonsense pages that  even a squadron of mature, experienced patrollers would not  be able to cope with  on  a daily  basis. NPP is a magnet for the least  experienced and least  mature of users, and is therefore a broken system, and careful long-term  research by  en.Wiki editors (and not  by  WMF projects such  as the Summer of Reserch), has already offered more than sufficient  evidence of it. --Kudpung 20:14, 18 September 2011 (UTC)

Lovely Root System - can we make it grow?
This idea has such a lot of useful ways in which it could grow. I'd personally like it if new accounts had a sandbox area automatically created for them, with links to a really good set of interactive tutorials covering the basics of what newbies need to know; screenshots with annotations for "how to do this", which newbies can read in conjunction with playing in their sandbox; a kind of "test yourself" auto-marked interactive quiz thing for each basic concept, a heap of stuff like that so that newbies can really get into practising "getting it right" from a very early stage. A links to "pick a mentor" (somewhere really easy to see, so they're likely to hit it, perhaps) would be a good idea. But really excellently-structured interactive video tutorials would be such a good thing to look into, from this start. One of the options in the article workflow "I don't feel confident yet - I want to do the tutorial" as one of the options. Probably the top one! ThatPeskyCommoner 08:02, 14 September 2011 (UTC)

Are we really examining the entire workflow here?
I realize that this change is happening now for reasons aside from new page patrol workload. That seems to be what's driving its implementation at the moment, though, so in the interest of brevity I'm going to address it primarily from that perspective.

I'd like to start with a brief anecdote: I had a discussion with Neilk recently about the community's response to UploadWizard. I was curious why so many people were opposed to what is an obvious, major improvement in usability for contributors. His perspective, in summary, is that we only solved part of the problem. While the old interface was confusing for new contributors, that wasn't an issue for the established community. The community had a problem with categorization and sorting. We did little to solve that problem. Instead we made it worse, from the perspective of the people in the trenches, by inviting a torrent of new contribution.

I think there's a lot to be learned from that. One lesson might be that improving UI is bad, and the technical barriers we have in place are beneficial because they keep out the riff raff. I think that's the wrong lesson. This entire movement is built on people feeling empowered to contribute.

Instead, I think we need to be more mindful of the entire, end-to-end workflow for new content creation. Clearly at this point, there's a mandatory review of all new pages (via NPP). Our current approach sees improvements to that step as out-of-scope. I really like the current proposal, but I don't think we're going to get broad support from page patrollers until we include them in it.

Furthermore, addressing only one part of the problem carries additional risk. For example: we hope that educating people about Wikipedia policies as part of the process will help reduce the overall workload on page patrollers. Currently, about 30% of all new pages are deleted. Let's assume we're successful, and our education efforts reduce that number to 18%, but our usability improvements double the number of new pages created. In a sense, this outcome would be wonderful, but it would actually result in an increased NPP workload.

I tried doing some NPP recently. I set up Twinkle, read the documentation, and checked out Special:NewPages. While I didn't mark any pages (I don't feel that I'm personally experienced enough), I saw many areas where the process could be improved with WMF support. I'm sure the experienced new page patrollers know way better than I do. The central issues, as I see them:


 * The documentation is long and confusing, which I bet keeps people from learning NPP
 * While there's a clear workflow, the available tools don't guide the patroller through that workflow
 * The existing process is inefficient (requires many clicks)
 * The existing process requires the patroller to keep a lot of minutae in their head
 * There does not appear to be automated infrastructure for recruiting new patrollers

Data shows that workload per new page patroller is actually decreasing, and has been for some time. However, it's clear that there's a perception that workload is increasing, or that the work itself is terrible. The source of this perception is a mystery to me, but I suspect it may stem from a sense of frustration and disempowerment that comes from years facing the same endless torrent of spam and vandalism. Perhaps if we were to work with these people to improve their tools and actively recruit new members to their ranks, the sense of futility would wane.

Of course, we can't do all things at once, and I'm not proposing that. Rather, I think we should broaden our scope when making plans, and balance improvements that create an influx of new users with features that make dealing with that influx easier. The most important thing is that we consider the entire process, and make it clear that we're doing so to the people who are affected.

tl;dr

Our current plan to address the NPP workload:


 * Improve the quality of new articles by effectively educating new users.

This proposal:


 * Improve the quality of new articles by effectively educating new users.
 * Improve the efficiency of the new page patrol workflow
 * Recruit additional new page patrollers by building tools for that purpose, and make software that helps to train them more effectively
 * Get the new page patrollers whatever else they need to work effectively

Raindrift 01:23, 15 September 2011 (UTC)
 * This is the last comment I'll make here, since I'm sick of being patronized. It's not addressing the problem of an influx of new users because people don't read what we give them.  Someone who wants to write about The Sound of Perseverance (band) (see WP:GARAGE) will do it, and someone who wants to write "Let's expose all burakus for what they are!!!!" and link to a website with a list of all the burakumin in Nagano Prefecture doesn't care what we want them to do, they'll do it.  This will enable them to do it and keep it around (and for those of you who think this is trivial, this is illegal in Japan, and the wrong person seeing it could lead to thousands of lives being destroyed; you're very lucky I caught it).  I could point out some other problems, but I want to see if people here can really solve this problem being so far removed from actual New Page Patrolling on en.wiki. The Blade of the Northern Lights  (話して下さい ) 02:59, 15 September 2011 (UTC)
 * In my opinion, Raindrift's proposal is the first step in the right direction. Any proposal of this scope needs to equally take into consideration both the needs of new users and the needs of experienced editors who will largely be managing the new users' contributions.  You may have followed the autoconfirmed trial debacle.  Our long-term plan was to first implement a trial restricting article creation to autoconfirmed users, and the second step was to start drafting improvements to NPP.  In any case, I've patrolled pages for awhile &mdash; not nearly as much as some others like Blade of the Northern Lights or Kudpung, but enough to know what's wrong with the system.  At the very least, I can offer my personal answers to some of your questions.
 * In particular, you mention that workload per patroller is decreasing while the perception is that it is increasing. I think I might be able to shed some light on this one.  Patrolling an article encompasses a wide spectrum of potential tasks, and which tasks are performed depends on the patroller.  For instance, on one end of the spectrum, many patrollers will simply check to make sure that the article is not overt vandalism, and mark it as patrolled if it passes this simple test.  On the other (ideal) end of the spectrum, a patroller will check everything on this checklist, and actually fix various problems with the article before moving on to the next one.  This side of the spectrum requires not only more time, but more experience from the patroller.
 * Anyway, the point of the story is that workload per patroller is decreasing because patrollers are doing less work on each article. The reason they are doing this is because of the constant pressure from knowing that if the Special:Newpages queue reaches 30 days, then articles are going to fall off the list and never get patrolled.  So, they feel like they are stuck between two bad choices: maximize their patrolling speed by skipping steps, or allow the queue to overflow.  The problems are relatively simple:
 * There are too many articles for not enough patrollers.
 * NPP is often seen by new users as a way to quickly increase your edit count, therefore patrollers are often inexperienced users who aren't equipped to recognize (no less fix) many problems with articles.
 * How do we solve this problem? Many of your suggestions will certainly help.  Better patrolling tools, better training for patrollers.  One idea that we've been throwing around is requiring patrollers to have a user right in order to mark a page as patrolled.  This will prevent very new, inexperienced users from marking inappropriate articles as patrolled.  It will also allow us to revoke the user right if we determine that a user is doing a particularly poor job at patrolling.  Another frequently requested feature is to make the "Mark this page as patrolled" link appear on any unpatrolled article, regardless of whether you accessed it via Special:NewPages or not.  This would allow patrollers to patrol even when they're doing other tasks and they happen to come upon a new, unpatrolled article.  My personal opinion is that we should combine both ideas and make the "Mark this page as patrolled" link much larger and more visible for users with the aforementioned user right.
 * NPP is definitely a broken process, and therefore I'm sure other patrollers have many more ideas than I do. You may want to consider posting a notice to this discussion at en:WT:NPP.  Snottywong 03:32, 15 September 2011 (UTC)
 * First of all, I should mention to everyone that there's been some discussion of how NPP works now and its issues over at Patroller Workload. It's worth reading that discussion.


 * To add to the list of issues, it seems there's a concurrency problem when multiple NPPers are working at the same end of the queue. Once there's more than 3-5 simultaneously, they start generating edit conflicts with one another.  I'm concerned that more patrollers on the current interface would make that worse.


 * Snottywong, thanks for your input. It sounds like the process itself actually involves many people with varying levels of skill, but the only differentiation in the task is which end of the queue one examines.  I wonder if we could start improving the tools by storing more granular information about which aspects of an article have been reviewed.  This way, we could promote greater specialization (doing NPP now requires a lot of knowledge), and the tool itself could be used to teach new skills.


 * For example, imagine a tool that shows one new article at a time, choosing it based on a set of criteria (age, what's been checked, who the patroller is, etc). NPPers could start out patrolling for vandalism and spam, which are easy to recognize.  After patrolling 25 articles for those criteria, the tool explains how to look for a good title and how to move an article (and so on).  This way, new patrollers are gently led along the learning curve while accomplishing useful work at the same time.  At some point, they could start only reviewing articles that had been vetted as not-vandalism and not-spam, on the assumption that new patrollers were handling those.


 * For very new patrollers, it might make sense to show their edits to someone more experienced to see if they agree.


 * Tracking the specific review steps would allow us to expose which steps an article had passed. This is handy for finding articles to patrol that match a patroller's skillset, but it might also help to reduce newbie-biting.  Instead of telling someone, "Your article lacks references," we could instead say, "Your article is notable, has a good title, is properly categorized, nicely formatted, has great spelling, but lacks references."  That kind of positive reinforcement is very specific, which makes it feel very sincere.


 * Well designed, I think such a process could actually be less work. Ideally people are looking at these things anyway, so it'd only be an extra moment to check a box when done.  In return, we'd see a significant reduction in duplicated effort.  Also, we could build a UI that keeps all the resources necessary for effectively patrolling a page handy, which would streamline the process further.  Then again, perhaps this wouldn't help at all and there's some important aspect of the process I just don't get yet.  I'm glad we're discussing it, though.


 * Secondly, it seems that a very small number of people do most of the page patrolling, and especially the more skilled (end of queue) portion. That's a pretty hefty burden for a small number of people.  Training new patrollers will help with that, but we also need to be able to call on them when they're needed.  I'm thinking a simple system that monitors the age of the queue, and starts notifying known NPPers when there are many unreviewed articles that match their skillset.


 * Lastly, the 30-day deadline for NPP seems like it might be useful for setting targets and motivating work, but could also be getting in the way and leading to necessary patrol not happening. With this hypothetical system, we could apply a more nuanced view of how long is reasonable for patrol to take.  For example, the deadline for spam-checking should probably be a day or less, but the deadline for reference checks could perhaps be more than 30 days without ill effect.  When a task runs past the deadline, automatically deciding to skip it is probably the wrong course of action.  The point here not that we should implement any specific change, but that it's worth examining whether this deadline (and how it's implemented) is serving us well, and if not, what could take its place.


 * Ideally, a tool for this sort of thing could be generalized for many types of patrols (ie. not just new page patrol), but I'm also wary of scope creep, and I realize that just because these things sound similar doesn't mean they are. I'd like to identify the areas where the process could find the greatest benefit, and address those things first. Raindrift 23:22, 15 September 2011 (UTC)


 * I'm encouraged that you're taking the time to independently discover many of the problems with NPP. You might be interested to learn that volunteers have taken it upon themselves to at least partially mitigate some of these problems.  For instance, I have created a tool which tells you how many people are working on each side of the newpages queue (see http://toolserver.org/~snottywong/cgi-bin/patrolreport.cgi).  I've also created a tool which tracks how close we've gotten to the 30-day limit (see http://toolserver.org/~snottywong/cgi-bin/patrolgraph.cgi) along with a defcon template which is updated hourly by my bot (see en:User:Snottywong/NPPdefcon).  My bot also runs a task which notices when the newpages queue overruns its 30-day limit, and keeps tracks of any unpatrolled articles which were forced off the list (see en:WP:NPP30).  Luckily, the queue hasn't been overrun for a few months, but earlier in the year there were hundreds of unpatrolled articles overflowing the queue every month.


 * Clearly, many of these problems could be solved more efficiently and effectively by changes to the mw software. Patrollers have been screaming about the 30-day limit for a long time, but have never convinced anyone to address it.


 * I think your idea about making multiple levels of patrolling is a step in the right direction, but I think I'd have to see a more detailed proposal before I could support it. My fear is that it will actually create more work, because 4 or 5 people will end up reading through the whole article as it goes through all of the different patrolling stages.  The important part is that we make patrolling a user right which must be requested and which can be taken away if the patroller consistently does a poor job.  Pair that with an intuitive, interactive education system on how to correctly patrol and I think we'll be making some progress.  Snottywong 00:02, 16 September 2011 (UTC)
 * And one last word of advice. Kudpung and I are the main presence at NPP; do not ask us for help unless forging ahead with ACTRIAL is a part of the deal. We spent a tremendous amount of time on it only to be spat in the face, and we do not want to waste our time if our ideas will be disregarded. The Blade of the Northern Lights (話して下さい ) 01:16, 16 September 2011 (UTC)
 * Snottywong: Thanks for linking to the tools you created. I'd seen some of them, but hadn't run across the patrolreport yet.  I think this work is really good, and should form part of the basis for a more integrated interface.


 * I have to say, it upsets me that you've needed to code a bot to keep track of unpatrolled articles that run off the end of the queue. It really demonstrates how Mediawiki hasn't been updated to reflect the reality of how people are using it in this area in a very long time.


 * The multiple levels of patrolling idea is just a brainstorm at this point. The underlying issue I see is that there are people with many different levels of skill, and coordinating them effectively and training them up is hard.  There are probably multiple ways to address that issue, and I'm going to ask if other people here can come up with more promising designs.  In any case, I feel like I'll need to understand the problem better and work more directly with people who do NPP regularly to craft a proposal that can actually work.  You're totally right that, implemented naively, such an interface could be worse than useless.


 * I like the idea of making patrolling a userright. I've read the discussions about how much damage a sloppy NPPer can do, though I'm still unclear on exactly how pervasive that problem is.  If anything, though, the userright would make identifying NPPers to request their help easy.  However, rather than making it request-only, I wonder if inviting people would lead to a greater number of NPPers and reduced overall workload.  I think we could come up with some criteria by which good potential patrollers could be identified (some combination of edit count and types of previous activity), and manage this part of the process automatically.  It could even be the sort of thing that is only triggered when the number of active NPPers gets below some threshold, to make sure we don't request help when it's not needed.  Conversely, triggering invitations could also be managed by people if there's no clear-cut way to do it with code.


 * The Blade of the Northern Lights: I hear you. I have about as much ability to directly implement ACTRIAL as you do, so I've intentionally avoided forming a strong opinion about it.  However, I think there are a lot of other angles from which this issue can be approached, and I'd like to focus on the ultimate goals (reduced NPP workload, higher-quality Wikipedia, more contribution in all areas from a more diverse userbase).  You have a lot of experience here, and your input, if/when you feel ready to give it, would be of value.  However, I also understand that you don't necessarily feel enough trust to do that right now.  Raindrift 19:10, 16 September 2011 (UTC)
 * greetings Raindrift,


 * Regarding the userright idea, at least initially there would probably be no need to actively recruit people to get the userright. History has shown that there is nothing more fashionable among Wikipedians than acquiring the latest new userright.  There will probably be a stampede of editors when it becomes available.  Many of them will likely not use it much, but I think that simply the act of making it a userright would initially serve as an adequate form of recruitment.  Snottywong 21:37, 16 September 2011 (UTC)


 * some thoughts on your post.


 * The documentation is long and confusing, which I bet keeps people from learning NPP


 * i found the docs to be fine. true, theres a fair bit to know but (like anything) you just focus on one part until you're comfortable then move on to the next thing.  for instance, i've been patrolling for a while but only yesterday made my first submission to afd.


 * While there's a clear workflow, the available tools don't guide the patroller through that workflow


 * not really sure what you mean here, as you say the flow is clear and i find the tools to be fine.


 * The existing process is inefficient (requires many clicks)


 * dont agree, i have no probem with the tools, just for the record i use twinkle and the "New Page Patroller" script, plus some firefox plugins like mouse gestures and tab mix plus.  i find it to be fast and efficient.


 * The existing process requires the patroller to keep a lot of minutae in their head


 * sure, but thats the nature of the beast no? and besides, as i said above when you're starting you're not going to memorise everything, you just start with a section, study G11, work out what exactly "uambiguous advertising" is before moving onto the next thing.  hmmm but maybe that approach should be suggested somewhere.


 * There does not appear to be automated infrastructure for recruiting new patrollers


 * not sure about this at all, what i see is people tryng it out and then most move on. pretty much every time i'm patrolling there are names in the patrol log i havent seen before, thats subjective tho, no data.


 * However, it's clear that there's a perception that workload is increasing, or that the work itself is terrible.


 * dont agree, i think theres a few people saying that yes, but if you look at the npp logs, most of the people doing npp are not complaining on the talk pages, at least i havent seen that. maybe i'm not looking at the right talk pages?


 * as to it being 'terrible', i think thats a hard call, it seems to me most people come to "build an encyclopedia" they're creative types, they're sociable, they like to help people out and they are very sincere and genuine in this. for them i think yes it can be a terrible job because a lot of what you do on npp can be *perceived* as quite negative and it really goes against the grain for them.


 * for myself, i enjoy it and every now and again when the torrent of crap gets too much, i take a break and that helps a LOT.


 * ok, so three things that i think would help :-


 * putting "Mark this page as patrolled" link on a page even if it you didnt get to it from the npp backlog. i suspect that just doing this would help a lot.


 * some way of filtering the npp log, so i only see unpatrolled articles that have certain words in. for example, i might know a lot about garage bands, so any articles with that phrase get put to the top of my list.  i think this would help because it would mean a npp could just focus on their area of expertise and get used to the rules surrounding that area.


 * this is a touchy one, admins ;) the docs only get you so far, you learn a lot (or at least i do) from just doing something and then paying attention to how its handled. for example, and again i have no data but...  i might tag some article with G11, it gets deleted and i think "ok, so thats valid for that corner case" then i tag a similar article G11 and BANG!  all hell breaks loose.  each admin themselves tends to be quite consistent, but amongst themselves, not so much.  this, imo, leads to confusion and may drive people away from npp cos you dont know if you're going to get slapped or not and it makes it hard to figure out the edge cases.


 * cheers -- The Elves Of Dunsimore 04:01, 16 September 2011 (UTC)
 * Good point, about how the link should always be present for patrollers, rather than just when they're doing NPP. It looks like this has been tried before, but had performance problems.  I think it could be implemented in a way that takes advantage of caching using JavaScript, so it's totally possible, and should definitely go on the list.


 * I agree it's essential that NPPers be able to filter the list of articles by some criteria, or have articles shown to them based on those criteria. That would streamline the process a lot.  It'd also help with the concurrency problem a little, though I think there are better ways to address that.  Regardless, people shouldn't waste their time searching for articles they know how to patrol.


 * It sounds like the current interface works pretty well for you. That makes a certain amount of sense, since you've chosen to do NPP.  It sounds like you're pretty fluent in how the web works, you don't mind tracking down tools to do the things you want (scripts, browser extensions, etc). Also, you seem to be pretty organized in how you learn from documentation, mastering each part in-turn rather then overwhelming yourself with information.  My comments came from approaching the subject as a new user, tempered by some experience designing UI for people who don't use computers the way I do.  I strongly suspect that the way things work is okay with a small number of people, but even little improvements could dramatically increase that number.


 * Just as an example, people seldom approach a new computing task by reading documentation first. Rather, they expect the software to either be immediately understandable, or to teach them.  We don't have any imperative to meet that expectation, but if we don't, we lose those contributors.


 * I'm trying to synthesize a number of perspectives to find the best way to make improvements, and I'm glad to have yours. It's good to hear from people for whom the current system works.


 * While it's totally not something you have to care about at all, if you have any interest in learning about how UI is designed, I recommend checking out Joel Spolsky's article series on it from way back in 2000. The examples are a little dated, but the content has stood the test of time.  It starts with Controlling Your Environment Makes You Happy.  In this specific case, the article about providing options instead of expecting people to remember things is particularly relevant. Raindrift 19:39, 16 September 2011 (UTC)

To be sure that it  does not  get  buried, I'll  repost  my message in the Moodbar comments section here, because Raindrift's comments synthesise  broader issues and do  not  address the specific problems outlined by  Blade and Snottywong that  need immediate solutions:
 * NPP is a broken process and needs a solution now

'Specific feedback' has already been provided, but Brandon and Erik Moeler, as evidenced by their comments at Bugzilla, apparently do not wish to entertain it. It should not be a foregone conclusion that  WMF office bods are more competent  than the volunteers who  do  the grunt  work, and if Steve wants experienced editors to  show up very very early in the design process and provide  you (the WMF) with specific feedback about the potential process and policy implications of new features, then a more civil  and friendly  approach should be considered by those who dismiss the good proposals  for solutions  that have already been made by  groups of highly competent editors who have hands-on, first-hand experience of the problems. All these vague suggestions above for 'new' solutions are so similar in other guises  to  the en:WP:ACTRIAL that  was reached by  an overwhelming  consensus of the community  at  en.Wiki, that  the original  trial  might  just  as well  be allowed to  take place - it  has many  safeguards that  are not  even being  mentioned here. A solution is required now for the en.Wiki, not in  two  or three years time -  which  is what  will  happen  if this discussion on  vague alternatives is allowed to  continue. Note that the goal  of  Snottywong  and Blade with  their ACTRIAL was never  to  delete poorly  written  articles, and is not  the product  of a cabal of  exclusionists as was suggested at  Bugzilla; it  was simply intended to  prevent  the creation  of thousands of  uncontroversially  useless new  pages, and at  the same time offer three parallel avenues for fast-tracking  serious articles to  live mainspace -  al  with  relatively  uncomplicated and minor changes to  the site software and user interface. The current decline in   potentially  serious new editors is due to  their being  driven away  by  the daunting  text-walls of dozens of pages of policy, guidelines, and other instructions, while we are leaving  the door open  to a deluge of spam, copyvio, vandal, test, attack, and nonsense pages that  even a squadron of mature, experienced patrollers would not  be able to cope with  on  a daily  basis. NPP is a magnet for the least  experienced and least  mature of users, and is therefore a broken system, and careful long-term  research by  en.Wiki editors (and not  by  WMF projects such  as the Summer of Reserch), has already offered more than sufficient  evidence of it. --Kudpung 20:28, 18 September 2011 (UTC)


 * Kudpung, I'm going to have to take you to task for your comments about people being "uncivil". Reading everything, the conversation has been fairly civil. I'll admit that I was aggressive in the bug when I said that NPP is often staffed by new editors who want to "rack up edit counts", and I'll accept chastisement for that statement (a statement which you appear to agree with, by the way).  You are clearly frustrated, and I wish that we could move beyond that and into a constructive dialog.
 * I have been engaged as much as I can be with people about this over the past several days. Many people are providing input to us, both on-wiki and on IRC. I've actually worked all weekend on an idea for a revamped interface.  We are absolutely trying to work with Good Faith here and are thinking hard about how this works, what's broken, and how it should work.
 * We would very much like your constructive help with this. We are trying to set up a system for people (New Page Patrollers, in this case) to record screencasts of how they go about working, say, for 15 minutes to half an hour - so that we can get as many experiences as possible to work from.  Your input would be very much appreciated with this.
 * Also, I don't think that this is a situation of "all or nothing". I'm fairly certain that there are other ways to go about combatting the problems you describe than adding another user right and applying ACTRIAL.  There is always another way.  Always.
 * (One idea that we've been turning over is a way to better crowd-source page patrol, where the "tagging" of articles doesn't result in higher edit count, and that an article has to reach a defined threshold of the same tag (given by different users) in order to have the tag go "Active". So, for example, if four people mark a new article as "spam" then it will be considered spam, but if only two people do, it won't hit the threshold.  This would help solve for the problem of inexperienced patrollers making bad judgement calls since the responsibility for accuracy would be distributed.)
 * For the record, just so identity is clear in this issue, I am employed by the Foundation. My title is "Senior Designer." Erik is my grand-boss of sorts - he is the Vice President of Engineering (and, I believe, still the Deputy Director of the Foundation).  Steven Walling and Ian (Raindrift) are also Foundation employees (Steven works for the Community Department as a Fellow, and Ian is a Software Developer in the Features team).
 * I really hope that you can bring your experience to play here. This is a very hard problem (the kind I enjoy solving), and the more brains we have the better.--Jorm (WMF) 22:16, 18 September 2011 (UTC)


 * I'm concerned that a lot of the solutions to the NPP problem that being thought up by the WMF involve an increase in the amount of work required to patrol an article. A fundamental concept that you need to understand is that there is already too much work for the current pool of patrollers to keep up with.  At this point, most articles exist for 20-30 days before one patroller gets a chance to look at them.  Any solution that is going to require multiple patrollers must be coupled with a guaranteed solution to increase the number of patrollers by a corresponding factor, or increase another variable in the equation.  For instance, if it's going to take 4 people to mark an article as patrolled before it is "fully" patrolled, then one (or a combination) of these things would need to happen concurrently:
 * The number of new page patrollers would need to increase by a factor of 4.
 * The Special:NewPages queue timeout would have to be increased by a factor of 4 (i.e. from 30 days to 120 days).
 * The time it takes to patrol an article would have to be reduced by a factor of 4.
 * Numbers 1 and 3 would be very difficult (if not impossible) to acheive, and number 2 is problematic because do we really want terrible articles floating around on Wikipedia for up to 120 days before someone gets a chance to look at it? While distributing responsibility for patrolling articles is a good idea in theory, in reality it is unlikely to work.


 * At the risk of sounding like a broken record, the ACTRIAL idea was an attempt to solve the problem from the other end; i.e. by reducing the number of articles that need to be patrolled. If you consider that nearly three-quarters of all articles created by non-autoconfirmed users are eventually deleted (and there is hard evidence to support that), this is the low-hanging fruit that we saw as the easiest solution.   On average, non-autoconfirmed users create 695 articles per day, only 191 of them are not deleted after they get patrolled.  This is an average of 504 articles per day that are injected not only into the workload of the new page patrollers, but also into the workload of admins who patrol speedy deletion requests, admins and users who patrol PROD requests, users who comment at AfD discussions, admins that close AfD discussions, and users who comment at DRV discussions.  The impact of finding a way to cut down on the number of inappropriate articles created per day (whether it be by the ACTRIAL method or some other way) would be huge, and it would lessen the workload of a lot more users than just new page patrollers.  This is a major inefficiency with the way wikipedia currently works, and it's only going to get worse as the number of new things to write articles about continues to decrease.  Attempts to improve methods of educating new users on how to create better new articles can only accomplish so much, and in my opinion it won't do nearly enough.  Snottywong 21:53, 19 September 2011 (UTC)
 * Item (2) won't do much good; if NPP can't keep up then it can't keep up. The size of the backlog is irrelevant. ErikHaugen 22:13, 19 September 2011 (UTC)
 * I'm aware of these issues; I actually have a design for a feature that operates from the other end of this spectrum - the patrolling interface itself. I'm in the process of writing it up and will publish it either late tonight or tomorrow.--Jorm (WMF) 21:58, 19 September 2011 (UTC)

Multiple Extensions/Projects
Looking over the proposal as is and how it could be, it is clear that there are several issues here which dovetail into one another. Many (most?) of these elements can be applied across several workflows in Wikipedia.

First, I'm going to Be Bold and say it: MediaWiki is great for what it is but it's really not designed for the scale of the workflow that is being demanded of it. The software has been very "generic" for the most part and not targeted at the specific needs of Wikipedia (such as the New Page Patrol workload). As Raindrift stated above, there are several "degrees" of NPP; the software was never designed to accomodate that.

The following is a list of identified features that may or may not be useful in the overall zeitgeist of the problem. Feel free to add to it.--Jorm (WMF) 18:46, 16 September 2011 (UTC)


 * Better Landing and Interstitial Pages - this design document currently focuses on this in particular.
 * Better Help System - this should be a context sensitive, targeted, and word-budgeted help system. Additional links can take the user to "Full Monty" documents, but for the most part editor education should be a gradual process rather than Encyclopedia Writing 101.
 * This feature would be an extension, smart, and be able to be inserted with parser hooks to sections on a page (e.g.,.
 * Additional features within this extension could include a stream of "did you know..." tips that can be paged through.
 * Article Tagging System - this should be an easy-to-use panel that allows a user to tag an article in many ways, such as "needs sources," "more photographs", "spam", etc. This system could be expanded to work within the main namespace on existing articles, possibly replacing the large Orange Boxes at the top of many articles (and thus possibly reducing server/template overhead)
 * Storing tags in a metadata format would allow for them to be easily searched ("Show me articles lacking sources within WikiProject Military History").
 * Such a tool could be easily converted into a "Work List". One of the issues known to plague new editors is that they simply don't know where to start working.   This, combined with an interests graph provided by GlobalProfile would go a long way towards showing people things that they can do - especially if we ask them questions like "Do you like copy-editing?," "Do you like researching sources?", etc.
 * The system could easily be built so that the tags could be defined by the community at large. With integrated localization, it could operate trans-wiki.
 * Content Analysis System - an AbuseFilter-like system that gets run across a new article and checks that it meets various criteria (
 * This should include as sophisticated a "copyright violation" search system as possible. I'm not sure that we can win big with this, but plugging it into multiple search engines for text matching may be doable.
 * Safe Harbor Workspace - this is described (vaguely) in the main design document.
 * Depending on configuration, this could reduce spam and vandalism pages if publication to the main namespace required review for all articles (e.g., you cannot publish your own article).
 * New Page Patrol Call-to-Action System - this would be a simple call-to-action banner system that would appear to users who have been identified as previous or current patrollers.
 * Ideally, this system would funnel people into addressing elements that they have done before (copyediting, reference checking, etc.)
 * Connection to Feedback Dashboard - this could be a way for new editors to request help.

Update on WMF conversations today
I'm not going to be able to respond in detail probably before next week, but I want to at least make an effort to counteract the normal bias that discussions happen internally at WMF, and aren't shared publicly, by recapping some of the conversations I've had today.

Earlier today, I met with Steven Walling, Ian Baker (User:Raindrift), Howie Fung (User:Howief), and Brandon (User:Jorm (WMF)) to further discuss the different problem areas we've been talking about (new page patrolling vs. straight article creation vs. article creation through an improved AFC-like mechanism). Essentially, we have different hypotheses how we may be able to positively influence inflow, quality and user experiences through targeted interventions, without necessarily creating new restrictions to page creation:

1) We hypothesize that improving the NPP tooling efficiency itself is likely going to be essential to keep up with the backlog, reduce NPP frustration, and bring more people into the fold.

2) We hypothesize that NPP patrolling tools could also be used to more effectively shepherd promising new users into environments where they can be mentored and supported.

3) We hypothesize that an initial interstitial as illustrated on the mock-ups on this page may help socially sort users into those who don't mind an experience that can get a bit rough (but who want instant gratification) vs. those who would like to do things in a guided and "correct" way, being recognized and supported along the way. This could reduce friction and frustration in the second group.

4) We hypothesize that an improved AFC process/tooling can not only help newbies make better pages, but it can also connect them with people who want to help them with their their specific article more quickly.

5) Last but not least, we hypothesize that friendlier messages/templates can go a long way in helping new good faith users feel welcome.

If we are to make a difference, we have to carefully stage interventions along these lines, and measure the impact. I've asked Howie to develop these hypotheses more clearly, and we'll probably start re-structuring the page consistent with that. Needless to say, the above is a loose characterization of what we've discussed, and I appreciate corrections/clarifications from the participants.

In order to improve NPP efficiency, we'll need more than first-hand experience -- we'll need to watch experienced users doing NPP work and explaining what they are doing. To do this with minimal cost, we want to explore using screencasting tools. We've worked with usertesting.com to get a simple version of their Java-based screencasting tool up and running so we can make it trivial to upload screencasts for people who aren't familiar with screencasting tools.

We'll talk more about that in a few relevant places as we finalize the tooling review, but if you want to already go ahead and record a 15-30 minute screencast of your NPP work, it would be tremendously helpful. Any explanations of the specific judgments you are making as you're doing the NPP work are appreciated.

As I mentioned a couple of times, we're serious about engaging with these problems. I hope we can formulate a set of hypotheses together that we all agree are worth testing. If, after doing so, we still think we need to test the autoconfirmed hypothesis, then we can revisit that question, but I suggest we start with hypotheses that are less controversial in light of our goals to attract new users and minimize barriers to participation.--Eloquence 01:58, 17 September 2011 (UTC)


 * I do  not  actually see a professional attempt  to  engage these problems. I  see a lot  of talk and positing  of new hypotheses, and I  see a marked resistance to  properly  examine the reports, stats,  empirical  findings, and solutions  that  have already been offered from  the fold of experienced volunteers outside the WMF office walls, and that  are the result  of long, intense research  since the beginning  of October 2010. There appears to be some difficulty at  MediaWiki in  understanding  that  the en.Wiki NPP  is a fundamentally broken process, because unless NPP is converted into  a user right  for mature and experienced editors, or unless en:WP:ACTRIAL is implemented, the problems will  persist and get  worse,  and it  will  never be known without  this trial, what  further or other solutions should be preferred and explored. What  en.Wiki basically  has at  present as its  firewall  against  totally  unacceptable new pages is an NPP system   operated mainly by a batallion of  the least experienced and mature editors, newbies themselves, and a very, very small platoon of seasoned users and admins trying  to put  right  what  they  are doing  wrong. Attempts have been made to  make the NPP  instructions as plain  as possible, and to  engage the patrollers in  it, and hundreds of friendly  requests have been placed on  patrollers' talk  pages asking  them to  improve their work, but  1. You  can lead a horse to  water  but  you  can't make it  drink, and 2. The NPP instructions are complex because they  are based on  the maze of walls of text  and instruction creep of en:WP:DELETION, and en:WP:CSD, and all  their variants and exceptions, and further  explanatory  essays. --Kudpung 20:55, 18 September 2011 (UTC)


 * Just to be clear on one point: we haven't really discussed making patrolling a userright at the WMF, either internally or in places such as Bugzilla, except that Raindrift (Ian) brought it up as an idea he supports. If English Wikipedia wants to improve the quality of patrolling by making it a userright that you control access to, I don't think anyone would oppose that. Steven Walling (WMF) &bull; talk   02:15, 19 September 2011 (UTC)


 * The user right idea was a solution originally intended to be implemented after ACTRIAL was implemented, simply because there is a valid fear that making patrolling a user right would lead to a decline in patrollers. There is already a shortage of patrollers with respect to the number of articles created per day, and while many of the patrollers are doing a particularly sub-standard job, some believe that any amount of patrolling is better than allowing the article to fall off the 30-day queue at Special:NewPages and never get patrolled by anyone.  I continue to believe that creating a user right for patrollers is a great idea, but I think it needs to be paired with either an increase in patrolling efficiency through interface changes (as proposed by Raindrift above) or a decrease in the number of inappropriate articles created per day (as proposed by ACTRIAL).  Snottywong 21:58, 19 September 2011 (UTC)

On required portals/introductions

 * "The new user may proceed only after viewing this content (this page will be dismissable)."

I hope there is a better way to introduce editors to this kind of info, without requiring an extra clickthrough and pageview. A two-column view for example, with the tutorial side-by-side with the editing screen. Extra required clickthroughs isn't fun the first time through, and becomes especially frustrating for anyone who edits as an IP from time to time. Sj 09:37, 17 September 2011 (UTC)

"Zoom" Interface
As promised, I have the stubs of a design document for a "Zoom" interface for new page patrolling.

This is really it's own project, however, so I have started its own design document at New Page Patrol Zoom Interface.

Please add comments and ideas there; this design is only in phase 1 of thinking. I don't normally like to post design documents that are this incomplete but I figure it is better to start early rather than later.--Jorm (WMF) 00:18, 20 September 2011 (UTC)

There still needs to be barriers to entry to article creation
I agree wholly with MZMcBride: MoodBar feedback belongs in /dev/null. I guess these were cherrypicked/edited for comprehensibility -- I saw the comments that MRG posted on en.wp:


 * bad interdace,not easy,realy bad
 * How do I email Tabercil - wiki admin on my personal bio page?
 * "of when i ask you are not answer me why?"
 * i want to write bt i can nt

Inviting users who can't even punctuate a sentence properly to create articles will only make things worse -- the community will find some other very bitey way to keep the garbage they create out. For this reason, we don't want to make this too easy to use (after all, editing Wikipedia requires a brain).

Running heuristics before page creation is the best aspect of this proposal. We've had ex-post heuristics for some time -- I created w:Template:Spamsearch some five years ago to identify spam pages in userspace. It is now an abuse filter. Intervention before the article is created stops nasty SD tags, talk page warnings and the sometimes unfriendly responses to questions regarding why blatantly unencyclopedic garbage gets deleted.

Something to consider is including an AJAX search box for the article title, e.g.

Article title: [Textbox] [throbber] First 5 search results for this title

We have plenty of users who don't STFW before creating an article (hence CSD A10). MER-C 05:50, 20 September 2011 (UTC)


 * The MoodBar is a tool that's presented to all new users after making an account and attempting their first edit. As such, it captures feedback from a wide variety of very different people: people of different ages, educational/cultural backgrounds, many folks for whom English is a second language, etc. Not all of them want to edit (they may have created an account for altogether different reasons, such as believing that it was required, and have found the edit function by accident); many of them shouldn't, but that doesn't justify discounting all the comments by those who do want to edit. Even the seemingly inarticulate ones may be examples of people who might be fluent in another language, or who are used to typing in slang in a feedback context, or who are dyslexic, or who simply have no mental map of Wikipedia yet (the comment you picked about Tabercil is a good example).


 * I would strongly dispute that the social and technical complexity of Wikipedia acts as any kind of neutral intelligence filter, and indeed, in addition to the MB comments telling us what problems people are reporting, we know (from user tests, surveys, etc.) that it filters out lots of people who we want as editors. We're going to keep an eye on the editors who've given us MB feedback and I'll bet you a non-virtual beer that among the folks you'd readily identify as inarticulate idiots are some who are or could be promising editors.


 * That doesn't discount the second part of what you're saying -- of course, if we increase the intake, we also have to improve intake management, and ideally in a way that's designed to avoid unnecessary biteyness, but that also takes into account that we have to show some people the door sooner rather than later. In case you haven't looked yet, New Page Patrol Zoom Interface goes in that direction. A part of the flow described on this current page that's easily overlooked is also that it proposes to sort people more quickly and effectively into those who are OK with a certain degree of roughness ("I know what I'm doing, lemme at it") vs. those who'd like a higher degree of human touch in a safe sandbox environment.


 * As for heuristics, I agree -- although again I would be careful not to pass judgment so quickly why/whether users have or haven't "STFW" yet. Empathy is a virtue. ;-) Some users have a harder time getting a full mental model of a site early; that doesn't make them stupid. There may be a red link they followed, or a search result page that invited them to create a page while they didn't notice the match.


 * It's certainly possible to provide more clues in the creation process, though. The Bugzilla implementation when filing a bug is a good example of a search to avoid duplicate reports integrated into the creation flow. I'd love to find a good, non-obtrusive way to include this type of functionality.--Eloquence 10:04, 20 September 2011 (UTC)


 * Regarding language issues: see here. Although it's from an OSS context, this part is extremely relevant. I can't speak for others, but I am much more likely to respond with heavy sarcasm to semi-literate, clueless comments like those seen on MB. That's why I wrote w:User:MER-C/SmartQuestions (it's still a draft). MER-C 13:52, 20 September 2011 (UTC)

Please record a screencast of yourself doing New Page Patrol
Hi, everyone. Thanks for all your ideas and feedback. I'm really excited about this project--I think we have a chance of making something really useful.

One of the things we've realized in talking with new page patrollers is that, in the absence of any real coherent UI, everyone has tailored their own custom interface from scripts, browser plugins, etc. This has its good and bad points, but it's also meant that we at the WMF can't actually examine the interface most of your are using, because we don't have it.

Any good software design effort starts with understanding what people are doing now. So, we're hoping that a few of you could send us screencasts of yourselves doing NPP. Here's specifically what we'd like you to do:


 * 1) Begin recording a screencast of your NPP session.  I think 10-15 minutes would be about right, but feel free to take more or less time as you see fit.
 * 2) Clearly state your username, so we can easily link your comments with your video.
 * 3) Start doing NPP.  Spend about the first half of the time moving slowly, talking through the process and explaining what you're doing, why you chose that specific tool, what you like/dislike about it, and any other options you tried in the past.  If you're using gestures or other off-screen interface elements, describe them.  Also tell us about what you're looking for in the article, the policy decisions you're making, and your thought process.  If you suspect copyvio, tell us what feature triggered that suspicion; if something seems like spam, what's spammy about it?  Show us your process.
 * 4) Spend the rest of the time running through NPP in your usual way.  This gives us an idea of the pace at which things happen, how long specific steps take, etc.  Don't try to go faster or slower than you normally would here.  The idea is to understand where the bulk of the time and effort go, so we can streamline that process.
 * 5) Wrap up the video.
 * 6) If you're using the usertesting.com tool, upload the video per the instructions below.  If you're using your own tool, send your video (the whole file, or a link if it's really big) to me at ian@wikimedia.org.
 * 7) Send any additional comments you have in an email to ian@wikimedia.org.
 * 8) Feel free to post a link to your video and your comments as a followup to this message.  This is encouraged, but a video of one's personal computer is, well, personal, and I want to protect your privacy as well.

If you want to user the usertesting.org tools (this is the quickest/easiest route if you don't already have screencasting software), follow these handy instructions from Jorm:


 * 1) Go to our usertesting.com launch page.
 * 2) Click the "start" link (it's the only link available)
 * 3) * Do not close this tab or window. It's okay to background it, but don't close it.
 * 4) The Java applet may require rights to run.  Grant the rights as needed.
 * 5) A few moments later, a "box" interface will appear.  Resize it to contain your entire work area (either your entire desktop or all the tools you use at once)
 * 6) Press the red button in the lower left of the interface to begin recording.
 * 7) A countdown will start.  Once the countdown has finished, the recording has begun.
 * 8) Speak aloud your thoughts as you work. This is very important.
 * 9) When you are finished, click the "Done" button (down near where the start button was)
 * 10) The browser window (that you didn't close earlier) will then refresh with a screen cap from your session and ask you to upload the file.
 * 11) Click "Upload this file".  The upload will begin.  This may take a while.
 * 12) You'll be presented a list of screencasts.  The filenames will be datestamps.  Please note which one is yours and list it below, if possible.

If you have your own screencasting software, feel free to use it, and send your video and comments to ian@wikipedia.org. If you can't get the usertesting.com stuff to work (it's still quite beta) or you just want to get some screencasting software anyway, here's a list of tools, and here's another one.

Once we've received what seems like a representative sample, I'll compile all videos, we'll look them over, and we'll let you all know what we've learned.

Thanks in advance for your help. This will be invaluable information for making a tool that's actually useful to existing new page patrollers.


 * There's also a great deal of material WikiProject Screencast, though it focuses on creating how-to screencasts for general consumption, not screencasts for user experience research. Do we want to make a MediaWiki-focused general documentation page for this here, including materials on usertesting.com? Steven Walling (WMF) &bull; talk   19:38, 20 September 2011 (UTC)