Talk:Article Creation Workflow/Design

Instructional text
This page looks good :-) You're absolutely right, of course, that we offer far too much instructional text. Users who are task-focused perceive the text as an impediment -- e.g., just this past weekend, I tried to test out using the Article Creation Wizard, and even though its text is relatively tight, I still found myself skipping past most of it. The trouble with our existing instructional text is that we try to accommodate all possible use cases [1], which makes the text really, really long. That backfires because users then skip all of it (like I did with the ACW). It'd be much more effective for us to keep instructional text super-succinct --covering just the most common use cases--, because that way, at least some people would get some information that could help them, rather than nobody getting any. Sue Gardner 01:59, 14 September 2011 (UTC)

[1] a laudable goal in software development, but maybe less of a laudable goal for instructional text :-)

MoodBar comments
Does anyone reading the MoodBar comments think that the articles created by these individuals would be kept on Wikipedia? I understand and appreciate the desire to make article creation less horrible, but it comes with a cost. Who's going to pay? --MZMcBride 04:11, 14 September 2011 (UTC)


 * Thanks for reading through the page and for the comments MZ. In my view, the two changes proposed here are:


 * 1) Adding clear instructional material and warnings through one or two interstitial pages that appear before the edit window when you're hitting a redlink
 * 2) Possibly providing a noindexed space outside the main namespace to create articles and strongly encouraging people to work there first
 * Neither of those changes seem like they will make anyone "pay". Rather, we'd like to tell newbies much more clearly upfront what the expectations are, and then ask them to start articles in a place other than the mainspace where both the article and their skills have time to develop. Even without the second step, what we want to do is reduce the costs placed on the community by making it clearer what we do and do not want in Wikipedia. Steven Walling (WMF) &bull; talk   06:50, 14 September 2011 (UTC)
 * Number two seems uncannily similar to AfC. Killiondude 07:38, 14 September 2011 (UTC)
 * Hey Killion. You're right, it is pretty similar, but if you get a chance to read through "Future Phases" section there's more detail there about the ways it's different (as currently proposed anyway). For starters, it would not necessarily require review in order to publish, because it's aimed at registered users rather than anonymous ones. If you want to pick apart that description and leave us some feedback, it'd be a big help. Steven Walling (WMF) &bull; talk   17:12, 14 September 2011 (UTC)
 * The only difference between the proposed workspace idea and AfC is that no review is required (or even encouraged, by the looks of it) by an experienced editor. This means that, while your warnings might occasionally convince a new editor to start their articles in the workspace, there is nothing stopping them from immediately moving it from the workspace to mainspace when they realize their article is not yet "live".  Therefore, we're just adding a superfluous step in the article creation process which won't stop anyone from creating an inappropriate article in mainspace, and won't provide for any type of review process by an editor who is familiar with WP policies.  A better process would be to require new users to go through AfC (or a similar AfC-like process) when they want to create a new article, until they have been properly educated on what is expected in a new article.  Thus, en:WP:ACTRIAL.  Snottywong 18:17, 14 September 2011 (UTC)
 * I think I might let Brandon reply about the particulars of the workspace design and how it might be adapted, but I do want to make a general point: when it comes to this or other WMF Projects, nothing has to be taken 100% as-is with an all-or-nothing decision. It's not substantially happening yet, but we'd like to live in a world where there is a lot more of this exact thing going on: experienced editors showing up very very early in the design process and giving us specific feedback about the potential process and policy implications of new features. Steven Walling (WMF) &bull; talk   18:32, 14 September 2011 (UTC)
 * The description of the future phases and how they work is deliberately vague in certain areas, especially regarding how policies work out. For example, it could easily be set up that Anonymous IP editors can be allowed to create articles in the Workspace but not publish to the Mainspace - this allows us to have articles created by IP users but not allow them to pollute the Main space.  That's only one policy change that could occur.  Another idea could include "not automatically deleting poorly-written articles but instead unpublishing and then tagging them" so that work isn't lost.--Jorm (WMF) 18:36, 14 September 2011 (UTC)
 * Yeah, where they can collect dust just like everything else that needs maintenance. As evidenced by the numerous backlogs on such essential problems as copyright and fair-use issues, we can't handle all of it at the rate it's coming in.  At the risk of being a dick, I'd urge you to click on the link Snottywong provided you above and give input there. The Blade of the Northern Lights  (話して下さい ) 22:01, 14 September 2011 (UTC)

Lovely Root System - can we make it grow?
This idea has such a lot of useful ways in which it could grow. I'd personally like it if new accounts had a sandbox area automatically created for them, with links to a really good set of interactive tutorials covering the basics of what newbies need to know; screenshots with annotations for "how to do this", which newbies can read in conjunction with playing in their sandbox; a kind of "test yourself" auto-marked interactive quiz thing for each basic concept, a heap of stuff like that so that newbies can really get into practising "getting it right" from a very early stage. A links to "pick a mentor" (somewhere really easy to see, so they're likely to hit it, perhaps) would be a good idea. But really excellently-structured interactive video tutorials would be such a good thing to look into, from this start. One of the options in the article workflow "I don't feel confident yet - I want to do the tutorial" as one of the options. Probably the top one! ThatPeskyCommoner 08:02, 14 September 2011 (UTC)

Are we really examining the entire workflow here?
I realize that this change is happening now for reasons aside from new page patrol workload. That seems to be what's driving its implementation at the moment, though, so in the interest of brevity I'm going to address it primarily from that perspective.

I'd like to start with a brief anecdote: I had a discussion with Neilk recently about the community's response to UploadWizard. I was curious why so many people were opposed to what is an obvious, major improvement in usability for contributors. His perspective, in summary, is that we only solved part of the problem. While the old interface was confusing for new contributors, that wasn't an issue for the established community. The community had a problem with categorization and sorting. We did little to solve that problem. Instead we made it worse, from the perspective of the people in the trenches, by inviting a torrent of new contribution.

I think there's a lot to be learned from that. One lesson might be that improving UI is bad, and the technical barriers we have in place are beneficial because they keep out the riff raff. I think that's the wrong lesson. This entire movement is built on people feeling empowered to contribute.

Instead, I think we need to be more mindful of the entire, end-to-end workflow for new content creation. Clearly at this point, there's a mandatory review of all new pages (via NPP). Our current approach sees improvements to that step as out-of-scope. I really like the current proposal, but I don't think we're going to get broad support from page patrollers until we include them in it.

Furthermore, addressing only one part of the problem carries additional risk. For example: we hope that educating people about Wikipedia policies as part of the process will help reduce the overall workload on page patrollers. Currently, about 30% of all new pages are deleted. Let's assume we're successful, and our education efforts reduce that number to 18%, but our usability improvements double the number of new pages created. In a sense, this outcome would be wonderful, but it would actually result in an increased NPP workload.

I tried doing some NPP recently. I set up Twinkle, read the documentation, and checked out Special:NewPages. While I didn't mark any pages (I don't feel that I'm personally experienced enough), I saw many areas where the process could be improved with WMF support. I'm sure the experienced new page patrollers know way better than I do. The central issues, as I see them:


 * The documentation is long and confusing, which I bet keeps people from learning NPP
 * While there's a clear workflow, the available tools don't guide the patroller through that workflow
 * The existing process is inefficient (requires many clicks)
 * The existing process requires the patroller to keep a lot of minutae in their head
 * There does not appear to be automated infrastructure for recruiting new patrollers

Data shows that workload per new page patroller is actually decreasing, and has been for some time. However, it's clear that there's a perception that workload is increasing, or that the work itself is terrible. The source of this perception is a mystery to me, but I suspect it may stem from a sense of frustration and disempowerment that comes from years facing the same endless torrent of spam and vandalism. Perhaps if we were to work with these people to improve their tools and actively recruit new members to their ranks, the sense of futility would wane.

Of course, we can't do all things at once, and I'm not proposing that. Rather, I think we should broaden our scope when making plans, and balance improvements that create an influx of new users with features that make dealing with that influx easier. The most important thing is that we consider the entire process, and make it clear that we're doing so to the people who are affected.

tl;dr

Our current plan to address the NPP workload:


 * Improve the quality of new articles by effectively educating new users.

This proposal:


 * Improve the quality of new articles by effectively educating new users.
 * Improve the efficiency of the new page patrol workflow
 * Recruit additional new page patrollers by building tools for that purpose, and make software that helps to train them more effectively
 * Get the new page patrollers whatever else they need to work effectively

Raindrift 01:23, 15 September 2011 (UTC)
 * This is the last comment I'll make here, since I'm sick of being patronized. It's not addressing the problem of an influx of new users because people don't read what we give them.  Someone who wants to write about The Sound of Perseverance (band) (see WP:GARAGE) will do it, and someone who wants to write "Let's expose all burakus for what they are!!!!" and link to a website with a list of all the burakumin in Nagano Prefecture doesn't care what we want them to do, they'll do it.  This will enable them to do it and keep it around (and for those of you who think this is trivial, this is illegal in Japan, and the wrong person seeing it could lead to thousands of lives being destroyed; you're very lucky I caught it).  I could point out some other problems, but I want to see if people here can really solve this problem being so far removed from actual New Page Patrolling on en.wiki. The Blade of the Northern Lights  (話して下さい ) 02:59, 15 September 2011 (UTC)
 * In my opinion, Raindrift's proposal is the first step in the right direction. Any proposal of this scope needs to equally take into consideration both the needs of new users and the needs of experienced editors who will largely be managing the new users' contributions.  You may have followed the autoconfirmed trial debacle.  Our long-term plan was to first implement a trial restricting article creation to autoconfirmed users, and the second step was to start drafting improvements to NPP.  In any case, I've patrolled pages for awhile &mdash; not nearly as much as some others like Blade of the Northern Lights or Kudpung, but enough to know what's wrong with the system.  At the very least, I can offer my personal answers to some of your questions.
 * In particular, you mention that workload per patroller is decreasing while the perception is that it is increasing. I think I might be able to shed some light on this one.  Patrolling an article encompasses a wide spectrum of potential tasks, and which tasks are performed depends on the patroller.  For instance, on one end of the spectrum, many patrollers will simply check to make sure that the article is not overt vandalism, and mark it as patrolled if it passes this simple test.  On the other (ideal) end of the spectrum, a patroller will check everything on this checklist, and actually fix various problems with the article before moving on to the next one.  This side of the spectrum requires not only more time, but more experience from the patroller.
 * Anyway, the point of the story is that workload per patroller is decreasing because patrollers are doing less work on each article. The reason they are doing this is because of the constant pressure from knowing that if the Special:Newpages queue reaches 30 days, then articles are going to fall off the list and never get patrolled.  So, they feel like they are stuck between two bad choices: maximize their patrolling speed by skipping steps, or allow the queue to overflow.  The problems are relatively simple:
 * There are too many articles for not enough patrollers.
 * NPP is often seen by new users as a way to quickly increase your edit count, therefore patrollers are often inexperienced users who aren't equipped to recognize (no less fix) many problems with articles.
 * How do we solve this problem? Many of your suggestions will certainly help.  Better patrolling tools, better training for patrollers.  One idea that we've been throwing around is requiring patrollers to have a user right in order to mark a page as patrolled.  This will prevent very new, inexperienced users from marking inappropriate articles as patrolled.  It will also allow us to revoke the user right if we determine that a user is doing a particularly poor job at patrolling.  Another frequently requested feature is to make the "Mark this page as patrolled" link appear on any unpatrolled article, regardless of whether you accessed it via Special:NewPages or not.  This would allow patrollers to patrol even when they're doing other tasks and they happen to come upon a new, unpatrolled article.  My personal opinion is that we should combine both ideas and make the "Mark this page as patrolled" link much larger and more visible for users with the aforementioned user right.
 * NPP is definitely a broken process, and therefore I'm sure other patrollers have many more ideas than I do. You may want to consider posting a notice to this discussion at en:WT:NPP.  Snottywong 03:32, 15 September 2011 (UTC)
 * First of all, I should mention to everyone that there's been some discussion of how NPP works now and its issues over at Patroller Workload. It's worth reading that discussion.


 * To add to the list of issues, it seems there's a concurrency problem when multiple NPPers are working at the same end of the queue. Once there's more than 3-5 simultaneously, they start generating edit conflicts with one another.  I'm concerned that more patrollers on the current interface would make that worse.


 * Snottywong, thanks for your input. It sounds like the process itself actually involves many people with varying levels of skill, but the only differentiation in the task is which end of the queue one examines.  I wonder if we could start improving the tools by storing more granular information about which aspects of an article have been reviewed.  This way, we could promote greater specialization (doing NPP now requires a lot of knowledge), and the tool itself could be used to teach new skills.


 * For example, imagine a tool that shows one new article at a time, choosing it based on a set of criteria (age, what's been checked, who the patroller is, etc). NPPers could start out patrolling for vandalism and spam, which are easy to recognize.  After patrolling 25 articles for those criteria, the tool explains how to look for a good title and how to move an article (and so on).  This way, new patrollers are gently led along the learning curve while accomplishing useful work at the same time.  At some point, they could start only reviewing articles that had been vetted as not-vandalism and not-spam, on the assumption that new patrollers were handling those.


 * For very new patrollers, it might make sense to show their edits to someone more experienced to see if they agree.


 * Tracking the specific review steps would allow us to expose which steps an article had passed. This is handy for finding articles to patrol that match a patroller's skillset, but it might also help to reduce newbie-biting.  Instead of telling someone, "Your article lacks references," we could instead say, "Your article is notable, has a good title, is properly categorized, nicely formatted, has great spelling, but lacks references."  That kind of positive reinforcement is very specific, which makes it feel very sincere.


 * Well designed, I think such a process could actually be less work. Ideally people are looking at these things anyway, so it'd only be an extra moment to check a box when done.  In return, we'd see a significant reduction in duplicated effort.  Also, we could build a UI that keeps all the resources necessary for effectively patrolling a page handy, which would streamline the process further.  Then again, perhaps this wouldn't help at all and there's some important aspect of the process I just don't get yet.  I'm glad we're discussing it, though.


 * Secondly, it seems that a very small number of people do most of the page patrolling, and especially the more skilled (end of queue) portion. That's a pretty hefty burden for a small number of people.  Training new patrollers will help with that, but we also need to be able to call on them when they're needed.  I'm thinking a simple system that monitors the age of the queue, and starts notifying known NPPers when there are many unreviewed articles that match their skillset.


 * Lastly, the 30-day deadline for NPP seems like it might be useful for setting targets and motivating work, but could also be getting in the way and leading to necessary patrol not happening. With this hypothetical system, we could apply a more nuanced view of how long is reasonable for patrol to take.  For example, the deadline for spam-checking should probably be a day or less, but the deadline for reference checks could perhaps be more than 30 days without ill effect.  When a task runs past the deadline, automatically deciding to skip it is probably the wrong course of action.  The point here not that we should implement any specific change, but that it's worth examining whether this deadline (and how it's implemented) is serving us well, and if not, what could take its place.


 * Ideally, a tool for this sort of thing could be generalized for many types of patrols (ie. not just new page patrol), but I'm also wary of scope creep, and I realize that just because these things sound similar doesn't mean they are. I'd like to identify the areas where the process could find the greatest benefit, and address those things first. Raindrift 23:22, 15 September 2011 (UTC)


 * I'm encouraged that you're taking the time to independently discover many of the problems with NPP. You might be interested to learn that volunteers have taken it upon themselves to at least partially mitigate some of these problems.  For instance, I have created a tool which tells you how many people are working on each side of the newpages queue (see http://toolserver.org/~snottywong/cgi-bin/patrolreport.cgi).  I've also created a tool which tracks how close we've gotten to the 30-day limit (see http://toolserver.org/~snottywong/cgi-bin/patrolgraph.cgi) along with a defcon template which is updated hourly by my bot (see en:User:Snottywong/NPPdefcon).  My bot also runs a task which notices when the newpages queue overruns its 30-day limit, and keeps tracks of any unpatrolled articles which were forced off the list (see en:WP:NPP30).  Luckily, the queue hasn't been overrun for a few months, but earlier in the year there were hundreds of unpatrolled articles overflowing the queue every month.


 * Clearly, many of these problems could be solved more efficiently and effectively by changes to the mw software. Patrollers have been screaming about the 30-day limit for a long time, but have never convinced anyone to address it.


 * I think your idea about making multiple levels of patrolling is a step in the right direction, but I think I'd have to see a more detailed proposal before I could support it. My fear is that it will actually create more work, because 4 or 5 people will end up reading through the whole article as it goes through all of the different patrolling stages.  The important part is that we make patrolling a user right which must be requested and which can be taken away if the patroller consistently does a poor job.  Pair that with an intuitive, interactive education system on how to correctly patrol and I think we'll be making some progress.  Snottywong 00:02, 16 September 2011 (UTC)