Talk:WMF product development process

Tracked in Phabricator
Task T125806

See /Archive 1 and /Archive 2 for previous page contents.

Sounds like waterfall

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

(Reposting, since the original was lost in the conversion to flow)

The process as written here seems to imply that a feature, even a large one like VE, would go through all the phases, roughly in order, as a single effort. Basically it would culminate in a "big bang" release of the whole thing, months or possibly years after the initial conceptual work was done. That contrasts sharply with the agile and FOSS approach of "release early and often".

I believe (as do most agile and open source practitioners) believe that whenever possible, small increments of work (at most a few developer-weeks of effort) should move through all the phases, including release, so that real end-user feedback can inform later increments of the same major feature. I hope that is how this was intended to be used, but if so, it needs a fair bit of clarifying text. If the intent is to shift to a more waterfall-style development process, then that's a whole other discussion we should have. KSmith (WMF) (talk) 23:01, 28 October 2015 (UTC)

I wanted to jump back in and say: Having worked pretty closely with Wes, I'm quite confident that he did not intend to propose a waterfall-like process. He is very familiar and comfortable with agile development, and my perception is that he is an advocate of it.

So despite how this page was initially worded, I am confident that our work will continue to be more on the agile side. KSmith (WMF) (talk) 00:29, 11 November 2015 (UTC)

Hmmm - I was looking at it differently, but I was focusing at the top of the table where it has iterations of bugs and feedback, so I was thinking of "Release" as being an iterative cycle. Can you describe more about what this would look like to make it seem more agile? Rdicerb (WMF) (talk) 23:09, 28 October 2015 (UTC)

Without context, this page pretty much exactly aligns with a multi-year waterfall process. Even in waterfall, there are some backtracks, exactly as the two "bugs and feedback" arrows indicate.

My main suggestion would be that this page should specifically advocate for splitting bigger features up into smaller ones. And to advocate iterative releases of relatively small bits of functionality, rather than one "big bang" release. (That is, when iterative releases are possible...which they almost always are.)

The page should also warn against performing extensive work in the Concept phase, or any other phase, before the first release. Because anything you dream of up front might end up invalidated by user feedback. You want that feedback loop to be as short as possible, to avoid wasted effort.

The best diagram I could find in 5 minutes was this one. The current wording of this page seems to advocate a waterfall approach, as opposed to the approach labelled "spiral" here: https://en.wikipedia.org/wiki/Software_development_process#In_practice (although I think the quadrants on that spiral diagram are in an odd sequence). KSmith (WMF) (talk) 23:25, 28 October 2015 (UTC)

Yes to splitting bigger features and develop on small increments, but there needs to be a definition of general checkpoints where community review is encouraged / required. In a regular agile project the team members will decide next steps on a bi-weekly basis or so, and "real end-user feedback" will start coming as soon as alpha/betas exist. In our context though, we cannot ask our communities to participate in every sprint planning/review, and we need their involvement already in the planning phase. How to solve this equation in a way that works for developer teams and our communities? Qgil-WMF (talk) 23:35, 28 October 2015 (UTC)

@KSmith (WMF) we added some additional text to help articulate things further. Thanks for the input. WMoran (WMF) (talk) 11:01, 29 October 2015 (UTC)

Thanks Wes. It still feels more waterfall-friendly than I would prefer, and doesn't emphasize "release early and often" as much as I would like. But the changes look good, and I definitely I like that it now mentions agile.

Per Quim's comment, and in other ways, it's unclear how this process would actually interact with an agile process like scrum or kanban, which many teams are already using. I look forward to seeing this evolve. KSmith (WMF) (talk) 17:20, 29 October 2015 (UTC)

@KSmith (WMF) - do you happen to have a table or some sort of imagery to this effect? Rdicerb (WMF) (talk) 23:01, 30 October 2015 (UTC)

+1 to @KSmith (WMF). It might help to show example features and how they move through this process. DStrine (WMF) (talk) 23:13, 29 October 2015 (UTC)

+1 to @KSmith (WMF). @Rdicerb (WMF) this imagery is a quick and dirty visualization of iterative cycles vs waterfall cycles:

Waterfall

https://upload.wikimedia.org/wikipedia/commons/e/e2/Waterfall_model.svg

Iterative

https://upload.wikimedia.org/wikipedia/commons/3/39/Iterative_development_model.svg

Also, I was surprised to see categories under "Maintain" and not see TPG in there:

"Retrospective," "Feedback," "Improvements."

Seems right up TPG's alley. MBinder (WMF) (talk) 23:48, 2 November 2015 (UTC)

I like those diagrams, Max, but they don't connect to each other. The waterfall one, like the table on this wiki page, could be used to represent 1 week or 2 years. Similarly, the second one has some implication of a faster cycle, but doesn't explicitly make that point. It too could apply to a project with releases every year or two.

I sent a sketch to Rdicerb which tries to show how phases work within an iterative process. Hopefully something like that can be incorporated here, or we can find a (far) better diagram online. KSmith (WMF) (talk) 01:44, 3 November 2015 (UTC)

i agree - i see the process represented as linear with maintenance as an end state. i would prefer to see a cycle / spiral of continuous improvement. i see some teams are doing this. Slowking4 (talk) 21:57, 7 November 2015 (UTC)

I like the spiral idea to help convey better and I need to figure out how I can do that in the html effectively. WMoran (WMF) (talk) 14:40, 9 November 2015 (UTC)

here are some cycle diagrams c:Category:PDCA or c:Category:Spiral_model_of_Boehm Slowking4 (talk) 23:51, 9 November 2015 (UTC)

Actually, the agile/iterative image like this one is the one that the product team is thinking about adopting. Does this look like it makes sense around an agile methodology? If you think about a product that is launched in some projects and in Beta in others - say VisualEditor - It's been in use for some years, but Citoid was developed earlier this year. There are languages that it is not ready for that the team has not yet started working on.

Does this image help to clarify? Rdicerb (WMF) (talk) 21:49, 12 November 2015 (UTC)

As the original creator of that image, I feel like it does communicate the point, but in a very crude and non-artistic way. I would love if someone with more design+art talent could do a better job. The image also overstates the maintenance burden over time--when done properly, a given feature really shouldn't require substantial "maintenance". (Users might request additional features, but that would be tracked as new functionality.)

If a better image isn't available, I think adding this image to the article would be a net positive. As long as it has a couple sentences describing what it means. KSmith (WMF) (talk) 00:12, 20 November 2015 (UTC)

Is there anything actionable left here or can be consider this topic resolved? Qgil-WMF (talk) 14:48, 18 November 2015 (UTC)

Personally, I think this page still *looks* like a waterfall process. The one mention of agile is nice, but for me doesn't overcome the impression left by the rest of the page. So for me the issue is definitely not resolved. I thought there was going to be another image or two which would help make the page more clearly non-waterfall. KSmith (WMF) (talk) 00:09, 20 November 2015 (UTC)

We need to conciliate the agile idea of constant iterations with a reality where hundreds of communities cannot be expected to be involved in a sprint process. The agile process is described in detail at WMF product development process/Proposal, and that is the document that aims to define the daily practice of WMF teams. Here we aim to provide an overview that should allow everybody to understand the different activities and stakeholders involved in the development process.

I think the emphasis of this overview should be put on checkpoints, the first time a project aims to enter a new stage and the requirements to pass each checkpoint. Once a project passes a checkpoint, then they can iterate as much as needed within that stage and the previous ones.

Let's take "Release" as an example for a checkpoint. No matter how agile a project is, it will require several iterations before thinking seriously about releasing a new product to real users. A requirement for that checkpoint could be a minimum release plan. No matter how agile a project is, it would not enter the release stage without a release plan publicly available.

This is not waterfall (in a waterfall you cannot swim back), but there is a sequence. If you allow the comparison, this is more like an adventure game, where a team has to solve problems in order to enter a new room, but they can go back to previous rooms at any time.  :) Qgil-WMF (talk) 07:50, 20 November 2015 (UTC)

@KSmith (WMF) I added a version of the time scale as a chart but it needs the explanation. Could you expand please? @Slowking4 I added an example cycle table to help illustrate the point. Thanks for the input. WMoran (WMF) (talk) 22:54, 20 November 2015 (UTC)

The discussion above is closed. Please do not modify it. No further edits should be made to this discussion.

PM/UX/ENG

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

Could these acronyms be expanded please? Legoktm (talk) 22:25, 2 November 2015 (UTC)

+1. I'd also like to link terms (like teams mentioned) to their corresponding pages whenever possible (and I'm happy to do the gruntwork), like "A/B' to https://en.wikipedia.org/wiki/A/B_testing AKlapper (WMF) (talk) 19:28, 3 November 2015 (UTC)

+1 added a few, feel free. WMoran (WMF) (talk) 23:59, 10 November 2015 (UTC)

The page looks good acronym-wise now. Closing this topic, further improvements can be done just editing the page. Thank you! Qgil-WMF (talk) 07:46, 11 November 2015 (UTC)

The discussion above is closed. Please do not modify it. No further edits should be made to this discussion.

"Design"

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

UX/UI and overall architecture are two very different things, why are they being lumped together? Legoktm (talk) 23:01, 2 November 2015 (UTC)

@Legoktm are you referring to WMF_product_development_process#Design and its related column? I think the term "Design" is used correctly there, with the meaning it has in the context of engineering processes (see i.e. w:Design). I agree it is inconsistent with the use of the term in mediawiki.org (see Design) as in strictly Visual/UX, but I think it is better to fix the latter, not the former. Having Architecture, Performance, and Operations as part of the Design stage makes sense to me.

@Brion VIBBER @Ori Livneh @Mark Bergsma (WMF) what do you think? Qgil-WMF (talk) 08:48, 11 November 2015 (UTC)

Yes. I think Brion pretty much summed up exactly what I wanted to say :) Legoktm (talk) 07:55, 13 November 2015 (UTC)

The current list of tasks under 'Design' seems mostly focused on UX, with "Architecture and operations" sorta thrown in randomly. :)

What is it that arch & ops would be expected to provide in terms of feedback? If what you're asking for is, say, feasibility assessment, it probably belongs back a step or two in Plan. If you're looking for advice on what technical directions to work in ("should this be an extension or in core?" "should this use a mysql table or a separate storage service?" "should this have a a web API?" "how much JS should we use?") then I think you need it in the build stage along with Performance. brooke (talk) 18:31, 11 November 2015 (UTC)

@WMoran (WMF), what do you think? Qgil-WMF (talk) 14:48, 18 November 2015 (UTC)

@Brion VIBBER good points. Planning and build are more logical for the respective examples. @Ori Livneh (WMF) and @Mark Bergsma (WMF)? WMoran (WMF) (talk) 22:39, 18 November 2015 (UTC)

@WMoran (WMF), I think it is safe to just proceed with the changes and move forward. Qgil-WMF (talk) 09:38, 2 December 2015 (UTC)

The discussion above is closed. Please do not modify it. No further edits should be made to this discussion.

A path for experiments, with/without eventual productization

It's clear that to allow agile experimentation, not every tool or experiment would, or could, involve all these stakeholders.

Perhaps it would be useful to sketch out what experimental tools or features would look like (i.e. a *minimalist* rather than maximalist mapping of stakeholders), and what, if successful/popular, eventual "productization" of that tool/feature might look like. Ijon (talk) 01:56, 3 November 2015 (UTC)

I wonder if a checklist of things that *could* be done would be helpful - steps, tools and actions as a practical list. That way should a product be more lightweight, there's an understanding of experiments and steps that a team may choose to skip, but at least they would actively be choosing to do so. Rdicerb (WMF) (talk) 05:49, 5 November 2015 (UTC)

The natural space for experimentation goes from Concept to Build and Release in Labs. I agree that the process should allow for experimentation without dragging the effort with overhead, and that is not clear now.

As soon as someone wants to deploy a piece of software in Wikimedia servers, things get more complicated. Maybe the checkboxes should be still in place, as an exercise of communication, even if their active involvement or approval would not be needed. "Just FYI".

If an experiment starts heading toward productization, committing resources, being prioritized in sprints and goals, aiming to be deployed in all Wikimedia projects... then those checkboxes become more important. What for a team might be "a very cool experiment" can turn into a trainwreck if the stakeholders didn't even have a chance to know that such experiment was happening.

We have precedents of cool experiments being drawn by Process, and also of experiments that turned out not to be so cool and caused crisis affecting many others. Qgil-WMF (talk) 15:01, 18 November 2015 (UTC)

It is becoming evident that the agility of movement between stages will be determined not by the potential stakeholders of every stage but by the checklist that will be placed at the entrance of every stage.

See for instance Phab:T120070, a task about the creation of a checklist for concept to enter the prioritization process. There it is suggested that some item of the checklist would be required, other optional. In the case of an experiment you want to run, there would be a question i.e. "Do you need funds / developers?" and the answer would be "No, thank you.", which would save to this concept a bunch a further questions related to the stakeholders that won't need to be involved.

@Ijon, are these explanations enough to resolve this topic? Qgil-WMF (talk) 09:31, 2 December 2015 (UTC)

Ijon's question needs some more thought. Reduced to the minimalist statement, the issue might look something like this:

It's good to do "experiments". It's good to "involve communities early". It is not always (or even usually) possible to do both of these things (well) at the same time.

If we set up "involve communities early" as the prime directive, then we are imposing significant, even destructive, burdens on experiments. We would be saying that devs may only conduct an experiment if they involve non-dev communities in it. That increases the costs (not just in money) as well as the likelihood of volunteer time being wasted on failed experiments, volunteers being disappointed by the cancellation of products that they want but which the devs need to abandon, and volunteers learning to ignore development because the cost:benefit ratio is unfavorable to them. We could adopt this, but we need to own the costs of such a position.

But if we encourage experiments, then we will frequently encounter a situation in which communities have no opportunity to be "involved early". We will have successful experiments that are presented "later", and which are rejected by some community members precisely on the grounds that these products began (as experiments) in advance of their involvement. More often, some volunteer communities may feel like discussions with them are merely for appearances, rather than true interest in their views. We could adopt this, but we need to own the costs of such a position.

IMO a better answer is a balance of the two: involving affected communities while the product is in active development, but not always (or even usually) in the experimental stage. There certainly are costs to this position, too: A "balance" means that some people will find out about projects later than they wish, some people will be asked for help much earlier than they wish, and some people will use "not early enough" as an excuse to object when the real problem is that they just don't like it or need it themselves. We could adopt this, too, but we need to own the costs of such a position.

(I also believe that it is inappropriate to have a one-size-fits-all method for involving non-dev communities, but having a process of "use your best judgment to decide when to involve communities" has all of these costs, too.)

There is IMO no perfect answer. However, I beleive we need to directly address this issue, by clearly stating a standard expectation here, regardless of whether that expectation is "You must never spend even 10 minutes thinking about an experiment before holding a formal community consultation" or "Communities can be ignored until 24 hours before the launch of any new product" or something in the sensible center of those extremes. Let's make a decision about what we're going to do, name and truly understand the disadvantages to our choice, try it out, and decide how to adjust our process from there. Whatamidoing (WMF) (talk) 01:26, 5 December 2015 (UTC)

@Whatamidoing (WMF) your comment is spot on! In order to draw the line, what about requiring community review only when there is a clear aim to end up deploying the project that is today an experiment. This "clear aim"could be defined by

• a plan for deployment
• a proposal to make this project a team goal
• a request for funds or people allocated to this project (new or moved from other projects) Qgil-WMF (talk) 13:32, 7 December 2015 (UTC)

That sounds like a practical, workable plan with concrete stages. I like it.

However, I expect some people to complain that this is "not early". We should think about ways to manage their expectations. We may want to formally and publicly define "early involvement of communities" as being equivalent to publishing information about the project on wiki at any of these three points in time. Whatamidoing (WMF) (talk) 19:58, 23 December 2015 (UTC)

What are all the related efforts?

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

Here are all of the active efforts to change our development process that I'm aware of. I'd like to replace the last paragraph of Overview with this list, and treat this as the authoritative list. Any objections? Anything I'm missing?

• SPDPP: WMF Engineering Process Proposal
  • related: TPG 2015/16Q2 goals for Burnup and Maintenance Fraction measurement project)
• How we prototype (ongoing meetings led by May)
• Codifying and promoting Design and development principles
• Design and development principles
• Quarterly Goals proposal (private doc)
• Discuss alternate ways to current quarter planning
• Roadmapping (see https://phabricator.wikimedia.org/T117800) JAufrecht (WMF) (talk) 22:02, 4 November 2015 (UTC)

Thank you for the compilation! These links should be all reflected in the main page, wherever appropriate. Qgil-WMF (talk) 20:11, 5 November 2015 (UTC)

The discussion above is closed. Please do not modify it. No further edits should be made to this discussion.

Beta features

(Copying a post by @Wittylama that was misplaced due to a pagemove problem)

Could you perhaps discuss in this document the role of the "Beta features" system?

This was created as a place for people to opt in to new tools but has been used somewhat randomly. Sometimes new features bypass the beta system altogether - going straight into production (a recent example being the new notification icons) and sometimes things sit in the beta system indefinitely. Personally I'd really like it if there was a consistency about how the beta system was used, and also some objective, public, and measurable goals applied to each feature as the criteria for 'graduation' being an opt-out/default feature. Currently the beta system tells you how many people have opted-in to a given too, but not the retention rate or how many people that is as a % of active users etc.etc. No one likes surprises, and you've got a perfect platform to test and receive feedback, but it's just not seemingly part of the standard procedures. Wittylama (talk) 18:11, 5 November 2015 (UTC)

@Wittylama, I agree that the use of Beta features should be systematic and documented. In my opinion, the use of Beta features should be part of the Release plan (as a requirement to enter the Release stage) and the intended results should be integrated in the quality criteria for release (as a requirement to move from individual opt-in to regular deployment).

See also the related discussion about Scope of "release" stage. Qgil-WMF (talk) 09:23, 12 November 2015 (UTC)

@Jdforrester (WMF), @Quiddity (WMF), could you help clarifying whether there is a documented process defining when and how Beta Features should be used? If not, is there a task in Phabricator to fix this? Qgil-WMF (talk) 09:37, 2 December 2015 (UTC)

See Beta Features/Package. The guidance is that Beta Features should only be used for bigger changes on which you're not sure; trivial things like changing icons are not in scope. The system was built back when desktop vs. mobile was still a thing, and there's no good plan for resolving that. In general, I would rather we killed Beta Features than added yet more bureaucracy to it. Jdforrester (WMF) (talk) 16:10, 2 December 2015 (UTC)

On the one hand what you say sounds quite reasonable, but then again the WMF did just recently rolled-out a change that was considered a cosmetic adjustment to buttons (splitting the notification icon into two) that had to be rolled-back for an important fix. It's a quite recent example of where a 2 week 'beta' period would have been perfectly sensible:

*the change wasn't urgent

*the audience for the change was active editors

*the success criteria for default rollout are easy to define (no slowdown in load time, no broken extensions, WCAG accessibility compliant

*whatever it was that the change was designed to improve in the first place).

I completely agree that minor changes should not have unnecessary bureaucracy, but that should speak more to the way that a 'beta testing' period should be handled as a smooth and natural transition. If the change isn't urgent, what's the harm in having people who opt-in testing it for two weeks? If there's no problem, no problem. If there's an issue raised then it can be quickly fixed without the embarrassment of having to undo a rollout.

P.s. Jdforrester (WMF) can you explain what you mean by "back when desktop vs. mobile was still a thing"? Wittylama (talk) 09:30, 4 December 2015 (UTC)

By "back when desktop vs. mobile was still a thing", I was making a slightly snippy off-hand reference to the out-dated idea that we wanted to build two different websites, one for desktop and one for mobile, instead of working towards a single site for users of any and all device sizes. What we would have considered acceptable then isn't something we'd be OK with now. Sorry for the confusion. Jdforrester (WMF) (talk) 17:45, 7 December 2015 (UTC)

It remains my understanding that the mobile development and the desktop development are undertaken separately, and that the features, user-experience and workflows are intentionally different. See, for example, this recent comment by @Jdlrobson on Phabricator https://phabricator.wikimedia.org/T118338#1840066. I'm not saying that either approach is wrong, but I am getting very mixed messages about how the mobile development and the desktop development are undertaken. Wittylama (talk) 13:50, 8 December 2015 (UTC)

Sorry, I don't run the Reading department, I was just relaying what I'd understood from them. Jdforrester (WMF) (talk) 01:59, 10 December 2015 (UTC)

Stop pretending, @Jdforrester (WMF) - we all know that you secretly run everything ;-) Wittylama (talk) 19:11, 10 December 2015 (UTC)

I actually highly doubt that such a problem would have made itself obvious in beta mode. If it wasn't obvious in mediawiki.org and test.wikipedia.org, then it wouldn't have been obvious in beta either.

It's actually a good example of what James meant I think: Beta will not protect you from bugs or implementation errors. Regression testing and performance metrics will.

Beta, at most it allows you to do gauge interest and gather feedback on something you are 'considering'. But unfortunately, it's not instrumented well enough to actually do so in practice.

And there is another problem, that in the current form Beta is not much more then 'gadgets on steroids', which also means it comes with implementation limitations, that require you to reimplement whatever you tested. If we ever want to use it more, it's entire architectural pinnings will need to be reworked to change that.... —TheDJ (Not WMF) (talk • contribs) 09:40, 4 December 2015 (UTC)

Thanks for the reply user:TheDJ, even if your response is rather depressing. If that's the case, it basically tells me that the Beta features system is pretty much a wasted project in the first place.

P.S. There's a related thread on Phabricator where I've just mentioned this discussion here: https://phabricator.wikimedia.org/T76573 Wittylama (talk) 10:29, 4 December 2015 (UTC)

Part of this has to do with that fact that we are 'hyper optimized'. Unfortunately, that also makes it very difficult to 'vary' inside the software stack, since the stack was never designed to allow for much variance. MediaWiki and Wikipedia specifically was always designed on the premise of "Everyone gets the same, where possible", and it's very difficult to change all those thousands of assumptions throughout the system into: "Everyone should be able to get something totally different" and not tumble the entire website.

I think we will have to change that going into the future, but it will require massive changes, not only in our software, but definitely also in our operations and hosting. What ops is doing with new deployment tools might help with that, but it's a long road. It's what is needed to make 'Beta' actually work and to make that a 'Core' feature. —TheDJ (Not WMF) (talk • contribs) 10:58, 4 December 2015 (UTC)

And I definitely wouldn't say it is useless/wasted. Every system has limitations, you just need to be aware of them and understand how they impact your usage of the system. But currently there are soooo many limitations, that the system is basically not worth the extra effort required to make use of it.

It can be improved, the linked ticket describes a few things that could be worthwhile improvements to at least get better/more understandable/interpretable feedback out of a deployed beta feature.

But using it to 'catch bugs' when only about 5% of a couple thousand active editors will ever use the system is unrealistic. You need the test group of 120000 active english Wikipedians at some point to really reach those 50 editors that, more than once a week, use the thing you changed (on Chrome 35). This is part of our challenge, we have areas of the software that really have a minute amount of users, yet those few people can be impacted greatly by even the smallest change. —TheDJ (Not WMF) (talk • contribs) 11:21, 4 December 2015 (UTC)

I think that being able to expose features gradually to our users has many benefits.

Some features may need more steps than others. For example, a feature can be available as opt-in for those users that look for it, then it can be announced in a relevant context inviting more users to try, later it can become the default with an option to opt-out, and finally become just the default experience. Other features may need less steps, or none at all for the most trivial cases (although the devil is in the details and most uncontroversial changes may have more ramifications than the ones expected initially).

Being able to try a new feature in a real context (not an example in a testing server that may or may not fit with your workflows) and having always a way out to go back to normal in case things go wrong, seems valuable and helps to set expectations about the feature.

Communication is a big part of this in order to make users aware about how the feature is evolving, indicate whether they consider the feature is working for them and describe which issues they experience.

I think many of the aspects mentioned above connect with several of the challenges teams face when launching products and result in much time spent afterwards. Beta features is not completely supporting all of them, but I think that what we need is to improve the platform. I think that problems such as few people joining, or the need to make the process more agile require less effort than the problems of not having a way to gradually early and often get our users on board. Pginer-WMF (talk) 14:09, 4 December 2015 (UTC)

Yup, this is right. Beta Features isn't for "beta testing" of whether the code works overall, or has performance issues like with the notification badge split (that's the job of the developers to get right before it ever is seen by a user). It's more like a tool for User acceptance testing of bigger changes. Jdforrester (WMF) (talk) 17:47, 7 December 2015 (UTC)

Beta Features is essentially intended for performing user acceptance testing, i.e. intended to answer the question "Does this feature meet the user's needs?". It's not intended to be used as a way of finding regressions like those in the recent notifications rollout, although of course any time anyone's using software bugs and regressions can be found.

test.wikipedia.org and en.wikipedia.beta.wmflabs.org are intended to find regressions and test integration, i.e. "Does this feature simply not work, or break other features?". In theory, the regression with notifications should've been caught here. However, it is known that these wikis are not really enough like production to catch these regressions. That's something that @Greg (WMF) et. al. have been interesting in improving for a while, but the problem is very complex and would take significant work. Dan Garry, Wikimedia Foundation (talk) 22:01, 28 December 2015 (UTC)

Let's try to summarize this long discussion. All what this product development process would define are checkpoints about public testing in the Develop and Release stages. These checkpoints could define some expectations:

• Software quality: prototype, alpha, beta, stable opt-in...
• Target groups for each announcement: tech ambassadors, Beta Feature users...
• Minimum duration of the testing
• Feedback channels used

This process should not require the use Beta Features or any other specific technology to get early feedback from users.

Is this a good summary? Qgil-WMF (talk) 08:52, 2 February 2016 (UTC)

I'd say that one thing missing from that bullet-point list is that it needs specific and measurable criteria for acceptance. Depending on the type of thing that might be something needing community consultation to define or might be a statistical measurement (like 'no slowdown in load time'). My primary criticism of the Beta Features system (other than confusion about when it is used or not used) is that there's no way to tell whether something is "successful" and should be promoted, or "unsucessful" and needs to be demoted. At present, things just live there in limbo-land. I refer again to my favourite ever rollout 'go/no go' critieria in Wikimedia history - the "Usability Initiative" (for building the Vector skin). Its published in advance critieria was "80% opt-in user retention". This was a measurable, achievable, and objective criteria. It also baked-in the concept of valuing the feedback from the early-adopters who chose to drop-out, which resulted in those people trying it again and eventually becoming advocates for it within the community (ping @Trevor Parscal (WMF) who was part of that team). Wittylama (talk) 16:52, 4 February 2016 (UTC)

@Wittylama I also think that was a good approach. One thing to note, however, is that we actively surfaced the feature's availability by adding links to the personal tools and running banners. This dramatically increases awareness of features, but it also requires we do this one-at-a-time and once-in-a-while or there will be advertisement fatigue. Trevor Parscal (WMF) (talk) 19:05, 9 February 2016 (UTC)

Community Engagement (Product)/Collaboration process/Draft

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

Nemo_bis asks a related question at Talk:Community_Engagement_(Product)/Collaboration_process/Draft#Other pages. [Editing content, as requested] Quiddity (WMF) (talk) 20:13, 5 November 2015 (UTC)

Hadn't seen this, responded, thank you. Rdicerb (WMF) (talk) 02:45, 17 November 2015 (UTC)

In this table

|Iteration

|

|

|<---- Analysis & Feedback

|<---- Bugs, Analysis & Feedback

|<---- Bugs, Analysis & Feedback

|

|- style="font-size: 16px;"

|

! style="background-color: #90f380" | Concept >

! style="background-color: #7cf069" | Plan >

! style="background-color: #67ee51" | Design >

! style="background-color: #53ec3a" | Build >

! style="background-color: #32de15" | Release >

! style="background-color: #2cc613;" | Maintain

|-

| rowspan="6" style="font-weight: bold;" | Tasks

|Ideation

|Output

|Prototypes & Labs

|Backend Development

|Stakeholder go/no go

|Retrospective

|-

|Research

|Outcomes (Metric)

|Mockups & A/B

|Frontend Development

|Announcements

|Feedback

|-

|Analyze Technical Debt

|Resources & People

|Surveys

|Performance Checks

|Community Events

|Metric (Impact)

|-

|Strategy

|Milestones

|User Sessions

|Code Quality

|Analysis

|Improvements

|-

|

|Dependencies

|Workflow

|

|Triage Issues

|

|-

|

|Maintenance

|

|

|

|

|-

| colspan="7" |

|-

| rowspan="9" style="font-weight: bold;" | WMF Stakeholders

|

|

|

|

|

|

|-

| style="border-left: 4px #1414C8 solid;" |Product Manager

User Experience

Engineering

| style="border-left: 4px #1414C8 solid;" |Product Manager

User Experience

Engineering

| style="border-left: 4px #1414C8 solid;" |Product Manager

User Experience

| style="border-left: 4px #1414C8 solid;" |Product Manager

User Experience

Engineering

| style="border-left: 4px #1414C8 solid;" |Product Manager

| style="border-left: 4px #1414C8 solid;" |Product Manager

|-

| style="border-left: 4px #ace solid;" |Community Liaisons

| style="border-left: 4px #ace solid;" |Community Liaisons

| style="border-left: 4px #ace solid;" |Community Liaisons

| style="border-left: 4px #68C814 solid;" |Operations

| style="border-left: 4px #ace solid;" |Community Liaisons

| style="border-left: 4px #ace solid;" |Community Liaisons

|-

| style="border-left: 4px #1486C8 solid" |Community Members

| style="border-left: 4px #C8AA14 solid" |Legal

| style="border-left: 4px #5614C8 solid" |Design Research

| style="border-left: 4px #90f380 solid" |API/Services

| style="border-left: 4px #14C850 solid" |Release Engineering

| style="border-left: 4px #5614C8 solid" |Design Research

|-

| style="border-left: 4px #5614C8 solid" |Design Research

| style="border-left: 4px #C81A14 solid" |Security

| style="border-left: 4px #AA14C8 solid" |Research & Data

| style="border-left: 4px #8CC814 solid" |Performance

| style="border-left: 4px #C8AA14 solid" |Legal

| style="border-left: 4px #AA14C8 solid" |Research & Data

|-

| style="border-left: 4px #AA14C8 solid" |Research & Data

| style="border-left: 4px #C8148C solid" |Communications

| style="border-left: 4px #C814C8 solid" |Data Analysts

| style="border-left: 4px #14C8C8 solid" |Team Practices

| style="border-left: 4px #C81A14 solid" |Security

| style="border-left: 4px #C814C8 solid" |Data Analysts

|-

| style="border-left: 4px #000 solid" |Partnerships

| style="border-left: 4px #14C8C8 solid" |Team Practices

| style="border-left: 4px #14C86E solid" |Architecture

|

| style="border-left: 4px #C8148C solid" |Communications

| style="border-left: 4px #8CC814 solid" |Performance

|-

| style="border-left: 4px #C8148C solid" |Communications

| style="border-left: 4px #14C850 solid" |Release Engineering

| style="border-left: 4px #8CC814 solid" |Performance

|

| style="border-left: 4px #68C814 solid" |Operations

| style="border-left: 4px #C81A14 solid" |Security

|-

|

| style="border-left: 4px #90f380 solid" |API/Services

| style="border-left: 4px #68C814 solid" |Operations

|

|

| style="border-left: 4px #68C814 solid" |Operations

|}

I see no mention whatsoever of where the wider editing community fit in. When do things get mentioned or discussed in any venue non WMF people are likely to see. I suggest you have a new section for editor engagement. Salix alba (talk) 08:47, 17 November 2015 (UTC)

@Salix alba, hmm - can you clarify for me whether you believe that the tasks need more definition, or do you mean that along with the "WMF Stakeholders" groups: Operations, Engineering, Community Liaisons, , Research & Data, you believe that Editor Engagement is missing? If it's the latter, community liaisons support community voices along the many of the tasks along the top level: Ideation, Dependencies, Triage issues, etc.

This is a very high level table that lacks details, but there are opportunities to drill down into specific tasks which may offer more detail. Rdicerb (WMF) (talk) 22:09, 17 November 2015 (UTC)

@Salix alba, your comment is not really related to the original post of this topic, is it? Details about the participation of communities during the development process are being discussed in other topics of this page. See Talk:WMF product development process#h-Community_input_prior_to_build_phase-2015-11-08T12:06:00.000Z and Talk:WMF product development process#h-Scope_of_"release"_stage-2015-11-05T20:14:00.000Z. Qgil-WMF (talk) 13:39, 18 November 2015 (UTC)

The discussion above is closed. Please do not modify it. No further edits should be made to this discussion.

Scope of "release" stage

The "release" stage in this model doesn't seem to distinguish between adding a feature that can be used, and adding a feature that must be used (or must be manually disabled/worked around to avoid it) -- a distinction that I think was at the core of past fights over MultimediaViewer and VisualEditor. brooke (talk) 20:14, 5 November 2015 (UTC)

Pushed post too early. :) Needs more explicit calling out of iterative releases: open-beta style opt-in testing, ability to see old and new versions side by side and select among them.

And also needs to call out what happens to "old" features -- discontinuing things that are obsoleted/replaced by a new feature, or new work that gets tested for a while and then discontinued, itself has consequences. (And failing to trim out those thigns has its own consequences.) brooke (talk) 20:17, 5 November 2015 (UTC)

Good points. The idea of this entire flow is that it could be applied to a beta/alpha/production. In that line of thought then something in Maintain as a task to graduate? or evaluate? Evaluate may be better as it could apply to old features or new beta's that perform. WMoran (WMF) (talk) 23:49, 10 November 2015 (UTC)

I agree that this stage is critical and needs clear definition. @JAufrecht (WMF) @Greg (WMF), your thoughts here are welcomed.

In the context of Wikimedia, releases are frequently very complex because they involve waves (not all wikis get the same releases at the same time), and different methods for opt-in / opt-out at a wiki level and at an individual level. I bet not many people at the WMF are aware of all the options available, leave alone among our communities, and as far as I know they are not documented in a single place.

When a project enters the Release stage, it is communicating an intention to become a new product/feature in production. This should trigger a communication to our communities, so they can prepare accordingly and optionally provide their feedback. Some requirements should be fulfilled to enter the Release stage, such as a functional prototype available, a list of release blockers and quality criteria, a release plan, and a communication plan.

Can we consider that a project enters the Release stage for the first time when it requests deployment to test.wikipedia.org? (equivalents must be defined for mobile apps etc). Labs prototypes would be part of Build. Beta features would be definitely part of Release.

The release plan could be as simple as following a default timeline for beta features, A/B testing, and release waves (to be defined, unless a systematic process already exists). Projects with special needs would customize this default process, proposing alterations in the deployment waves (e.g. some wikis to be prioritized, some wikis to be moved to later waves), providing special methods for user opt-in / opt-out, etc.

Communities should have a chance to influence this plan, proposing release blockers and non-blockers specific to them (not all issues impact equally all types of projects and languages), or choosing a different wave (some communities will be socially more excited about a certain product, or more stressed about it).

When the release plan is defined, another communication to the communities would be sent. From that point, releasing as planned should be a matter of checking against the quality criteria agreed, and deploying when the criteria have passed. Qgil-WMF (talk) 09:06, 12 November 2015 (UTC)

FYI, there are two tasks in Phabricator that depend on the details about the Release stage:

• Document checklist for releasing a new feature
• Provide a reply to Letter to Wikimedia Foundation: Superprotect and Media Viewer Qgil-WMF (talk) 08:55, 1 December 2015 (UTC)

Setting Priority in Phabricator

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

Hi, There was recently a disagreement about what the "Priority" tag means in Phabricator. I learnt that developers change this when they mean to work on a given bug. This is not the purpose I expected. I initially thought that this means what the bug reporter thinks, i.e. how much priority should be given to this bug. So we may need a new parameter, to enable volunteers who report and comment on bugs to express what they think, which obviously may not match what developers finally do.

PS: Sorry if this is not the right place for this. Yann (talk) 20:19, 5 November 2015 (UTC)

The expected use of the Priority field is described at Phabricator/Project_management#Setting_task_priorities, and the discussion over this point would belong to that page and not here. However, linking to that page from this overview makes total sense. Qgil-WMF (talk) 22:08, 5 November 2015 (UTC)

The discussion above is closed. Please do not modify it. No further edits should be made to this discussion.

Community input prior to build phase

Could someone please clarify what the WMF's vision is here, for community input prior to the build phase? Alsee (talk) 12:06, 8 November 2015 (UTC)

The Vision is being defined as we speak, and this is the place to define it (or maybe a specific page focusing on community participation in the development process?) Let me try to capture our state of mind in the discussions that lead to this draft:

• Communities have a simple way to watch projects going through stages, resting assured that they won't miss any important step. To be defined and implemented.
• Communities have a say during each stage. What feedback is expected and how it is provided, needs to be defined. This is an open source software development process, and therefore FLOSS and standard engineering practices are likely to be more efficient than Wikimedia's tradition in wiki management. Contributing must / should / could criteria in the Concept phase and identifying blockers before / during the Build phase is more powerful and effective than providing a first and final community resolution in the form of a support/oppose RfC when a product is basically ready to be deployed.
• If everybody has done their homework up to the Build phase, the Release phase should be a simple matter of going through an agreed list of remaining blockers and quality criteria.
• The role of Beta features, A/B tests, user opt-in / opt-out needs to be defined, the idea being that all these ways to test with real users and soft release inform the developers and stakeholders about the status of a new product. Having solved "philosophical" discussions between the Concept and Design stage should allow to proceed with all these test releases with a shared interest from the communities in gathering new data and feedback.
• How soon and how exactly a community receives a new product depends on the specific plan for beta features / A/B tests / opt-in / opt-out / deployment phases... We need to organize these elements in a clear and predictable way. Communities should have a way to manifest their interest in being among the first ones, or their preference to be included in a later wave, when technical and social details got more time to be polished. Deployments should follow paths of least resistance whenever possible.
• Communities have the final say on the deployment of a product in their wiki. This is a right and a responsibility. A community blocking the deployment of a product as planned is expected to provide proof of community consensus and actionable blockers, to be open to discuss the blockers in product terms, and to be involved in their resolution. The aim to provide a consistent user experience across Wikimedia projects should always prevail. Moving from a deployment wave to another should not be a big deal.
• If the team in charge of the deployment of a product suspects that the blocking in a project is not fair, they will report the situation. Where and to whom is to be defined, but it must be a current or an adapted Wikimedia community process, like reporting to the administrators board of a project, the stewards board... An example of unfair blocking would be a single active admin blocking a new product in a small project based on personal taste, without community discussion.

In order to participate effectively as software partners, the WMF teams need to change in some way and learn to interact better, but this is also true for the communities. Non-technical contributors need to understand the basics of the product development process and their role in it. Communities need to understand that they are not the only stakeholder, and that "The Community" is a very complex stakeholder difficult to read due to the many communities and the many voices in each community (including those voices that follow but don't speak because of many structural and occasional factors). The proactive involvement of tech ambassadors, tech-curious contributors, and administrators is essential in order to keep a healthy and productive collaboration.

There are other ideas about systemic bias in community discussions, respectful communication, opportunities for wider segments of the communities to provide feedback in simple/private/non-demanding ways... But this post is getting long, and I hope that at least it is useful to start defining The Vision, and the homework that we need to do in order to implement it.

PS: er... I got carried away and went through the process until deployment, not just up to Build. There are many discussion that can start from here anyway, so I hope you don't mind.  :) Qgil-WMF (talk) 10:49, 11 November 2015 (UTC)

Thanx Qgil. There's a lot to like in that "state of mind". The WMF seems to have thoroughly discussed and documented many of the more... mechanical aspects of the process... but I wish the working-as-partners aspects would be better addressed. Sometimes partners disagree, and sometimes it takes effort to sort out those disagreements. I don't want to rehash old problems, I don't want to assume the worst, but I also don't want to assume the best. I'm hoping for clarity on some points that are still "to be defined".

Individuals, admins, wiki community consensus, global community consensus:

1. Individuals: The WMF can shoo away bad ideas by saying "come back with a consensus".
2. Admins: Admins are individuals. "Come back with a consensus" is perfectly appropriate.
   • Regarding an admin who acts without consensus to block a new product: An admin who disregards consensus is not just going against WMF, that is an abuse against Community. I will firmly support nuking his admin bit, even if I agreed with what he wanted. Admin tools are to be used on behalf of community and consensus.
3. Wiki-consensus: On EnWiki central community consensus is established at Village Pump. I assume other wikis have similar central community pages. If a consensus is asserted elsewhere, it is appropriate and often advisable for the WMF to say "come back with a central community consensus".
   • It is absolutely reasonable for the WMF to request/expect elevated levels of advertisement for important RFC's. It's not reasonable to reject a consensus merely because someone imagines a "silent majority" would have given an answer more to their liking.
4. Global community consensus: For matters that affect the global community it may be reasonable to say that evidence of broader consensus is needed, especially when dealing with a smaller wiki. However that is a rather thin argument regarding EnWiki. EnWiki is about 44%[1] of the active editor community.
   • The WMF deployed Single User Logon relatively recently, the brand new translation extension looks rather successful, cross-wiki notifications are in the works, and it looks like Community Tech team might pick up cross-wiki transclusions (or similar) as a new project. With these tools the community may be able to solve the cross-community complexities for you.

• Contributing must / should / could criteria in the Concept phase and identifying blockers before / during the Build phase is more powerful and effective

I particularly asked about pre-build because it's the best and most painfree stage to address issues. To quote something the WMF said over two years ago:

Flow/Community_engagement#What_we.27ve_learnt_so_far Bringing users into the conversation at the end of the development process makes it hard for their feedback to be incorporated. Ideas gather inertia, particularly when implemented, so it's hard to change course.

In the best case the WMF_product_development_process acknowledges that the community might want a significant change in course, and it acknowledges that "expected community feedback" could be a consensus that a product would be unwanted or detrimental.

To be clear myself, my concept here is dialog. The WMF announces a concept, and in most cases the community thinks it's good or great. *IF* there are community concerns and if consensus is shown, the WMF could either agree, or we need more dialog to get on the same page. If partners disagree on something, both sides probably have good reasons.

Is WMF's idea of "expected community feedback" merely bugreports and feature requests? Or does it contemplate the possibility of feedback opposed to the project itself?

• The aim to provide a consistent user experience across Wikimedia projects should always prevail.

Limiting support to a single configuration can be an absolutely legitimate technical decision, and it absolutely is a valid basis to override minority resistance at the end. However if the WMF follows the "path of least resistance" with initial deployment to small wikis (some of which never seem to object to anything), then hits consensus community rejection, imposing a single configuration would mean rolling back the initial deployment to the first 10% percent of editors. "Consistency" isn't a legitimate basis to force deployment against-consensus to the remaining 90%.

• A community blocking the deployment of a product as planned is expected to provide proof of community consensus and actionable blockers

Proof of consensus: absolutely agreed.

Actionable blockers: Objection. Setting the system font to ComicSans would have a serious negative impact on reader's perception of Wikipedia as a credible source of information. There is no actionable blocker on that. No list of bugreports or feature enhancements will fix it. Can you please confirm that an objection can be both valid and non-actionable? A legitimate issue can't be disregarded simply because it is non-fixable.

• There are other ideas about systemic bias in community discussions

That would be an interesting topic. I would be happy to contribute some ideas about systematic bias at WMF as well. Grin.

In any case, we share the same mission. We're all on the same team. We will reach much better outcomes if we work as partners. Alsee (talk) 23:34, 16 November 2015 (UTC)

It looks like we start needing an own page to draft and discuss these steps in more detail. Your opinion is welcomed at Talk:WMF product development process#h-Subpages_for_each_stage?-2015-11-12T09:15:00.000Z.

Is WMF's idea of "expected community feedback" merely bugreports and feature requests? Or does it contemplate the possibility of feedback opposed to the project itself?

In my opinion, opposition to an entire project should be possible, but ideally it would appear at the Concept / Plan phases, before spending lots of donors money developing a project in a certain direction. It should come with constructive arguments and actionable proposals, the same expectations put on any stakeholder during the product development process.

The constructive arguments should reply questions like Do we agree on the problem but not on the planned solution? Do we actually disagree on the problem itself? Is the plan correct but not its prioritization because something else should be addressed first?

Actionable proposals should provide a way forward to resolve the problem as good software partners. From the WMF point of view, if one community is objecting radically to a new product with a solid argumentation, it is probable that at least some of that argumentation will be echoed by other communities and by other stakeholders at the WMF. In that case, changes to the plan should we logical and welcomed.

Can you please confirm that an objection can be both valid and non-actionable? A legitimate issue can't be disregarded simply because it is non-fixable.

If a community finds that Comic Sans in a new product is a horrible choice, then the blocker is "Remove Comic Sans from Product X". I believe all valid and legitimate blockers can be made actionable? The point is to avoid the risk of features being blocked with subjective or collateral arguments without any practical way forward, leaving the developer team with no margin to discuss or improve anything.

systematic bias at WMF

Sure, nobody is exempt of the risk of systemic and systematic bias. If trust, common sense, and healthy partnership prevails, then telling each other how to be a better partner becomes a lot easier. We need to allow to ourselves flexibility and common understanding to put this process in practice progressively, before having defined and demonstrated every aspect for it, improving it whenever something didn't go as expected. Qgil-WMF (talk) 14:46, 18 November 2015 (UTC)

Between "Concept" and "Plan" - Prioritize?

In a world of infinite resources, once the WMF decided that a specific proposal was a good idea, it would move that concept into the planning stage, and continue on through to Release and Maintenance. But, of course, we don't live in that world. So, the question: what are the factors that go into deciding what moves forward, and what is deferred?

Related to the question of who decides is estimating the cost of implementation. If two proposals have about the same level of expected benefits, but one will cost two or three times more to implement, then the WMF should probably implement the lower-cost proposal, and possibly defer the higher-cost proposal. (Note: the Concept phase now includes "technical debt", which is the eventual consequences of any system design, software architecture or software development within a codebase. That's not at all the same thing as cost: a proposal might require a large number of staff-hours to implement, and yet be relatively easy to maintain, and affect few other parts of MediaWiki code.

In short, if the WMF truly wants to be transparent, there should be a visible decision process. That doesn't mean that the community has to vote (or !vote) on most (or even any) products being developed, but it does mean that decision-making should be more than a group of developers saying "Okay, what do we want to work on next?", with regard to feature development.

What might this look like, on the chart? Perhaps a Prioritize phase:

• List of benefits
• Estimate of costs
• Synergies with other product development
• Setting priority: Very high to very low

Obviously, "benefits" typically would not be described in dollar terms (WMF is not a business). But WMF should make clear who it sees as benefiting: novice editors, editors using specialized tools, wikignomes, prospective editors, etc. That in turn encourages a discussion about the relative merits of different proposed products. John Broughton (talk) 17:58, 8 November 2015 (UTC)

Thank you for this., @John Broughton :) I agree that we can say that the prioritization question is one that has been asked frequently and one that has potential to be included within this process, though it may also make sense to have the prioritization/decision be separate item (at least, as far as "decision making" is concerned.

As far as the bullet points you mention, I think those are a great start, and want to draw @WMoran (WMF) and @JAufrecht (WMF)'s attention to them.

I don't know that this process page is the right place to hash out prioritization (what do you think?), but it's definitely worth discussing. It's been raised several times in the past (the link is only one example), but I don't find a dedicated page here on MW. Before making one more page (there are, er, so many), I'll poke around. It would be great to have a clearly defined description of how projects are decided on and how they are prioritized. Rdicerb (WMF) (talk) 00:39, 10 November 2015 (UTC)

I agree that the prioritization step may be the dominant part of this lifecycle for the Foundation, and it's the least well-understood step. I understand the "<---- Analysis & Feedback" back-arrows to encompass that, but in terms of pixel ink, they are much smaller now than they should be, and they don't have specific Task names. One big problem here is that the real work of prioritization is saying "no" or "not yet" or "not much" or "nobody else agrees with you" to most things (the visual editor backlog is at least three to five years long, and more likely decades). There are more incentives for each individual person involved in the process say or believe "yes" or "maybe", even though this leads to an unrealistic plan that fails everybody. So this is a tough step to realistically model. What are the Tasks involved? "Estimate Costs and Benefits" could be one, but "Setting Priority" is circular; set priority how? What examples are there in recent Foundation/community history of doing this work of prioritization well that we can use as models? JAufrecht (WMF) (talk) 05:57, 10 November 2015 (UTC)

A known problem in organizations is the risk of rushing to Prioritize even before Concept (top management decides X is very important, team A is tasked to work on X from scratch). Even before getting in the discussion about how to prioritize publicly and how to involve our communities, it would be useful to make a conscious decision about Prioritize being part of this product development process (or not) and requiring at least a first iteration of Concept (or not).

Note that Concept identifies Community Members as stakeholders, and therefore any concept willing to become a product should be documented publicly. Having all the concepts waiting for prioritization publicly available and well organized would be helpful in many ways:

• knowing all the candidates for prioritization is a prerequisite to prioritize well and to explain/understand the decisions made
• it is also a prerequisite to involve our communities in any sensible exercise of prioritization
• beyond the WMF, other organizations (i.e. chapters, 3rd party developers) and individuals (i.e. volunteers in a hackathon) could submit their own concepts and/or prioritize existing concepts with their own resources

In any case, I agree that clarifying Concept and Prioritize is essential. We need to define a clear starting point for new projects under this new WMF product development process. Adapting all the ongoing projects to the new process will take significant effort, but we can start containing that effort by channeling new concepts through the new process. Qgil-WMF (talk) 09:34, 10 November 2015 (UTC)

"beyond the WMF, other organizations (i.e. chapters, 3rd party developers) and individuals (i.e. volunteers in a hackathon) could submit their own concepts and/or prioritize existing concepts with their own resources"

I agree this is very important, and I think it bears some explicit calling-out and support in how the process is defined. Wikimedia Foundation Engineering a) isn't a monolith and b) isn't the only game in town. We as members of a global movement need to be able to talk about, reason about, prioritize, and help assign some subset of our collective resources to, important issues that might not be picked up by any individual 'vertical' in WMF Engineering.

There are other organizations like the chapters which have varying resources and may have different specific interests, and WMF itself runs grants. I would love it if we had more organized collective work on non-Wikimedia uses of MediaWiki as well -- we basically ignore those use cases inside WMF engineering, while we reap the benefit of having hired a bunch of engineers out of the open source community around MediaWiki.

I'd possibly be happier with a movement-wide process that WMF enthusiastically participates in, rather than a WMF-focused one. brooke (talk) 18:24, 11 November 2015 (UTC)

Agreed. That is where all of our energy should be and it is absolutely worth the effort. This is just the high-level and it will never be a perfect fully filled in explanation of the steps within each phase. This is more of a guideline to get things moving or as @Rdicerb (WMF) pointed out could be a checklist. The tasks or path for ideas from concept to plan is much better left to a specific page around prioritization, and historical success or fails seem like a great way to learn. Interested to see what you find there @Rdicerb (WMF). The whole high level table could be linked up like this to specific focus areas, that way while we evolve the parts the whole can also evolve. I like some of the elements of both the Community Wishlist Survey and the slightly different style on the Developer Summit phabricator board. @Qgil-WMF are you suggesting then to create a new column like I believe @John Broughton is suggesting or a distinct line as a decision point? @JAufrecht (WMF) did you see this specific round as part of the SPDPP and perhaps a task on the table with an analysis loop on top of plan as well? WMoran (WMF) (talk) 23:24, 10 November 2015 (UTC)

@WMoran (WMF), I have a strong opinion about "Concept exists and has been shared with Community Members" as a requirement to enter Prioritization, but I don't have any good ideas about how Prioritization should be reflected in the draft and the diagram. Qgil-WMF (talk) 23:38, 10 November 2015 (UTC)

There are wishlists and surveys a-plenty. I'm primarily interested in the methodology by which projects and products are prioritized.

Like @John Broughton, I also find a couple of bullet points relevant to how the movement would prioritize developing one feature over another, sooner over another.

Things like:

• Impact, or severity of the problem (We should be prioritizing our biggest problems or opportunities)
  • Data
    • Security/Risk
    • User request
• Potential cost (people + time = money)
• Blockers (sometimes in order to accomplish one thing, another thing must be handled first. SUL is a good example - now that usernames are standard through the projects, cross-wiki notifications and global accounts can be created)

Obviously the overall themes like alignment with movement values and long-term goals also apply. Rdicerb (WMF) (talk) 21:49, 11 November 2015 (UTC)

The basic way I imagine this implemented within SPDPP is that someone at the Foundation who wants to start a project picks from a few different bundles of Milestones, which are derived from past projects (successes and failures). Each bundle would start with a set of Milestones forming a pathway through conceptualization, prioritization, and planning. Each Milestone is a checklist, so all of these bundles would have checkpoints for stuff like "most recent attempt to do something similar identified and 'lessons learned' documentation recovered" and "previous normative community discussion on this topic identified, if any". One bundle might be the path for starting something off the community wishlist; another might be for an apparent greenfield (never been tried before) project, and so on. Each would have multiple "go back and try again" points. We might also want some metrics around what we think the funnel should look like: should the Foundation proceed into full funding and execution for 100% of initiated projects? 50%? 10%? JAufrecht (WMF) (talk) 22:26, 11 November 2015 (UTC)

@JAufrecht (WMF) - following up on this - this section is about prioritization, and I'm trying to correlate your most recent statement with this topic. Could you clarify how the SPDPP connects to Prioritization process being discussed here? Or are you saying that Prioritization needs to have a bundle of actions as a Milestone? Rdicerb (WMF) (talk) 23:29, 17 November 2015 (UTC)

I think Prioritization has to be first-level, not in a bundle of Milestones. The way Prioritization connects to SPDPP is that there should be a few places in the SPDPP process where Prioritization happens, but SPDPP so far just has that as a blank arrow coming in. SPDPP should be able to say how often and and what level of prioritization needs to happen to work well with the Foundation's existing processes, but should be pretty neutral to exactly how that happens. JAufrecht (WMF) (talk) 00:21, 18 November 2015 (UTC)

@JAufrecht (WMF), the key point of this thread is to define the conditions for a Concept to become a priority and start receiving the attention and resources to define a Plan, request resources, become part of quarterly goals and annual plans...

Before being prioritized, teams should have a green field for agile experimentation without having to compromise with process requirements, strategy alignment, community consensus, project documentation... However, in order to be prioritized as a result of an informed decision, these teams will need to do some homework.

The filter proposed tries to avoid that projects are prioritized without the minimum requirements defined, out of process. If we agree on this principle, then it should be reflected in the product development process. I'm not sure whether Prioritize should be an own column between Concept and Plan. Perhaps it should be a set of criteria to enter Build? Qgil-WMF (talk) 13:35, 18 November 2015 (UTC)

No matter how, we need a checklist for project concepts to be considered for prioritization. I just created phab:T120070 to track this action. Qgil-WMF (talk) 09:13, 2 December 2015 (UTC)

Subpages for each stage?

Currently only specialists can jump comfortably from the necessary simplicity of the WMF product development process to the very detailed and specialized documentation at WMF product development process/Proposal. What about creating subpages for every stage? These pages would provide enough information for every stakeholder involved, and especially for our communities and volunteer developers, who don't need to dig deep into WMF's agile methodologies to contribute efficiently to the process. Qgil-WMF (talk) 09:15, 12 November 2015 (UTC)

@Rdicerb (WMF), @WMoran (WMF), any thoughts? Objections? Qgil-WMF (talk) 13:15, 18 November 2015 (UTC)

Following on the feedback from @Salix alba and others, it would be useful to have an additional page focusing on the participation of our communities, explaining the points where they are expected to participate and how, linking to the appropriate pages or sections of the product development process with more details.

For the average editor, the expression "product development process" is confusing enough, leave alone the whole description of the process. We can present a community-centric view of this process highlighting when and how they are expected to get involved, promoting Wikimedia community terminology and trying to minimize software development terminology. Qgil-WMF (talk) 13:44, 18 November 2015 (UTC)

I think it is a great idea. What about a simple FAQ per stage? How do I submit a feature? How do I provide feedback? How can I participate? Would that be better? WMoran (WMF) (talk) 21:02, 18 November 2015 (UTC)

OK, one page for every stage.

A FAQ in the stage pages focusing on community members would give the impression that this documentation is only for them. The docs are the same for everybody and need to speak to everybody. I still think that a subpage "WMF product development process/For Communities" might be a better approach. Qgil-WMF (talk) 23:43, 18 November 2015 (UTC)

I have a page that is retooling, and there was a Design Research workshop this morning that took staff through a spreadsheet that describes each teams actions during the stages of development. That could be another way of doing it.

I do think that a goal at the moment is to update the table, but my table/markup skills are lacking. I did upload the chart that was discussed to Commons the other day - not sure if that's helpful (I feel like that clarification may be needed prior. Rdicerb (WMF) (talk) 23:38, 20 November 2015 (UTC)

@Rdicerb (WMF), I think you are right. Let's keep the overview simple, focusing only on the key aspects of the process. Then different audiences may have their own related pages with all the details affecting to them directly. Modular Milestone-driven Development is a good example of a page geared toward Engineering teams more than our communities. We should have a page targeted to our communities. Should this page be created or is there an existing page that could be reused as starting point? Qgil-WMF (talk) 09:20, 2 December 2015 (UTC)

We discussed this problem in a meeting with Wes, Rachel, Keegan, Abbey and Jonathan, and we agreed on the following:

• Each stage will have a subpage (coming soon) with a general intro and sections specific for each audience (i.e. Communities, Performance, Security...)
• Each audience will have a subpage as well, so readers caring mainly about Communities, Performance, Security... have a clear landing page.
• In order to avoid duplicated work and mismatches, we will transclude the sections from the stage subpages into audience subpages, using labelled section transclusion.

Let's see how it goes. Qgil-WMF (talk) 09:09, 2 February 2016 (UTC)

What binds process to reality

What I'm missing is: how are we measuring progress and quality, what do we do with interruptions, how do we deal with the fact that we are understaffed, etc etc. How do we deal with the fact that each product is 10000 pieces all being in another phase of this defined process ?

And how do we then COUPLE these events to the development process. Because otherwise it will be very difficult to 'recognize' these phases and then they aren't really useful.

Are we gonna label each ticket with a phase ? Will we have 'process' workboards per product with these phases ? (And then the team boards being a the agile board ?). I'm just looking for practical implementations that will help me recognize this process. —TheDJ (Not WMF) (talk • contribs) 10:05, 4 December 2015 (UTC)

Thank you for this, @TheDJ. Very relevant points, and ones that I've been thinking about. @WMoran (WMF) and I will be in an office hours on Thursday, and I think your questions, and others, would be good things to address. Rdicerb (WMF) (talk) 18:26, 7 December 2015 (UTC)

I also agree these are all valid points. This again is a high level draft that is being informed and evolved through the work with  SPDPP (Software Product Development Process Proposal). That more detailed process will have more concrete examples. WMoran (WMF) (talk) 17:21, 17 December 2015 (UTC)

What about something like this:

• Every software project going through this process has a project page in mediawiki.org
• Every project page should have a subpage for every stage that project is going through, containing the checklist for that stage, any additional notes from the team, and a discussion page. Small projects and big feature may opt for a single subpage for all stages.
• If a project just started, only an /Understand subpage/section is expected. If a project is about to enter the Release stage, all the previous subpages/sections will be expected.
• The main project pages has an infobox with the standard project information, including the status (OK, IN PROGRESS...) for the relevant stages.

I don't think we need to bring much overhead to Phabricator (as in labelling all tickets). However, at least for big projects and complex features it might be useful to have tasks like "Product X to enter Release stage" in order to identify the main blocking tasks. To be discussed. Qgil-WMF (talk) 09:04, 2 February 2016 (UTC)

How to publish and track project concepts entering prioritization?

In wikitech-l, Ideas for tech-related IdeaLab Campaigns? derives into a discussion about whether initial concepts should be published in IdeaLabs or Phabricator.

There are different places where a project concept could be sensibly published (mediawiki.org, IdeaLab, Phabricator) and there are different processes for prioritization (WMF quarterly goals, IEG grants process, Possible-Tech-Projects, Communtiy Tech and TCB-Team wishlists...). Even if a central location for publishing project concepts might sound a good idea "simpler for those proposing new projects and for those discussing them", in practice there is a risk of disrupting process that are working decently well today and also a risk of alienating the participants of this process. Not a simple question.

However, it seems that we would all benefit from a central log that people could watch to learn about new candidates for prioritization and check to know the prioritization status of existing proposals. Nowadays it takes almost a full time job to follow everything. No idea how this central place would look like or how it should be build, but we could start with a sensible requirement for the proposals pushed by WMF teams...? Qgil-WMF (talk) 10:56, 8 December 2015 (UTC)

Waterfall or agile/scrum

As a documentation starter:

  - The phab:T124022 ticket has been announced by Quim Gil on wikimedia-l

  - That post links to WMF product development process

The WMF product development process very much looks like a waterfall process or method.  Is that intentional or inevatible? Isn't that at odds with agile / scrum software development? Ad Huikeshoven (talk) 16:26, 21 January 2016 (UTC)

Thanks for your question. Had you already read the topic "Sounds like waterfall"? Elitre (WMF) (talk) 16:48, 21 January 2016 (UTC)

Quim has pointed to that long discussion on the phab ticket discussion. I've seen it :) Ad Huikeshoven (talk) 13:58, 23 January 2016 (UTC)

I wonder whether agile is too – I'm not sure how to explain this. With a large project, in agile, you're really doing everything at once. When you are releasing part 1, you're also building part 2, designing part 3, planning part 4 – and also re-building part 1, re-designing part 1, re-planning how to integrate the changes from part 1 into your current plans for parts 2, 3, and 4, etc.

And it's kind of hard to explain this "do everything at once" as a "process" to someone who is thinking in terms of, say, the process used by a factory line or the process used in a court of law.

Would it make more sense to present it as several interconnected, simultaneous focus areas, rather than anything in a line? (For example, six blobs with lines between them rather than six lists in a row.) Whatamidoing (WMF) (talk) 02:34, 28 January 2016 (UTC)

At least when it comes to the conversations with our communities, Understand and Plan come before Release and Maintain, and not vice versa. There is a sequence. The purpose of the checkpoints for every stage is to check whether a project is fit enough for a next stage.

Another example. Once a project is in Develop, they can go back to Understand, Plan, etc as often as they wish, but still they cannot jump into Release without going through the related checkpoint. Again, there is a sequence for new products and new big features requiring community checks. Qgil-WMF (talk) 08:44, 2 February 2016 (UTC)

We probably agree, but your brief explanation here sounds rather "waterfall-y". Perhaps VisualEditor is an example: "Release" and "Maintain" citations came before "Understand" and "Plan" table editing. Therefore, "Release and Maintain" (something) actually did come before "Understand" and "Plan" (something else). From the communities' perspectives, all of these things happened at the same time. Whatamidoing (WMF) (talk) 16:37, 2 February 2016 (UTC)

We probably agree indeed. From the point of view of the communities, they want to know about features like "VE citations" or "VE table editing", and some people might be interested in A but not in B. Each feature follows more or less that sequence (at least ideally).

I think this topic will be less of a problem when we move from one draft in a wiki page to some real action. Qgil-WMF (talk) 21:57, 2 February 2016 (UTC)

I think it would be helpful for this discussion to have clear definitions into what is waterfall and what is agile from a person on TPG.

Also, we have drawn this process as a line, and really it is more of a circle. There are places in the process that do not neatly check off boxes and then move to the next stage without going back and forth for a while. For example when building citations in VE, there were several concepts built and tested, and iterated, and tested (several circles, back and forths between parts of the process) until people were able to use it (it was usability tested and people were able to accomplish creating a citation without difficulty, considering all the details and edge cases), there were no bugs (this was a combo of QA and usability testing - which are different), then, it was released to production.

It might help to explain this process in a more visually representative way, showing that it really is a circle (we may some day decide to add functionality or remove functionality to the product called citations) that sometimes has circles (iterative testing until ready for release) within each part of the process. Drawn as a straight line with not back and forth between parts of the process (iteration), it communicates a less iterative process that does not reflect the reality of product development inclusive of the necessary activities to create usable, bug free, quality software. I think it is important to call out the back and forth between stages with arrows pointing backwards in the, at this point, unidirectional flow. Perhaps a visual designer could help us with creating a more representative visual to describe this concept. ARipstra (WMF) (talk) 11:46, 25 February 2016 (UTC)

End of life

This process should include end-of-life requirements: If you stop maintaining an extension, you should post a plan for a graceful removal/information about when support stopped (e.g., "last tested on this version of MediaWiki"). Whatamidoing (WMF) (talk) 17:26, 3 February 2016 (UTC)

Good point. I guess this would belong to the Maintain stage. Qgil-WMF (talk) 14:09, 4 February 2016 (UTC)

It should probably be its own stage. Everything, even something as durable as Microsoft Windows, is likely to have an End of Life stage some day. Whatamidoing (WMF) (talk) 16:46, 4 February 2016 (UTC)

It is a good point. A Retire stage could be a nice addition. What types of tasks and stakeholders would be involved? WMoran (WMF) (talk) 20:51, 4 February 2016 (UTC)

For tasks, I suppose you'd need to figure out any dependencies (both directions: Did you add something else to support this product, that could be removed with it? Is anything else relying upon your code?) and make suitable announcements and updates to documentation.

For stakeholders, I would guess this short list:

Product Manager – to make the decision

Community Liaisons – to find out whether anyone at the WMF wikis is using it

Operations – to deal with practical stuff about removal Whatamidoing (WMF) (talk) 00:09, 7 February 2016 (UTC)

Just FYI, I have added this discussion as blocker of Phab:T125806 (Consolidate the WMF product development process overview). Qgil-WMF (talk) 09:22, 11 February 2016 (UTC)

An example of the need for quality assurance: Defective WMF harassment survey

(Copied from Jimbo's talk page. My comments are below).

The mystery of the obviously incorrect "revenge porn" results of the WMF Harassment Survey has been solved on Wikipediocracy by Belgian poster Drijfzand... Basically, this survey of 3,845 Wikipedians across a range of WMF projects (45% of whom were from En-WP) generated 2,495 responses to a question asking whether they personally experienced harassment. Of these, 38% (about 948 people) said yes. (pg. 15). However, on page 17, in what is purported to be a breakdown of the forms of harassment experienced by these editors, an astounding 61% (about 578 people) are said to have claimed to be victims of "revenge porn." This, to anyone who ponders the number for more than 6 seconds, appears patently absurd — bearing in mind that the survey respondents were about 88% male and that the great majority of Wikipedians maintain some degree of anonymity.

Drijfzand observed that the number of responses for doxxing, revenge porn, hacking, impersonation, and threats of violence all fell within a range of 5% — which she or he said "simply can't happen." I theorized that the problem was a software glitch and Drijfzand identified the problem as a set of defective sliders in the survey form which refused to accept a value of 0, a bug identified by Burninthruthesky on November 3 and which was apparently remedied on November 4. LINK. Unfortunately, the survey was not launched on En-WP until Day 5 (to allow more responses from smaller Wikis so as to reduce the weight of the large projects, see pg. 2), meaning that bad data was generated on some projects for nearly a week. Whereas the survey should have been aborted and restarted, it apparently was not, and so the data presented on page 17 (and any conclusions derived therefrom) is a case of Garbage-In-Garbage-Out.

Once again: a failure to adequately beta-test software is evident. There is one saving grace, and that is we have a very good snapshot of the magnitude of the gender gap based on survey respondents (a ratio 88:12 for those who indicated a gender, with some 7 % of survey participants declining to respond). Assuming a heavier-than-average percentage of women than men in the "decline to respond group," this means we are probably in the ballpark of 86:14 or 85:15. There is also, for the first time ever as far as I am aware, a decent survey of age of Wikipedians. Your takeaway numbers: 35% of respondents (and presumably Wikipedians in general) are age 45 or over; only 24% are under the age of 25. All the fresh faces, many on travel grants, at Wikimania are deceiving — it appears that the median age of Wikipedians is right around 31 years old, give or take. So the expenditure on the harassment survey wasn't a total loss even if it failed at its intended mission (at least in part) due to bad software (leaving aside the very real question of sketchy survey design).

--Originally posted by User:Carrite at User talk:Jimbo Wales on the English Wikipedia at at 19:50, 3 February 2016 (UTC), reposted here by Guy Macon

-------------------------------------------------------------------------------

I would like to discuss the comment "Once again: a failure to adequately beta-test software is evident" in the above.

The WMF has many roles, and one of those roles is "software developer". One of our ongoing problems is the low quality of the software we develop. I would note that this is almost certainly not the fault of the individual developers or the managers one or two levels above them, but rather an institutional problem that flows down from top management. I would also note that top management almost certainly do not realize that they are the root cause of our low quality software.

The Wikipedia community has some extremely skilled project managers and software developers, but we have no way of helping the WMF to address this problem. I have personally tried every avenue that anyone suggested to try to get a technical proposal considered, (details available on request, but right now I am addressing the larger problem) but have been stonewalled. There should have been a way for me to get an answer, even if the answer was "no".

I would very much like to be able to report in a few months time that this has been solved and that the lines of communication are opening. Let's talk about how to make that happen. --~~~~ Guy Macon (talk) 20:28, 3 February 2016 (UTC)

@Guy Macon, testing software is a crucial activity, but I wonder which specific recommendations from this message we can extract for the product development process being discussed here. The survey mentioned was handled with Qualtrics, which is a product that we have not developed. Qgil-WMF (talk) 14:02, 4 February 2016 (UTC)

Are you implying that using an external product relieves the WMF of the responsibility of doing some basic functionality testing of the software that handles a survey before releasing said software upon the Wikipedia community?

I would also note that the actual question I asked ("The Wikipedia community has some extremely skilled project managers and software developers, but we have no way of helping the WMF to address this problem. .. I would very much like to be able to report in a few months time that this has been solved and that the lines of communication are opening. Let's talk about how to make that happen.") has, as is our tradition, gone completely unanswered. Guy Macon (talk) 18:09, 6 February 2016 (UTC)

I think that everyone agrees that QA testing is good. However:

1. The software that was being used was not written by or supported by the WMF,
2. the software itself was functioning correctly,
3. the survey isn't a "product" and isn't being "developed", and
4. the survey was not produced by or associated with any Product team.

Consequently, it's unclear to me why you think the Product department should address this, or how this error should affect the development of software products by the WMF Product department (=the subject of this page).

Or, to put it in terms that may be more familiar with Wikipedia editors: you have posted your comments at the {{wrong venue}}. (If you need help finding the right venue, then leave a note on my talk page.) Whatamidoing (WMF) (talk) 00:02, 7 February 2016 (UTC)

What's a "product"

This is a product development process, and that means that we need a good, shared understanding of what a product is.

Below is an alphabetized list of things that might be "products" for the purpose of this process; I believe that most of them are. Are there any items in this list that you think should not be covered by this process? Are there any items in this list that you suspect would receive little or no engagement from typical contributors? Are there any products (on or off this list) that you would recommend a different process for?

• Article Feedback Tool
• Compact Personal Bar
• ContentTranslation
• Echo (Notifications)
• Education Program
• Engagement metrics
• FlaggedRevisions and PendingChanges
• Flow
• Forced HTTPS
• Gather
• GettingStarted
• GuidedTour
• HHVM
• Image filter
• LiquidThreads
• MediaViewer
• MobileApp
• MobileFrontend
• MoodBar
• PageImages
• PageTriage
• Parsoid
• QuickSurveys
• RelatedArticles
• Scribunto
• SUL
• SUL finalisation
• TextExtracts
• Thanks
• Typography refresh
• UniversalLanguageSelector
• UserMerge
• VisualEditor
• WikiGrok
• WikiLove
• Winter Whatamidoing (WMF) (talk) 21:01, 11 February 2016 (UTC)

I'm not sure that SUL finalisation counts as a "product" for the purpose of this proposal. There's no possibility of a "maintenance phase". It was a one-time event, rather than a product (although some software and scripts were created to automate some of the tasks).

What do you think? Whatamidoing (WMF) (talk) 18:24, 16 February 2016 (UTC)

I know it's not this simple, but maybe products are things people touch our interface with, while the other things are infrastructure?

• HHVM - infrastructure
• Echo - product
• SUL implementation - product
• SUL as a thing moving forward - infrastructure CKoerner (WMF) (talk) 19:48, 17 February 2016 (UTC)

"Any tech change which is bound to impact wikis users' experience"? Elitre (WMF) (talk) 13:01, 25 February 2016 (UTC)

"Any tech change that affects a user's experience" means, well, "any tech change".

@CKoerner (WMF), I think that major infrastructure projects are meant to be covered. Wikipedia editors might not care about infrastructure, but it should presumably follow this process anyway (just with a reasonable expectation that the core volunteer communities won't care very much about most infrastructure changes). Whatamidoing (WMF) (talk) 06:44, 1 March 2016 (UTC)

I'm pretty sure there may be changes which people would have no way to detect unless we told them. That a certain feature is now written in a programming language rather than in another may be the easiest example. Elitre (WMF) (talk) 15:27, 1 March 2016 (UTC)

I think it is useful to offer a common process for all Product and Technology projects, where each project takes the checklist points that affect to them and skips the rest. If a project doesn't have any impact on end users, only on developers, then they could skip the part related to involving end users and editors, but they still would need to go through other checks, like architecture, performance, deployment plan... Qgil-WMF (talk) 09:15, 2 March 2016 (UTC)

A process that allows teams to completely skip some points may be sensible, but it tends to make the process unpredictable. Also, what if I decide that (for example) performance is not relevant for my project, so I skip that step, but someone who works in that area later decides that I was wrong? How can I be certain that it's safe/appropriate to skip a potential stakeholder without actually contacting that stakeholder? Whatamidoing (WMF) (talk) 05:28, 3 March 2016 (UTC)

Every stakeholder should define the conditions of their requirements in their checklists. For instance, Community Liaison's requirements (the ones we put on behalf of the communities) should be reflected at WMF product development process/Communities. If a condition is ambiguous (i.e. one could say "Relevant stakeholders have been invited" leaves too much room to interpretation) then we can edit the checklists and make them more precise.

Also, reviews are expected along the process. If a project didn't have a performance check but then later in the process a stakeholder has a problem with this (or the tests results show performance issues), they will need to get that performance check, or at least have a discussion whether they should do that check or not. Qgil-WMF (talk) 06:42, 3 March 2016 (UTC)

If we create a flexible process, then we create a process in which you can take reasonable steps and I can claim later that you acted badly.

If we create a "precise checklist", then we create a process that overburdens most products in the hope of being barely adequate for the biggest.

I am not certain that having a single process is necessarily appropriate. What's needed for, say, PageTriage, is really quite different in both quality and quantity from what's needed for, say, MobileApp, which in turn is different from ULS. At an abstract level, of the sort that you'd find in the first chapter of a beginner's textbook on software development, there are certainly some commonalities, but the application is quite different. PageTriage needed to make a dozen highly communicative English Wikipedians happy. MobileApp needs to make thousands of non-editors happy. ULS needs to avoid making the wikis unreadable for readers and editors alike. Whatamidoing (WMF) (talk) 16:45, 16 March 2016 (UTC)

Calendar request

I use English Wikipedia. Is there any calendar logging when features are added or removed to the "beta features" user menu?

I would like to have access to a record of when features are added, when they are removed, and the circumstances under which they are removed. I was wondering how many beta features become standard offerings, and how many are ended without being integrated.

Thanks. Blue Rasberry (talk) 15:31, 26 February 2016 (UTC)

No, sorry, I don't think such a calendar exists. Jdforrester (WMF) (talk) 23:05, 26 February 2016 (UTC)

git log -L is your friend: P2684 Tgr (WMF) (talk) 03:03, 29 February 2016 (UTC)

Thanks - this is at the edge of my ability to understand but nice to know that this is what is available. I appreciate it. Blue Rasberry (talk) 19:00, 2 March 2016 (UTC)

Technical Collaboration Guideline

In order to avoid confusion and excessive dependencies, the recommendations on collaboration between Product teams and communities are being developed at Technical Collaboration Guideline. Qgil-WMF (talk) 11:39, 20 April 2016 (UTC)

Connecting with other initiatives

There does not seem to have been much activity on this topic in recent months. Could somebody with authority to do so please make sure the page is still up to date – it would be especially helpful to give more detail of the status of this process (mandatory, preferred, advisory, optional, draft, ...), scope (universal, default, specific, limited, ...) and the relationship between this and various other proposals such as Design and development principles and Technical Collaboration Guideline/Community decisions Rogol Domedonfors (talk) 21:11, 29 October 2016 (UTC)

Regarding the status of this process, the top of the page says it's a draft. Focus is currently on the Technical Collaboration Guideline (TCG). For the relation to the TCG, see Talk:WMF product development process#h-Technical_Collaboration_Guideline-2016-04-20T11:39:00.000Z. Malyacko (talk) 22:00, 29 October 2016 (UTC)

Thank you, I could see that. My question is, what is it supposed to be a draft of? Is it a draft of a mandatory process, a draft of a preferred process, a draft of an advisory process, a draft of an optional process, a draft of a draft framework or what? Is it a working draft of a process that will come into force (what ever force it has) as soon as it is finished, or is a draft of a draft framework to be adapted ad hoc by multiple developers in any given project scenario, or perhaps it is the intended end product a draft submission to some higher authority who will further refine it or enact it as mandatory, preferred, etc? Or is it a draft of something which is not yet currently decided but when finished there will be a further discussion to decide what sort of thing – mandatory, preferred, advisory, optional – enactment, framrework, submission – it is?

I can see the topic currently immeditely below this one too, thanks. My request was for "more" details, not to be told to read again that single sentence – more details of the relationship between this and other proposals such as Design and development principles and Technical Collaboration Guideline/Community decisions, the first of which you did not address at all.

You say the focus is currently on another piece of work. Will the fous return to this process at some stage for further work, or has it served its purpose, whatever that is, or has it failed and been quietly abandoned? Please tell us, so we know where and how to collaborate with you.

It seems strange that this piece of work has been kicked off without anyone able to immediately answer these questions. Presumably everyone was clear what they were trying to do before they started the work? Anyway, I ask not out of idle curioisity nor to test your undertstanding, nor to point the finger at yet another piece of WMF work that seems to be going nowhere, but because if you publish more information about what you are trying to do with this, you are likely to get more and better collaboration from the other stakeholders, both in the drafting and in the operation of this process. And that is what you want, isn't it? Rogol Domedonfors (talk) 07:47, 30 October 2016 (UTC)

Status?

What's the status of this initiative? Is it already being applied?

At any rate, we would like to add Performance to the Plan, Develop and Release stages. Fixing Performance after the fact in the Maintenance phase isn't a sustainable model. GDubuc (WMF) (talk) 10:39, 13 December 2016 (UTC)

I'd say focus is on the Technical Collaboration Guideline currently. AKlapper (WMF) (talk) 11:02, 13 December 2016 (UTC)