User:AKlapper (WMF)/Sandbox

T101686, T78768 - Reduce CR queues and time to review; prioritizing CR of patches submitted by volunteers
See User:AKlapper_(WMF)/Code_Review.

T113707 - Service Level Agreement (SLA) of WMF staff on time to review for CR? (not specific to "first patch" though)
This is blocked on sorting out "Unclear responsiblities" in T101686, T78768
 * ❌ Requires clear (single person, and/or gerrit-wrangler who needs institutional knowledge and have people in each team to delegate to?) deterministic responsibilities who owns what -- how do we decide who the owner is? Bus factor? Brian proposes "the review-owner person should do one of these actions in no more then 1 week time since the patch is submitted."
 * ❌ A query of patches without review within a week, automated message to wikitech-l or so?
 * but "Difficult to enforce that volunteers must follow some deadline."

T113706 - Cut-off date to abandon patches in CR

 * Clarify first which problem this potential solution should target
 * Nothing found in literature. Andre blogged: https://blogs.gnome.org/aklapper/2015/12/01/volunteer-contributions/
 * Very similar to a lowest priority vs WONTFIX/declined discussion in Maniphest/Bugzilla?

Stuff we know

 * "If a project does not make a good first impression, newcomers may wait a long time before giving it a second chance."
 * 3 different profiles of newcomers: "Newcomers can be novice developers who are starting their career, people who are experienced developers from industry but are not used to OSS projects, or people who are migrating from other OSS projects."
 * Entry barriers are not always bad: Some lead to improved contributions in the long run.
 * "Successful FOSS projects grow their communities outward to drive contribution to the core project. To build that community, a project needs to develop three onramps for software users, developers, and contributors, and ultimately commercial contributors."
 * "Sometimes, a contribution barrier cannot be lowered, but instead, the FLOSS project may shift the contribution barrier to the inside of the community's onion model."
 * Dialogs shown to first time contributors trigger additional contributions.
 * T73357: Gerrit: Add Welcomer bot to Gerrit - ❌ Does something similar exist for Differential?
 * AFAIK no study exists to compare successful and unsuccessful contributors specifically.
 * Annoying little bugs: Labuschagne and Holmes looked at Mozilla's 'Good First Bugs' (GFB) and 'Mentored Bugs' (MeB; something that Wikimedia does not have) programs taking quantitative data from Bugzilla and a qualitative survey with 11 developers with at least 10 contributions who started in one of these programs.
 * "newcomers who make 10 contributions take an average of 203 days to complete this work"
 * Interviewees appreciated mentorship, list a variety of starting points (Bugsahoy, Bugzilla search, IRC, Codefirefox.com), and underestimated the required effort to contribute compared to size and impact of the contribution. ❌ Advertise https://www.whatcanidoforwikimedia.org/ (T91633) more?
 * "The likelihood that a developer's first contribution is successful" is 86% for Meb&&GFB, 80% for MeB, 67% for GFB, 73% for not in any program, hence "The dropout rate for program participants is much higher than for developers that did not start in a program" and "the data suggests that while developers in onboarding programs are more likely to succeed with their first attempts they are generally less likely to become long-term contributors". Possible reasons: "attract individuals who would not otherwise have attempted to make a contribution at all" who are "less equipped to transition into long-term contributors". "downplaying the difficulty of tackling a GFB may in fact increase the chances of failure since the expectation may be that these bugs should be 'easy'".
 * Note that "it is unclear whether these successful contributors would have started contributing without these programs, this paper provides quantitative evidence that these programs alone do not automatically improve the odds of a new developer becoming a long-term contributor" but newcomers might potentially "continue to be actively involved in the project in other ways, such as reporting bugs" (which was not measured in this paper).

Barriers faced by FOSS newcomers
Incorporate appropriate CR items into other section, leave "stuff happening before starting to write the first patch" here.

Steinmacher et al. went through 291 studies and considered 20 relevant. The majority relies on quantitative past data instead of qualitative studies. Via grounded evidence they analyzed 15 newcomer contribution barriers in 5 categories with 3 origins (Newcomers, Community, Product). The following items and quotes are from Steinmacher et al. if not noted differently.


 * 1) Documentation
 * 2) Outdated docs. Not indicative for FOSS projects as no specific studies. Creates uncertainty whether docs can be trusted. Potential waste of time on already existing features. ❌ "make newcomers aware of the status of the documents": Encourage using mw:Template:Outdated? (related: For missing docs, use mw:Template:Documentation needed?)
 * 3) Too much docs. Supported by two experimental studies. Overwhelming information overload. ❌ "The projects need to provide easy ways to find and navigate the information provided by the projects, linking different sources of information and enabling the recommendation of relevant parts of the group memory" for the newcomer's task.
 * 4) Unclear code comments. Rather irrelevant.
 * 5) Social interaction.
 * 6) Lack of social interaction with project members. Newcomer's social network; social status and need to build an identity. All 7 studies "show a correlation between the centrality of necomers' social relationships and newcomers' successful permanence of a contributor. However, there is no clear evidence of the causal relationship between social network centrality and newcomer success."
 * 7) Receiving an improper answer. "Three studies brought evidence of the negative impact of the content of answers received by newcomers" when it comes to answering "politely or positively". "[N]ewcomers demand attention and friendly hands to start contributing".
 * 8) Not receiving a (timely) answer. Feeling demotivated or unimportant. (Studies with contradictory results, however:) "absence of responses, improper answers, and not receiving recognition [...] can lead to newcomer dropout." "[...] could nominate people with social skills to receive newcomers in the communication channels.", "avoiding the use of project specific terms and jargons", "need to receive proper directions in a positive way". Potentially "make newcomers aware of the average time to receive answers [...] to help manage their expectations."
 * 9) Newcomers' previous knowledge
 * 10) Lack of domain expertise. Not indicative. People who contribute are nearly always users. No specific study exists about domain expertise though!
 * 11) Lack of technical expertise. Practical hands-on experience strongly associated with continued contribution; knowledge received via academic education not significant for successful contributing. A contributor can benefit from presenting skills and demonstrating expertise by sending mail or patches to influence the perception of the community. "[B]efore contributing to a project, newcomers must, for example, verify whether their skills math wtih skills needed". "Newcomers that showed proactivity [...] were better received by the community" and "the content of a message influences the reception of a newcomer". "[S]ocial and political behavior was important for newcomers to become long-term contributors or to be accepted [...] as members."
 * 12) Lack of knowledge of project practices. Not indicative, but "making it clear what is expected from them and what the process and practices are that must be followed to contribute" by having a "more informative and less technical initial environment" which looks "less geeky/daunting for newcomers". --- ❌ Could this be turned into an actionable task? Or not?
 * 13) Onboarding: Finding a way to start
 * 14) Annoying little bugs: Difficulty to find an appropriate first task. "the community wants the newcomers to pick the task themselves; however, newcomers have no clue of how to do this." Capiluppi and Michlmayr report that it is easier for new contributors to work on newer codebases than on older codebases. . ❌ Again, do not list MediaWiki core as first item on Annoying little bugs?
 * 15) Difficulty to find a mentor. Not indicative for FOSS projects as no specific studies. Not common in FOSS to offer formal mentorship. ❌ Document the fact that there is no formal mentorship (except for ) in Docs for Contributors?
 * 16) Technical hurdles (in general rather poorly studied)
 * 17) Onboarding: Setting up a local development environment. Getting stuck setting it up due to disproportionate configuration effort, reuse which creates dependencies and complicates building, platform diversity due to build configurations, constant changes in the build process requiring the developer who once mastered has to constantly keep up, prior experience, (not applicable:) interpreted languages, nobody in charge to simplify the build configuration. . "Make it easy to contribute by making the software easy to configure, build, and test to a known state. The more time you save outside developers that might be interested in contributing, the more time they have to work on the contribution they want to make, rather than losing time and possibly interest in trying to get past building the software." Hence provide a VM like Vagrant which has good tutorial-like documentation that also covers MS Windows sufficiently. ❌
 * 18) Code complexity. (Too hard to fix in this scope; action item already covered before.) "code complexity negatively influenced newcomers' decision to contribute". Hence "directing newcomers to peripheral modules" is better. ❌ Again, do not list MediaWiki core as first item on Annoying little bugs? Furthermore, fear of introducing new issues and feeling embarrassed, introducing platform specific bugs (e.g. different database backends), only superficially testing and creating unpredictable side effects can be reduced by providing easy access to unit tests via CI (Jenkins). ✅
 * 19) Software Architecture complexity. Providing visual information (e.g. class diagrams) reduces newcomers' challenges. Some people prefer more "visual learning styles". ❌ Check if we can provide better visual information in technical documentation?

Hospitality and tone

 * cf. https://en.wikisource.org/wiki/Hospitality,_Jerks,_and_What_I_Learned
 * "the level of politeness in the communication process among developers does have an effect on the time required to fix issues and, in the majority of the analysed projects, it has a positive correlation with attractiveness of the project to both active and potential developers. The more polite developers were, the less time it took to fix an issue, and, in the majority of the analysed cases, the more the developers wanted to be part of project, the more they were willing to continue working on the project over time."; "Issue fixing time for polite issues is faster than issue fixing time for impolite issues for 10 out of 14 analysed projects."; "Findings. In the majority of projects Magnet and Sticky are positively correlated with Politeness."; ”Politeness is the practical application of good manners or etiquette. It is a culturally defined phenomenon and therefore what is considered polite in one culture can sometimes be quite rude or simply eccentric in another cultural context. The goal of politeness is to make all of the parties relaxed and comfortable with one another.”
 * "Would you be interested in contributing a fix and a test case for this as well?" style instead of "this isn't the forum to clarify support requests"?; "The latest changeset isn't applying cleanly to git master anymore - could you resubmit it please?"
 * Cultural differences: Gupta et al tested four strategies (Brown and Levinson's theory): Direct ("Do X", "You could do X"), Approval ("Would it be possible/Could you do X"), Autonomy ("I'm wondering whether it would be possible if"/"Could you possibly do X"), Indirect ("X is not done yet", "Someone should do X") on 26 people (11 british, 15 indian, mixed gender, 20-30y) asked to rate on a scale between overpolite to excessively rude. While Brown and Levinson posit that the indirect strategy should be the politest form, to Indian people it sometimes sounds "like a complaint or sarcasm" as "English and Indian native speakers of English have different perceptions of politeness". Still "utterances to strangers need to be much more polite than those to friends".

T78639 - Addressing the long tail of low priority tasks in active projects
Saha et al. looked at 7 FOSS projects using Bugzilla and tried to find out why 125 manually analyzed bug reports open for more than a year take so long to get fixed. (Andre challenges some smaller assumptions but in general the authors have a clue, e.g. that triaging does not always and immediately take place, that severity and priority fields often keep their default values etc). Reasons are diverse:
 * Hard to understand and to find the right place where to fix in the code
 * Uncertain how to fix due to missing the bigger picture or related technical debt
 * Hard to fix due to required complex solution or architecture
 * Risky to fix with regard to release schedule
 * Incomplete fix due to missing corner cases
 * Importance not realized until duplicates were reported or someone comments, then activity follows
 * Hard to reproduce, e.g. steps to reproduce missing
 * Scheduling: Developer's workload and personal to-do lists
 * Unaware of corresponding task for a code change fix, hence not closed
 * Infrequent use case: Destructive bug but in a not very common area
 * Others, e.g. blocked by other tasks; developer vacations etc
 * No specific reasons / as-usual delay: limited resources etc, "there are many bug-fixes that got delayed without any specific reason".
 * Added by Andre: Misclassification in wrong basket/project hence not on the screen of devs?
 * Solution for more important tasks: Careful prioritization, predicting severity, change effort, and change impact. ''Added by Andre: Also by just saying "No until you do it yourself?"
 * Misc: "40% of long-lived bug fixes involved only a few changes in only one file."
 * ❌ Expectation setting, again - maybe "The developers have a lot to do but if you feel strongly about this particular issue, please consider a code contribution which the developers would be happy to review."''
 * ❌ How to set more realistically more tasks to Priority=low?
 * ❌ How to explain? See T87411: "Add help link to explain meaning of priority levels"

Bounties / Crowdfunding

 * cf. T88265 -- ✅: Relevant parts merged into comment there


 * "All bounty programs are characterized by a winner-take-all incentive structure. [...] a developer must consider if they should spend a considerable amount of time working for an uncertain prize of not enter." (page 176; cf. game theory).
 * Advantages from Company/corporation perspective: Can reduce development costs; can lead to greater number of alternative implementation designs due to competition/rivalry; can create broader interest in product; might change priorities for developer community. (pages 174--179)
 * Disadvantages from developer perspective: Risk of not making it due to getting paid only in case of success; hard for developer to evaluate difficulty of task and quality of documentation (pages 174--179)
 * Disadvantages from Company/corporation perspective: Can be hard to define the amount of money - if amount is too low might not attract qualified developers; potentially attracting students who wish to learn; potentially attracting developers in countries where salaries are low (cf. GSoc?); in FOSS it could be work that someone else might have done for free at some point been (cf. Free rider problem); time spend to judge contribution is a cost (though also with non-bounty reviews). (pages 174--179); Target group: "Typically, bounties have been won by a few people who have worked on projects for a long time." (page 177)
 * Risks for project perspective: might influence direction that project is heading; short-term orientation may create direction that is not scalable and creates maintenance costs (but also a non-bounty issue?) hence long-term orientation required; potential rivalry with volunteer developers and potential withdrawing of them; potential bypassing of maintainers if bounty organizers decide which tasks are pushed/prioritized instead of actual maintainers/community; attracting developers which do not know the direction and context of project. (page 177--178)

Other related tasks

 * T114320, T114311 - CR migration to Differential
 * T114419 - Make CR not suck, especially for volunteers
 * T89907 - MW developer community governance model
 * T102920 - Unmaintained/inactive repos
 * T115659 - Collaboration on prioritization

Misc

 * Large single vendor governed FOSS projects tend to be controversial; non-profit foundations seem to be successful
 * "Solid Engineering Practices + Strong Community Governance + Clear IP Management enables Growth."
 * Magnetism and Stickiness: A project is Magnetic (magnetism) if it attracts new developers over time. A project is Sticky (stickiness) if it keeps its developers over time.
 * Magnetism is the portion of new active developers during the observed time interval, in our example 2/10 (dev 6 + dev 7 were active in 2011 but not in 2010).
 * Stickiness is the portion of active developers that were also active during next time interval, in our example 3/7 (dev 1, dev 2, dev 3 were active in 2011 and in 2012).
 * Covered by korma's Demographics page
 * Time to merge
 * OpenStack: Number of active core reviewers growing, 50% of the changesets that landed into master were merged in less than 3 days since the review process was opened.
 * Linux Kernel: 33% of the patches make it into the Linux kernel within 3-6 months; Factors are submission time, affected subsystems, number of requested reviewers
 * Who works on what and how to find out and contact (WMF) teams?
 * Key Wikimedia software projects extremely outdated when it comes to areas, teams, projects, links. Transclude from each team page (if existing) for WMF stuff at least? More automatic approach needed, e.g. list of deployed extensions and adding/removing mw:Category:Extensions used on Wikimedia from extension homepages on mediawiki.org
 * T115853 - Every wiki page of a WMF engineering and product team should have a "Contact" section
 * Teams to consistently (structure!) document and update their codebase responsibilities on-wiki, like Reading in https://lists.wikimedia.org/pipermail/mobile-l/2015-November/009926.html ?
 * Teams to consistently (structure!) document and update their codebase responsibilities in Phabricator, listing their "sub"projects in Team project descriptions?
 * Teams to update mediawiki.org extension homepages?
 * Wikimedia Engineering and Staff to link to corresponding team wiki pages
 * Apply structure of each section of Wikimedia Product also on team wiki pages? Proposed in mw:Topic:Su7ud4jz6a4338wx
 * (Tool) Labs
 * T64837: Community-maintained projects on Labs are hard to track
 * Labs_labs_labs/future
 * mw:User:BDavis_(WMF)/Projects/Tool_Labs_support
 * https://etherpad.wikimedia.org/p/toolssupport from Jan 2016