Talk:New Developers/Quarterly/2017-10

• Review the tutorials explaining the code contribution process and investigate what we can improve further. Such as: include a list of easy tasks requiring one-line fixes, etc. that might bring some value to the first contribution made by newcomers
• Getting Wikipedia Android Apps project listed as a featured project on the New Developers page
• Better tracking information of new developers
  • Quim's idea to start with developers from workshops in Africa via CRM
• Shorten the survey, change the timeframe, and send it not just at the end of the quarter but a few times?
• Can we do anything about this issue "Fix the IP ban that blocks access to Phabricator"?

@AKlapper (WMF) @Rfarrand (WMF) @Qgil-WMF What do you think?

@SSethi (WMF):

• Having real one-liner tasks (e.g. typo fixes) offered instead of just "check out Examples and make a change that does not fix any real bug" is exactly what https://phabricator.wikimedia.org/T146960 is about. We'd need folks to do this (categorize #easy tasks, I can try to remember but it's hard to judge if you're not the dev), and then using that in https://phabricator.wikimedia.org/T161901.
• Survey timeframe: Up to you, but if you want to go for monthly sending out, please update our docs in https://phabricator.wikimedia.org/T177522

• Re Phab IP ban: It is meant to be only about specific Wikipedia Zero providers. Which might have bugs which someone needs to fix. Such as https://phabricator.wikimedia.org/T173537 or https://phabricator.wikimedia.org/T174342. To read more about the IP block, see https://phabricator.wikimedia.org/T170200#3424774...

@SSethi (WMF) Thanks so much for putting this together. It't really valuable. :)

• For point #9: perhaps if this is the case, we can hold yearly (??) "tell your friends about contributing to Wikimedia" volunteer recruting drive. Maybe using similar tactics to how other non-profit organizations ask for money.
  • We could write both an email and a facebook post with links that people could copy and paste and then just send that around
  • We could ask people to think about people that they know that might like to get interested & if they don't want to do anything - we could approach them ourselves. "Hi NAME, we heard from Quim Gil that you might be interested in learning about volunteer opportunities to contribute to............... if you have any questions about getting started, please don't hesitate to ask"
  • We could even be crazy and ask Fundraising to add some details in some of their banners "Donate money OR volunteer your developer skills"
• For point #8: maybe we can send personal messages to some newcomers asking them to fill it out next time. Shortening it, like you plan, is probably going to help also.
• For point #7: certainly we should, as a team, look at what questions we want to ask people during hackathon registration. Registration for Barcelona will sneak up on us pretty quickly here.
• For point #3 can we work with the Community Tech team to set up a "newcomers-code-review-day" once every quarter? Or even make it wikitech-l wide and send an email out with code written by newcomers that has not been reviewed and / or needs help?
• I think we should have an action item to check in with other small hackathon organizers (Ghana, Greece, Netherlands, etc) both just a week or so in advance of their events and again afterwards.
  • The pre event check in would be to offer any last minute support in any capacity
  • The post event check in should be to see if they want to write a blog or if they found our tasks useful, or if they discovered other tasks that still need work, or if there is anything else they want to tell us. We should also ask them where they are collecting their lessons learned and add a link to that to our hackathons documentation. Maybe also, specifically, what they wished they had known in advance of organizing the event.

@AKlapper (WMF) @Qgil-WMF & @SSethi (WMF), any thoughts? ^^ :)

I'm wondering.. if we should make the following tasks as subtasks of https://phabricator.wikimedia.org/T178630

• Gerrit/Tutorial https://phabricator.wikimedia.org/T161901
• Phabricator IP/Ban https://phabricator.wikimedia.org/T173537
• CRM for TC: Start with new developers from workshops in Africa https://phabricator.wikimedia.org/T177583
• Follow up with organizers of WikiFemHack?
• I created already one about making Wikipedia Android app as a featured project for new developers https://phabricator.wikimedia.org/T178633
• Shortening the survey is already now a TODO in https://phabricator.wikimedia.org/T178627

@Rfarrand (WMF) All your points 1-3 are some promising ideas for improving outreach, that we could utilize in https://phabricator.wikimedia.org/T176488

@AKlapper (WMF) Better triaging for easy tasks sounds interesting, but I am not sure if we currently have the bandwidth to work on this, and if we should carve out a tiny bit of it for the purpose of improving the Gerrit tutorial.

@SSethi (WMF): Re Follow up with organizers of WikiFemHack: https://phabricator.wikimedia.org/T176861 might be usable for this (as that task needs an update anyway)

Slight change > Make new tasks as subtasks of https://phabricator.wikimedia.org/T167085 as Quim suggested in T178630#3699139

Looking at https://phabricator.wikimedia.org/T167085 which is called "Release a "New Developers Quarterly Report" including metrics, trends, lessons learned, and recommendations by Oct 2017", those tasks listed above do not block releasing a Quarterly Report by Oct 2017 and that task is already resolved. Hence they are not subtasks (tasks that need to be done to resolve a parent task) in my understanding - also see Phabricator/Help#Parent tasks and subtasks. So I guess I disagree with Quim here. :)

Hello,

Thanks for compiling this report − very interesting!

I am curious about the retention of events. How is the number of developers attracted (eg for the Wikimedia Hackathon, 37) computed? Is that based on the registration survey? (For comparison, the report of that event mentions 56 newcomers)

From there, the 3 retained is based on a Kibana query.

• the query only contains 19 usernames. I guess it is because we don’t have the usernames of the 18 others?
• the query appears to look through Gerrit data ; is that a deliberate choice? I would have thought that contribution to any Technical space would count as retained − the same query against Maniphest appears to return 5 users.

@Jean-Frédéric Thanks for your question! Yes, we are relying on the registration survey. Members of TC went through the events registration spreadsheet to remove folks who may not be new developers such as staff members, who we didn't see at the event, folks who have been already active for a while, etc. The final list had 37 developers (from Vienna). We tried to look up for more information for people (whom we didn't have any Phabricator, Gerrit username info) in Wikimedia Biterg using their email address. Then we could gather usernames of only 19 people. (some more info here)

For this report, we have target new developers who have contributed at least one patch in the last 90 days. For the survey that we sent out, for the metrics we gathered, we considered the same audience. And, so the query containing 19 usernames looks through Gerrit data. 

Having said that, yes we need to improve on measuring this data better, key finding 7 in the report New Developers/Quarterly/2017-10#Key findings :)

@AKlapper (WMF) might be able to shed more light on this if I've missed anything as he is the master mind behind pulling all this together :P

@Jean-Frédéric: Good questions. And thanks for your interest. :)

In addition to what Srishti already wrote:

19 usernames indeed because we could not find any activity in the data for the 18 other usernames. But that might be because https://wikimedia.biterg.io for example does not cover activity on GitHub where some projects are located.

Concentrating on Gerrit data became a choice at some point. I first queried on the mainpage of https://wikimedia.biterg.io for the usernames but then had to realize https://phabricator.wikimedia.org/T177566#3663047. The pleasure of working with systems that you need to understand. :P Also note that the Maniphest query will only show you users who created a task, but not users who for example commented on a task. See for example https://phabricator.wikimedia.org/T161926 or https://phabricator.wikimedia.org/T161928.

And https://phabricator.wikimedia.org/T175854 provides even more technical details about the Kibana queries that you might not want to dive into. :)

Wrong number corrected; clarified legend of graph as it already does display what is proposed here.

I cannot map the numbers shown in the graph with the numbers seen in the page following the link. However, the graph seems to be counting number of contributions. Wouldn't it be more useful to count number of developers?

FICTIONAL EXAMPLE: If 2 developers contribute 10 patches each to Wikipedia Android app and 10 developers contribute 2 patches each to MediaWiki Core, both make a total of 20 patches, but it is a lot more interesting to see that MediaWiki Core got 10 newcomers.

Focusing on people rather than patches also diminishes the problem of comparing patches by complexity. Instead, the complexity of attracting a new developer is quite comparable across people and projects.

@AKlapper (WMF)

@Qgil-WMF Uh, thanks. Indeed 25 instead of 28 repositories. That's because excluding staff not marked as volunteers is... still WIP as I ran into new puzzling data corruption issues yesterday night (I already reported them to Bitergia).

The graph counts what you expect it to count: 51 developers contributed to 25 repos. It does not count the number of contributions. The number of contributions is shown in the "New Authors" list widget. If you click on the 3rd biggest item in the pie chart (apps/android/wikipedia with a value of 3), you will see 3 authors listed under "New Authors". With in total 24 contributions. (That the list of "new Authors" still includes staff is a different issue I still need to solve, see my second sentence in this comment.)

I have adjusted the legend under User:SSethi (WMF)/New Developers Quarterly Report (July - September 2017)#Projects with most received contributions by new developers

So I guess this is resolved.

I still find the title confusing. Why not simply "Projects with most new volunteers"?

Yeah, good point. Done.

Y Done

@SSethi (WMF), @AKlapper (WMF) I think the New Developers/Quarterly/2017-10 report is good to go!

Yay! we are live :)

"More autonomy for development teams: build a trust-based DevOps culture allowing teams to adopt Continuous Integration/Continuous Delivery practices and deliver value in small iterations (current deployment guide mentions a lead time of over six weeks for new services!)."

Does this sound like something that a new volunteer would say? Not a new professional developer hired?

This makes me think. Was the list of participants in the survey vetted, in order to assure that they were volunteers only, not new hires?

The comment below about Phragile in the topic below is also from the same individual response. This response is from a developer with a wikimedia.de address.. so yeah I sent the emails to everybody then from the list I had. Six people from July and August, who received the survey were staff members. Ahha, hope not to repeat the same mistake next time, but now to make up for this, should I just add a point in the Key findings as the next step towards improvement or something else? 

Right. No problem, another lesson learned. We need to have a documented sequence of steps, where we first agree on the list of newcomers reflected in the metrics and events organized, and then we send the survey to those newcomers.

Point noted here (https://phabricator.wikimedia.org/T177522) under Survey section.

@AKlapper (WMF) @Qgil-WMF Perhaps in a day or two time to send it away from my home to elsewhere.. but where Meta/ MediaWiki? Also, how about the title, it be the same?

[[New Developers/Report]] for the main page and then /October for this edition?

"New Developers Quarterly Report" sounds good to me. If we remove "quarterly" then it is shorter, but at the same time might give the impression that is a one-time report.

Please do not send it before the three of us are happy with it!  :)

Not sending it for right now, making sure, it will then be [[New Developers/Quarterly Report/October]]??

Based on the name you proposed for the new newsletter (which I like a lot), the location could be

New Developers/Quarterly/2017-10

"YYYY-MM" is a unique identifier (I had missed this little detail).  :) Starting with the year will assure an automatic chronological classification in lists.

I wonder whether we should keep that recommendation, considering that it si so generic that it is not actionable.

I feel like it is a known fact and there is a recommendation already about looking into improving documentation on a specific topic in the key findings. 

IMO there is no value in this page User:SSethi (WMF)/New Developers Quarterly Report (July - September 2017)/Survey analysis, do you see any? It is not conveying anything extra other than the graphs which can be anyways hard and dangerous to interpret with n=9. Should we get rid of this page?

I would keep it as documentation, of course.

Percentages were removed.

@SSethi (WMF) I mentioned this briefly but wanted to log it: Looking at User:SSethi (WMF)/New Developers Quarterly Report (July - September 2017)#Contributions :

As this is a multiselect option, you and I know that 9 developers checked a total of 31 checkboxes, so you calculate 100/31*6. But nobody else knows that number '31'. Hence I only know that 9 people replied in total (=100%), and 6 people out of those 9 said they 'fix bugs'. And that is not 19.35% but 66.67%?

I had added this text earlier "Participants could select more than one option," I thought it was sufficient but may be not? 

@ SSethi (WMF) If no readers (apart from you and me) know that the magic number is 31 this chart might become one of the greatest unresolved mysteries of... ah well, if 6 out of 9 people say "Yes I do X" to answer X, that answer X got 66.67%. Regardless of some hidden dependency on the popularity of saying "Yes I also do Y" to answer Y. Multi-option in my understanding means that the sum must be ≥100%.

I agree that this is confusing. For instance, "35.48% of respondents of this survey said they fix or report bugs"... is this really true? They could also be 19%, with people clicking both options. I would remove percentages altogether. They don't tell much anyway.

I would also remove percentages in "Challenges" and "Experience...". When the total universe is 9 people, talking about "55.65%" or "33.33%" feels odd. "Motivations" keeps the points in a more generic way, and that sounds more frank and humble.

Added an additional column "Count" in "Types of Contributions" that helps explain why the total count of responses=31 when n=9. Removed percentages in "Challenges" and "Experience".

I have removed the percentages altogether. They provide a "scientific flair" that in this case we cannot sustain.

Y Done

These two separate sections seem to have the same goal. Why not joining them? Some items under "Recommendations for next steps" could be shorter.

Made this change accordingly.