Google Code-in/Lessons learned

This page lists feedback and ideas for potential improvement.

2016

 * Andre would have also loved to see tasks like T154198 to T154201, but they did not even get marked as easy. cf. T149564
 * Marking a task on the GCI site as "ready" did not always work. Having to ping admins to publish a task on the GCI site was cumbersome. We need a better workflow.
 * Mailing list or IRC channel to contact org admins specifically, instead of random private pings on IRC and emails answering mentor questions and publishing tasks when wanted in a timely manner, plus also to gather feedback in the end?
 * We had no "concept how to offer harder tasks at the end"; cf. learning curve. It was pure luck that we got mentors (like TTO) for harder tasks at the end.
 * We had way more mentors this year at the beginning. but this did not necessarily result in more tasks :(
 * Hangout session for mentors at the beginning worked well. (Srishti's idea) T150636
 * We need better examples for really well written tasks; e.g. jayvdb pointed to https://github.com/coala/coala/wiki/Google-Code-In-Task-Use-coala
 * In 2016, we went 16 times over the 36h review/feedback deadline which is way more than in previous years (however never longer than 44h; I've heard stories of 72h from other orgs).
 * Some input from TTO (not always exact quotes):
 * a lot of the tasks that have been offered have been almost too trivial even for GCI... Maybe before GCI 2017 we (as Wikimedia) should discuss what standard the GCI tasks should aim to be. Obviously there will always be a mix of easier and harder tasks, but tasks that require changing one or two lines of code don't seem suitable as standalone non-beginner tasks.
 * A recurring issue in our participation in GCI is availability of org admins. Last year I pointed out that all the org admins were in Europe, creating difficulties for mentors and students in the US and Asia. This year, the situation was a lot better, as mentors could now edit and accept tasks they weren't mentoring. But it still proved challenging to get new tasks published; waits of more than 24 hours, during periods when we were short on tasks, were often experienced. [...]
 * Provide clear list of org admins and ways to contact them on MediaWiki.org. It would be useful to place a separate table on the mentors subpage with this info.
 * The biggest issue for me, though, was the statement on the mentors subpage that "Tasks are supposed to take 2-3 hours to an experienced contributor." A number of the tasks created by other mentors were trivial, and would have taken the best part of 5 or 10 minutes for an experienced contributor to fix. Most of the students I worked with felt that they weren't learning anything from these tasks, and they were clamoring for some more juicy tasks to sink their teeth into. The fact that one student completed tasks at a rate of more than one per day on average is another sign that some tasks were too simple or involved too little work. I think org admins might need to take a bit of a harder line on task difficulty next time around, to ensure Wikimedia's GCI effort stays high-quality and more students remain engaged right through to the end. For example, a task could have required the student to remove a deprecated method in three extensions instead of one, or six extensions instead of three, etc.
 * jayvdb in https://codein.withgoogle.com/tasks/5785685814411264/ says "Set up a Vagrant instance (your development environment) - see instructions." ; it is tagged "vagrant". Rather than change that task, maybe retitle it to be vagrant, and create other tasks about installing the Docker, installing from the package manager, and have the participant report any problems.  In each of these "setup a [dev|prod] environment" tasks, we should say something like "if you run into an unreported bug in the process, you should abandon this task and do [this task](link to create a bug) task instead, and then maybe re-start this task again later."
 * AFAIK Dan / Analytics wanted a earlier heads-up for GCI (at least one month before) to have time to prepare tasks.
 * Consider an optional Hangout session after GCI has finished, to gather more feedback and Lessons Learned??
 * Work already performed by students cannot retroactively become a GCI task (GCI contest rules, section 4.2). However before GCI starts we plan potential tasks in public so there is a bit of a conflict here we cannot easily avoid (students could investigate tasks before claiming them but it is their risk if someone else is faster to claim the GCI task once available).
 * MovingBlocks' 'Introduce yourself to the community on IRC' tasks have a "Definition of Done" and "Suggested next task".

2015

 * We had tasks about extensions that do not list their software license (and how to add it). However students often did not check all files and the task turned out to be more complex when it came to required investigations than initially thought.
 * We edited some on-wiki developer documentation but results were mixed as students often performed (more or less acceptable) copy&paste on each page without actually understanding what problem they were solving by their wiki edits.
 * The CI whitelist caused confusion (and waste of time as the student had to wait for half a day due to timezone differences just so someone else can say "recheck"). We either need more documentation on how to run tests locally, or ideally add all students to the whitelist at the beginning. Without adding, only a subset of the tests is run by jenkins and that can cause confusion. cf. https://gerrit.wikimedia.org/r/#/c/261322/
 * Vagrant docs might need better documentation for Windows platform?
 * How to encourage having more tasks with more than one mentor?
 * Better workflow for mentors creating tasks to tell admins they are ready to publish? Or is "Either add a "[READY]" prefix to the task summary to let org admins know, or contact them explicitly." sufficient?
 * Part of GCI is "getting work/tasks done", part is "recruiting new contributors / community members"
 * We miss a "pipeline" how to direct students to further stuff (try GSoC if you are 17 years old?) Point to local hackathons? / "GCI is missing a follow-up program for the best contributors. (Or maybe the MediaWiki/Wikimedia technical community is missing a non-time-limited mentorship system?)" Every year there are GCI participants with a "hire on sight" competence level and it would be great to keep them engaged.
 * Students sometimes still have no way to gauge the complexity / required skill level of a task, and making a task description perfect can be the enemy of good.