Engineering Community Team/Meetings/2014-07-08

ECT IRC meeting at 16:00 UTC via.

Topics:

= #wikimedia-office: Engineering Community office hour - July 2014 =

Meeting summary
= #wikimedia-office: ECT office hour 8 july 2014 | Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE) | https://meta.wikimedia.org/wiki/IRC_office_hours =

Meeting started by sumanah at 16:05:02 UTC. The full logs are available at https://tools.wmflabs.org/meetbot/wikimedia-office/2014/wikimedia-office.2014-07-08-16.05.log.html .


 * LINK: https://www.mediawiki.org/wiki/Engineering_Community_Team/Meetings/2014-07-08 (sumanah, 16:07:57)
 * Developer hub revamp https://www.mediawiki.org/wiki/Data_%26_Developer_Hub (sumanah, 16:08:48)


 * Non-Sumana ECT activities :) (sumanah, 16:10:49)


 * Developer hub revamp (sumanah, 16:12:13)
 * LINK: https://www.mediawiki.org/wiki/Data_%26_Developer_Hub (sumanah, 16:12:21)
 * LINK: https://www.mediawiki.org/wiki/Wikimedia_Engineering/2014-15_Goals#Engineering_Community as you can see our goals for the end of September 2014 are: (sumanah, 16:14:13)
 * ACTION: sumanah to ask Analytics which APIs are most used (sumanah, 16:18:10)
 *  most people don't know about the CommonsMetadata api, for example. I myself only learnt about the very useful assert= last few days (sumanah, 16:20:29)
 * Yuvi says: the query interface our API has is also rather powerful (with generators and all) but not too well known, in the sense people don't utilize the full power of it (since it is a bit hard to fully grok) (sumanah, 16:22:11)
 * LINK: http://juliuszgonera.com/wddh/showcase/wikistream.html Prototype: http://juliuszgonera.com/wddh/ (temporary) (sumanah, 16:25:22)
 * GlobalUsage: "View the global usage of images in a wiki farm with shared image repository. Used by Multimedia: https://www.mediawiki.org/wiki/Multimedia/Architecture (sumanah, 16:27:26)
 * LINK: http://lists.wikimedia.org/pipermail/wikitech-l/2014-July/077406.html Wikidata's search API interface - currently a work in progress (sumanah, 16:28:42)
 * Niklas suggests the Translate extension's translation memory API (sumanah, 16:29:12)
 * HELP: tell Sumana which APIs you & your teams have had the most delay in picking up due to poor documentation (autogen + human-centric tutorials) (sumanah, 16:50:26)
 * main questions: * how do we publish low-level API docs? 3rd party tool or MediaWiki solution? * whatever is the first answer, how do we integrate these low-level docs and sandbox with tutorials and showcases? (what tools?) (sumanah, 16:50:59)
 * LINK: Swagger demo http://petstore.swagger.wordnik.com/ (sumanah, 16:51:15)
 * LINK: https://www.mediawiki.org/wiki/Requests_for_comment/Content_API#Structured_API_specs additional structured API tools/formats (sumanah, 16:51:27)
 * important to continue ensuring that docs reflect the current configuration & to continue ensuring that enabling an API end point should also make its docs visible. Gabriel suggests: "one way to ensure that is to couple the docs with the module registration, as is currently the case for PHP API modules and the Content API prototype at  "  (sumanah, 16:59:54)

Meeting ended at 17:00:23 UTC.

Action items

 * sumanah to ask Analytics which APIs are most used

Action items, by person

 * sumanah
 * sumanah to ask Analytics which APIs are most used

People present (lines said)

 * sumanah (73)
 * qgil (33)
 * gwicke (19)
 * YuviPanda (15)
 * wm-labs-meetbot (4)
 * Nikerabbit (3)
 * guillom (3)
 * Lydia_WMDE (1)
 * andre__ (1)
 * Tpt (1)
 * marktraceur (1)

Generated by MeetBot 0.1.4 (http://wiki.debian.org/MeetBot)

Full log
16:05:02 #startmeeting ECT office hour 8 july 2014 | Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE) | https://meta.wikimedia.org/wiki/IRC_office_hours 16:05:02  Meeting started Tue Jul 8 16:05:02 2014 UTC and is due to finish in 60 minutes. The chair is sumanah. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:05:02  Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 16:05:02  The meeting name has been set to 'ect_office_hour_8_july_2014___channel_is_logged_and_publicly_posted__do_not_remove_this_note____https___meta_wikimedia_org_wiki_irc_office_hours' 16:05:07 #chair qgil sumanah 16:05:07  Current chairs: qgil sumanah 16:05:10 qgil: ok! go. 16:05:39 Alrigt, thank you 16:05:55 This is the Engineering Community Team IRC meeting -- https://www.mediawiki.org/wiki/Engineering_Community_Team 16:06:36 (I apologize for the problems, I just landed in Germany, I still don't have decent Internet at home, etc...) 16:07:01  so far no problems appeared :) 16:07:01 My sympathies qgil 16:07:37 Well, one problem is that I could not prepare the boilerplate text :) 16:07:47 because I just got internet running at home 10 minutes ago :) 16:07:57 #link https://www.mediawiki.org/wiki/Engineering_Community_Team/Meetings/2014-07-08 16:08:28 Anyway, the main topic planned for today is the Developer Hub revamp: 16:08:48 #topic Developer hub revamp https://www.mediawiki.org/wiki/Data_%26_Developer_Hub 16:08:49 are you ready sumanah ? 16:09:07 qgil: actually let's maybe quickly cover what's up with the other projects first? 16:09:14 the other activities I mean 16:09:42 (mentorship, Phabricator, Wikimania planning) 16:10:03 We can... but I guess the rest of us will improvise an update. 16:10:35 nod 16:10:49 #topic Non-Sumana ECT activities :) 16:11:26 Just checking, I believe some weeks ago we had agreed that the main topic for today would be the Developer Hub 16:11:50 yes 16:12:03 but I assumed we'd sort of clear the other stuff out first - no matter, I can go first 16:12:13 #topic Developer hub revamp 16:12:21 #link https://www.mediawiki.org/wiki/Data_%26_Developer_Hub 16:12:21 Starting in mid-July, I'm working on this central developer hub, focusing on our web APIs. 16:12:28 I haven't got time to check the announcement yet, had we mentioned the develloer hub there? 16:12:42 (I'm currently finishing a bunch of work on the security and architecture guidelines.) 16:13:18 Working with me are Brad Jorsch (anomie), and (on a more part-time basis) Moiz Syed and Juliusz Gonera 16:13:38 consulting as well with Dario Taraborelli & Jared Zimmerman 16:13:41 Sorry again for the confusion. I mean, I thought we could have an IRC meeting focusing on the Developer Hub, previously announced. 16:14:00 No prob 16:14:13 #link https://www.mediawiki.org/wiki/Wikimedia_Engineering/2014-15_Goals#Engineering_Community as you can see our goals for the end of September 2014 are: 16:14:19 Developer Hub prototype 16:14:19 Landing page, 3 projects showcased, 3 APIs documented 16:14:19 API sandbox functional prototype 16:14:19 Contribution process defined 16:15:25 At the moment Moiz and Juliusz are working on a prototype just to help us understand what the interface will look like. For sample data/a sample API to document, they're starting with the recent changes streaming API -- stream.wikimedia.org is close to ready 16:17:00 And my first task specific to this hub will be to interview engineering managers + community to find out which APIs people want to use and have trouble using. Also I'll prioritize Gabriel's new content API. DEADLINE: late July 16:17:52 * YuviPanda suggests also asking analytics to see which APIs are most used 16:17:55 * YuviPanda waves as well 16:18:02 YuviPanda: this is a good idea! 16:18:10 #action sumanah to ask Analytics which APIs are most used 16:18:29 YuviPanda: in a sense the highest-priority ones are the ones where there's a big delta between how often they're used and the quality of the existing docs 16:18:45  sumanah: indeed, since they might not have been used much because people didn't know they exist. 16:18:49 yes 16:19:38 So those are some of the facts .... YuviPanda I can start the more interactive portion here by asking you :) whether there are APIs where you think they're very underappreciated 16:20:14  indeed. most people don't know about the CommonsMetadata api, for example. I myself only learnt about the very useful assert= last few days 16:20:29 #info  most people don't know about the CommonsMetadata api, for example. I myself only learnt about the very useful assert= last few days 16:20:57  the query interface our API has is also rather powerful (with generators and all) but not too well known 16:21:31  in the sense people don't utilize the full power of it (since it is a bit hard to fully grok) 16:21:42 Lydia_WMDE: marktraceur Nikerabbit ragesoss and Tpt - I would also be curious about your answers. What Wikimedia APIs do you think are underappreciated, where there's a lot of possibility there but people don't know about them? 16:21:55 * YuviPanda mentions the wikidata API as well 16:22:11 #info Yuvi says: the query interface our API has is also rather powerful (with generators and all) but not too well known, in the sense people don't utilize the full power of it (since it is a bit hard to fully grok) 16:22:40  Strong +1 for the query interface of the API 16:23:24 I believe that my intern Frances, who had already figured out how to get data out of Wikipedia and Wikivoyage, had a more difficult time figuring out how to get data out of Wikidata - the structure of things, really 16:23:25  WikiData as well, especially once the (currently WIP) search API interface is made available. I'd say when that becomes available it'll be the most powerful/useful API we expose 16:23:33 YuviPanda: link? 16:23:36  sumanah: moment 16:23:38 thanks 16:23:58  sumanah: http://lists.wikimedia.org/pipermail/wikitech-l/2014-July/077406.html 16:24:10 ah, yes 16:24:12  sumanah: wikidata obviously :P 16:24:23 Lydia_WMDE: :-) 16:25:22 #link http://juliuszgonera.com/wddh/showcase/wikistream.html Prototype: http://juliuszgonera.com/wddh/ (temporary) 16:25:25 Quick question: sumanah, when you mention "Contribution process defined" as one of the goals, does that include determining how to leverage all the existing documentation on mediawiki.org to avoid duplication? 16:25:44 <Nikerabbit> sumanah: to your earlier question: all of them ;) 16:25:45 guillom: The short answer is yes 16:25:54 Nikerabbit: OK, now get serious :) 16:25:57 ok :) Thanks 16:26:34 sumanah: GlobalUsage is pretty neat. 16:26:35 Nikerabbit: I imagine not all of them are equally "hidden diamonds". I'm looking for which are the most hidden-diamond-y 16:27:10 guillom, the API documentation of the Developer Hub shouldn't be just editable like a regular wiki page, but it cannot be editable only by e.g. Sumana. There must be a point in between, that should be defined by this contribution process. 16:27:26 #info GlobalUsage: "View the global usage of images in a wiki farm with shared image repository. Used by Multimedia: https://www.mediawiki.org/wiki/Multimedia/Architecture 16:28:04 qgil: Agreed, and I mentioned FlaggedRevs at some point in Zürich. I know there are some strong opinions on this among the team working on this hub, so I'm glad to know that sumanah is keeping an eye on the issue :) 16:28:10 :-) 16:28:23 <Nikerabbit> sumanah: well, for example Translate extension has the translation memory API 16:28:36 <Nikerabbit> can't think of anything else right now, have to go ;) 16:28:42 #link http://lists.wikimedia.org/pipermail/wikitech-l/2014-July/077406.html Wikidata's search API interface - currently a work in progress 16:28:55 Nikerabbit: got it. https://www.mediawiki.org/wiki/Talk:Data_%26_Developer_Hub - leave thoughts there too! 16:29:09 At this point everything needs to be defined, e.g. part of the documentation might be generated directly by the source code, and therefore would be editable only though patches. 16:29:12 #info Niklas suggests the Translate extension's translation memory API 16:29:57 fwiw, for the content API I am planning to use one of the fairly structured tools / formats linked at https://www.mediawiki.org/wiki/Requests_for_comment/Content_API#Structured_API_specs; this will be the pure API docs, without much tutorial content 16:30:38 the nice thing is that tools like swagger auto-generate a sandbox & browsable docs based on the API spec 16:30:51 I love that ... yeah I was about to say, sandboxes and mocks 16:31:01 * YuviPanda notes that our current APISandbox could use some love 16:31:32 YuviPanda: you may have seen, in the goals for end of September on this project: "API sandbox functional prototype" 16:32:05 <YuviPanda> sumanah: right, was wondering if the 'prototype' means a complete rewrite or fixes to the current one 16:32:09 here's a swagger demo: http://petstore.swagger.wordnik.com/ 16:33:43 YuviPanda: I think that is a good question and my personal professional opinion is "it depends" - if you have facts/opinions to share that would lean one way or the other, feel free 16:34:48 A deep problem to solve: a common way for all the APIs to be documented and showcased. For instance, it is good that gwicke and his team are using Swagger now for their API, but what about the rest? 16:34:57 <YuviPanda> sumanah: right. from the POV of a developer, I'd think the sandbox would be useful only if it is complete, and the current one is (because it autogenerates itself). so anything that ensures it is complete without much human intervention is good, wether it be generic new one or improvements to this one 16:35:52 qgil: I'm not 100% decided between swagger & json schema hypermedia yet 16:36:25 the latter looks more powerful, but I haven't evaluated the front-end options for it yet 16:36:36 I any case, it would be good that "your" decision and other "local" decisions made by other dev teams would converge in the Developer Hub 16:36:41 it has more buy-in from the likes of Google & Heroku though 16:37:29 what do you think, sumanah ? 16:37:55 qgil: I feel as though I'd like to talk with Design about it 16:39:13 gwicke: do you have a favorite demo for the json schema hypermedia approach, so I can compare? 16:39:23 It would be good to see what gwicke ìs looking for when thinking of Swagger / json schema hypermedia, and see how far MediaWiki + APIsandbox is from there, 16:39:41 I played with Swagger but it was a few years back so I should check it out again 16:39:45 and compare API Sandbox 16:39:56 and whether we should try to solve everything with repos + MediaWiki or bet on a 3rd party tool as well 16:40:06 qgil: I'm mostly looking for a machine-readable API spec, which can also be used to auto-generate rich docs, sandboxes etc 16:41:22 with evolving APIs consistency between docs & implementation is important, and having a single machine-readable spec can help with avoiding divergence 16:41:31 yup 16:42:37 gwicke, in our Developer Hub discussions, we have been talking about a way to export strict API documentation from repos to MediaWiki, while writing tutorials etc directly on MediaWiki. 16:43:03 I don't know how far we are from this scenario, and how desirable it is compared to using a 3rd party tool, though 16:43:39 I think there is a need for both low-level API docs, and higher-level tutorials / entry points 16:43:52 yes 16:43:53 The idea is that such extension + editing tutorials in MediaWiki would allow us to have everthing under the same (search) umbrella) 16:44:14 gwicke: yup, I've been saying that for years ;-) glad to have a chance to work on both 16:44:33 gwicke, agreed, the point is, can we use a MediaWiki extension to publish the low-level API docs without needing a 3rd party tool that will create a separate site? 16:44:53 <YuviPanda> should be wary of NIH 16:45:13 something like swagger is easy to hook up 16:45:20 hope so 16:45:49 the main work is figuring out which format / tool to use, and creating the spec 16:46:07 and then having a process that ensures that the spec remains in sync with the code 16:46:13 YuviPanda, indeed, I'm just saying that the end result would be better something that developers can search and find in a single place, integrated with tutorials etc, rather than a separate site 16:47:08 We have about 15 min left in case anyone else has any particular questions or APIs they want to highlight 16:47:29 It looks like we have some big questions that need agreed answers early in the process: 16:47:55 how do we publish low-level API docs? 3rd party tool or MediaWiki solution? 16:48:23 whatever is the first answer, how do we integrate these low-level docs and sandbox with tutorials and showcases? 16:49:36 There seem to be very different (and sometimes strong) opinions about the tools to use, and we will need to decide one, hopefull 16:49:37 y 16:50:24 otherwise developers will keep not finding the interesting APIs, not knowing what they can do, not knowing how 16:50:26 #help tell Sumana which APIs you & your teams have had the most delay in picking up due to poor documentation (autogen + human-centric tutorials) 16:50:59 #info main questions: * how do we publish low-level API docs? 3rd party tool or MediaWiki solution? * whatever is the first answer, how do we integrate these low-level docs and sandbox with tutorials and showcases? (what tools?) 16:51:15 #link Swagger demo http://petstore.swagger.wordnik.com/ 16:51:27 #link https://www.mediawiki.org/wiki/Requests_for_comment/Content_API#Structured_API_specs additional structured API tools/formats 16:51:53 qgil: whether the API docs are served through MW or not is a technical detail IMO -- what's more important is that they are searchable (search engines) and linked from the entry points on-wiki 16:52:57 being able to link to individual API doc sections is an important feature IMO 16:53:01 gwicke, sumanah, YuviPanda how could we get the relevant people to discuss and agree the right solution ref these main questions? 16:53:22 ( qgil: this topic seems to have died down mostly so I suggest we open it up for other ECT questions/updates/etc if desired) 16:53:38 <YuviPanda> A good way is for you guys to decide if the primary 'user' of this system is to be the devs writing the APIs or the users consuming them, and then put it to wikitech-l 16:54:08 again, this feels like something where I want to talk to Design 16:54:10 The Developer Hub is primarily for the users consuming the APIs 16:54:40 YuviPanda: write-only API docs sound kind of useless ;) 16:55:50 another consideration I'd like to add is that docs should reflect the current configuration 16:56:07 which means that enabling an API end point should also make its docs visible 16:56:42 it's important that the end result includes easy-to-use ways for developers to write and document new APIs AS WELL AS easy-to-use ways for novice devs to lean how to use them. And yes, I plan on dropping notes to wikitech-l 16:57:03 gwicke: how is this different from how things are now? 16:57:43 one way to ensure that is to couple the docs with the module registration, as is currently the case for PHP API modules and the Content API prototype at https://github.com/gwicke/restface/blob/master/handlers/frontend/genhandler.js#L34 16:58:20 sumanah: it is not different, but it creates a constraint on possible solutions 16:58:28 Got it. 16:59:05 One minute left. sumanah, thank you for driving an interesting presentations and discussion about this project. 16:59:26 The community needs to be aware of this project! 16:59:54 #info important to continue ensuring that docs reflect the current configuration & to continue ensuring that enabling an API end point should also make its docs visible. Gabriel suggests: "one way to ensure that is to couple the docs with the module registration, as is currently the case for PHP API modules and the Content API prototype at https://github.com/gwicke/restface/blob/master/handlers/frontend/genhandler.js#L34 " 17:00:20 ok, I'm calling this done. thank you qgil 17:00:23 #endmeeting