Engineering Community Team/Meetings/2014-07-08

From mediawiki.org

ECT IRC meeting at 16:00 UTC via #wikimedia-office connect.

Topics:

#wikimedia-office: Engineering Community office hour - July 2014[edit]

Meeting summary[edit]

#wikimedia-office: ECT office hour 8 july 2014 | Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE) | https://meta.wikimedia.org/wiki/IRC_office_hours[edit]

Meeting started by sumanah at 16:05:02 UTC. The full logs are available at https://tools.wmflabs.org/meetbot/wikimedia-office/2014/wikimedia-office.2014-07-08-16.05.log.html .


  • Non-Sumana ECT activities :) (sumanah, 16:10:49)


Meeting ended at 17:00:23 UTC.


Action items[edit]

  • sumanah to ask Analytics which APIs are most used


Action items, by person[edit]

  • sumanah
    • sumanah to ask Analytics which APIs are most used


People present (lines said)[edit]

  • sumanah (73)
  • qgil (33)
  • gwicke (19)
  • YuviPanda (15)
  • wm-labs-meetbot (4)
  • Nikerabbit (3)
  • guillom (3)
  • Lydia_WMDE (1)
  • andre__ (1)
  • Tpt (1)
  • marktraceur (1)


Generated by MeetBot 0.1.4 (http://wiki.debian.org/MeetBot)

Full log[edit]

Meeting logs

16:05:02 <sumanah> #startmeeting ECT office hour 8 july 2014 | Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE) | https://meta.wikimedia.org/wiki/IRC_office_hours
16:05:02 <wm-labs-meetbot> Meeting started Tue Jul  8 16:05:02 2014 UTC and is due to finish in 60 minutes.  The chair is sumanah. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:05:02 <wm-labs-meetbot> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:05:02 <wm-labs-meetbot> The meeting name has been set to 'ect_office_hour_8_july_2014___channel_is_logged_and_publicly_posted__do_not_remove_this_note____https___meta_wikimedia_org_wiki_irc_office_hours'
16:05:07 <sumanah> #chair qgil sumanah
16:05:07 <wm-labs-meetbot> Current chairs: qgil sumanah
16:05:10 <sumanah> qgil: ok! go.
16:05:39 <qgil> Alrigt, thank you
16:05:55 <qgil> This is the Engineering Community Team IRC meeting -- https://www.mediawiki.org/wiki/Engineering_Community_Team
16:06:36 <qgil> (I apologize for the problems, I just landed in Germany, I still don't have decent Internet at home, etc...)
16:07:01 <andre__> so far no problems appeared :)
16:07:01 <sumanah> My sympathies qgil
16:07:37 <qgil> Well, one problem is that I could not prepare the boilerplate text  :)
16:07:47 <qgil> because I just got internet running at home 10 minutes ago  :)
16:07:57 <sumanah> #link https://www.mediawiki.org/wiki/Engineering_Community_Team/Meetings/2014-07-08
16:08:28 <qgil> Anyway, the main topic planned for today is the Developer Hub revamp:
16:08:48 <sumanah> #topic Developer hub revamp https://www.mediawiki.org/wiki/Data_%26_Developer_Hub
16:08:49 <qgil> are you ready sumanah ?
16:09:07 <sumanah> qgil: actually let's maybe quickly cover what's up with the other projects first?
16:09:14 <sumanah> the other activities I mean
16:09:42 <sumanah> (mentorship, Phabricator, Wikimania planning)
16:10:03 <qgil> We can... but I guess the rest of us will improvise an update.
16:10:35 <sumanah> nod
16:10:49 <sumanah> #topic Non-Sumana ECT activities :)
16:11:26 <qgil> Just checking, I believe some weeks ago we had agreed that the main topic for today would be the Developer Hub
16:11:50 <sumanah> yes
16:12:03 <sumanah> but I assumed we'd sort of clear the other stuff out first - no matter, I can go first
16:12:13 <sumanah> #topic Developer hub revamp
16:12:21 <sumanah> #link https://www.mediawiki.org/wiki/Data_%26_Developer_Hub
16:12:21 <sumanah> Starting in mid-July, I'm working on this central developer hub, focusing on our web APIs.
16:12:28 <qgil> I haven't got time to check the announcement yet, had we mentioned the develloer hub there?
16:12:42 <sumanah> (I'm currently finishing a bunch of work on the security and architecture guidelines.)
16:13:18 <sumanah> Working with me are Brad Jorsch (anomie), and (on a more part-time basis) Moiz Syed and Juliusz Gonera
16:13:38 <sumanah> consulting as well with Dario Taraborelli & Jared Zimmerman
16:13:41 <qgil> Sorry again for the confusion. I mean, I thought we could have an IRC meeting focusing on the Developer Hub, previously announced.
16:14:00 <sumanah> No prob
16:14:13 <sumanah> #link https://www.mediawiki.org/wiki/Wikimedia_Engineering/2014-15_Goals#Engineering_Community as you can see our goals for the end of September 2014 are:
16:14:19 <sumanah> Developer Hub prototype
16:14:19 <sumanah> Landing page, 3 projects showcased, 3 APIs documented
16:14:19 <sumanah> API sandbox functional prototype
16:14:19 <sumanah> Contribution process defined
16:15:25 <sumanah> At the moment Moiz and Juliusz are working on a prototype just to help us understand what the interface will look like. For sample data/a sample API to document, they're starting  with the recent changes streaming API -- stream.wikimedia.org is close to ready
16:17:00 <sumanah> And my first task specific to this hub will be to interview engineering managers + community to find out which APIs people want to use and have trouble using. Also I'll prioritize Gabriel's new content API. DEADLINE: late July
16:17:52 * YuviPanda suggests also asking analytics to see which APIs are most used
16:17:55 * YuviPanda waves as well
16:18:02 <sumanah> YuviPanda: this is a good idea!
16:18:10 <sumanah> #action sumanah to ask Analytics which APIs are most used
16:18:29 <sumanah> YuviPanda: in a sense the highest-priority ones are the ones where there's a big delta between how often they're used and the quality of the existing docs
16:18:45 <YuviPanda> sumanah: indeed, since they might not have been used much because people didn't know they exist.
16:18:49 <sumanah> yes
16:19:38 <sumanah> So those are some of the facts .... YuviPanda I can start the more interactive portion here by asking you :) whether there are APIs where you think they're very underappreciated
16:20:14 <YuviPanda> indeed. most people don't know about the CommonsMetadata api, for example. I myself only learnt about the very useful assert= last few days
16:20:29 <sumanah> #info <YuviPanda> most people don't know about the CommonsMetadata api, for example. I myself only learnt about the very useful assert= last few days
16:20:57 <YuviPanda> the query interface our API has is also rather powerful (with generators and all) but not too well known
16:21:31 <YuviPanda> in the sense people don't utilize the full power of it (since it is a bit hard to fully grok)
16:21:42 <sumanah> Lydia_WMDE: marktraceur Nikerabbit ragesoss and Tpt - I would also be curious about your answers. What Wikimedia APIs do you think are underappreciated, where there's a lot of possibility there but people don't know about them?
16:21:55 * YuviPanda mentions the wikidata API as well
16:22:11 <sumanah> #info Yuvi says: the query interface our API has is also rather powerful (with generators and all) but not too well known, in the sense people don't utilize the full power of it (since it is a bit hard to fully grok)
16:22:40 <Tpt> Strong +1 for the query interface of the API
16:23:24 <sumanah> I believe that my intern Frances, who had already figured out how to get data out of Wikipedia and Wikivoyage, had a more difficult time figuring out how to get data out of Wikidata - the structure of things, really
16:23:25 <YuviPanda> WikiData as well, especially once the (currently WIP) search API interface is made available. I'd say when that becomes available it'll be the most powerful/useful API we expose
16:23:33 <sumanah> YuviPanda: link?
16:23:36 <YuviPanda> sumanah: moment
16:23:38 <sumanah> thanks
16:23:58 <YuviPanda> sumanah: http://lists.wikimedia.org/pipermail/wikitech-l/2014-July/077406.html
16:24:10 <sumanah> ah, yes
16:24:12 <Lydia_WMDE> sumanah: wikidata obviously :P
16:24:23 <sumanah> Lydia_WMDE: :-)
16:25:22 <sumanah> #link http://juliuszgonera.com/wddh/showcase/wikistream.html Prototype: http://juliuszgonera.com/wddh/ (temporary)
16:25:25 <guillom> Quick question: sumanah, when you mention "Contribution process defined" as one of the goals, does that include determining how to leverage all the existing documentation on mediawiki.org to avoid duplication?
16:25:44 <Nikerabbit> sumanah: to your earlier question: all of them ;)
16:25:45 <sumanah> guillom: The short answer is yes
16:25:54 <sumanah> Nikerabbit: OK, now get serious :)
16:25:57 <guillom> ok :) Thanks
16:26:34 <marktraceur> sumanah: GlobalUsage is pretty neat.
16:26:35 <sumanah> Nikerabbit: I imagine not all of them are equally "hidden diamonds". I'm looking for which are the most hidden-diamond-y
16:27:10 <qgil> guillom, the API documentation of the Developer Hub shouldn't be just editable like a regular wiki page, but it cannot be editable only by e.g. Sumana. There must be a point in between, that should be defined by this contribution process.
16:27:26 <sumanah> #info GlobalUsage: "View the global usage of images in a wiki farm with shared image repository. Used by Multimedia: https://www.mediawiki.org/wiki/Multimedia/Architecture
16:28:04 <guillom> qgil: Agreed, and I mentioned FlaggedRevs at some point in Zürich. I know there are some strong opinions on this among the team working on this hub, so I'm glad to know that sumanah is keeping an eye on the issue :)
16:28:10 <sumanah> :-)
16:28:23 <Nikerabbit> sumanah: well, for example Translate extension has the translation memory API
16:28:36 <Nikerabbit> can't think  of anything else right now, have to go ;)
16:28:42 <sumanah> #link http://lists.wikimedia.org/pipermail/wikitech-l/2014-July/077406.html Wikidata's search API interface - currently a work in progress
16:28:55 <sumanah> Nikerabbit: got it. https://www.mediawiki.org/wiki/Talk:Data_%26_Developer_Hub - leave thoughts there too!
16:29:09 <qgil> At this point everything needs to be defined, e.g. part of the documentation might be generated directly by the source code, and therefore would be editable only though patches.
16:29:12 <sumanah> #info Niklas suggests the Translate extension's translation memory API
16:29:57 <gwicke> fwiw, for the content API I am planning to use one of the fairly structured tools / formats linked at https://www.mediawiki.org/wiki/Requests_for_comment/Content_API#Structured_API_specs; this will be the pure API docs, without much tutorial content
16:30:38 <gwicke> the nice thing is that tools like swagger auto-generate a sandbox & browsable docs based on the API spec
16:30:51 <sumanah> I love that ... yeah I was about to say, sandboxes and mocks
16:31:01 * YuviPanda notes that our current APISandbox could use some love
16:31:32 <sumanah> YuviPanda: you may have seen, in the goals for end of September on this project: "API sandbox functional prototype"
16:32:05 <YuviPanda> sumanah: right, was wondering if the 'prototype' means a complete rewrite or fixes to the current one
16:32:09 <gwicke> here's a swagger demo: http://petstore.swagger.wordnik.com/
16:33:43 <sumanah> YuviPanda: I think that is a good question and my personal professional opinion is "it depends" - if you have facts/opinions to share that would lean one way or the other, feel free
16:34:48 <qgil> A deep problem to solve: a common way for all the APIs to be documented and showcased. For instance, it is good that gwicke and his team are using Swagger now for their API, but what about the rest?
16:34:57 <YuviPanda> sumanah: right. from the POV of a developer, I'd think the sandbox would be useful only if it is complete, and the current one is (because it autogenerates itself). so anything that ensures it is complete without much human intervention is good, wether it be generic new one or improvements to this one
16:35:52 <gwicke> qgil: I'm not 100% decided between swagger & json schema hypermedia yet
16:36:25 <gwicke> the latter looks more powerful, but I haven't evaluated the front-end options for it yet
16:36:36 <qgil> I any case, it would be good that "your" decision and other "local" decisions made by other dev teams would converge in the Developer Hub
16:36:41 <gwicke> it has more buy-in from the likes of Google & Heroku though
16:37:29 <qgil> what do you think, sumanah ?
16:37:55 <sumanah> qgil: I feel as though I'd like to talk with Design about it
16:39:13 <sumanah> gwicke: do you have a favorite demo for the json schema hypermedia approach, so I can compare?
16:39:23 <qgil> It would be good to see what gwicke ìs looking for when thinking of Swagger / json schema hypermedia, and see how far MediaWiki + APIsandbox is from there,
16:39:41 <sumanah> I played with Swagger but it was a few years back so I should check it out again
16:39:45 <sumanah> and compare API Sandbox
16:39:56 <qgil> and whether we should try to solve everything with repos + MediaWiki or bet on a 3rd party tool as well
16:40:06 <gwicke> qgil: I'm mostly looking for a machine-readable API spec, which can also be used to auto-generate rich docs, sandboxes etc
16:41:22 <gwicke> with evolving APIs consistency between docs & implementation is important, and having a single machine-readable spec can help with avoiding divergence
16:41:31 <sumanah> yup
16:42:37 <qgil> gwicke, in our Developer Hub discussions, we have been talking about a way to export strict API documentation from repos to MediaWiki, while writing tutorials etc directly on MediaWiki.
16:43:03 <qgil> I don't know how far we are from this scenario, and how desirable it is compared to using a 3rd party tool, though
16:43:39 <gwicke> I think there is a need for both low-level API docs, and higher-level tutorials / entry points
16:43:52 <sumanah> yes
16:43:53 <qgil> The idea is that such extension + editing tutorials in MediaWiki would allow us to have everthing under the same (search) umbrella)
16:44:14 <sumanah> gwicke: yup, I've been saying that for years ;-) glad to have a chance to work on both
16:44:33 <qgil> gwicke, agreed, the point is, can we use a MediaWiki extension to publish the low-level API docs without needing a 3rd party tool that will create a separate site?
16:44:53 <YuviPanda> should be wary of NIH
16:45:13 <gwicke> something like swagger is easy to hook up
16:45:20 <sumanah> hope so
16:45:49 <gwicke> the main work is figuring out which format / tool to use, and creating the spec
16:46:07 <gwicke> and then having a process that ensures that the spec remains in sync with the code
16:46:13 <qgil> YuviPanda, indeed, I'm just saying that the end result would be better something that developers can search and find in a single place, integrated with tutorials etc, rather than a separate site
16:47:08 <sumanah> We have about 15 min left in case anyone else has any particular questions or APIs they want to highlight
16:47:29 <qgil> It looks like we have some big questions that need agreed answers early in the process:
16:47:55 <qgil> how do we publish low-level API docs? 3rd party tool or MediaWiki solution?
16:48:23 <qgil> whatever is the first answer, how do we integrate these low-level docs and sandbox with tutorials and showcases?
16:49:36 <qgil> There seem to be very different (and sometimes strong) opinions about the tools to use, and we will need to decide one, hopefull
16:49:37 <qgil> y
16:50:24 <qgil> otherwise developers will keep not finding the interesting APIs, not knowing what they can do, not knowing how
16:50:26 <sumanah> #help tell Sumana which APIs you & your teams have had the most delay in picking up due to poor documentation (autogen + human-centric tutorials)
16:50:59 <sumanah> #info main questions: * how do we publish low-level API docs? 3rd party tool or MediaWiki solution? * whatever is the first answer, how do we integrate these low-level docs and sandbox with tutorials and showcases? (what tools?)
16:51:15 <sumanah> #link Swagger demo http://petstore.swagger.wordnik.com/
16:51:27 <sumanah> #link https://www.mediawiki.org/wiki/Requests_for_comment/Content_API#Structured_API_specs additional structured API tools/formats
16:51:53 <gwicke> qgil: whether the API docs are served through MW or not is a technical detail IMO -- what's more important is that they are searchable (search engines) and linked from the entry points on-wiki
16:52:57 <gwicke> being able to link to individual API doc sections is an important feature IMO
16:53:01 <qgil> gwicke, sumanah, YuviPanda how could we get the relevant people to discuss and agree the right solution ref these main questions?
16:53:22 <sumanah> ( qgil: this topic seems to have died down mostly so I suggest we open it up for other ECT questions/updates/etc if desired)
16:53:38 <YuviPanda> A good way is for you guys to decide if the primary 'user' of this system is to be the devs writing the APIs or the users consuming them, and then put it to wikitech-l
16:54:08 <sumanah> again, this feels like something where I want to talk to Design
16:54:10 <qgil> The Developer Hub is primarily for the users consuming the APIs
16:54:40 <gwicke> YuviPanda: write-only API docs sound kind of useless ;)
16:55:50 <gwicke> another consideration I'd like to add is that docs should reflect the current configuration
16:56:07 <gwicke> which means that enabling an API end point should also make its docs visible
16:56:42 <sumanah> it's important that the end result includes easy-to-use ways for developers to write and document new APIs AS WELL AS easy-to-use ways for novice devs to lean how to use them. And yes, I plan on dropping notes to wikitech-l
16:57:03 <sumanah> gwicke: how is this different from how things are now?
16:57:43 <gwicke> one way to ensure that is to couple the docs with the module registration, as is currently the case for PHP API modules and the Content API prototype at https://github.com/gwicke/restface/blob/master/handlers/frontend/genhandler.js#L34
16:58:20 <gwicke> sumanah: it is not different, but it creates a constraint on possible solutions
16:58:28 <sumanah> Got it.
16:59:05 <qgil> One minute left. sumanah , thank you for driving an interesting presentations and discussion about this project.
16:59:26 <qgil> The community needs to be aware of this project!
16:59:54 <sumanah> #info important to continue ensuring that docs reflect the current configuration & to continue ensuring that  enabling an API end point should also make its docs visible. Gabriel suggests: "one way to ensure that is to couple the docs with the module registration, as is currently the case for PHP API modules and the Content API prototype at https://github.com/gwicke/restface/blob/master/handlers/frontend/genhandler.js#L34 "
17:00:20 <sumanah> ok, I'm calling this done. thank you qgil
17:00:23 <sumanah> #endmeeting
Retrieved from "https://www.mediawiki.org/w/index.php?title=Engineering_Community_Team/Meetings/2014-07-08&oldid=5422473"