Architecture meetings/RFC review 2014-04-29

22:00-23:00 UTC on Tuesday, 29 April at.

Requests for Comment to review

 * 1) Requests for comment/MediaWiki libraries
 * 2) Requests for comment/Third-party components

Summary and logs
We have some open questions for Markus regarding Composer, and for administrators of MediaWiki installations. Would Composer work in your environment?

And it sounds like we only want to replace an artisanal, homegrown MediaWiki component with an externally maintained library if the feature is complicated, upstream is friendly and responsive and aligned with us, and so on, on a case-by-case basis.

https://gerrit.wikimedia.org/r/#/c/119939/ still needs review.

Meeting summary

 * LINK: https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-04-29 (sumanah, 22:00:57)
 * asking for help with a few RfCs (sumanah, 22:01:50)
 * Jon Robson needs help writing the initial code for https://www.mediawiki.org/wiki/Requests_for_comment/Allow_styling_in_templates (sumanah, 22:02:00)
 * https://www.mediawiki.org/wiki/Requests_for_comment/Scoping_site_CSS and https://www.mediawiki.org/wiki/Requests_for_comment/Book_management need new coauthors to move forward (sumanah, 22:02:15)


 * MediaWiki libraries (sumanah, 22:03:01)
 * LINK: https://www.mediawiki.org/wiki/Requests_for_comment/MediaWiki_libraries Ryan Lane's proposal (better handling and versioning libraries that are MW extensions) (sumanah, 22:03:12)
 * See https://www.mediawiki.org/wiki/Talk:Requests_for_comment/MediaWiki_libraries - we just need to merge "Add Composer managed libraries" https://gerrit.wikimedia.org/r/#/c/119939/ in order to complete part I of this. (sumanah, 22:03:46)
 * Then Part II is actually moving everything else into that directory, and also making it reasonable to move some extensions into that directory. (sumanah, 22:03:47)
 * the Composer RfC: https://www.mediawiki.org/wiki/Requests_for_comment/Extension_management_with_Composer (sumanah, 22:08:44)


 * Extension management with Composer (TimStarling, 22:19:31)
 * LINK: https://www.mediawiki.org/wiki/Requests_for_comment/Extension_management_with_Composer (TimStarling, 22:19:37)
 * LINK: http://www.bn2vs.com/blog/2013/11/24/introduction-to-composer-for-mediawiki-developers/ (sumanah, 22:24:03)
 * The ops concerns can be dealt with for the WMF cluster using a technique like I used in the Scholarships app (which uses Composer and is live on the cluster). The technique in patch https://gerrit.wikimedia.org/r/#/c/119939/ is very similar to what Scholarships does to freeze the external libs. (sumanah, 22:31:33)
 * Technically it should be trivial to convert most extensions to a composer library (sumanah, 22:31:50)
 * Wikidata team has some Composer experience per aude's explanation (sumanah, 22:32:48)
 * discussion of whether to deal with multiple packaging systems or just one, use something like https://github.com/derrabus/composer-webui for a web UI, CLI vs web (doing web UI securely is ?) (sumanah, 22:33:38)
 * IDEA: question for composer-for-extensions: best way to fit in with our source-control-based deployment? (brion, 22:37:11)
 * IDEA: question for composer-for-extensions: web ui possible for small installations? (compare with wordpress’s installer & extension manager) avoid chmod 777 (brion, 22:37:11)
 * IDEA: question for composer-for-extensions: avoid proliferation of package managers, but make sure we can still install extensions manually if need be or for migrations (brion, 22:37:12)
 * A composer extension can register several customization scripts to hooks -- https://getcomposer.org/doc/articles/scripts.md (sumanah, 22:40:59)
 * ACTION: sumanah to get input from Wikia, ShoutWiki, wikiHow, & other 3rd-party folks on the Composer RfC talk page (sumanah, 22:42:16)
 * ACTION: sumanah to bring this spirited discussion to the attention of Markus and ask him if he can move forward, incorporate suggestions, maybe talk to aude from Wikidata & bd808 for specific HOWTO recommendations to incorporate (sumanah, 22:43:01)


 * Third-party components (sumanah, 22:43:16)
 * LINK: https://www.mediawiki.org/wiki/Requests_for_comment/Third-party_components (sumanah, 22:43:20)
 * Tyler Romeo's proposal (replacing some MW components with more widely used libraries, especially Symfony libraries) (sumanah, 22:43:26)
 * parent5446 is looking for feedback on the general concept.  (sumanah, 22:43:41)
 * LINK: https://www.mediawiki.org/wiki/Upstream_projects (sumanah, 22:56:08)
 * this seems like something that�ll be easier to treat once we have a consistent dependency manager :D (sumanah, 22:57:36)
 * we feel mixed about the general concept of choosing to reuse external stuff instead of writing it ourselves - depends on code quality, which you gotta take on a case-by-case basis (sumanah, 22:58:41)
 * part of the issue with the way we do things now is that, while being potentially superior along many vectors (performance, reliabilitiy, security), we seem to fall down on reusability, which is unfortunate (sumanah, 22:59:00)
 * also, we're more likely to be interested in a replacement if the feature is quite complicated, e.g., JS minification (sumanah, 22:59:52)
 * examples of good PHP code out there that's reusable: monolog, Symfony2 stuff (sumanah, 23:00:16)

Meeting ended at 23:01:25 UTC.

Action items

 * sumanah to get input from Wikia, ShoutWiki, wikiHow, & other 3rd-party folks on the Composer RfC talk page
 * sumanah to bring this spirited discussion to the attention of Markus and ask him if he can move forward, incorporate suggestions, maybe talk to aude from Wikidata & bd808 for specific HOWTO recommendations to incorporate

Full log
22:00:34 #startmeeting RfC review: MediaWiki libraries and third-party components | Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE). https://meta.wikimedia.org/wiki/IRC_office_hours 22:00:34  Meeting started Tue Apr 29 22:00:34 2014 UTC and is due to finish in 60 minutes. The chair is sumanah. Information about MeetBot at http://wiki.debian.org/MeetBot. 22:00:34  Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 22:00:34  The meeting name has been set to 'rfc_review__mediawiki_libraries_and_third_party_components___channel_is_logged_and_publicly_posted__do_not_remove_this_note___https___meta_wikimedia_org_wiki_irc_office_hours' 22:00:49 #chair sumanah brion andrewbogott 22:00:49  Current chairs: andrewbogott brion sumanah 22:00:57 #link https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-04-29 22:01:35 Thanks for coming all 22:01:43 First, an announcement of a few RfCs that need some help 22:01:50 #topic asking for help with a few RfCs 22:02:00 #info Jon Robson needs help writing the initial code for https://www.mediawiki.org/wiki/Requests_for_comment/Allow_styling_in_templates 22:02:09 jdlrobson - tell me if that's wrong :) 22:02:15 #info https://www.mediawiki.org/wiki/Requests_for_comment/Scoping_site_CSS and https://www.mediawiki.org/wiki/Requests_for_comment/Book_management need new coauthors to move forward 22:02:23 so check those out. 22:02:57 OK, now on to the first topic today, Ryan_Lane's MediaWiki libraries RfC. Then we'll move on to parent5446's RfC on 3rd-party components 22:03:01 #topic MediaWiki libraries 22:03:12 #link https://www.mediawiki.org/wiki/Requests_for_comment/MediaWiki_libraries Ryan Lane's proposal (better handling and versioning libraries that are MW extensions) 22:03:12 * marktraceur frows at Book Management 22:03:36 #chair sumanah brion andrewbogott TimStarling 22:03:36  Current chairs: TimStarling andrewbogott brion sumanah 22:03:41 So I have a question about this RfC. It mentions LdapAuthentication as an example. What library in LdapAuthentication would we be packaging with core? 22:03:46 #info See https://www.mediawiki.org/wiki/Talk:Requests_for_comment/MediaWiki_libraries - we just need to merge "Add Composer managed libraries" https://gerrit.wikimedia.org/r/#/c/119939/ in order to complete part I of this. 22:03:47 #info Then Part II is actually moving everything else into that directory, and also making it reasonable to move some extensions into that directory. 22:04:04 B/c I understand the RFC in the context of the Gerrit patch 22:04:19 Libraries that are being used in core will need to be managed somehow 22:04:23  parent5446: a number of extension use LDAP as a library 22:04:31 i’m not sure i understand this rfc; is the idea to package specific libraries in mediawiki core? 22:04:39  I'd split the necessary functionality out of the LDAP extension into a common library 22:04:44 or to provide a general place to put libraries? 22:04:47 ...how? Are there extensions that use LDAP other than LdapAuthentication? 22:04:53  I think the idea is to create a separate class of extensions that you put in libs/ instead of extensions/ 22:05:05  TimStarling: yep 22:05:13 first question: why? 22:05:25 second question: what needs to be different other than the filename ‘extensions’ -> ‘libs’? 22:05:36  because there's now a lot of extensions being used as libraries and they aren't being versioned with mediawiki 22:05:40 OK, but the gerrit patch makes it look like we'd be packaging them with core. So it makes it sound like we'd be packaging libraries within core that are not actually used in core. 22:05:45 third question: does that patch adding a libs directory help or hinder this rfc’s recommendations? 22:05:53 (the patchset is by bd808 in case he wants to comment) 22:05:57  most of these could be bundled with core 22:06:11  we have includes/libs already for things that are bundled 22:06:13  and should definitely be versioned with releases 22:06:28  TimStarling: that's for things that core depends on, but not for things extensions may need to depend on 22:06:45 so is this directory’s contents something is fixed, that extensions can depend on always being present? 22:06:55 or is it something extensible, that extensions can rely on having a way to install things there? 22:07:09 I don't really think it's a good idea to package extension libraries as part of core if core is not using it. 22:07:23 The best solution for Ryan's RFC would probably be enabling the full use of Composer (or a similar system) with wm-core 22:07:27 Maybe an extension needs a different library version, and can't use it because core decided to package it 22:07:48 * sumanah invited Jeroen to discuss composer-y stuff with us in this chat but - ah, there's JeroenDeDauw! 22:08:03  Just saw calendar notification now 22:08:07  Anout to head off 22:08:14 <Ryan_Lane> so, maybe I should describe my issue, rather than this specific solution? 22:08:18 JeroenDeDauw: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/20140429.txt starting at 2200 in case you wanna see our talk till now 22:08:26 s/talk/chatter/ 22:08:29 <TimStarling> with composer, I'd like to know whether you think extensions should be in mediawiki/* or mediawiki/extensions/* 22:08:32 Ryan_Lane: yes please :D 22:08:42 <TimStarling> in the package names 22:08:44 #info the Composer RfC: https://www.mediawiki.org/wiki/Requests_for_comment/Extension_management_with_Composer 22:08:57 <Ryan_Lane> so, we have extensions. those extensions for a long time only had a dependency on a specific mediawiki version 22:09:02 <TimStarling> because JeroenDeDauw put a few in mediawiki/*, but it seems a bit weird to me 22:09:14 <Ryan_Lane> now we have extensions that only exist as libraries for use by extensions 22:09:14 <JeroenDeDauw> TimStarling: vendor/package 22:09:18 <JeroenDeDauw> TimStarling: two levels 22:09:24 (many extensions require external dependencies as well…) 22:09:52 extensions can also be hooks onto other extensions -- but that doesn't make the 'parent' extension a library 22:11:03 * aude waves 22:11:09 <TimStarling> hmm 22:11:12 this seems to be mostly an argument about our release process... in theory, everything in a release is stably versioned and has release branches for backports 22:11:43 <Ryan_Lane> well, it's more an issue that we have no reasonable way to specify extension compatibility 22:12:00 <Ryan_Lane> and now we need to specify extension compatibility with mediawiki and with other library extensions 22:12:02 I mean, this is basically what composer was made to do 22:12:21 Ryan_Lane: don’t you just grab them from the same version branch? 22:12:21 Except people don't want to use composer in core 22:12:34 * sumanah sees parent5446's point. If this is a configuration/dependency management problem, maybe we should go with the thing that does that 22:12:40 <Ryan_Lane> brion: very few people use version branches 22:12:52 Ryan_Lane: how would this solve that? 22:12:59 we're cutting the version branches automatically now 22:13:14 <greg-g> mwalker: but everyone else still uses tarballs 22:13:15 <Ryan_Lane> my proposal was to have the common libraries shipped with mediawiki and locked with the version 22:13:22 … 22:13:31 so something that’s common to a group of extensions, and has no relation to mediawiki core... 22:13:34 … would be shipped with core? 22:13:38 Shipping common libraries is kind of like a band-aid to the actual problem 22:13:39 <TimStarling> I think it's fine to have extensions that depend on other extensions 22:13:50 parent5446: I think we are open to using composer in core, but new need to figure out how to do that in a way that works for JeroenDeDauw's use case and mine (which is basically the same as Ryan_Lane's) 22:14:07 <Ryan_Lane> the issue is that most of these library extensions probably should just be core 22:14:09 <TimStarling> drupal has a DIY system for this 22:14:18 <TimStarling> or of course we could use composer 22:14:25 I thought the main problem was that people didn't like how it didn't interact properly with Git? 22:14:29 <Ryan_Lane> if 20 extensions are using an extension, why isn't it a core feature? 22:14:30 Emufarmers: Dantman: would love to have your input here as people who work on non-WMF MediaWiki installations 22:14:32 <TimStarling> or support any other package management system, like PEAR or .deb or whatever 22:14:50 Ryan_Lane: are those 20 extensions all specific and only used by a tiny minority of installations? 22:14:54 greg-g, we bundle extensions into our release tarballs... 22:14:55 or are they used all the time by most? 22:15:05 <Ryan_Lane> to be fair, I'd be fine with composer, if it can be used reasonably 22:15:23 Well many extensions have already been made to work as composer libraries 22:15:24 <TimStarling> if a facility is widely used, I think it can be in core, you know that I am not one of the people who has a narrow view of what core should be 22:15:25 <greg-g> mwalker: yeah. (/me doesn't want to derail, it was a minor point) 22:15:27 it feels like the solution here is “a dependency-managing installation system for extensions" 22:15:38 <Ryan_Lane> TimStarling: I'm not saying you are :) 22:15:42 <TimStarling> but yes, we need extension dependencies for various reasons 22:15:42 Which means we should check out https://www.mediawiki.org/wiki/Requests_for_comment/Extension_management_with_Composer 22:15:51 and maybe expand it 22:16:04 <Dantman> sumanah: I'm not 100% certain of the topic, or even whether "extensions as composer modules" is included in this or excluded. 22:16:18 <TimStarling> not just for libraries -- some extensions hook others, for example 22:16:28 <Ryan_Lane> yep 22:16:41 <Ryan_Lane> it's something we've been punting on forever 22:16:43 Dantman: sounds like we're talking about including it, maybe 22:16:44 <TimStarling> some use RL modules from another extension 22:17:09 Dantman: the topic is https://www.mediawiki.org/wiki/Requests_for_comment/MediaWiki_libraries Ryan Lane's proposal (better handling and versioning libraries that are, or are used by, MW extensions) 22:17:22 <Ryan_Lane> so, let's drop this proposal and maybe talk about the composer suggestions? 22:17:43 <Ryan_Lane> or is that not how this process works? heh 22:17:52 Well if we need to itemize this to dependencies on individual RL modules and hooks, there's not really any system that handles that 22:18:05 With composer it's either you depend on the entire extension or you don't 22:18:12 <Dantman> Well this idea of extensions as composer modules has annoyed me over and over again. 22:18:14 Unless you separate into further libraries 22:18:15 Ryan_Lane: I'm fine with the process working like that, if you want to abandon your RfC and become a coauthor on the Composer one :) 22:18:53 Extension management with composer would allow managing dependent libraries as well, for the extensions. My use-case of wanting an external library for core is slightly different, but could pervert the system by using a basically empty extension just to get the library management. 22:19:00 andrewbogott: I guess one question for you as Ops representative is how Ops feels about Composer (although I assume if Ryan's fine with it then you would be too) 22:19:02 <Ryan_Lane> wasn't that another RfC that was in the queue for today? 22:19:21 Ryan_Lane: no, it wasn't; it is one that I wanted people to have fresh in their memory because I predicted it would interrelate 22:19:24 <Ryan_Lane> the current implementation of composer doesn't fit with our deployment process 22:19:31 <TimStarling> #topic Extension management with Composer 22:19:37 <TimStarling> #link https://www.mediawiki.org/wiki/Requests_for_comment/Extension_management_with_Composer 22:19:45 Yeah, I was going to say: I don't think Ryan_Lane is fine with it :) 22:19:47 but since Markus isn't here and JeroenDeDauw needs to head off, it's suboptimal. Ah well 22:20:22 <Ryan_Lane> composer, as currently implemented, means you need to run composer on every node you deploy to. it also makes it impossible to disable any individual extension 22:20:30 <TimStarling> I don't think there's any reason to use composer in production 22:20:38 Ryan_Lane: no 22:20:44 you make a build 22:20:57 with all the stuff installed 22:20:59 run tests on it 22:21:00 The ops concerns can be dealt with for the WMF cluster using a technique like I used in the Scholarships app (which uses Composer and is live on the cluster). 22:21:05 then deploy that 22:21:14 all the stuff checked in, including composer.lock 22:21:17 <Ryan_Lane> what about other people using similar deployment methods? 22:22:06 what aude has described is basically how we're having to deploy Node.JS apps 22:22:08 it's also possible to have an internal satis (packagist) system 22:22:24 The technique in patch https://gerrit.wikimedia.org/r/#/c/119939/ is very similar to what Scholarships does to freeze the external libs. 22:22:24 that only knows about our own stuff 22:22:26 <TimStarling> I'd rather not have any kind of external dependency for source in production -- just our own git servers 22:22:33 <Ryan_Lane> so, does this assume that all WMF extensions will also have composer data? 22:22:48 can composer use private repos? 22:22:50 <TimStarling> the wikidata thing is to run composer locally and then push the result into git, right? 22:22:51 probably our own satis + a build would work 22:22:54 brion: yes 22:22:57 Technically it should be trivial to convert most extensions to a composer library 22:22:59 yes looks like good 22:23:00 :D 22:23:14 TimStarling: yes 22:23:28 so what does it look like to composer-ize an extension? what sort of migration path would we be looking at? 22:23:38 including it's important to carefully check composer lock and the diff 22:23:40 and would this just handle fetching files, or would it also handle installation, enabling, etc? 22:23:45 You make a composer.json file that tells composer to include the main extension file 22:23:47 brion, http://www.bn2vs.com/blog/2013/11/24/introduction-to-composer-for-mediawiki-developers/ 22:23:54 plus all our tests run and then only after they pass, it gets merged 22:23:59 <TimStarling> composer fetches files and handles class autoloading 22:23:59 thanks 22:24:03 #link http://www.bn2vs.com/blog/2013/11/24/introduction-to-composer-for-mediawiki-developers/ 22:24:36 <TimStarling> it has its own autoload system, and does clever things like scanning .php files on install for class definitions 22:24:41 some of a problem will be when some other extension wants to use same library that wikidata uses 22:24:58 <TimStarling> in addition to PSR-0 of course 22:25:01 would be nice to have it registered in a shared place 22:25:01 related question: 22:25:12 obviously small installations would love an extension installation manager. 22:25:23 would tying to composer force us to require CLI for extension setup? 22:25:37 or would it allow for a web control panel as well, given appropriate permissions on server? 22:25:42 ok, it sounds like there is a lot of interest in Composer usage. Ryan_Lane does this interest you enough to make you wanna collaborate on investigating things after today, working on the RfC with Markus Glaser or whoever else wants to work on this? JeroenDeDauw maybe? 22:25:43 TimStarling: we also use composer option to dump a classmap (helps optimize) 22:25:54 because we're not gonna figure all this out and approve it today 22:25:57 so in the end it is like AutoLoader.php 22:25:58 <TimStarling> we can support any number of pacakge management systems in parallel 22:26:01 brion: Yes, unless we make a PHP interface for composer, which we can 22:26:13 parent5446: that works for me 22:26:13 <TimStarling> we don't need to support composer exclusively 22:26:20 ^that as well 22:26:26 hmmmmmmmmmmm 22:26:30 i hate mixing package managers though 22:26:38 +1 to brion 22:26:40 brion: a php interface would be cool, but not aware of one 22:26:45 <TimStarling> as long as we document dependencies as well as putting them in composer.json, people can keep on installing things the old way 22:26:50 <Ryan_Lane> sumanah: I can't really spend time on this. I need this as a user of mediawiki, though 22:26:50 <Dantman> I also have doubts about being able to do a web interface around composer. 22:26:50 probably would be fairly simple 22:27:00 <Dantman> Especially a secure one. 22:27:01 Ryan_Lane: totally fair - just wanted to check 22:27:27 Dantman: secure is the hard part. wordpress updates for instance seem to jump through weird hoops to ssh into itself or some weird shit :D 22:27:32 it scares me 22:27:38 but is an oh-so-useful feature 22:28:04 <TimStarling> it's a bit more secure than just using the web user for installation 22:28:25 chmod 777 22:28:26 <TimStarling> at least after installation, the source tree is not writable by the webserver 22:28:31 yeah :D 22:28:35 Wordpress also allows using FTP to update 22:28:39 <Dantman> Most of the important metadata is still in PHP. So you can't read it without first fetching and installing the extension. 22:28:41 *shudder* 22:28:45 Google found https://github.com/derrabus/composer-webui for me 22:29:36 Looking at the code for what bd808 just posted, you can basically just include composer itself, and use it's internal interface 22:29:36 hexmode: you talk to Markus a lot, I assume :-) Do you know whether Markus has time to respond to any of these points soon? 22:29:43 (since he proposed Composer-y stuff) 22:30:11 <Dantman> It's also a dead end if you can't do shell execution from php. (Don't we still sort of support that?) 22:30:12 sounds like we’ve got a lot of questions and ideas to queue up for that :D 22:30:30 brion: help me summarize for the notes? 22:30:36 <Dantman> Side note on composer itself. 22:30:44 You wouldn't need shell access to use composer from PHP 22:31:00 <Dantman> Hmmm... right. 22:31:33 #info The ops concerns can be dealt with for the WMF cluster using a technique like I used in the Scholarships app (which uses Composer and is live on the cluster). The technique in patch https://gerrit.wikimedia.org/r/#/c/119939/ is very similar to what Scholarships does to freeze the external libs. 22:31:44 <Dantman> Composer is similar to other package managers that install things locally. 22:31:50 #info Technically it should be trivial to convert most extensions to a composer library 22:31:52 <Dantman> Like npm and bower. 22:32:37 <Dantman> However it's unique (in a flawed way) at being the only one that requires that every installed package be present inside the .json file. 22:32:48 #info Wikidata team has some Composer experience per aude's explanation 22:33:32 <Dantman> Both bower and npm can install packages locally without adding them to the .json file you commit. (If you want to add it you use --save) 22:33:36 <TimStarling> how would registration be handled? 22:33:38 #info discussion of whether to deal with multiple packaging systems or just one, use something like https://github.com/derrabus/composer-webui for a web UI, CLI vs web (doing web UI securely is ?) 22:33:45 You can add arbitrary libraries using Composer at the command line, but it writes them into the lock file 22:34:06 [15:28:39] <Dantman> Most of the important metadata is still in PHP. So you can't read it without first fetching and installing the extension. <-- Yeah, I was thinking about that as well. I had an idea that we could just put all the extension registration stuff (hooks, classes, basically mw:User:Legoktm/extension_registration) into a JSON file, so anything can read it without actually installing the extension 22:34:21 Which is very similar to using a require.txt file with pip in python or ruby's bundler 22:35:14 <TimStarling> would composer installation somehow register the extension? or would registration require a line to be added manually to LocalSettings.php? 22:35:32 TimStarling: can be done wither way 22:35:59 i prefer manually registration for prodution 22:35:59 <TimStarling> that may be a question for discussion on the RFC page then 22:36:20 TimStarling: If you set the autoloader to include the main file, it would not need lines added to LocalSettings 22:36:25 so we can enable something for one wiki but not an other (e.g.) 22:36:35 <Dantman> My point is, if it weren't for that flaw in composer. It would have never been necessary to delete the composer.json that we used to have in core. 22:36:36 like we do w/o composer 22:36:45 A composer extension can register several customization scripts to hooks -- https://getcomposer.org/doc/articles/scripts.md 22:37:11 #idea question for composer-for-extensions: best way to fit in with our source-control-based deployment? 22:37:11 #idea question for composer-for-extensions: web ui possible for small installations? (compare with wordpress’s installer & extension manager) avoid chmod 777 22:37:11 i hope i’m using the right magic tags 22:37:12 #idea question for composer-for-extensions: avoid proliferation of package managers, but make sure we can still install extensions manually if need be or for migrations 22:37:12 brb 22:37:38 brion: yeah, you are :-) https://wiki.debian.org/MeetBot #idea works 22:37:46 ok ... 22:37:53 did we get the key points? 22:38:05 also #info #action #help #link 22:38:11 <TimStarling> maybe we should have our own admin console for registration 22:38:21 whoa colloquy just dropped the last 5 minutes on me in a batch :D 22:38:37 <TimStarling> and it would detect extensions installed by composer for the purposes of displaying the list 22:38:53 Dantman: Part of the problem is that core is both a library and an application. Composer doesn't grok that. 22:38:55 <TimStarling> that way you can disable them, but it is still easy to use 22:38:57 TimStarling: i like that 22:39:08 composer for physical installation, but we have a separate activation as we do now 22:39:19 this works well for multi-site farms as well as for shipping defaults etc 22:40:36 <TimStarling> you could pull the list of enabled extensions out of the DB or some other web-writable place 22:40:46 * sumanah invited folks from Wikia, ShoutWiki, & wikiHow to come talk about this; will try to get them to talk on the talkpage 22:40:59 #info A composer extension can register several customization scripts to hooks -- https://getcomposer.org/doc/articles/scripts.md 22:41:01 <TimStarling> iterate, invoke some autoloaded static function for registration 22:42:16 #action sumanah to get input from Wikia, ShoutWiki, wikiHow, & other 3rd-party folks on the Composer RfC talk page 22:42:21 so, i guess... with wikidata, we disabled autoloading of wikibase client and repo by default 22:42:39 those are then loaded via CommonSettings based on config 22:42:54 <TimStarling> should we talk about parent5446' 22:42:55 once one of those is loaded, all the dependencies are autoloaded 22:42:59 <TimStarling> should we talk about parent5446's RFC now? 22:43:01 #action sumanah to bring this spirited discussion to the attention of Markus and ask him if he can move forward, incorporate suggestions, maybe talk to aude from Wikidata & bd808 for specific HOWTO recommendations to incorporate 22:43:06 I think so TimStarling 22:43:10 #topic MediaWiki libraries 22:43:16 #topic Third-party components 22:43:20 #link https://www.mediawiki.org/wiki/Requests_for_comment/Third-party_components 22:43:26 #info Tyler Romeo's proposal (replacing some MW components with more widely used libraries, especially Symfony libraries) 22:43:39 it would be cumbersome to manually register all the dependencies of wikibase 22:43:41 #info parent5446 is looking for feedback on the general concept. 22:43:46 which requires a package manager :) 22:44:17 I don't see any one replacement jumping out at me yet as "oh yes, let's do that replacement right now" 22:44:19 Hence the advantage of discussing these two RFCs together 22:44:24 bd808: yep 22:44:41 but I like the idea. As parent5446 says: "other open source projects have created entire libraries that do the same thing as the MediaWiki components, except are better maintained and much more functional. This RFC aims to replace one or more MediaWiki components with third-party open source components." 22:44:50 *nod* 22:45:01 <TimStarling> I don't think I am in favour of this general direction 22:45:02 this seems like something that’ll be easier to treat once we have a consistent dependency manager :D 22:45:06 hmm 22:45:26 <Dantman> I like the idea of using a full mail library. But I don't like the replacement of WebRequest. 22:45:29 <TimStarling> when we maintain our own small libraries, we can make changes to them 22:45:42 <TimStarling> the more we use external libraries, the more we will be blocked waiting for upstream 22:46:04 <TimStarling> with our own code, we maintain some flexibility 22:46:26 <TimStarling> for complicated things like JS minification, sure, use an external library 22:46:26 we can maintain local patches but that way lies madness 22:46:35 <TimStarling> but I don't think it makes sense for simple things like curl wrappers 22:46:42 How are those other (external) libs hosted? Would managing them drastically increase the scope of what composer has to do? 22:46:48 the more we use external libraries; the more we'll start running into libraries that pull in other libraries (which at worst are not compatible and at best are redundant) 22:47:15 isn't this chain of logic the way to wheel reinvention hell? 22:47:31 I wouldn't use the extreme as the norm 22:47:41 wheel not reinvented here :) 22:47:41 Not all libraries are dependency hell or hard to get things fixed upstream 22:47:57 maybe one way to balance this is to actually do the kind of inventory that parent5446 has done every so often, and check ourselves 22:48:03 <TimStarling> I don't really believe in that phrase 22:48:18 <TimStarling> I think there's a difference between making a wheel and inventing a wheel 22:48:38 <TimStarling> having studied many wheels, a good engineer can make a wheel for a specific purpose without much difficulty 22:48:40 Making a custom mail library when SwiftMailer already exists in re-inventing the wheel 22:48:50 <TimStarling> the same is true of libraries 22:49:04 Is there a general-case question implied by that RFC? Or is it just a question of considering each possible replacement case-by-case? 22:49:11 I agree with parent5446 that there are good externals and bad. We should only use the good ones. :) 22:49:23 * andrewbogott doesn't know if relying on any one external library already involves overhead that isn't done yet 22:49:26 <TimStarling> "don't reinvent the wheel", taken literally, encourages study, not recycling 22:50:00 as parent5446 says on the talkpage, "This RFC is not advocating replacing all of MediaWiki with third-party components; it's just an attempt to consider the possibilities and see what might be worth replacing." 22:50:40 do we have anyone here who has maintained jQuery in core? I feel it is a good case study 22:50:47 so it sounds like one principle is: the more complicated the kind of thing we're trying to do, the more likely it is we ought to check what external libraries are available 22:51:01 I think I could have gotten my implementation for the Structured logging RFC merged already if I had just written my own logging library, but I'd rather not. 22:52:13 <TimStarling> we can discuss individual proposals on their merits, of course 22:52:57 <TimStarling> but parent5446 is asking for general thoughts, and so this is my position, pretty much at the other end of the spectrum as far as opinions on reuse go 22:53:40 parent5446: the individual proposals I presume are most congruent with the "replace complicated things" principle would be around config and maintenance/console. What's your assessment? 22:54:03 <TimStarling> I have read a lot of code, and most of it was worse than what we have in the MW core 22:54:07 <TimStarling> and subject to less review 22:54:20 ^ the review this is important for fundraising 22:54:21 (other principles that are probably pretty obvious: the replacements should be high quality code, friendly & responsive upstreams, etc. HHVM would be an example?) 22:54:46 Yeah I haven't really looked at code, so I can't make a statement on specific libraries 22:54:59 (not a replacement really, but us choosing to work with an existing project instead of going it alone) 22:56:08 #link https://www.mediawiki.org/wiki/Upstream_projects 22:56:11 part of the issue with the way we do things now is that, while being potentially superior along many vectors (performance, reliabilitiy, security), we seem to fall down on reusability, which is unfortunate 22:56:49 The larger PHP community has made great strides in code quality in the last 4-5 years. There is still a lot of nasty stuff out there, but in the PHP 5.4+ library space there are a lot of high quality components. 22:57:11 I got to head out. Sorry for leaving a bit early on my own RFC. 22:57:25 It's ok parent5446 22:57:31 * sumanah decides to sum up 22:57:36 #info this seems like something that�ll be easier to treat once we have a consistent dependency manager :D 22:57:51 i also agree with bd808, while there is alot of crap php out there, there is also a ton of clean, modular code with minimal or no dependencies 22:58:02 and that has all happened in only the last couple years 22:58:10 +1 22:58:33 monolog being good example 22:58:41 #info we feel mixed about the general concept of choosing to reuse external stuff instead of writing it ourselves - depends on code quality, which you gotta take on a case-by-case basis 22:59:00 #info part of the issue with the way we do things now is that, while being potentially superior along many vectors (performance, reliabilitiy, security), we seem to fall down on reusability, which is unfortunate 22:59:42 Most of the Symphony2 code I've read is solid. There's some "javification" in parts but in general that happens when you build flexible libraries. 22:59:52 #info also, we're more likely to be interested in a replacement if the feature is quite complicated, e.g., JS minification 23:00:16 #info examples of good PHP code out there that's reusable: monolog, Symfony2 stuff 23:00:47 All right, I think that's about it today - next week we have no IRC meeting, because of the Zurich hackathon 23:01:00 yay, hackathon! 23:01:00 where we'll talk about performance guidelines and, I hope, thumbnails 23:01:09 Thank you all 23:01:19 woo! 23:01:22 I really appreciate your consistent thoughtfulness 23:01:25 #endmeeting