Architecture meetings/RFC review 2014-04-29

From mediawiki.org

22:00-23:00 UTC on Tuesday, 29 April at #wikimedia-office connect.

Requests for Comment to review[edit]

  1. Requests for comment/MediaWiki libraries
  2. Requests for comment/Third-party components


Summary and logs[edit]

We have some open questions for Markus regarding Composer, and for administrators of MediaWiki installations. Would Composer work in your environment?

And it sounds like we only want to replace an artisanal, homegrown MediaWiki component with an externally maintained library if the feature is complicated, upstream is friendly and responsive and aligned with us, and so on, on a case-by-case basis.

https://gerrit.wikimedia.org/r/#/c/119939/ still needs review.

Meeting summary[edit]

  • Extension management with Composer (TimStarling, 22:19:31)
    • LINK: https://www.mediawiki.org/wiki/Requests_for_comment/Extension_management_with_Composer (TimStarling, 22:19:37)
    • LINK: http://www.bn2vs.com/blog/2013/11/24/introduction-to-composer-for-mediawiki-developers/ (sumanah, 22:24:03)
    • <bd808> The ops concerns can be dealt with for the WMF cluster using a technique like I used in the Scholarships app (which uses Composer and is live on the cluster). The technique in patch https://gerrit.wikimedia.org/r/#/c/119939/ is very similar to what Scholarships does to freeze the external libs. (sumanah, 22:31:33)
    • <parent5446> Technically it should be trivial to convert most extensions to a composer library (sumanah, 22:31:50)
    • Wikidata team has some Composer experience per aude's explanation (sumanah, 22:32:48)
    • discussion of whether to deal with multiple packaging systems or just one, use something like https://github.com/derrabus/composer-webui for a web UI, CLI vs web (doing web UI securely is ?) (sumanah, 22:33:38)
    • IDEA: question for composer-for-extensions: best way to fit in with our source-control-based deployment? (brion, 22:37:11)
    • IDEA: question for composer-for-extensions: web ui possible for small installations? (compare with wordpress’s installer & extension manager) avoid chmod 777 (brion, 22:37:11)
    • IDEA: question for composer-for-extensions: avoid proliferation of package managers, but make sure we can still install extensions manually if need be or for migrations (brion, 22:37:12)
    • <bd808> A composer extension can register several customization scripts to hooks -- https://getcomposer.org/doc/articles/scripts.md (sumanah, 22:40:59)
    • ACTION: sumanah to get input from Wikia, ShoutWiki, wikiHow, & other 3rd-party folks on the Composer RfC talk page (sumanah, 22:42:16)
    • ACTION: sumanah to bring this spirited discussion to the attention of Markus and ask him if he can move forward, incorporate suggestions, maybe talk to aude from Wikidata & bd808 for specific HOWTO recommendations to incorporate (sumanah, 22:43:01)
  • Third-party components (sumanah, 22:43:16)
    • LINK: https://www.mediawiki.org/wiki/Requests_for_comment/Third-party_components (sumanah, 22:43:20)
    • Tyler Romeo's proposal (replacing some MW components with more widely used libraries, especially Symfony libraries) (sumanah, 22:43:26)
    • parent5446 is looking for feedback on the general concept. (sumanah, 22:43:41)
    • LINK: https://www.mediawiki.org/wiki/Upstream_projects (sumanah, 22:56:08)
    • <brion> this seems like something thatďż˝ll be easier to treat once we have a consistent dependency manager :D (sumanah, 22:57:36)
    • we feel mixed about the general concept of choosing to reuse external stuff instead of writing it ourselves - depends on code quality, which you gotta take on a case-by-case basis (sumanah, 22:58:41)
    • <robla> part of the issue with the way we do things now is that, while being potentially superior along many vectors (performance, reliabilitiy, security), we seem to fall down on reusability, which is unfortunate (sumanah, 22:59:00)
    • also, we're more likely to be interested in a replacement if the feature is quite complicated, e.g., JS minification (sumanah, 22:59:52)
    • examples of good PHP code out there that's reusable: monolog, Symfony2 stuff (sumanah, 23:00:16)


Meeting ended at 23:01:25 UTC.


Action items[edit]

  • sumanah to get input from Wikia, ShoutWiki, wikiHow, & other 3rd-party folks on the Composer RfC talk page
  • sumanah to bring this spirited discussion to the attention of Markus and ask him if he can move forward, incorporate suggestions, maybe talk to aude from Wikidata & bd808 for specific HOWTO recommendations to incorporate


Full log[edit]

Meeting logs


22:00:34 <sumanah> #startmeeting RfC review: MediaWiki libraries and third-party components | Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE). https://meta.wikimedia.org/wiki/IRC_office_hours
22:00:34 <wm-labs-meetbot> Meeting started Tue Apr 29 22:00:34 2014 UTC and is due to finish in 60 minutes.  The chair is sumanah. Information about MeetBot at http://wiki.debian.org/MeetBot.
22:00:34 <wm-labs-meetbot> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
22:00:34 <wm-labs-meetbot> The meeting name has been set to 'rfc_review__mediawiki_libraries_and_third_party_components___channel_is_logged_and_publicly_posted__do_not_remove_this_note___https___meta_wikimedia_org_wiki_irc_office_hours'
22:00:49 <sumanah> #chair sumanah brion  andrewbogott
22:00:49 <wm-labs-meetbot> Current chairs: andrewbogott brion sumanah
22:00:57 <sumanah> #link https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-04-29
22:01:35 <sumanah> Thanks for coming all
22:01:43 <sumanah> First, an announcement of a few RfCs that need some help
22:01:50 <sumanah> #topic asking for help with a few RfCs
22:02:00 <sumanah> #info Jon Robson needs help writing the initial code for https://www.mediawiki.org/wiki/Requests_for_comment/Allow_styling_in_templates
22:02:09 <sumanah> jdlrobson - tell me if that's wrong :)
22:02:15 <sumanah> #info https://www.mediawiki.org/wiki/Requests_for_comment/Scoping_site_CSS and https://www.mediawiki.org/wiki/Requests_for_comment/Book_management need new coauthors to move forward
22:02:23 <sumanah> so check those out.
22:02:57 <sumanah> OK, now on to the first topic today, Ryan_Lane's MediaWiki libraries RfC. Then we'll move on to parent5446's RfC on 3rd-party components
22:03:01 <sumanah> #topic MediaWiki libraries
22:03:12 <sumanah> #link https://www.mediawiki.org/wiki/Requests_for_comment/MediaWiki_libraries Ryan Lane's proposal (better handling and versioning libraries that are MW extensions)
22:03:12 * marktraceur frows at Book Management
22:03:36 <sumanah> #chair sumanah brion  andrewbogott TimStarling
22:03:36 <wm-labs-meetbot> Current chairs: TimStarling andrewbogott brion sumanah
22:03:41 <parent5446> So I have a question about this RfC. It mentions LdapAuthentication as an example. What library in LdapAuthentication would we be packaging with core?
22:03:46 <sumanah> #info See https://www.mediawiki.org/wiki/Talk:Requests_for_comment/MediaWiki_libraries - we just need to merge "Add Composer managed libraries" https://gerrit.wikimedia.org/r/#/c/119939/ in order to complete part I of this.
22:03:47 <sumanah> #info Then Part II is actually moving everything else into that directory, and also making it reasonable to move some extensions into that directory.
22:04:04 <parent5446> B/c I understand the RFC in the context of the Gerrit patch
22:04:19 <parent5446> Libraries that are being used in core will need to be managed somehow
22:04:23 <Ryan_Lane> parent5446: a number of extension use LDAP as a library
22:04:31 <brion> i’m not sure i understand this rfc; is the idea to package specific libraries in mediawiki core?
22:04:39 <Ryan_Lane> I'd split the necessary functionality out of the LDAP extension into a common library
22:04:44 <brion> or to provide a general place to put libraries?
22:04:47 <parent5446> ...how? Are there extensions that use LDAP other than LdapAuthentication?
22:04:53 <TimStarling> I think the idea is to create a separate class of extensions that you put in libs/ instead of extensions/
22:05:05 <Ryan_Lane> TimStarling: yep
22:05:13 <brion> first question: why?
22:05:25 <brion> second question: what needs to be different other than the filename ‘extensions’ -> ‘libs’?
22:05:36 <Ryan_Lane> because there's now a lot of extensions being used as libraries and they aren't being versioned with mediawiki
22:05:40 <parent5446> OK, but the gerrit patch makes it look like we'd be packaging them with core. So it makes it sound like we'd be packaging libraries within core that are not actually used in core.
22:05:45 <brion> third question: does that patch adding a libs directory help or hinder this rfc’s recommendations?
22:05:53 <sumanah> (the patchset is by bd808 in case he wants to comment)
22:05:57 <Ryan_Lane> most of these could be bundled with core
22:06:11 <TimStarling> we have includes/libs already for things that are bundled
22:06:13 <Ryan_Lane> and should definitely be versioned with releases
22:06:28 <Ryan_Lane> TimStarling: that's for things that core depends on, but not for things extensions may need to depend on
22:06:45 <brion> so is this directory’s contents something is fixed, that extensions can depend on always being present?
22:06:55 <brion> or is it something extensible, that extensions can rely on having a way to install things there?
22:07:09 <parent5446> I don't really think it's a good idea to package extension libraries as part of core if core is not using it.
22:07:23 <bd808> The best solution for Ryan's RFC would probably be enabling the full use of Composer (or a similar system) with wm-core
22:07:27 <parent5446> Maybe an extension needs a different library version, and can't use it because core decided to package it
22:07:48 * sumanah invited Jeroen to discuss composer-y stuff with us in this chat but - ah, there's JeroenDeDauw!
22:08:03 <JeroenDeDauw> Just saw calendar notification now
22:08:07 <JeroenDeDauw> Anout to head off
22:08:14 <Ryan_Lane> so, maybe I should describe my issue, rather than this specific solution?
22:08:18 <sumanah> JeroenDeDauw: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/20140429.txt starting at 2200 in case you wanna see our talk till now
22:08:26 <sumanah> s/talk/chatter/
22:08:29 <TimStarling> with composer, I'd like to know whether you think extensions should be in mediawiki/* or mediawiki/extensions/*
22:08:32 <brion> Ryan_Lane: yes please :D
22:08:42 <TimStarling> in the package names
22:08:44 <sumanah> #info the Composer RfC: https://www.mediawiki.org/wiki/Requests_for_comment/Extension_management_with_Composer
22:08:57 <Ryan_Lane> so, we have extensions. those extensions for a long time only had a dependency on a specific mediawiki version
22:09:02 <TimStarling> because JeroenDeDauw put a few in mediawiki/*, but it seems a bit weird to me
22:09:14 <Ryan_Lane> now we have extensions that only exist as libraries for use by extensions
22:09:14 <JeroenDeDauw> TimStarling: vendor/package
22:09:18 <JeroenDeDauw> TimStarling: two levels
22:09:24 <brion> (many extensions require external dependencies as well…)
22:09:52 <mwalker> extensions can also be hooks onto other extensions -- but that doesn't make the 'parent' extension a library
22:11:03 * aude waves
22:11:09 <TimStarling> hmm
22:11:12 <mwalker> this seems to be mostly an argument about our release process... in theory, everything in a release is stably versioned and has release branches for backports
22:11:43 <Ryan_Lane> well, it's more an issue that we have no reasonable way to specify extension compatibility
22:12:00 <Ryan_Lane> and now we need to specify extension compatibility with mediawiki and with other library extensions
22:12:02 <parent5446> I mean, this is basically what composer was made to do
22:12:21 <brion> Ryan_Lane: don’t you just grab them from the same version branch?
22:12:21 <parent5446> Except people don't want to use composer in core
22:12:34 * sumanah sees parent5446's point. If this is a configuration/dependency management problem, maybe we should go with the thing that does that
22:12:40 <Ryan_Lane> brion: very few people use version branches
22:12:52 <brion> Ryan_Lane: how would this solve that?
22:12:59 <mwalker> we're cutting the version branches automatically now
22:13:14 <greg-g> mwalker: but everyone else still uses tarballs
22:13:15 <Ryan_Lane> my proposal was to have the common libraries shipped with mediawiki and locked with the version
22:13:22 <brion> …
22:13:31 <brion> so something that’s common to a group of extensions, and has no relation to mediawiki core...
22:13:34 <brion> … would be shipped with core?
22:13:38 <parent5446> Shipping common libraries is kind of like a band-aid to the actual problem
22:13:39 <TimStarling> I think it's fine to have extensions that depend on other extensions
22:13:50 <bd808> parent5446: I think we are open to using composer in core, but new need to figure out how to do that in a way that works for JeroenDeDauw's use case and mine (which is basically the same as Ryan_Lane's)
22:14:07 <Ryan_Lane> the issue is that most of these library extensions probably should just be core
22:14:09 <TimStarling> drupal has a DIY system for this
22:14:18 <TimStarling> or of course we could use composer
22:14:25 <parent5446> I thought the main problem was that people didn't like how it didn't interact properly with Git?
22:14:29 <Ryan_Lane> if 20 extensions are using an extension, why isn't it a core feature?
22:14:30 <sumanah> Emufarmers: Dantman: would love to have your input here as people who work on non-WMF MediaWiki installations
22:14:32 <TimStarling> or support any other package management system, like PEAR or .deb or whatever
22:14:50 <brion> Ryan_Lane: are those 20 extensions all specific and only used by a tiny minority of installations?
22:14:54 <mwalker> greg-g, we bundle extensions into our release tarballs...
22:14:55 <brion> or are they used all the time by most?
22:15:05 <Ryan_Lane> to be fair, I'd be fine with composer, if it can be used reasonably
22:15:23 <parent5446> Well many extensions have already been made to work as composer libraries
22:15:24 <TimStarling> if a facility is widely used, I think it can be in core, you know that I am not one of the people who has a narrow view of what core should be
22:15:25 <greg-g> mwalker: yeah. (/me doesn't want to derail, it was a minor point)
22:15:27 <brion> it feels like the solution here is “a dependency-managing installation system for extensions"
22:15:38 <Ryan_Lane> TimStarling: I'm not saying you are :)
22:15:42 <TimStarling> but yes, we need extension dependencies for various reasons
22:15:42 <sumanah> Which means we should check out https://www.mediawiki.org/wiki/Requests_for_comment/Extension_management_with_Composer
22:15:51 <sumanah> and maybe expand it
22:16:04 <Dantman> sumanah: I'm not 100% certain of the topic, or even whether "extensions as composer modules" is included in this or excluded.
22:16:18 <TimStarling> not just for libraries -- some extensions hook others, for example
22:16:28 <Ryan_Lane> yep
22:16:41 <Ryan_Lane> it's something we've been punting on forever
22:16:43 <sumanah> Dantman: sounds like we're talking about including it, maybe
22:16:44 <TimStarling> some use RL modules from another extension
22:17:09 <sumanah> Dantman: the topic is https://www.mediawiki.org/wiki/Requests_for_comment/MediaWiki_libraries Ryan Lane's proposal (better handling and versioning libraries that are, or are used by, MW extensions)
22:17:22 <Ryan_Lane> so, let's drop this proposal and maybe talk about the composer suggestions?
22:17:43 <Ryan_Lane> or is that not how this process works? heh
22:17:52 <parent5446> Well if we need to itemize this to dependencies on individual RL modules and hooks, there's not really any system that handles that
22:18:05 <parent5446> With composer it's either you depend on the entire extension or you don't
22:18:12 <Dantman> Well this idea of extensions as composer modules has annoyed me over and over again.
22:18:14 <parent5446> Unless you separate into further libraries
22:18:15 <sumanah> Ryan_Lane: I'm fine with the process working like that, if you want to abandon your RfC and become a coauthor on the Composer one :)
22:18:53 <bd808> Extension management with composer would allow managing dependent libraries as well, for the extensions. My use-case of wanting an external library for core is slightly different, but could pervert the system by using a basically empty extension just to get the library management.
22:19:00 <sumanah> andrewbogott: I guess one question for you as Ops representative is how Ops feels about Composer (although I assume if Ryan's fine with it then you would be too)
22:19:02 <Ryan_Lane> wasn't that another RfC that was in the queue for today?
22:19:21 <sumanah> Ryan_Lane: no, it wasn't; it is one that I wanted people to have fresh in their memory because I predicted it would interrelate
22:19:24 <Ryan_Lane> the current implementation of composer doesn't fit with our deployment process
22:19:31 <TimStarling> #topic Extension management with Composer
22:19:37 <TimStarling> #link https://www.mediawiki.org/wiki/Requests_for_comment/Extension_management_with_Composer
22:19:45 <andrewbogott> Yeah, I was going to say:  I don't think Ryan_Lane is fine with it :)
22:19:47 <sumanah> but since Markus isn't here and JeroenDeDauw needs to head off, it's suboptimal. Ah well
22:20:22 <Ryan_Lane> composer, as currently implemented, means you need to run composer on every node you deploy to. it also makes it impossible to disable any individual extension
22:20:30 <TimStarling> I don't think there's any reason to use composer in production
22:20:38 <aude> Ryan_Lane: no
22:20:44 <aude> you make a build
22:20:57 <aude> with all the stuff installed
22:20:59 <aude> run tests on it
22:21:00 <bd808> The ops concerns can be dealt with for the WMF cluster using a technique like I used in the Scholarships app (which uses Composer and is live on the cluster).
22:21:05 <aude> then deploy that
22:21:14 <aude> all the stuff checked in, including composer.lock
22:21:17 <Ryan_Lane> what about other people using similar deployment methods?
22:22:06 <mwalker> what aude has described is basically how we're having to deploy Node.JS apps
22:22:08 <aude> it's also possible to have an internal satis (packagist) system
22:22:24 <bd808> The technique in patch https://gerrit.wikimedia.org/r/#/c/119939/ is very similar to what Scholarships does to freeze the external libs.
22:22:24 <aude> that only knows about our own stuff
22:22:26 <TimStarling> I'd rather not have any kind of external dependency for source in production -- just our own git servers
22:22:33 <Ryan_Lane> so, does this assume that all WMF extensions will also have composer data?
22:22:48 <brion> can composer use private repos?
22:22:50 <TimStarling> the wikidata thing is to run composer locally and then push the result into git, right?
22:22:51 <aude> probably our own satis + a build would work
22:22:54 <aude> brion: yes
22:22:57 <parent5446> Technically it should be trivial to convert most extensions to a composer library
22:22:59 <brion> yes looks like good
22:23:00 <brion> :D
22:23:14 <aude> TimStarling: yes
22:23:28 <brion> so what does it look like to composer-ize an extension? what sort of migration path would we be looking at?
22:23:38 <aude> including it's important to carefully check composer lock and the diff
22:23:40 <brion> and would this just handle fetching files, or would it also handle installation, enabling, etc?
22:23:45 <parent5446> You make a composer.json file that tells composer to include the main extension file
22:23:47 <mwalker> brion, http://www.bn2vs.com/blog/2013/11/24/introduction-to-composer-for-mediawiki-developers/
22:23:54 <aude> plus all our tests run and then only after they pass, it gets merged
22:23:59 <TimStarling> composer fetches files and handles class autoloading
22:23:59 <brion> thanks
22:24:03 <sumanah> #link http://www.bn2vs.com/blog/2013/11/24/introduction-to-composer-for-mediawiki-developers/
22:24:36 <TimStarling> it has its own autoload system, and does clever things like scanning .php files on install for class definitions
22:24:41 <aude> some of a problem will be when some other extension wants to use same library that wikidata uses
22:24:58 <TimStarling> in addition to PSR-0 of course
22:25:01 <aude> would be nice to have it registered in a shared place
22:25:01 <brion> related question:
22:25:12 <brion> obviously small installations would love an extension installation manager.
22:25:23 <brion> would tying to composer force us to require CLI for extension setup?
22:25:37 <brion> or would it allow for a web control panel as well, given appropriate permissions on server?
22:25:42 <sumanah> ok, it sounds like there is a lot of interest in Composer usage. Ryan_Lane does this interest you enough to make you wanna collaborate on investigating things after today, working on the RfC with Markus Glaser or whoever else wants to work on this? JeroenDeDauw maybe?
22:25:43 <aude> TimStarling: we also use composer option to dump a classmap (helps optimize)
22:25:54 <sumanah> because we're not gonna figure all this out and approve it today
22:25:57 <aude> so in the end it is like AutoLoader.php
22:25:58 <TimStarling> we can support any number of pacakge management systems in parallel
22:26:01 <parent5446> brion: Yes, unless we make a PHP interface for composer, which we can
22:26:13 <brion> parent5446: that works for me
22:26:13 <TimStarling> we don't need to support composer exclusively
22:26:20 <parent5446> ^that as well
22:26:26 <brion> hmmmmmmmmmmm
22:26:30 <brion> i hate mixing package managers though
22:26:38 <mwalker> +1 to brion
22:26:40 <aude> brion: a php interface would be cool, but not aware of one
22:26:45 <TimStarling> as long as we document dependencies as well as putting them in composer.json, people can keep on installing things the old way
22:26:50 <Ryan_Lane> sumanah: I can't really spend time on this. I need this as a user of mediawiki, though
22:26:50 <Dantman> I also have doubts about being able to do a web interface around composer.
22:26:50 <aude> probably would be fairly simple
22:27:00 <Dantman> Especially a secure one.
22:27:01 <sumanah> Ryan_Lane: totally fair - just wanted to check
22:27:27 <brion> Dantman: secure is the hard part. wordpress updates for instance seem to jump through weird hoops to ssh into itself or some weird shit :D
22:27:32 <brion> it scares me
22:27:38 <brion> but is an oh-so-useful feature
22:28:04 <TimStarling> it's a bit more secure than just using the web user for installation
22:28:25 <brion> chmod 777
22:28:26 <TimStarling> at least after installation, the source tree is not writable by the webserver
22:28:31 <brion> yeah :D
22:28:35 <parent5446> Wordpress also allows using FTP to update
22:28:39 <Dantman> Most of the important metadata is still in PHP. So you can't read it without first fetching and installing the extension.
22:28:41 <brion> *shudder*
22:28:45 <bd808> Google found https://github.com/derrabus/composer-webui for me
22:29:36 <parent5446> Looking at the code for what bd808 just posted, you can basically just include composer itself, and use it's internal interface
22:29:36 <sumanah> hexmode: you talk to Markus a lot, I assume :-) Do you know whether Markus has time to respond to any of these points soon?
22:29:43 <sumanah> (since he proposed Composer-y stuff)
22:30:11 <Dantman> It's also a dead end if you can't do shell execution from php. (Don't we still sort of support that?)
22:30:12 <brion> sounds like we’ve got a lot of questions and ideas to queue up for that :D
22:30:30 <sumanah> brion: help me summarize for the notes?
22:30:36 <Dantman> Side note on composer itself.
22:30:44 <parent5446> You wouldn't need shell access to use composer from PHP
22:31:00 <Dantman> Hmmm... right.
22:31:33 <sumanah> #info <bd808> The ops concerns can be dealt with for the WMF cluster using a technique like I used in the Scholarships app (which uses Composer and is live on the cluster). The technique in patch https://gerrit.wikimedia.org/r/#/c/119939/ is very similar to what Scholarships does to freeze the external libs.
22:31:44 <Dantman> Composer is similar to other package managers that install things locally.
22:31:50 <sumanah> #info <parent5446> Technically it should be trivial to convert most extensions to a composer library
22:31:52 <Dantman> Like npm and bower.
22:32:37 <Dantman> However it's unique (in a flawed way) at being the only one that requires that every installed package be present inside the .json file.
22:32:48 <sumanah> #info Wikidata team has some Composer experience per aude's explanation
22:33:32 <Dantman> Both bower and npm can install packages locally without adding them to the .json file you commit. (If you want to add it you use --save)
22:33:36 <TimStarling> how would registration be handled?
22:33:38 <sumanah> #info discussion of whether to deal with multiple packaging systems or just one, use something like https://github.com/derrabus/composer-webui for a web UI, CLI vs web (doing web UI securely is ?)
22:33:45 <bd808> You can add arbitrary libraries using Composer at the command line, but it writes them into the lock file
22:34:06 <legoktm> [15:28:39] <Dantman> Most of the important metadata is still in PHP. So you can't read it without first fetching and installing the extension. <-- Yeah, I was thinking about that as well. I had an idea that we could just put all the extension registration stuff (hooks, classes, basically [[mw:User:Legoktm/extension_registration]]) into a JSON file, so anything can read it without actually installing the extension
22:34:21 <bd808> Which is very similar to using a require.txt file with pip in python or ruby's bundler
22:35:14 <TimStarling> would composer installation somehow register the extension? or would registration require a line to be added manually to LocalSettings.php?
22:35:32 <aude> TimStarling:  can be done wither way
22:35:59 <aude> i prefer manually registration for prodution
22:35:59 <TimStarling> that may be a question for discussion on the RFC page then
22:36:20 <parent5446> TimStarling: If you set the autoloader to include the main file, it would not need lines added to LocalSettings
22:36:25 <aude> so we can enable something for one wiki but not an other (e.g.)
22:36:35 <Dantman> My point is, if it weren't for that flaw in composer. It would have never been necessary to delete the composer.json that we used to have in core.
22:36:36 <aude> like we do w/o composer
22:36:45 <bd808> A composer extension can register several customization scripts to hooks -- https://getcomposer.org/doc/articles/scripts.md
22:37:11 <brion> #idea question for composer-for-extensions: best way to fit in with our source-control-based deployment?
22:37:11 <brion> #idea question for composer-for-extensions: web ui possible for small installations? (compare with wordpress’s installer & extension manager) avoid chmod 777
22:37:11 <brion> i hope i’m using the right magic tags
22:37:12 <brion> #idea question for composer-for-extensions: avoid proliferation of package managers, but make sure we can still install extensions manually if need be or for migrations
22:37:12 <brion> brb
22:37:38 <sumanah> brion: yeah, you are :-) https://wiki.debian.org/MeetBot #idea works
22:37:46 <brion> ok ...
22:37:53 <brion> did we get the key points?
22:38:05 <sumanah> also #info #action #help #link
22:38:11 <TimStarling> maybe we should have our own admin console for registration
22:38:21 <brion> whoa colloquy just dropped the last 5 minutes on me in a batch :D
22:38:37 <TimStarling> and it would detect extensions installed by composer for the purposes of displaying the list
22:38:53 <bd808> Dantman: Part of the problem is that core is both a library and an application. Composer doesn't grok that.
22:38:55 <TimStarling> that way you can disable them, but it is still easy to use
22:38:57 <brion> TimStarling: i like that
22:39:08 <brion> composer for physical installation, but we have a separate activation as we do now
22:39:19 <brion> this works well for multi-site farms as well as for shipping defaults etc
22:40:36 <TimStarling> you could pull the list of enabled extensions out of the DB or some other web-writable place
22:40:46 * sumanah invited folks from Wikia, ShoutWiki, & wikiHow to come talk about this; will try to get them to talk on the talkpage
22:40:59 <sumanah> #info <bd808> A composer extension can register several customization scripts to hooks -- https://getcomposer.org/doc/articles/scripts.md
22:41:01 <TimStarling> iterate, invoke some autoloaded static function for registration
22:42:16 <sumanah> #action sumanah to get input from Wikia, ShoutWiki, wikiHow, & other 3rd-party folks on the Composer RfC talk page
22:42:21 <aude> so, i guess... with wikidata, we disabled autoloading of wikibase client and repo by default
22:42:39 <aude> those are then loaded via CommonSettings based on config
22:42:54 <TimStarling> should we talk about parent5446'
22:42:55 <aude> once one of those is loaded, all the dependencies are autoloaded
22:42:59 <TimStarling> should we talk about parent5446's RFC now?
22:43:01 <sumanah> #action sumanah to bring this spirited discussion to the attention of Markus and ask him if he can move forward, incorporate suggestions, maybe talk to aude from Wikidata & bd808 for specific HOWTO recommendations to incorporate
22:43:06 <sumanah> I think so TimStarling
22:43:10 <sumanah> #topic MediaWiki libraries
22:43:16 <sumanah> #topic Third-party components
22:43:20 <sumanah> #link https://www.mediawiki.org/wiki/Requests_for_comment/Third-party_components
22:43:26 <sumanah> #info Tyler Romeo's proposal (replacing some MW components with more widely used libraries, especially Symfony libraries)
22:43:39 <aude> it would be cumbersome to manually register all the dependencies of wikibase
22:43:41 <sumanah> #info parent5446  is looking for feedback on the general concept.
22:43:46 <bd808> which requires a package manager :)
22:44:17 <sumanah> I don't see any one replacement jumping out at me yet as "oh yes, let's do that replacement right now"
22:44:19 <parent5446> Hence the advantage of discussing these two RFCs together
22:44:24 <aude> bd808: yep
22:44:41 <sumanah> but I like the idea. As parent5446 says: "other open source projects have created entire libraries that do the same thing as the MediaWiki components, except are better maintained and much more functional. This RFC aims to replace one or more MediaWiki components with third-party open source components."
22:44:50 <brion> *nod*
22:45:01 <TimStarling> I don't think I am in favour of this general direction
22:45:02 <brion> this seems like something that’ll be easier to treat once we have a consistent dependency manager :D
22:45:06 <brion> hmm
22:45:26 <Dantman> I like the idea of using a full mail library. But I don't like the replacement of WebRequest.
22:45:29 <TimStarling> when we maintain our own small libraries, we can make changes to them
22:45:42 <TimStarling> the more we use external libraries, the more we will be blocked waiting for upstream
22:46:04 <TimStarling> with our own code, we maintain some flexibility
22:46:26 <TimStarling> for complicated things like JS minification, sure, use an external library
22:46:26 <brion> we can maintain local patches but that way lies madness
22:46:35 <TimStarling> but I don't think it makes sense for simple things like curl wrappers
22:46:42 <andrewbogott> How are those other (external) libs hosted?  Would managing them drastically increase the scope of what composer has to do?
22:46:48 <mwalker> the more we use external libraries; the more we'll start running into libraries that pull in other libraries (which at worst are not compatible and at best are redundant)
22:47:15 <robla> isn't this chain of logic the way to wheel reinvention hell?
22:47:31 <parent5446> I wouldn't use the extreme as the norm
22:47:41 <brion> wheel not reinvented here :)
22:47:41 <parent5446> Not all libraries are dependency hell or hard to get things fixed upstream
22:47:57 <sumanah> maybe one way to balance this is to actually do the kind of inventory that parent5446 has done every so often, and check ourselves
22:48:03 <TimStarling> I don't really believe in that phrase
22:48:18 <TimStarling> I think there's a difference between making a wheel and inventing a wheel
22:48:38 <TimStarling> having studied many wheels, a good engineer can make a wheel for a specific purpose without much difficulty
22:48:40 <parent5446> Making a custom mail library when SwiftMailer already exists in re-inventing the wheel
22:48:50 <TimStarling> the same is true of libraries
22:49:04 <andrewbogott> Is there a general-case question implied by that RFC?  Or is it just a question of considering each possible replacement case-by-case?
22:49:11 <bd808> I agree with parent5446 that there are good externals and bad. We should only use the good ones. :)
22:49:23 * andrewbogott doesn't know if relying on any one external library already involves overhead that isn't done yet
22:49:26 <TimStarling> "don't reinvent the wheel", taken literally, encourages study, not recycling
22:50:00 <sumanah> as parent5446 says on the talkpage, "This RFC is not advocating replacing all of MediaWiki with third-party components; it's just an attempt to consider the possibilities and see what might be worth replacing."
22:50:40 <mwalker> do we have anyone here who has maintained jQuery in core? I feel it is a good case study
22:50:47 <sumanah> so it sounds like one principle is: the more complicated the kind of thing we're trying to do, the more likely it is we ought to check what external libraries are available
22:51:01 <bd808> I think I could have gotten my implementation for the Structured logging RFC merged already if I had just written my own logging library, but I'd rather not.
22:52:13 <TimStarling> we can discuss individual proposals on their merits, of course
22:52:57 <TimStarling> but parent5446 is asking for general thoughts, and so this is my position, pretty much at the other end of the spectrum as far as opinions on reuse go
22:53:40 <sumanah> parent5446: the individual proposals I presume are most congruent with the "replace complicated things" principle would be around config and maintenance/console. What's your assessment?
22:54:03 <TimStarling> I have read a lot of code, and most of it was worse than what we have in the MW core
22:54:07 <TimStarling> and subject to less review
22:54:20 <mwalker> ^ the review this is important for fundraising
22:54:21 <sumanah> (other principles that are probably pretty obvious: the replacements should be high quality code, friendly & responsive upstreams, etc. HHVM would be an example?)
22:54:46 <parent5446> Yeah I haven't really looked at code, so I can't make a statement on specific libraries
22:54:59 <sumanah> (not a replacement really, but us choosing to work with an existing project instead of going it alone)
22:56:08 <sumanah> #link https://www.mediawiki.org/wiki/Upstream_projects
22:56:11 <robla> part of the issue with the way we do things now is that, while being potentially superior along many vectors (performance, reliabilitiy, security), we seem to fall down on reusability, which is unfortunate
22:56:49 <bd808> The larger PHP community has made great strides in code quality in the last 4-5 years. There is still a lot of nasty stuff out there, but in the PHP 5.4+ library space there are a lot of high quality components.
22:57:11 <parent5446> I got to head out. Sorry for leaving a bit early on my own RFC.
22:57:25 <sumanah> It's ok parent5446
22:57:31 * sumanah decides to sum up
22:57:36 <sumanah> #info <brion> this seems like something that�ll be easier to treat once we have a consistent dependency manager :D
22:57:51 <ebernhardson> i also agree with bd808,  while there is alot of crap php out there, there is also a ton of clean, modular code with minimal or no dependencies
22:58:02 <ebernhardson> and that has all happened in only the last couple years
22:58:10 <aude> +1
22:58:33 <aude> monolog being good example
22:58:41 <sumanah> #info we feel mixed about the general concept of choosing to reuse external stuff instead of writing it ourselves - depends on code quality, which you gotta take on a case-by-case basis
22:59:00 <sumanah> #info <robla> part of the issue with the way we do things now is that, while being potentially superior along many vectors (performance, reliabilitiy, security), we seem to fall down on reusability, which is unfortunate
22:59:42 <bd808> Most of the Symphony2 code I've read is solid. There's some "javification" in parts but in general that happens when you build flexible libraries.
22:59:52 <sumanah> #info also, we're more likely to be interested in a replacement if the feature is quite complicated, e.g., JS minification
23:00:16 <sumanah> #info examples of good PHP code out there that's reusable: monolog, Symfony2 stuff
23:00:47 <sumanah> All right, I think that's about it today - next week we have no IRC meeting, because of the Zurich hackathon
23:01:00 <aude> yay, hackathon!
23:01:00 <sumanah> where we'll talk about performance guidelines and, I hope, thumbnails
23:01:09 <sumanah> Thank you all
23:01:19 <brion> woo!
23:01:22 <sumanah> I really appreciate your consistent thoughtfulness
23:01:25 <sumanah> #endmeeting
Retrieved from "https://www.mediawiki.org/w/index.php?title=Architecture_meetings/RFC_review_2014-04-29&oldid=3849467"