Parsoid/Packaging

This page is used to develop possible solutions for bug 53723.

We would like to


 * build Debian packages
 * still allows operations to review root-level code
 * support current deployment methods (git-deploy)
 * minimize manual dependency tracking and contrib updates

Option 1: node_modules as submodule

 * Use submodules to manage the debian/, puppet/ and contrib/ files. Operations has merge rights on debian/ and puppet/ so that root-level files can still be reviewed.
 * Automatically build debs for third parties from the full repository, either using launchpad or a labs instance.
 * Continue to use git-deploy to push out the full repository including contrib/, but deploy the config files using puppet. Submodules are currently disabled for Parsoid, but can be re-enabled when the external cpp submodules are removed.

Option 2: deploy repo with code as submodule
node will search the path for node_modules, so we can also place it in a parent directory.

Deploy repo layout:
 * node_modules : dependencies, updated with
 * parsoid : parsoid js code as submodule
 * debian : Debianization and upstart / systemd config submodule. Repo controlled by ops. Configs can also be directly referenced by puppet.
 * package.json : symlink to parsoid/package.json
 * localsettings.js : web api configuration, loaded with --config switch to the web api

With this model multiple independent deploy branches (architecture, node version etc) are possible.

Decision: go with option 2
In the Parsoid team meeting Wednesday, December 11, 2013 we dicided to go with option 2. This will be implemented in the next days.

See also / random notes

 * npm modules can be converted to basic debs with https://npmjs.org/package/npm2debian
 * Simple deb packages can be converted to rpms with alien. If we include node_modules, then we should only depend on node and (optionally) logrotate. Node seems to be called 'nodejs' in most distros:
 * http://debdelta.debian.net/ looks like a bandwidth-efficient way to update Debian packages
 * mini-dinstall looks like a potential alternative to reprepo, which we are currently using for the wmf repos. Reprepo only keeps the very last version of each package, which makes downgrades.. difficult.

Deployment
[13:14]  well, we can deploy the code itself using trebuchet [13:14]  puppet can install the upstart and dependencies [13:15]  you don't need to understand salt to use trebuchet ;) [13:15]  https://wikitech.wikimedia.org/wiki/Trebuchet#Adding_a_new_repo [13:16]  also look at how parsoid is configured in puppet: manifests/role/deployment.pp [13:17]  and note that parsoid has a really shitty init script and that instead you can just use service.restart in the checkout_module_calls config