Talk:Local development quickstart

macOS and PHP

Since macOS Monterey, PHP is no longer preinstalled. So perhaps that should change to reflect an install from Homebrew or Macports or something else if that is preferred. TTaylor (WMF) (talk) 13:50, 31 October 2023 (UTC)

Reference: php.net installation on macOS TTaylor (WMF) (talk) 13:51, 31 October 2023 (UTC)

Thanks! I'll fix that. APaskulin (WMF) (talk) 16:26, 1 November 2023 (UTC)

I'd really like to add apcu for Mac, but that requires several additional steps. Perhaps I should add that to Manual:Running MediaWiki on macOS. —TheDJ (Not WMF) (talk • contribs) 10:17, 23 November 2023 (UTC)

I do think that it's too many extra steps for this guide, but I think adding it to Manual:Running MediaWiki on macOS is a great idea. APaskulin (WMF) (talk) 21:10, 4 December 2023 (UTC)

Serving

There's a composer script called serve for running php -S localhost:4000, if that's easier. It doesn't make it possible to change the port, is probably the main drawback. Sam Wilson 03:26, 22 November 2023 (UTC)

Good point! I had considered this but felt that it obscured the function of the command unnecessarily and that using the full command provided the option to change the port if the user needed to. But maybe it would be worth including it as an option since it is easier to type. APaskulin (WMF) (talk) 18:54, 22 November 2023 (UTC)

@APaskulin (WMF): Yeah, I do like the fact that it makes it clear that it's a pretty simple command, that's a good point. Sam Wilson 06:36, 23 November 2023 (UTC)

I'd also suggest going with composer serve, for the same reason that we also wrap the contents of the install.php invocation. Also, we might want to update the composer servecommand to include includes/dev/router.php. KHarlan (WMF) (talk) 07:22, 24 November 2023 (UTC)

@APaskulin (WMF) @Krinkle comopser serve now sets the log directory, prints logs to the console, and sets the number of CLI workers. So IMHO we should update the docs to use composer serve. KHarlan (WMF) (talk) 12:56, 16 January 2024 (UTC)

Thanks, @KHarlan (WMF)! Switching to composer serve sounds good to me. Feel free to make the updates, otherwise I can make them next week. APaskulin (WMF) (talk) 02:02, 20 January 2024 (UTC)

Node / npm

You really need node/npm, even grunt for some of the development work. Do we want to add that here as well ? Shouldn't be too many additional packages right ?

If not, I suggest we add a section with what this does NOT do (npm, performance optimisation, php debugging, MariaDB, thumbnailing etc) and then from that section, we can link to further instructions ? —TheDJ (Not WMF) (talk • contribs) 10:19, 23 November 2023 (UTC)

Or we point people directly at fresh.. but that brings a few very MW specific things into the picture again, which I guess we don't really want here ? —TheDJ (Not WMF) (talk • contribs) 10:39, 23 November 2023 (UTC)

That's a good point, but considering the extra steps, I'd rather link out to docs that cover this setup in the Next steps section. For now, we're aiming for parity with MediaWiki Docker, which doesn't include this extra setup. We currently have a link to Manual:How to debug, which covers some of what you mentioned. Are there any other pages that you think would be useful for contributors new to MediaWiki? APaskulin (WMF) (talk) 21:23, 4 December 2023 (UTC)

Use shallow clone

I suggest using --clone-depth=1 for cloning MW and Vector. That takes 13 seconds or so instead of several minutes. After the initial clone, you could make the MW repo "unshallow" (a full clone). For example, this command would use a shallow clone, and then put the multi-minute process of obtaining a full clone in the background:

git clone https://gerrit.wikimedia.org/r/mediawiki/core.git --depth=1 mediawiki && git clone --depth=1 https://gerrit.wikimedia.org/r/mediawiki/skins/Vector.git mediawiki/skins/Vector && cd mediawiki && echo "Fetching full MediaWiki code in background; you may proceed" && git fetch --unshallow > /dev/null 2>&1 & KHarlan (WMF) (talk) 07:10, 24 November 2023 (UTC)

On further reflection, the git fetch --unshallow > /dev/null 2>&1 & is maybe too confusing and magical. What if we use the --clone-depth=1 argument and then in "Next steps" we offer to the user that they should run git fetch --unshallow in the core and Vector directories? KHarlan (WMF) (talk) 08:43, 29 November 2023 (UTC)

I love this idea! But unfortunately, I think it's too confusing to be worth the few minutes it saves. APaskulin (WMF) (talk) 21:32, 4 December 2023 (UTC)

Multiple workers for PHP server

We might want to suggest to the user that they set e.g. PHP_CLI_SERVER_WORKERS=4 per the documentation in https://www.php.net/manual/en/features.commandline.webserver.php. So the command would be PHP_CLI_SERVER_WORKERS=4 composer serve Not sure if it will make a big different in practice. KHarlan (WMF) (talk) 07:24, 24 November 2023 (UTC)

This is a good point. I wasn't able to reproduce any slowness or stalling with the single-threaded process when I tested using Visual Editor locally. Have you observed any slowness or stalling? I think it's a good idea to add PHP_CLI_SERVER_WORKERS=4 within the composer serve command, but I'd like to verify that we're seeing issues because of this limitation. APaskulin (WMF) (talk) 21:34, 4 December 2023 (UTC)

I don't see any appreciable difference with multiple CLI server workers (roughly tested by loading Main_Page as a logged out user with HTTP cache disabled and looking at DOMContentLoaded / load times, with and without the multiple workers flag). So, I think we can leave out this flag. cc @Krinkle in case you have a different opinion. KHarlan (WMF) (talk) 13:12, 12 December 2023 (UTC)

@KHarlan (WMF) The next thread on this page talks about the "blocked requests" caveat, which is what raising the default worker count would address. If we're comfortable with the default, and are confident enough that this does not warrant a note of any kind, we could leave it that way. However, for those using the composer serve command, it'd cost nothing to add it regardless.

In my own testing, I do see a clear difference in timeline, but the overall load times are indeed quite close, especially if you're used to a Docker or Vagrant VM.

• Page load Before, After.
• VisualEditor load Before, After.

Note how "After" isn't just a bit faster, the latency numbers actually reflect the time for that logical request to generate a respond from MediaWiki, instead of accumulating the queue time for the previous request, which is not obvious to most devs I imagine.

Broadly:

• More production-like. It makes waterfall charts more realistic, more comparable between environments, and between local and production.
• Less confusing. When analyzing performance on a single-threaded server, concurrent requests will appear to be slower when really they are just waiting. You and I know this, but it's a needless barrier that people have to be "in the know" about, before which they will make incorrect measurements and conclusions. They might even conclude that "composer" is slower than some other environment, when it's actually baremetal and close to the fastest it can be.
• A tiny bit faster. Every millisecond counts :)
• Allows benchmarking out of the box. I'm working with Derick at the moment on benchmarking the impact of a change. In doing so, I'm suggesing to use the composer setup instead of Docker, as it has far less variance and more consistent response times, however, as written he'd have to set ab -c1 (to avoid contaminating data with stalled requests) or to dirty the working state by setting custom workers. If we instead set PHP_CLI_SERVER_WORKERS=8 by default, then you can actually run a reasonably large benchmark in a matter of seconds rather than minutes. It's one thing to be able to notice "hey this sets workers to 8" and use that, then to have it not set anything, and having to know that that means there is no concurrency. The former is more transparent and obvious imho. I'd almost recommend setting WORKERS=1, except that if we do, we might as well set it to 8.

These are all minor points, but given it's so easy to add, I'd say why not? Krinkle (talk) 16:00, 19 December 2023 (UTC)

Sounds good to me. (Went to make a patch and then found https://gerrit.wikimedia.org/r/c/mediawiki/core/+/984225, thanks @Krinkle!) KHarlan (WMF) (talk) 07:20, 20 December 2023 (UTC)

Warn about blocked requests?

Maybe this is out of scope for the quick start document, but wanted to flag that the built-in server manual warns:

The web server runs only one single-threaded process, so PHP applications will stall if a request is blocked.

In the past, I saw this cause problems for extensions like VisualEditor, which made internal API requests as part of the process for saving an edit. That doesn't seem to be an issue anymore, but we might want a "Troubleshooting" section which explains that if the page stops loading, one possible cause is the single-threaded process. KHarlan (WMF) (talk) 07:58, 24 November 2023 (UTC)

This is related to my comment in response to your previous comment, but I think adding it to a troubleshooting section is a great idea. APaskulin (WMF) (talk) 21:41, 4 December 2023 (UTC)

Load DevelopmentSettings.php from the outset

Without that, it's really hard for a novice to figure out why errors are occurring, or to get more detailed feedback from someone else. I'd suggest a one-liner as part of composer mw-install:sqlite that adds this to LocalSettings.php. KHarlan (WMF) (talk) 08:00, 24 November 2023 (UTC)

Patch proposed in https://gerrit.wikimedia.org/r/c/mediawiki/core/+/977136 KHarlan (WMF) (talk) 08:44, 29 November 2023 (UTC)

Great idea! Thanks for the patch! APaskulin (WMF) (talk) 21:41, 4 December 2023 (UTC)

Help user set up git for pushing patches

At the end of this guide, the user will have cloned core and Vector with https, but in order to push a patch for review, they'll need to switch to e.g. ssh://USERNAME@gerrit.wikimedia.org:29418/mediawiki/core in their .git/config. It might be worth calling that out specifically (Gerrit/Tutorial#Set up Git) in next steps. KHarlan (WMF) (talk) 08:48, 29 November 2023 (UTC)

Great point! I'll add a note about this. APaskulin (WMF) (talk) 21:45, 4 December 2023 (UTC)

@php or php

I was looking at the composer magic commands (because of @putenv), and it made me wonder if we should switch composer to use @php instead of directly calling php. It seems that is how laravel is doing it https://github.com/laravel/laravel/blob/07fdfbc8d8d88440634bd1c42075653dbb5df402/composer.json#L38 —TheDJ (Not WMF) (talk • contribs) 09:21, 20 December 2023 (UTC)

Let's go with @php per the Composer docs. KHarlan (WMF) (talk) 09:30, 22 December 2023 (UTC)

job runner

The job runner rate is rather low by default, which can get in the way. Maybe we can provide some sort of default setup for this as well ? I was looking if maybe we could run parallel commands, to also have a job runner when running serve, but that doesn't seem to be very easy to do with composer. And if we need to list it as a separate composer command we might as well list it as a direct instruction or link to it. Anyone else have ideas ? —TheDJ (Not WMF) (talk • contribs) 09:32, 20 December 2023 (UTC)