Talk:Requests for comment/Watch Categorylinks

Watchlist

This seems slightly to be mixing watchlist and notices ui. If a user watches a category, shouldnt they find out about removals and additions in the watchlist, not in echo?

Do you plan to support showing who added/removed the category (i think thats impossible to do, at least in all cases, and i dont think it should be a requirement, but if you have a solution for that, that would be awesome). Bawolff (talk) 16:24, 30 March 2015 (UTC)

Hi Bawolff, since a WikiPage object is passed to the event handler, determining and displaying the corresponding author shouldn't be a problem. We need to make sure to keep the notification simple and respect the recommendation of having not more than two links in the notification (see: Echo_(Notifications)/Developer_guide#Defining_a_notification). Kai Nissen (WMDE) (talk) 13:00, 31 March 2015 (UTC)

not all category change events are directly associated with an edit. Bawolff (talk) 13:22, 31 March 2015 (UTC)

The terminology was unclear, so I made it consistent. The linked page clearly asks for watchlisting (Beobachten). Notifications (enotifwatchlist) are given "for free" by the watchlist. --Nemo 14:03, 31 March 2015 (UTC)

How will watching category membership changes integrate with Special:Watchlist? I'm not sure about using Echo/notifications for this. --MZMcBride (talk) 03:33, 15 April 2015 (UTC)

Setting

Pro But please make sure the default setting (notification via Web/Mail on/off) can be configured so that if a comunity wants to change it, it can be done without editing in the source-code. -- MichaelSchoenitzer (talk) 11:09, 31 March 2015 (UTC)

Since it sounds like this will be an Echo notification, that would be a simple case of altering your $wgDefaultUserOptions. --Krenair ^{(talk • contribs)} 11:16, 31 March 2015 (UTC)

Finally something useful

Congratulations to WMDE for trying to work on the most crucial piece of our wikis after action=edit, the RecentChanges/Watchlist mechanism. This is the heart of wikis, where all the collaboration and activity lives. --Nemo 14:03, 31 March 2015 (UTC)

Does this need to be its own extension?

Given the Echo features that you want to use (bundling is a good one, for instance), this probably can't be in MW core, but is there a reason why this should be in its own extension, rather than being part of the Echo extension? --Roan Kattouw (WMF) (talk) 20:43, 1 April 2015 (UTC)

I'm not sure if it's a good idea to stuff all notification types into it. Echo provides ways to easily define events and create notifications. If we start to add all types of notifications in Echo itself, wouldn't we bloat it? Kai Nissen (WMDE) (talk) 09:43, 2 April 2015 (UTC)

Well, Echo already takes over some watchlisting features of core, namely enotiftalk. --Nemo 09:47, 2 April 2015 (UTC)

Feature thoughts

Having recently implemented a category membership watchlist tool on labs, some thoughts:

• Allow to monitor additions only, removals only, or both for a given category.
• Allow to match partial category names.
• If a given category is added and removed from the same page the same number of times, suppress it. ie. reverts
• Provide an atom feed if this is not integrated with current page watchlists. --Bamyers99 (talk) 23:41, 6 April 2015 (UTC)

@Bamyers99: Could you make that tool to work on other wikis as well? Helder 14:20, 15 April 2015 (UTC)

Meeting outcome

Thanks Matt for the update on the meeting outcome. I'm not sure about point 1 (template changes), but on 2 I think it's fine to require a specific watchlist table setting to watch category members changes. The new field might contain serialised parameters à la log_params to be future-proof: later, this would allow to add an UI to customise category watching experience as well as to add more "smart watchlisting" features in other cases. I don't think it's necessary to provide a user-facing option to not watch category member changes from the very first iteration. --Nemo 06:13, 18 April 2015 (UTC)

IRC meeting log 2015-04-15

21:00:59 <TimStarling> #startmeeting RFC meeting
21:01:00 <wm-labs-meetbot> Meeting started Wed Apr 15 21:00:59 2015 UTC and is due to finish in 60 minutes.  The chair is TimStarling. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:01:00 <wm-labs-meetbot> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:01:00 <wm-labs-meetbot> The meeting name has been set to 'rfc_meeting'
21:01:21 <TimStarling> #topic Watch Categorylinks | RFC meeting | Wikimedia meetings channel  | Please note: Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE) | Logs: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/
21:01:29 <TimStarling> #link https://www.mediawiki.org/wiki/Requests_for_comment/Watch_Categorylinks
21:03:29 <TimStarling> Kai_WMDE: hello
21:03:37 <Kai_WMDE> hi there
21:03:56 * gwicke implemented such functionality in a previous life
21:04:02 <DanielK_WMDE> hehe
21:04:04 <TimStarling> it looks like a nice feature
21:04:16 <DanielK_WMDE> i'm wondering whether it should go via the recentchanges table
21:04:26 <superm401> Hey, Kai_WMDE.
21:04:27 <DanielK_WMDE> the current implementation is based on echo, right?
21:04:28 <superm401> I'm on the Collaboration team (maintains Echo).
21:04:36 <SMalyshev> question: how this is different from just watching category page? isn’t every addition/deletion a change for category page?
21:04:41 <Kai_WMDE> correct, DanielK_WMDE
21:04:46 <superm401> DanielK_WMDE, what current implementation?
21:04:49 <superm401> SMalyshev, no.
21:04:50 <DanielK_WMDE> SMalyshev: it isn't. that's exactly what kai wants to fix
21:05:12 <superm401> I'm a little concerned this is going to result in too many notifications.
21:05:17 <DanielK_WMDE> superm401: the one kai proposes. it's inprogress afaik
21:05:34 <SMalyshev> so this is just functionality so that watching category page actually does what it is expected to do?
21:05:34 <DanielK_WMDE> hey RoanKattouw
21:05:40 <RoanKattouw> Hey sorry for the delay
21:05:41 <superm401> What's the reason this is proposed to use Echo when the watchlist does not (mostly for the same reason, the watchlist is way more active than typical Echo traffic).
21:05:45 <DanielK_WMDE> SMalyshev: pretty much
21:05:49 <RoanKattouw> My desk is two floors away from the previous meeting's room
21:05:57 <legoktm> o/
21:06:13 <DanielK_WMDE> SMalyshev: the questions is mainly how to make it scale, not nly one the database, but also in terms of ux
21:06:26 <gwicke> I personally think that events like 'was added to category' would best be distributed through a event queue with pub/sub support
21:06:36 <TimStarling> "The users are able to configure the desired notification channel (email, web). Both of them are disabled by default to keep the current behaviour. The notifications can be held back for daily/weekly digests" -- these are proposed features, not existing features of echo?
21:06:46 <DanielK_WMDE> gwicke: that's what the recentchanges table is :)
21:06:55 <superm401> TimStarling, email/web is implemented.  Bundling is implemented.  Daily/weekly digests not at all.
21:07:01 <legoktm> what's the rationale for using Echo instead of the watchlist?
21:07:10 <superm401> +1, this is a key question for me as well.
21:07:17 <DanielK_WMDE> but yea, a generic pubsub infrastructure would be *very* nice
21:07:38 <SMalyshev> I wonder though what happens if somebody adds a popular category to a widely used template…
21:07:40 * aude waves
21:07:52 <DanielK_WMDE> SMalyshev: lots of notifications :)
21:07:59 <Kai_WMDE> SMalyshev: that is a possible drawback i mentioned in the rfc
21:08:00 <superm401> DanielK_WMDE, there is already a hook.  You could implement pub/sub today.
21:08:10 <superm401> I see that as separate from the RFC.
21:08:10 <TimStarling> well, we do have pubsub for recentchanges
21:08:16 <DanielK_WMDE> exactly
21:08:17 <SMalyshev> I wonder if template ones shouldn’t be separate setting...
21:08:32 <superm401> There's also a hook specifically for CategoryAdded/CategoryRemoved.
21:08:35 <TimStarling> it's not implemented using pubsub, it's just a hook that extensions can plug pubsub support into
21:08:36 <DanielK_WMDE> SMalyshev: templates are syntactical macros. you can't differentiate
21:08:57 <gwicke> one important bit is buffering in a queue
21:08:59 <Kai_WMDE> afaik the hooks is called on every category embedding that template as well
21:09:07 <DanielK_WMDE> SMalyshev: the parser doesn't (always/reliable) know whether something came from a template
21:09:07 <gwicke> so that bursts of activity can be absorbed
21:09:18 <Kai_WMDE> isn't echo already utilizing a job queue?
21:09:50 <TimStarling> the big difference between watchlist and echo is that watchlist is pull and echo is push, right?
21:10:05 <legoktm> yeah ^
21:10:16 <DanielK_WMDE> is it, internally? you register handler  functions to echo?
21:10:19 <TimStarling> the recentchanges table performs that push/pull inversion, it is the event queue
21:10:21 <DanielK_WMDE> or do you mean on the user level?
21:10:30 <superm401> Kai_WMDE, no, not in production.
21:10:35 <superm401> Except test wikis and MW.org
21:10:40 <Kai_WMDE> i see
21:10:42 <DanielK_WMDE> Kai_WMDE: i thought echo uses redis directly, but i could be wrong. perhaps superm401 knows better
21:10:43 <TimStarling> I mean RecentChange pushes events into the recentchanges table, which is effectively a queue
21:10:52 <TimStarling> then it is presented to the user as a pull
21:10:55 <legoktm> DanielK_WMDE: no it just uses a database
21:11:06 <superm401> TimStarling, can we start with why this is proposed to use Echo?
21:11:10 <DanielK_WMDE> so... pretty much the same as recentchanges
21:11:16 <TimStarling> recentchanges is efficient because it has a single queue per wiki
21:11:34 <TimStarling> we don't duplicate events by the number of listening users
21:11:38 <gwicke> it also only has a single event type
21:11:47 <DanielK_WMDE> not really
21:11:51 <DanielK_WMDE> there is a type field
21:11:55 <legoktm> gwicke: no, it has rc_source so you can put it other stuff
21:12:04 <legoktm> put in*
21:12:10 <DanielK_WMDE> though it currently only supports a handlful of hard coded types
21:12:22 <gwicke> so we'd add 'page x became member of category x' in there?
21:12:25 <Kai_WMDE> custom types and type categories can be defined
21:12:40 <Kai_WMDE> (even by other extensions)
21:13:03 <DanielK_WMDE> gwicke: why not? there's a blob field too, for the extra info you need
21:13:13 <TimStarling> that would potentially be a lot of events
21:13:16 <TimStarling> more than edits
21:13:28 <gwicke> yeah, I don't see how that could perform well
21:13:37 <DanielK_WMDE> TimStarling: via template edits, yes
21:14:10 <DanielK_WMDE> gwicke: i also see the performance issue. it's the same whether we use echo or rc, right?
21:14:21 <superm401> DanielK_WMDE, it's not the same at all.
21:14:22 <Kai_WMDE> so events triggered by template edits definitely have to be filtered/ignored
21:14:44 <DanielK_WMDE> superm401: echo would be worse if it has one queue per user. or am i missing something?
21:14:55 <TimStarling> #info Kai_WMDE: so events triggered by template edits definitely have to be filtered/ignored
21:15:00 <superm401> The Echo table has an event entry for every user/event pair.
21:15:06 <DanielK_WMDE> Kai_WMDE: yes, but I don't see a good mechnism to do this
21:15:07 <gwicke> that'd be a work-around
21:15:30 <DanielK_WMDE> superm401: so echo is a lot worse for events targetet at a lot of people
21:15:45 <superm401> DanielK_WMDE, it's not designed for a watchlist scenario.
21:15:47 <TimStarling> DanielK_WMDE: why is it hard to filter template changes?
21:16:10 <Kai_WMDE> the hook doesn't pass that info
21:16:21 <Kai_WMDE> it has to be determined, which might be expensive as well
21:16:23 <superm401> As far as I know, the watchlist is just implemented by filtering the RC table on titles in the user's watchlist table.  So the only per-user data stored is the watchlist table, which is solely user->page.
21:16:25 <DanielK_WMDE> TimStarling: when the page including the template is re-parsed, how do you know which categories come from the template?
21:17:01 <TimStarling> distinguish by who calls LinksUpdate
21:17:13 <TimStarling> if it is the main reparse on save, emit RC events
21:17:22 <DanielK_WMDE> so, some calls to LinksUpdate would just not trigger the notifications?
21:17:28 <aude> DanielK_WMDE: rc_source is not for hardcoded things
21:17:28 <TimStarling> yes
21:17:34 * aude catching up
21:17:34 <TimStarling> or have LinksUpdate return an event list
21:17:41 <SMalyshev> or trigger only for those that explicitly asked for it
21:17:46 <TimStarling> which the caller will either dispatch to RC or not
21:17:49 <aude> rc_type was hardcoded things and is essentially deprecated
21:18:10 <DanielK_WMDE> aude: i wous thinking of rc_type
21:18:13 <DanielK_WMDE> *was
21:18:57 <legoktm> but wasn't that replaced by rc_source?
21:19:15 <superm401> legoktm, it's in-progress/stalled.
21:19:15 <aude> legoktm: yes
21:19:19 <DanielK_WMDE> there's also rc_log_type, it's all very confusing :)
21:19:23 <aude> the old thing is not gone yet
21:19:26 <superm401> https://phabricator.wikimedia.org/T74157
21:19:40 <aude> but people should stop using it (e.g. core, wikibase)
21:19:48 <DanielK_WMDE> so... is there consensus that this should use RC rather than echo, because echo doesn't scale well in a multicast scenario?
21:20:16 <superm401> DanielK_WMDE, I would favor that.  Also from a user experience point of view, Kai_WMDE has not explained why they want to use Echo.
21:20:50 <legoktm> +1 on using watchlist/rc
21:21:04 <Kai_WMDE> superm401: i assumed there was some kind of job queue
21:21:16 <superm401> Kai_WMDE, there is, but it's an optional feature not turned on everywhere.
21:21:45 <superm401> Only problem with watchlist is that you might want two ways to filter ("Only show things affecting categorization" vs. "Only show things in namespace X that are added to/removed from categories").
21:21:53 <TimStarling> what happens if someone adds, say, 10000 categories to a page in a single edit?
21:21:57 <superm401> If desired, that could be done by improving the watchlist UI (though it might be a little complicated).
21:22:05 <TimStarling> should the edit just be rejected?
21:22:10 <DanielK_WMDE> TimStarling: as a DoS attack?
21:22:15 <gwicke> I'm still sceptical about performance, and consider filtering a bit of a hack
21:22:23 <superm401> If definitely can't be rejected, it would break templating.
21:22:29 <legoktm> superm401: 10k categories??
21:22:46 <superm401> Oh, 10,000 categories to a page, I thought you meant 10,000 pages affected by a cat change to a template.
21:22:48 <TimStarling> deliberate or accidental DoS, people do strange things sometimes
21:22:49 <superm401> My bad.
21:23:05 <DanielK_WMDE> they do indeed :P
21:23:08 <legoktm> TimStarling: either reject it or only send notifications for the first 100 or other arbitrary number?
21:23:09 <superm401> Yeah, that could be rejected.
21:23:21 <SMalyshev> maybe there should be a limit on how many notifications are sent?
21:23:24 <TimStarling> if you truncate the list, then what happens when the change is reverted?
21:23:40 <Kai_WMDE> notification limitation is also a feature e
21:23:42 <superm401> Does anyone actually still favor using Echo for this?
21:23:43 <Kai_WMDE> of echo
21:23:57 <DanielK_WMDE> TimStarling: notifications about the revert would again be truncated, so they would only be sent for the first 100 or so
21:24:02 <superm401> Kai_WMDE, what limit are you referring to in Echo?
21:24:21 <Kai_WMDE> configurable maximum number of notifications
21:24:39 <superm401> Kai_WMDE, total number of notifications per user?
21:24:43 <Kai_WMDE> yes
21:24:52 <superm401> I think people are more talking about number of notifications triggered by an edit (across all users).
21:25:30 <TimStarling> probably truncation is the best we can do
21:25:33 <superm401> It might make sense to use a different table, so as not to flood recentchanges, but a similar model.
21:26:01 <superm401> recent_category_changes, and then keep the watchlist table as is.
21:26:05 <superm401> Not sure if that would help.
21:26:07 <TimStarling> well, there is worst case event flow, like adding 10k categories to a page
21:26:15 <TimStarling> and there is typical event flow, which is probably more relevant
21:26:25 <TimStarling> I don't know if typical event flow will actually flood RC
21:26:37 <TimStarling> especially if we don't log updates due to template changes
21:26:39 <gwicke> from a user perspective I'd prefer a collapsed, but complete listing for template changes that add a category to many pages
21:27:00 <legoktm> gwicke: enhanced watchlist?
21:27:02 * DanielK_WMDE is annoyed by Konversation giving superm401, TimStarling and Kai_WMDE all the same color.
21:27:27 <superm401> DanielK_WMDE, yeah, I hate that you can't reconfigure the bucketing.
21:27:31 <gwicke> legoktm: it sounds like it would be incompatible with truncation / filtering though
21:27:41 <superm401> TimStarling, if we just ignored template changes a lot of things would get missed.
21:28:53 <DanielK_WMDE> gwicke: so, since the change event would be tied to the category, it could be a single even (soingle row in rc) for all the pages?
21:29:09 <DanielK_WMDE> there is rc_params, you can fit a lot of bytes in there
21:29:22 <gwicke> DanielK_WMDE: maybe, yeah
21:29:42 <TimStarling> if we had an event in MW marking the end of all refreshLinks jobs for a given edit, that would be feasible
21:29:43 <gwicke> definitely a lot more efficient
21:29:55 <TimStarling> assuming we wanted the RC event to fire at that time
21:29:57 <DanielK_WMDE> the problem is of course that currently, the hook would be fired by each page individually, when each gets re-rendered.
21:30:24 <DanielK_WMDE> TimStarling: isn't there a  concept of sub-jobs?
21:30:34 <TimStarling> yes, it just doesn't do this exact thing
21:30:36 <DanielK_WMDE> wouldn't that allow a bracket atround all the links update jobs?
21:30:45 <DanielK_WMDE> maybe it could be extended to do that
21:30:49 <TimStarling> yes
21:30:53 <gwicke> there's a root job id
21:31:05 <DanielK_WMDE> is aaron around?
21:31:18 <superm401> If we don't need to filter on the page side (only show Project namespaces pages that are added to categories I'm watching), the rc_params idea could work.
21:31:29 <DanielK_WMDE> Kai_WMDE: did we lose you along the way?
21:31:30 <superm401> 356 pages were added to Category:Foo, which you're watching.
21:31:34 <gwicke> I don't think it's straightforward for a job runner to figure out when the last job of a split job is processed though
21:31:40 <superm401> And show the exact pages if it's a small number.
21:31:40 <gwicke> there are retries etc to consider
21:31:47 <hoo> DanielK_WMDE: He's on IRC... can invite him, if needed
21:31:52 <Kai_WMDE> I'm following :)
21:32:15 <DanielK_WMDE> hoo: me might have ideas on how to best do this
21:32:40 <DanielK_WMDE> anyway...
21:32:53 <superm401> Hook is not necessarily the right way to do this.
21:33:12 <DanielK_WMDE> there is two extreme cases: a category being add to a lot of pages via a template. could be ignored. and a lot of categories being added to a page. could be truncated.
21:33:15 <gwicke> but, the root job could have a link to an id or the like that makes it possible to look up the list of affected pages separately
21:33:33 <gwicke> and jobs could just use the same id to extend the list of affected pages
21:33:56 <DanielK_WMDE> gwicke: could use the revision id to tie them all together
21:33:57 <gwicke> so rc_params would have the id rather than the actual titles
21:34:09 <superm401> I think ignoring changes caused by a template reduces the utility.  At the very least, need to show the root change "Template:Foo was added to Category:Bar", even if the consequent changes are ignored.
21:34:19 <gwicke> DanielK_WMDE: yup
21:34:47 <TimStarling> gwicke: yeah, it is possible, at the expense of at least 3x longer development time on top of Kai's proposal
21:34:50 <DanielK_WMDE> superm401: that's not necessarily true, categorization is often done in <includeonly> sections
21:35:11 <superm401> DanielK_WMDE, true, so in some cases that root change could not be shown.
21:35:20 <DanielK_WMDE> #info <superm401> I think ignoring changes caused by a template reduces the utility.  At the very least, need to show the root change "Template:Foo was added to Category:Bar", even if the consequent changes are ignored.
21:35:55 <DanielK_WMDE> superm401: since it doesn't affect the category, it wouldn't be a change to that category. there would still be the edit to the template itself recorded in rd
21:35:56 <gwicke> TimStarling: not sure; with such a scheme it might be feasible to keep everything in recentchanges
21:35:58 <DanielK_WMDE> *rc
21:36:05 <gwicke> and only move out the affected pages
21:36:21 <gwicke> each template edit would only cause a single RC entry
21:36:36 <superm401> #info However, with <includeonly>, the template itself is not always affected.
21:36:50 <gwicke> a long time ago I lobbied for having a timestamp in categorylinks
21:37:00 <TimStarling> yeah, what you are saying is possible, and 3x longer is not the end of the world
21:37:27 <gwicke> if that's still there, maybe that could even be used as a cheap way to figure out which pages were affected
21:37:43 <gwicke> revid would be better of course
21:38:13 <DanielK_WMDE> I propose a baseline implenmentation: make RC entries when pages are added to / removed from a category. cut off if there is too many categories being added/removed. ignore hook calls if the parse was triggered by a template edit.
21:38:18 <TimStarling> categorylinks has always had a timestamp
21:38:32 <gwicke> hehe
21:38:37 <TimStarling> by long ago do you mean like a decade?
21:38:38 <gwicke> not always ;)
21:38:39 <DanielK_WMDE> that would give us a baseline that is easy enough to implement, easy to expand on, and safe enough
21:38:54 <gwicke> TimStarling: yeah, roughly
21:39:05 <gwicke> 2005 sounds about right
21:39:12 <DanielK_WMDE> :P
21:39:19 <TimStarling> gwicke thinking ahead as always
21:39:41 <superm401> DanielK_WMDE, baseline should also specify how it's exposed in the user interface.
21:39:43 <gwicke> I used it for a category-based news feed for wikinews back then
21:40:33 <gwicke> for this use case the timestamp wouldn't necessarily be unique
21:41:51 <DanielK_WMDE> superm401: in the watchlist, as usual. but i'm not sure how you'd subscribe to these events
21:42:05 <TimStarling> watch the category
21:42:14 <gwicke> what would happen if the watchlist looked for all pages in watched categories added after last watchlist TS in categorylinks?
21:42:29 <DanielK_WMDE> TimStarling: no way to watch just the category page, not the contents?
21:42:40 <DanielK_WMDE> i mean, i'm all for it, but people *will* complain
21:42:40 <TimStarling> no
21:42:57 <TimStarling> meh
21:43:12 <superm401> gwicke, what about "removed from category"?
21:43:20 <DanielK_WMDE> we'll be killing some uncommon but important use case :)
21:43:21 <TimStarling> this is a 4 digit bug number we're solving here
21:43:24 <gwicke> superm401: yeah, that part would work less well
21:43:24 <TimStarling> people will love us
21:43:27 <superm401> :)
21:43:43 <gwicke> superm401: be positive, only show new shiny categories ;)
21:43:44 <superm401> I don't think "watch cat page, not contents" is needed (it might be if we went the Echo route).
21:43:53 <DanielK_WMDE> TimStarling: i was thinking of the visible whining minority :)
21:44:07 <superm401> On the watchlist, we could probably allow filtering ("Don't show categorization changes").
21:44:08 <DanielK_WMDE> but yea, i'm pretty sure most people will love it
21:44:31 <DanielK_WMDE> superm401: yes, you are right. except you have to implement this twice
21:44:41 <DanielK_WMDE> because there are two watchlist implementations
21:44:41 <superm401> DanielK_WMDE, implement what twice?
21:44:49 <superm401> Oh, enhanced?
21:44:52 <DanielK_WMDE> yea
21:44:55 <DanielK_WMDE> isn't it lovely?
21:44:58 <TimStarling> whether there is some important use case for watching category page contents is a question for product managers, right?
21:45:09 <aude> DanielK_WMDE: more than 2
21:45:17 <superm401> DanielK_WMDE, oh, believe me, we know plenty about watchlist and RC proliferation from Flow.
21:45:18 <aude> mobile stuff, feeds, etc
21:45:19 <DanielK_WMDE> oh rly?
21:45:27 * aude thinks of feeds as another view
21:45:30 <aude> then api
21:45:35 <superm401> TimStarling, there are no product managers for "general core" anymore, right?
21:45:38 <DanielK_WMDE> aude: true
21:46:30 <DanielK_WMDE> implementing this filter in all the right places is going to be harder than the actual functionality >_<
21:46:32 * aude wants some of this code more unified
21:46:41 <aude> and vary where needed on presentation things etc
21:46:56 <DanielK_WMDE> yes, we should take that opportunity to consoliidate this somewhat
21:47:05 <DanielK_WMDE> baby steps..
21:47:17 <aude> yet, also remember recent changes / related changes share code with watchlist
21:47:19 <TimStarling> so I think we are...
21:47:31 <aude> which is good yet can present challenges
21:47:36 <TimStarling> #agreed use RC not echo
21:47:36 <DanielK_WMDE> aude: maybe you could work with kai directly on this
21:48:05 <aude> maybe though have bazillion things to do :o
21:48:20 <aude> but can help with questions, at least or something
21:48:46 <TimStarling> #info Two basic implementation options: "baseline" with refreshLinks ignored, and "full" with refreshLinks merged per edit, end of job batch event
21:48:48 <DanielK_WMDE> yea, i was mostly thinking of helping him to dig in when needed
21:49:01 * aude nods
21:49:32 <TimStarling> #info UI question unresolved: is it necessary to continue to allow watching of category page contents only
21:49:39 <superm401> TimStarling, RC line per cat-page pair, or one per category?
21:49:45 <superm401> Latter with rc_params.
21:49:49 <DanielK_WMDE> #info watching a category page subscribes to pages being added/removed. filtering should be possible in all the various rc/watchlist feeds
21:49:58 <gwicke> #info alternative to end of job batch event would be to point from RC entry to ID and lazily store information about changes from jobs under ID somewhere
21:50:31 <TimStarling> gwicke: a mutable event?
21:50:49 <gwicke> no, an event storing an id of associated information
21:50:51 <TimStarling> if you see the RC entry early, it will not have all the page changes in it
21:50:52 <gwicke> which is built lazily
21:51:08 <DanielK_WMDE> a delay + coalescing mechanism, maybe...
21:51:35 <TimStarling> resourcing?
21:51:43 <gwicke> end of batch is difficult to establish afaik
21:52:12 <gwicke> we've had parsoid update jobs being retried for about a year
21:52:32 <gwicke> not sure if they are still in there
21:52:36 <DanielK_WMDE> TimStarling: wmde comitted to implement the feature, though the resourcing is limited
21:52:57 <TimStarling> anyone from WMF here interested in helping?
21:53:07 <DanielK_WMDE> Kai_WMDE: how much time is planned for this, do you know?
21:53:11 <TimStarling> superm401?
21:53:29 <superm401> TimStarling, I can be available to provide support/code review.
21:53:42 <Kai_WMDE> DanielK_WMDE: since the concept isn't completed, there is no time planning yet
21:54:39 <TimStarling> DanielK_WMDE: WMDE will still be committed if we require the full template-aware feature?
21:55:47 <DanielK_WMDE> TimStarling: help from WMF with the job queue side of things would be very much appreciated in that case, I think
21:56:03 <TimStarling> ok
21:56:26 <TimStarling> RFC status?
21:56:47 <Kai_WMDE> needs to be updated, i guess
21:56:57 <Kai_WMDE> rfc itself
21:57:08 <DanielK_WMDE> TimStarling: do you think we need another discussion about the updated rfc, or can kai just go ahead?
21:57:18 <gwicke> so, is this something that WMF would use?
21:57:22 <superm401> Might be good to have 15 minutes to review updated RFC in the future.
21:57:33 <aude> gwicke: definitely think so
21:57:35 <TimStarling> gwicke: yes
21:57:40 <aude> somethign the community wants
21:57:59 <TimStarling> WMDE are doing this because the German Wikipedia wants it
21:58:18 <gwicke> I see; what are the minimum requirements for a WMF deployments then?
21:58:25 <gwicke> *deployment
21:58:33 <gwicke> just baseline, or category support?
21:58:38 <TimStarling> I guess we can decide that in a second meeting
21:58:50 <superm401> gwicke, you mean baseline or template support, right?
21:58:51 <TimStarling> also we should have a wikitech-l post
21:59:00 <gwicke> superm401: yeah, sorry
21:59:05 <gwicke> multitasking ;)
21:59:14 <superm401> TimStarling, summarizing the meeting?  I can send that if you want.
21:59:35 <TimStarling> superm401:  yeah, especially the two open design questions
21:59:42 <gwicke> if our minimum requirement is template support, then I think it would be good to see whether we'd have the resources to make that happen
21:59:57 <gwicke> before starting work
22:00:14 <superm401> TimStarling, two being:
22:00:21 <superm401> 1. Whether to ignore template changes, or merge them at batch end?
22:00:34 <superm401> 2. Whether to allow watching only cat page, while ignoring categorization/decategorization
22:00:36 <superm401> ?
22:00:40 <TimStarling> yes
22:01:05 <TimStarling> I'm glad we had a full hour for this RFC
22:01:13 <DanielK_WMDE> simpler: ignore the individual updates triggered by a template edit, and just generate an update for all pages that include the template
22:01:26 <TimStarling> we're out of time now, thanks for coming everyone
22:01:27 <superm401> DanielK_WMDE, wasn't that the batch end option?
22:01:28 <DanielK_WMDE> much easier than collecting info from multiple events
22:01:37 <TimStarling> thanks for writing the RFC, Kai_WMDE
22:01:50 <superm401> Yep, thanks.
22:01:54 <Kai_WMDE> thank you all
22:02:01 <DanielK_WMDE> superm401: well, batching jobs/events after they have been queued is much harder than posting a single "bulk" event directly
22:02:35 <superm401> DanielK_WMDE, but how do you know what to put in the bulk event.  In theory, a template could cause every page transcluding it to go into a different cat, right?
22:03:07 <TimStarling> #endmeeting

Tim Starling (talk) 05:11, 21 August 2025 (UTC)