Talk:Synchronizer

Edit summaries
@Sophivorus: I tested this out a few times and it seems like it is not always creating edit summaries correctly. MediaWiki:Synchronizer.js appears to be trying to create a link with an interwiki prefix (with things like ) however it looks like it sometimes bases this off of Wikidata sitelinks which use database names and not interwiki prefixes. I do not think you want to attempt to maintain a complete mapping. You would be better off linking through Wikidata since it already knows how to link to each page via its sitelinks, e.g.,: Notice how the database name  is now useful when used in conjunction with   at Wikidata.
 * Edit a summary that seems good: 1144856459
 * Edits with problematic summaries: 71665579 13075854 24722031 5827927
 * Should have had more useful summaries like: Update from master using Synchronizer
 * Edits with problematic summaries: 71665579 13075854 24722031 5827927
 * Should have had more useful summaries like: Update from master using Synchronizer

You should do away with hackery like  (luckily it seems this is only used to create the "master" wikitext link in the summary). Instead, use master wiki database id combined with the Wikidata item id to create the "master" wikitext link for the summary by going through d:Special:GoToLinkedPage (using something like  and   if I am reading the code correctly).

I would attempt to edit it myself but I do not have access as I am not an interface/UI editor like you are (at least for the next few months anyway). —Uzume (talk) 00:51, 16 March 2023 (UTC)


 * You might want to consider adding a hashtag to these edits too. See Hashtags. Even better (but much more difficult) would be figuring out how to add a revision/change tag. See Manual:Tags and Help:Tags. —Uzume (talk) 01:15, 16 March 2023 (UTC)
 * The definition of which site is "master" also seems fragile (and perhaps flawed), e.g., contains text   and yet "master" seems to be defined for that at User:Sophivorus/sync by master on the template and module at the button creation site (which can be different from page to page across different wikis). Perhaps you should consider a more centralized method of defining that like a sitelink badge. —Uzume (talk) 12:42, 16 March 2023 (UTC)
 * Another problem with this tool is lack of attribution since you are not importing changes (likely due to permission issues). This makes me wonder if this tool would not be better implemented as a bot at toolforge or similar. —Uzume (talk) 12:42, 16 March 2023 (UTC)
 * @Uzume Thanks so much! I was unaware of d:Special:GoToLinkedPage and it was just what was needed. I just updated the code to get rid of that ugly getInterwikiPrefix method.
 * I was also unaware of Hashtags! I just added #synchronizer to the edit summary.
 * As to change tags, I agree they are much more difficult. Adding them to the API edit request is trivial, but as you probably know, the tag needs to be proposed and approved independently in each wiki for it to work in that wiki. I did that for Proveit on several wikis, and it's just not worth the effort. However, if you're interested in tracking edits in a specific wiki, feel free to request the tag there and I'll be happy to add it to the API edit request.
 * As to the inconsistent master with Module:Transcluder, you're right, I was transitioning to enwiki as the master version and you caught me mid-way. I just started updating the comments to point to enwiki as the master version. As to sitelink badges, I didn't know about those! They sound great, but I wonder if the benefits they promise are worth the added complexity for new users of Synchronzier. Perhaps by having a fallback system in which if no sitebadge is set, the "master" parameter is checked, and else the current wiki is considered the master? I'll give it some thought and would be happy to hear yours!
 * As to lack of attribution, perhaps the tool could check the usernames of the revisions being copied and write an edit summary like "Updating from master via Synchronizer, contributions by Foo, Bar and Baz"?
 * Thanks for all the help and new info I was unaware of! I've been editing and hacking wikis for 10+ years and I'm always amazed to keep finding new stuff! I'm looking forward to your thoughts and replies, and please let me know of any more issues you find or features you'd like to see. Cheers! Sophivorus (talk) 13:01, 16 March 2023 (UTC)
 * @Sophivorus: Cool, that looks much better:
 * 741359519
 * And it will work with any sitelinks at Wikidata even the more esoteric ones like wikimania and it is future proof too.
 * As for sitelink badges, I believe it would make sense to get a badge made for Wikimedia multi-site master page. They are just flags on the sitelinks. As such they do not enforce that only one sitelink for a Wikidata item has such, so yes, some methodology would be need to manage that. You mentioned a "fallback system" for ensuring such but I would rather the script instead provide a means to edit the Wikidata item (not unlike how its sends edits to the other sites for for synchronization) specifying such when an item does not adhere to "exactly one sitelink needs to have the master badge". It could also provide a small button at the top of the synchronization table to allow one to respecify such later affecting the ability to "rehome" like you are doing with (from   to  ).
 * Another thing you might want to consider is having the button generation be more error proof. This only works with Wikidata but the button generator does nothing to ensure such (the Scribunto Lua module could check Wikidata at page render time instead of the script failing at button press time). I would much prefer the interface to the script be the Wikidata item ID and not sitelink and database siteid. The button generator could allow specification of the Wikidata item via either its QID or sitelink with optional siteid (defaulting to local; see ). The latter provides for ease and backwards compatibility and it makes sense for the button generator to provide an error when it is unable to come up with a valid QID. I believe the button label should default to the Wikidata item label instead of the provided master sitelink (as it currently does).
 * And on that note, if the script has a Wikidata item edit capability it could ensure other things like the language labels be synchronized to the sitelink values (somehow; that might get tricky) and it could add claims of or  to its  statements or something too.
 * —Uzume (talk) 15:16, 16 March 2023 (UTC)
 * @Uzume Hi!
 * I just updated Synchronizer to use Wikidata IDs rather than titles. However, instead of using mw.wikibase.getEntityIdForTitle for backwards compatibility, I updated all the (few) pages that used Template:Synchronize to the new usage with Wikidata IDs. I figured that this way it'll keep the code and documentation simpler, and requiring a Wikidata ID will also help users understand that an associated Wikidata entity needs to exist (I already had someone asking why Synchronizer wouldn't work for a JS page with no Wikidata entity associated). I also added some error checking to Module:Synchronizer and started using the Wikidata label for the button, rather than the master page title. So thanks again for your ideas, they're a great step forward!
 * Regarding badges, I was surprised to find that a master version badge already exists (see the last of the list of badges). However, it doesn't pop up when editing sitelinks. It seems to be used as a regular property instead (example). It'd be nice to have it behave like the other badges, but I guess that's just a detail. Your ideas regarding the use of badges and an interface to set and change them, and using Synchronizer to add claims and language labels to Wikidata are all great and I'll try to implement them eventually, but they're a bit difficult so perhaps I'll wait until more people actually use the tool heh
 * Next I'm thinking on adding a button to update all non-forked modules at once, and another button next to each forked module to see the diff since it forked. I think that'd make the tool considerably more useful, at least for me. Thoughts?
 * Once again, thanks for all your ideas and please don't hesitate to share more! Sophivorus (talk) 20:24, 21 March 2023 (UTC)
 * @Sophivorus: Cool, that also solves a problem with the previous arrangement. Before Synchronizer is only deployed to a very few sites and the only way to specify a Wikidata item was via a local site link. What if you wanted to synchronize pages but the pages did not exist at any of the sites Synchronizer was deployed at? Now that it is specified by Wikidata item QID, the issues goes away. I also find curious. Maybe it used to be a sitelink badge but was never fully implemented or removed for some reason (otherwise why call it a badge? or possibly that is how badges worked before they were implemented as being attached to the sitelinks). —Uzume (talk) 20:37, 21 March 2023 (UTC)
 * @Uzume I just implemented a first version of the button to update all non-forked modules. Since it's a very powerful feature, the button is hidden from non-autoconfirmed users, and any feedback, testing or wise advice would be welcome. Perhaps the button should be at the bottom rather than the top? Perhaps it should be smaller, or red? Perhaps the warning when clicking should be different? Cheers! Sophivorus (talk) 00:30, 22 March 2023 (UTC)
 * @Sophivorus: I would be extremely hesitant to even test that with any real live code (unless I had been updating them all recently already). Each site has its own potentially complex dependencies that would be very hard to foresee. It looks nice though. I would probably prefer if it had the same styling as the other "Update" buttons, darker with the rounded corners. I think I would put the button in the space for the "Update" button on the master entry instead of floating above or below the main table. I don't have much of an opinion on whether you want to make it another color like red as I can see it either way. —Uzume (talk) 01:09, 22 March 2023 (UTC)
 * @Sophivorus: Okay, I tried it. I thought the safest bet was since none of the sites were far behind and none of them were forked. Wow! That is a really great way to get spammed with notifications about your first edit at a pile of sites—*laughs*. Anyway, I liked the dialog warning and it seemed to work, save some permission issues at two sites. I noticed on one of the two sites a lock icon is shown but the other did not. d:Special:GoToLinkedPage/eswiki/Q96679044 does not show a lock icon for me but it also fails to update with "No Permission". The site itself states: "No tienes permiso para editar las páginas del espacio de nombres Módulo." or "You do not have permission to edit pages in the Módulo namespace.". —Uzume (talk) 01:27, 22 March 2023 (UTC)
 * @Uzume Yes! Moving the button next to the master row was just what was needed, the interface is much better now. Thanks for testing it out! I'm glad it went ok. As to the problem with eswiki, I just improved Synchronizer to detect protected namespaces, so thanks for that too. Cheers! Sophivorus (talk) 11:17, 22 March 2023 (UTC)

@Sophivorus: I am glad to be of help. Yes, it looks much better in the "Action" column. I notice the "Update" buttons go away when the "Status" column is "Updated". Shouldn't the "Update all non-forked" also go away when all the non-forked are status "Updated"? Also, maybe this is a small nitpick but should the "Action" column be left justified? I was thinking center or right perhaps. I also think everything could use some i18n and l10n, especially the edit summaries which would help if they were done in the language matching either the page content language or the site content language for things that are forced to a single language (like Scribunto modules). I noticed you updated eswiki but the edit summary stands out in English amongst the past edit summaries. I am not sure how you would go about adding this to translatewiki.net. —Uzume (talk) 12:43, 22 March 2023 (UTC)


 * @Uzume Yes I agree, the "Update all non-forked" button should go away or not appear if there aren't any modules to update, but doing so is not trivial nor crucial so I left it for later. As to i18n and l10n, I totally agree, especially about the edit summaries. I'll tackle that next (but probably not today heh). Cheers and thanks again! Sophivorus (talk) 12:59, 22 March 2023 (UTC)
 * @Sophivorus: You might also want to put this into a real revision control system like git. That would allow you to take updates from translatewiki.net which could then be deployed by something with such rights. I know that at the moment you are doing that yourself but I would think there must be a privileged bot to do such things from git repos. —Uzume (talk) 13:07, 22 March 2023 (UTC)
 * @Sophivorus: Then you can take patches from others too (e.g., people could submit to changes in gerrit). On the i18n/l10n front, where doing an update, a diff is presented. I noticed this seems to use the default language of whatever site is being updated, however, it would be helpful if you used  to change the interface to the language of the user using the buttons or at least the page the synchronizer button is on. For example, in my case that would be "en" from User:Sophivorus/sync at mediawikiwiki but I can see how a user on eswiki (where this is also deployed) would likely want "es". Perhaps someone from here (a multilingual wiki) might also want "es" or something else. —Uzume (talk) 13:39, 22 March 2023 (UTC)
 * @Sophivorus: I may have found a bug. It seems it does not always detect "forked" properly (which I imagine would affect "Update all non-forked"). Case in point: go to User:Sophivorus/sync, use the last button "Synchronize Module:ScribuntoUnit", find "ptwiki" entry and click on "Update". Can you tell me why that sort of diff could potentially be in the history of enwiki (the master)? Those clearly contain Portugese l10n changes that I cannot imagine would every have been on the enwiki master. But the ptwiki entry is labelled as "8 revisions behind" and not as "Forked". —Uzume (talk) 15:50, 22 March 2023 (UTC)
 * @Uzume Hi! I think you may be misunderstanding what the diff shows. The diff shows the difference between the current forked version at ptwiki, and the current version at master (in other words, the change you're about to make if you hit Save), so it's reasonable ok to see Portuguese characters on the left. Perhaps you were expecting the diff between the last "shared version" (the version before the fork happened) and the current one at master? Or between the last shared version and the current one at ptwiki? They all seem relevant and useful, and my intention with the "Diff since fork" button I mentioned before was to show the latter diff. Perhaps each diff should include some text explaining exactly what is being shown. Sophivorus (talk) 19:13, 22 March 2023 (UTC)
 * @Sophivorus: It is not the diff that is the problem. It is the fact that the ptwiki entry says "8 revisions behind" and not "Forked" (not to mention what that implies for the "Update all non-forked" button). Synchronizer is treating the ptwiki version is part of the history of enwiki master but the enwiki master would never have any Portugese in it so it is clearly forked (the diff just exposes the flawed logic; the diff itself is fine). There is something wrong with your forked vs. historical revision logic. —Uzume (talk) 19:54, 22 March 2023 (UTC)
 * @Uzume Ah! I understand now! Curiously, it seems that enwiki actually did have a version with Portuguese characters exactly 8 revisions ago, lol Sophivorus (talk) 20:00, 22 March 2023 (UTC)
 * @Sophivorus: Oh, my mistake then. I should have checked that. I just assumed it couldn't have. So the logic is fine—just the revisions at enwiki are weird. I am sorry to have bothered you with a false report. —Uzume (talk) 20:04, 22 March 2023 (UTC)
 * @Sophivorus: On that note though. It might be nice to work on the "Forked" cases and compare revisions at the target as well. That could have many compares so I am not sure how to optimize the logic (since that ups things from $$O(n)$$ to potentially $$O(n \cdot m)$$ or $$O(n^2)$$; it should be much smaller on average though), but it would be really nice if it said things like "Forked 2 revisions back" or similar. This would also mean you could determine if the code is entirely different and never forked (i.e., no common ancestors) which would also be good to know (this is actually not that uncommon when having many templates/modules with similar names that get mislinked in Wikidata). It should be possible to cache these compares somewhere though as in theory old revisions never change. I am not sure where is the best place to cache such though (maybe in the users browser but it would be even better to share that across users). —Uzume (talk) 20:28, 22 March 2023 (UTC)

@Sophivorus: bug report: edit summary is wrong in enwiki:1145928504. Wikidata QID missing in d:Special:GoToLinkedPage/mediawikiwiki/undefined linked on the word "master". —Uzume (talk) 21:01, 22 March 2023 (UTC)


 * @Uzume Hi! I just implemented a first version of the "Diff since fork" button, let me know what you think! Showing "Forked 2 revisions back" would be great but as you noted, would require an extra request per forked module. Perhaps it's better to show that kind of info only when clicking the "Diff since fork" button? Same with detecting no common ancestor: it'd be great but would require that extra request per forked module. Perhaps it's not that big of a deal, I'll give it some thought and would be happy to hear yours.
 * As to the bug report, I'm pretty sure it was because that edit was made with a broken version of Synchronizer while I was developing. It shouldn't happen again.
 * Next I'll definitely tackle i18n and l10n, as there seems to be some forking already heh. Cheers! Sophivorus (talk) 23:59, 23 March 2023 (UTC)
 * @Sophivorus: I am not quite sure what that is doing and in most cases it just hangs with "Loading diff...", however, in come cases it seemed quite useful in identifying diffs since the fork point, however, if you know where it forked I am not sure why you don't just say "Forked X revisions back". So I am confused. —Uzume (talk) 00:20, 24 March 2023 (UTC)
 * @Uzume Apologies, I guess I was tired and didn't test it enough. The tool hanged when there was no common ancestor. I just fixed that. I also added helpful info to the dialog about how many revisions ago the fork happened. The wording and UX/UI can surely improve but I think it's already useful. Thanks as always! Sophivorus (talk) 02:10, 24 March 2023 (UTC)
 * @Sophivorus: Okay, cool. So you are doing a nearest common ancestor search (once you have decided it is forked). I like the new dialogues, e.g.:
 * 13 revisions to this module since the fork happened.
 * 34 revisions to master since the fork happened.
 * I wonder if "module" should be "page" though.
 * I just happened to be sync'ing a Scibunto module but that is probably not what was meant by "module" in the preceding.
 * The "Diff since fork" button probably should go away once you get the "This module has no common history with master." (again I quest the term "module") message. And maybe change the "Status" column should be updated after using that button too either reflect no common ancestry somehow or how many revisions back is the nearest common ancestor (e.g., Forked X revisions back).
 * In fact, if you want to delay the nearest common ancestor search from the main button (as you seem to be), you could have a second button (maybe "Analyze forks") on the master "Action" column to run such for all the forked and update the "Status' column as well as the "Action" column adding the "Diff since fork" button (or even "Diff since 2 revisions back").
 * If you still prefer it on each entry (instead of across all forks) you could instead have "Forked" entries have a "Analyze fork" button that then updates the status and provides the "Diff since fork" when there is a nearest common ancestor.
 * Anyway, some thoughts. Cheers, —Uzume (talk) 06:12, 24 March 2023 (UTC)
 * Thanks, again I agree with your assessment. I just updated the tool so that it does that extra request for Forked modules to determine if they're Unrelated to master or not. I could now also add "Forked X revisions back" to the status of Forked modules, but I'm afraid that'd be ambiguos (it could mean X revisions to master since the fork happened, or X revisions to the module since the fork happened) and I can't decide which is more important and what's the best wording. What do you think?
 * I also changed the button label to "Analyze" (much better, I agree) and I also changed the dialogs from process dialogs to simple alert and confirm dialogs, which clarified, reduced and simplified the code considerably.
 * Regarding the use of the word "page" rather than "module", I'm not sure because I expect that 99% of pages updated using this tool will be Lua modules. The only other kind of page I can think of where this tool could be useful would be JS (and maybe CSS) pages. Thoughts? Cheers! Sophivorus (talk) 14:56, 24 March 2023 (UTC)
 * @Sophivorus: Okay, I like the diff diaglogs better, however, you need to work on labeling (which goes along with i18n/l10n), e.g., "Update all non-forked" is a tad confusing since there are now "Unrelated" which also should not be updated by that. Also buttons on the "Update" diff now say "Cancel" and "OK". "OK" is very deceptive. I liked "Save" or "Save changes" before. This is an opportunity to use something like  or   and   (or better yet get your own translations setup there). —Uzume (talk) 21:45, 24 March 2023 (UTC)
 * @Uzume Hi! Well, I agree with your observations as usual, so I changed the default button labels for better ones until I manage to get some i18n and l10n going. I also changed the text of outdated modules to "Outdated (X revisions behind)" and that of forked modules to "Forked (X revisions ahead)". I take "ahead" to mean what it means in git, namely X revisions done to the forked module since it forked. I expect our audience to be mostly familiar with git so I'm ok with the decision and the wording. I guess it'd be cleaner to have just "Outdated" and "Forked" but I think the parenthesis adds too much value to just get rid of it. I then changed the "Update all non-forked" button to "Update all outdated" to avoid the ambiguity with Unrelated modules. If nothing else gets in the way, next time I'll tackle i18n and l10n, cheers! Sophivorus (talk) 22:54, 24 March 2023 (UTC)

@Sophivorus: That looks great. I still wonder about the value of making this a Javascript add-on via Snippets/Load JS and CSS by URL when this could very easily be a user directed bot at toolforge (it could even be mostly in browser-side Javascript possibly with some node.js server-side if you wanted). It could authenticate via OAuth using Wikimedia SUL. There there would be no porting issues and the code could easily live in a git repo and you could take patches (either through Gerrit changes here or GitHub PRs). If you did that you could also cache the revision hashes and speed things up for multiple runs even across users, etc. —Uzume (talk) 23:18, 24 March 2023 (UTC)


 * @Uzume Hi! I've been giving some thought to what you said and did some changes to the tool accordingly. I got rid of Module:Synchronizer, see Synchronizer for the new loading method. The new system will also allow for a better and more powerful UI over time. This first version is admittedly limited, but decent enough I believe. Future versions could include a lookup field or autosuggest for modules you've loaded in the past, automatic detection of the master module with the badge from Wikdata, etc.
 * Although there's still a lot to improve, I trust you'll agree this is a step in the right direction. It's not a tool in Toolforge yet, but I think this change would make a port easier. That being said, I'm still not convinced that the benefits of a port would outweigh the downsides. I think that keeping the tool simple is crucial to avoid what happened to DiBabel. That's why I also resist the idea of adding a backend or database.
 * On that same line, I wonder how much i18n is actually needed, since all users of this tool will probably know English. However, I do agree that the edit summary should definitely be localized, but I'm thinking that maybe a translation table at Commons would suffice for that, again to keep it simple.
 * Anyway, still experimenting and open to ideas, talk and feedback, cheers! Sophivorus (talk) 15:21, 25 March 2023 (UTC)

/doc
This is a great tool, very nice work on the interface. I am using it for en:Template:NUMBEROF, en:Module:NUMBEROF, and 4 other sub-pages which are Lua, installed on 80+ wikis. Each update requires these 6 files + 2 /doc pages. 8 pages x 80 sites = 640 updates each time there is a change (not all changes touch 8 pages but sometimes it does). Thus a tool like this is indispensable.

The problem is maintaining the /doc pages. The tool requires each page have a WikiData item and /doc pages are not considered notable on WikiData so it's not possible to create an item. Reserved sub-page names are /doc, /XML, /meta, /sandbox, /testcases and /TemplateData. Is there another solution for these sorts of pages? -- GreenC (talk) 01:22, 10 September 2023 (UTC)


 * @GreenC Hi, I'm glad you're finding the tool useful! I guess Synchronizer could be improved to handle this kind of case, but I can't think of an elegant way to do it yet. Also, I think this may be an exceptional case, because /doc pages are usually written in the language of the local wiki, and even NUMBEROF /doc subpages may be translated to the local languages eventually and synchronization may cease to be desirable, right? Thus, perhaps a better solution overall would be to request an exception to the Wikidata policy for NUMBEROF /doc pages, citing point 3 of d:Wikidata:Notability ("It fulfills a structural need"). I'm confident the request would be reasonable enough to warrant an exception (if you decide to request it, please link me to it so I can support it). Thoughts? Cheers! Sophivorus (talk) 15:38, 11 September 2023 (UTC)
 * Sometimes users will localize the doc page, but for the most part it's stayed pretty consistent over the past 3 years. Even then, Synchronizer is helpful for visualizing and organizing what can be synced as-is and what needs manual work. A lot easier then manually opening each page and trying to determine if there are diffs. Plus one of these doc pages is for the Module so it's usually not messed with locally.
 * I guess it will come to a consensus discussion to change the notability rules. I'm not sure they will give an exception for NUMBEROF since there is no argument why it's exceptional (other than the use case of Synchronizer), it would be for every module/template. -- GreenC (talk) 17:20, 11 September 2023 (UTC)
 * @GreenC I've given it some thought and tried a few things but I can't come up with a good solution yet. I'll continue to think about it in the back of my head, but let me know if you come up with something! Sophivorus (talk) 19:47, 11 September 2023 (UTC)