Extension talk:Graph/Plans

Table of usage
I'm not sure how accurate using "Pages with disabled graphs" is to get a count. Graphs have been disabled for a while now, presumably people have been removing them from pages, and thus also removing the category. Maybe looking at an older dump of the category table might be better but probably quite a bit of work. Bawolff (talk) 23:18, 14 July 2023 (UTC)


 * I'm not sure how accurate using "Pages with disabled graphs" is to get a count. Graphs have been disabled for a while now, presumably people have been removing them from pages, and thus also removing the category.
 * Mmm, great spot, @Bawolff. It looks like @TheDJ spotted a similar issue.
 * As a next step, I'm going to ask some folks internally about how we might go about generating a more accurate measure of usage. Of course, if new ideas emerge in your mind for how we might go about doing this and/or if volunteers who you think would be well equipped to help out with this, please let me know. PPelberg (WMF) (talk) 16:58, 28 July 2023 (UTC)

Including personal opinions
@Bawolff: I appreciate you both being bold in adding this statement and asking whether it's appropriate to do so.

What if you prefaced the bit beginning with "The elephant in the room is that graph is a high maintenance extension... with your username so as to show this is a personal opinion that may or may not be something the information we end up gathering substantiates? How does this sound to you? PPelberg (WMF) (talk) 23:54, 14 July 2023 (UTC)


 * SGTM. Bawolff (talk) 23:59, 14 July 2023 (UTC)
 * Awesome. PPelberg (WMF) (talk) 00:03, 15 July 2023 (UTC)

Generating graphs from Wikidata
hi – two things!


 * 1) Thank you for contributing helpful information to the Extension:Graph/Plans page  🙏🏼
 * 2) Related to "1.", are you able to add an example or perhaps describe in a bit more detail what an up-to-date graph from Wikidata data generated using the Graph Extension looked like? I ask as this is the first time I'm hearing of the extension being used in this way.

Ok, thank you ^ _ ^ PPelberg (WMF) (talk) 21:37, 28 July 2023 (UTC)


 * Hi you can find one example here (see the wikicode). In short, when the graph extension was working, this template get the neutron lifetime from Wikidata and then plot these values as a function of the year.
 * So that, when I add a new value on Wikidata, the graph is automatically updated. Pamputt (talk) 21:46, 28 July 2023 (UTC)
 * Ah, I see – thank you for following up with this added context and for doing so so promptly, @Pamputt!
 * Sure enough, just after I posted the above, I also saw that @Nux helpfully added the example of en:Template:Airport-Statistics.
 * In any event, @Pamputt can you please have a quick look at the below and let me know if there is anything missing, unnecessary, and/or inaccurate about how I'm currently understanding how Wikidata, the Graph Extension, and on-wiki templates can be used to generate auto-updating graphs?
 * How Peter (me) currently understands the relationships between Wikidata, the Graph Extension, and on-wiki templates
 * 1. Use SPARQL to retrieve the information you're seeking from Wikidata. ''[https://query.wikidata.org/#%23%20Scroll%20down%20and%20hit%20blue%20arrow%20down%20to%20run%20and%20see%20the%20results%20%2B%20the%20sources%0ASELECT%20%3Fyear%20%3Fitem%20%3Fshortname%20%28MAX%28%3Fnumber%29%20AS%20%3Fpassengers%29%20%20%20%28SAMPLE%28COALESCE%28%3Freference_URL%2C%20%3Fmonthly_reference_URL2%29%29%20AS%20%3Fsample_reference_URL%29%0AWITH%0A%7B%20%20SELECT%20%3Fitem%20%3Fstatement%20%3Fdate%20%3Fyear%20%3Ftimevalue%20%3Fnumberperperiod%20%3Freference_URL%0A%20%20WHERE%20%20%7B%20%20%20%20%3Fitem%20wdt%3A%20%3Fairport_code%0A%20%20%20%20VALUES%20%3Fairport_code%20%20%20%20%20%7B%20%22%22%20%20%20%20%7D%0A%20%20%20%20%3Fitem%20p%3AP3872%20%3Fstatement.%0A%20%20%20%20%3Fstatement%20pqv%3AP585%20%3Ftimevalue%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20ps%3AP3872%20%3Fnumberperperiod.%0A%20%20%20%20%3Ftimevalue%20wikibase%3AtimeValue%20%3Fdate.%0A%20%20%20%20OPTIONAL%20%7B%20%3Fstatement%20pq%3AP518%20%3Fapplies.%20%7D%0A%20%20%20%20OPTIONAL%20%7B%20%3Fstatement%20prov%3AwasDerivedFrom%20%2F%20%28pr%3AP854%7Cpr%3AP4656%29%20%3Freference_URL.%20%7D%0A%20%20%20%20FILTER%20%28BOUND%28%3Fapplies%29%3Dfalse%20%7C%7C%20%3Fapplies%20%3D%20wd%3AQ2165236%20%29%0A%20%20%20%20MINUS%20%7B%20%3Fstatement%20wikibase%3Arank%20wikibase%3ADeprecatedRank%20%7D%0A%20%20%20%20BIND%20%28YEAR%28%3Fdate%29%20AS%20%3Fyear%29%0A%20%20%20%20FILTER%20%28%3Fyear%20%3E1949%29.%20%20%20%20FILTER%20%28%3Fyear%20%3C%20YEAR%28NOW%28%29%29%29%0A%20%20%7D%20%7D%20AS%20%25airport%0AWHERE%0A%7B%20%20%20%7B%20%20%20%20%23%20Get%20the%20sum%20of%20monthly%20values%20within%20a%20year%0A%20%20%20%20SELECT%20%3Fitem%20%3Fyear%20%28SUM%28%3Fmax_numberperperiod%29%20AS%20%3Fnumber%29%20%28SAMPLE%28%3Fmonthly_reference_URL%29%20AS%20%3Fmonthly_reference_URL2%29%0A%20%20%20%20WHERE%0A%20%20%20%20%7B%20%20%20%20%20%20%23%20Get%20the%20maximal%20value%20and%20a%20sample%20reference%20URL%20for%20each%20unique%20month%0A%20%20%20%20%20%20%7B%20%20%20%20%20%20%20%20SELECT%20%3Fitem%20%3Fyear%20%28MAX%28%3Fnumberperperiod%29%20AS%20%3Fmax_numberperperiod%29%20%28SAMPLE%28%3Freference_URL%29%20AS%20%3Fmonthly_reference_URL%29%0A%20%20%20%20%20%20%20%20WHERE%20%20%20%20%20%20%20%20%7B%20%20%20%20%20%20%20%20%20%20INCLUDE%20%25airport%0A%20%20%20%20%20%20%20%20%20%20%3Ftimevalue%20wikibase%3AtimePrecision%20%3Fprec.%0A%20%20%20%20%20%20%20%20%20%20FILTER%20%28%3Fprec%20%3E%209%29%23%20precision%20more%20precise%20or%20equal%20to%20month%0A%20%20%20%20%20%20%20%20%7D%20%20%20%20%20%20%20%20GROUP%20BY%20%3Fitem%20%3Fyear%20%3Fdate%0A%20%20%20%20%20%20%7D%20%20%20%20%7D%20%20%20%20GROUP%20BY%20%3Fitem%20%3Fyear%0A%20%20%7D%20%20UNION%20%20%7B%20%20%20%20%3Ftimevalue%20wikibase%3AtimePrecision%209%20.%20%20%20%20BIND%20%28%3Fnumberperperiod%20AS%20%3Fnumber%29%20%20%20%20BIND%20%28%3Freference_URL%20AS%20%3Fsample_reference_URL%29%0A%20%20%20%20INCLUDE%20%25airport%0A%20%20%7D%0A%20%20OPTIONAL%20%7B%3Fitem%20wdt%3AP1813%20%3Fthis.%20%20%20%20%20%20%20%20%20%20%20%20%23%20has%20shortname%0A%20%20%20%20FILTER%28LANG%28%3Fthis%29%3D%22en%22%29%20%20%7D%0A%20%20SERVICE%20wikibase%3Alabel%20%7B%20bd%3AserviceParam%20wikibase%3Alanguage%20%22%5BAUTO_LANGUAGE%5D%2Cen%2Cen%22.%20%3Fitem%20rdfs%3Alabel%20%3FitemLabel.%7D%0ABIND%28COALESCE%28%3Fthis%2C%3FitemLabel%29%20as%20%3Fshortname%29%0A%7D%20GROUP%20BY%20%3Fitem%20%3Fshortname%20%3Fyear%20ORDER%20BY%20%3Fitem%20DESC%20%28%3Fyear%29 Example SPARQL query]''
 * 2. Use a template to ingest the data the query you wrote in "1." will have returned and convert that data into a format (e.g. JSON) that the Graph extension can understand
 * 3. Generate a graph using the Graph Extension by "calling" the data you generated in "2." by adding the  notation to the page that you'd like the graph to appear on PPelberg (WMF) (talk) 21:59, 28 July 2023 (UTC)
 * Yes, this is how it worked. The SPARLQ query directly returns the data in the JSON format. From this JSON, the key are the ones given in the SPARQL query (i.e. "main_val", "lower", etc.), and so we can call them in the graph code to format the data as we wish. Pamputt (talk) 04:00, 29 July 2023 (UTC)
 * Thank you for following up with this additional clarity, @Pamputt.
 * To be doubly certain, it sounds like it's two steps (listed below) rather than three steps (as I'd thought above), does that sound accurate?
 * Use SPARQL to retrieve the information you're seeking from Wikidata. ''[https://query.wikidata.org/#%23%20Scroll%20down%20and%20hit%20blue%20arrow%20down%20to%20run%20and%20see%20the%20results%20%2B%20the%20sources%0ASELECT%20%3Fyear%20%3Fitem%20%3Fshortname%20%28MAX%28%3Fnumber%29%20AS%20%3Fpassengers%29%20%20%20%28SAMPLE%28COALESCE%28%3Freference_URL%2C%20%3Fmonthly_reference_URL2%29%29%20AS%20%3Fsample_reference_URL%29%0AWITH%0A%7B%20%20SELECT%20%3Fitem%20%3Fstatement%20%3Fdate%20%3Fyear%20%3Ftimevalue%20%3Fnumberperperiod%20%3Freference_URL%0A%20%20WHERE%20%20%7B%20%20%20%20%3Fitem%20wdt%3A%20%3Fairport_code%0A%20%20%20%20VALUES%20%3Fairport_code%20%20%20%20%20%7B%20%22%22%20%20%20%20%7D%0A%20%20%20%20%3Fitem%20p%3AP3872%20%3Fstatement.%0A%20%20%20%20%3Fstatement%20pqv%3AP585%20%3Ftimevalue%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20ps%3AP3872%20%3Fnumberperperiod.%0A%20%20%20%20%3Ftimevalue%20wikibase%3AtimeValue%20%3Fdate.%0A%20%20%20%20OPTIONAL%20%7B%20%3Fstatement%20pq%3AP518%20%3Fapplies.%20%7D%0A%20%20%20%20OPTIONAL%20%7B%20%3Fstatement%20prov%3AwasDerivedFrom%20%2F%20%28pr%3AP854%7Cpr%3AP4656%29%20%3Freference_URL.%20%7D%0A%20%20%20%20FILTER%20%28BOUND%28%3Fapplies%29%3Dfalse%20%7C%7C%20%3Fapplies%20%3D%20wd%3AQ2165236%20%29%0A%20%20%20%20MINUS%20%7B%20%3Fstatement%20wikibase%3Arank%20wikibase%3ADeprecatedRank%20%7D%0A%20%20%20%20BIND%20%28YEAR%28%3Fdate%29%20AS%20%3Fyear%29%0A%20%20%20%20FILTER%20%28%3Fyear%20%3E1949%29.%20%20%20%20FILTER%20%28%3Fyear%20%3C%20YEAR%28NOW%28%29%29%29%0A%20%20%7D%20%7D%20AS%20%25airport%0AWHERE%0A%7B%20%20%20%7B%20%20%20%20%23%20Get%20the%20sum%20of%20monthly%20values%20within%20a%20year%0A%20%20%20%20SELECT%20%3Fitem%20%3Fyear%20%28SUM%28%3Fmax_numberperperiod%29%20AS%20%3Fnumber%29%20%28SAMPLE%28%3Fmonthly_reference_URL%29%20AS%20%3Fmonthly_reference_URL2%29%0A%20%20%20%20WHERE%0A%20%20%20%20%7B%20%20%20%20%20%20%23%20Get%20the%20maximal%20value%20and%20a%20sample%20reference%20URL%20for%20each%20unique%20month%0A%20%20%20%20%20%20%7B%20%20%20%20%20%20%20%20SELECT%20%3Fitem%20%3Fyear%20%28MAX%28%3Fnumberperperiod%29%20AS%20%3Fmax_numberperperiod%29%20%28SAMPLE%28%3Freference_URL%29%20AS%20%3Fmonthly_reference_URL%29%0A%20%20%20%20%20%20%20%20WHERE%20%20%20%20%20%20%20%20%7B%20%20%20%20%20%20%20%20%20%20INCLUDE%20%25airport%0A%20%20%20%20%20%20%20%20%20%20%3Ftimevalue%20wikibase%3AtimePrecision%20%3Fprec.%0A%20%20%20%20%20%20%20%20%20%20FILTER%20%28%3Fprec%20%3E%209%29%23%20precision%20more%20precise%20or%20equal%20to%20month%0A%20%20%20%20%20%20%20%20%7D%20%20%20%20%20%20%20%20GROUP%20BY%20%3Fitem%20%3Fyear%20%3Fdate%0A%20%20%20%20%20%20%7D%20%20%20%20%7D%20%20%20%20GROUP%20BY%20%3Fitem%20%3Fyear%0A%20%20%7D%20%20UNION%20%20%7B%20%20%20%20%3Ftimevalue%20wikibase%3AtimePrecision%209%20.%20%20%20%20BIND%20%28%3Fnumberperperiod%20AS%20%3Fnumber%29%20%20%20%20BIND%20%28%3Freference_URL%20AS%20%3Fsample_reference_URL%29%0A%20%20%20%20INCLUDE%20%25airport%0A%20%20%7D%0A%20%20OPTIONAL%20%7B%3Fitem%20wdt%3AP1813%20%3Fthis.%20%20%20%20%20%20%20%20%20%20%20%20%23%20has%20shortname%0A%20%20%20%20FILTER%28LANG%28%3Fthis%29%3D%22en%22%29%20%20%7D%0A%20%20SERVICE%20wikibase%3Alabel%20%7B%20bd%3AserviceParam%20wikibase%3Alanguage%20%22%5BAUTO_LANGUAGE%5D%2Cen%2Cen%22.%20%3Fitem%20rdfs%3Alabel%20%3FitemLabel.%7D%0ABIND%28COALESCE%28%3Fthis%2C%3FitemLabel%29%20as%20%3Fshortname%29%0A%7D%20GROUP%20BY%20%3Fitem%20%3Fshortname%20%3Fyear%20ORDER%20BY%20%3Fitem%20DESC%20%28%3Fyear%29 Example SPARQL query]''
 * Generate a graph using the Graph Extension by "calling" the data you generated in "1." by adding the  notation to the page that you'd like the graph to appear on
 * PPelberg (WMF) (talk) 00:14, 1 August 2023 (UTC)
 * Yes, that's it :) Pamputt (talk) 04:48, 1 August 2023 (UTC)
 * Yes, that's it :) Pamputt (talk) 04:48, 1 August 2023 (UTC)

Update: 11 August
hey y'all – this update is an effort to help us:
 * 1) Align on what we've come to know and think about the Graph Extension
 * 2) Work together to find the information needed to address the remaining questions/uncertainties

I'm thinking the above will help us arrive at the requirements needed to distinguish viable from non-viable solutions and for the WMF (by way of me) to propose some paths forward for us to consider in the next update, planned for 15 September.

With this in mind, if you see something below that you disagree with, think could benefit from additional clarity/specificity, etc., please say as much by commenting below!

...this update is meant to identify any gaps and inconsistencies in what we're collectively thinking so that we can move forward together.

Now, before getting into what's become more clear since the 14 July update and what new uncertainties/questions surfaced: thank you!

Thank you to @Ahecht, @Bawolff, @Bouzinac, @Edu!, @Matma Rex, @Msz2001, @Novem Linguae, @Nux, @Pamputt, @Pppery, @Quiddity, @RobinLeicester, @User:SBassett (WMF), @Snævar, @Tgr, @TheDJ, and @User:VPoundstone-WMF for patiently and generously sharing the information and resources I've needed to understand and orient myself within the decisions we find ourselves needing to make.

Now onto the update… PPelberg (WMF) (talk) 18:55, 11 August 2023 (UTC)


 * Help needed: what question do we need help answering?
 * How might we go about inviting non-English speakers into this conversation and making them feel welcomed to contribute?
 * How might we go about making more people aware that this conversation is happening? It's important that we increase the likelihood that people who have experiences with and information about the Graph Extension be aware of, and ideally, a part of this conversation.
 * How might we go about forming a more accurate understanding of how widely used the Graph Extension was in relation to features that offer similar functionality?
 * PPelberg (WMF) (talk) 18:56, 11 August 2023 (UTC)
 * Clarity: what do we agree to be true about the Graph extension and the needs it was meant to meet?
 * There seem to be three distinct features the Graph Extension offered for which there are no current workarounds/alternatives:
 * Generating interactive maps/charts/graphs
 * Where "interactive" in this context refers to functionality/features like: enabling individual people viewing a graph to adjust the underlying parameters that determine the data they see and how it is visualized.
 * Generating maps/charts/graphs that update automatically
 * This is most often done by pulling in data from Wikidata.
 * Updating maps/charts/graphs on-wiki
 * The Graph Extension enables people to generate graphs using data stored on-wiki. This means, people can update graphs by editing the data on wiki, similar to how they might edit any other page/piece of information on the wikis.
 * In addition to the "distinct features" the Graph Extension offered, it also enabled people to:
 * Specify how this data is represented (e.g. chart type, colors, legend, mouseover events/interactions, etc.)
 * Store data within templates on pages separate from articles
 * Embed/transclude graphs within articles using templates
 * Write Lua modules that:
 * Convert JSON data into a well formatted wiki table
 * Extract data needed for graph and outputs as JSON
 * Create graph templates that can consume data and plot it
 * Pull in data from external sources via URLs
 * From generating article page view graphs to showing air traffic volume, volunteers depended on the Graph Extension in a variety of reader- and editor-focused contexts.
 * PPelberg (WMF) (talk) 18:57, 11 August 2023 (UTC)
 * Impressions: what seems like it could be true? 
 * Absent accessible and easy-to-use features that enable people to store, collaborate on, and present data on-wiki:
 * The data we host/present on our projects is more vulnerable to being outdated/inaccurate
 * The content we offer is more likely to lack corresponding data visualizations
 * In order for the Movement as a whole, and the various communities that comprise it, to evaluate the impact of the work it's doing, they need ways of generating, updating, and visualizing data.
 * To use the Graph Extension, you need/needed a relatively high degree of technical expertise on top of a foundation of data literacy.
 * Related to "3.", the Graph Extension has remained a relatively niche feature that one could, theoretically, never interact with (directly or indirectly) throughout your tenure as an editor.
 * PPelberg (WMF) (talk) 18:57, 11 August 2023 (UTC)