Extension talk:Math/Roadmap

Some comments from mathematics content contributors
User:Jdforrester (WMF)‎ seemed to think it signficant that no content contributors had commented on this page. Accordingly, I'm posting a personal summary of a discussion at en:Wikipedia_talk:WikiProject_Mathematics which addresses issues that might be considered relevant. There are links to several other discussions on related topics at that page for the interested reader. Deltahedron (talk) 17:24, 21 June 2014 (UTC)


 * OK, well done for copy-pasting, but where's the consensus between wikis? Jdforrester (WMF) (talk) 16:03, 23 June 2014 (UTC)
 * Since you ask, I invited participants at French and German language equivalents and you will see that there were contributions from participants at both. I made, and make, no claims that this is a cross-wiki consensus: it is the summary of a discussion.  If it is important to obtain that, then WMF community advocate staff could help you.   Deltahedron (talk) 17:30, 23 June 2014 (UTC)

Background
About 1% of Wikipedia's 4.5 million articles are assessed as being in "Mathematics and Logic". Probably a similar number are in theoretical physics and in computer science. So in about a hundred thousand articles, the ability to render mathematics is indispensible to the reader: the ability to write and edit mathematics is indispensible to the author and editor.

Currently the predominant mathematics markup system in all forms of document preparation is some flavour of LaTeX. It may be presumed that any serious mathematics content contributor will be thoroughly familiar with LaTeX. LaTeX is rendered on web pages in a variety of ways: currently Wikipedia uses two of the more popular methods, rendering formulae as PNG images and rendering dynamically using MathJax. There are deficiencies in the current implementation of each of these methods.

The stability and usefulness of current mathematics rendering is reduced by the following
 * Incremental development of reader and editor interfaces is apt to degrade the reader or editor experience without warning.
 * Major changes in editor interfaces, such as the introduction of VisualEditor and Flow, may be radically incompatible with existing LaTeX markup practices.
 * Effort to support mathematics editing and rendering comes entirely from the volunteer community. Currently one volunteer is working on mathematics rendering, and support for mathematics editing in VE consisted of one GSoC summer volunteer.

WMF planning
We are reliably informed that WMF has no plans for development of mathematics rendering and editing. That is, there is no plan to coordinate volunteer effort; no plan to integrate volunteer effort into existing products; no plan to ensure the sustainability of mathematics rendering and editing through major changes to the software and user interface.

As a consequence of the lack of plans, there is no allocation of WMF developer effort to the maintenance, sustainability or enhancement of mathematics rendering and editing. It is assumed that volunteer developers will undertake any tasks that are necessary, even though there is no plan to coordinate those efforts.

It is reasonable to say that there is considerable expertise and experience in mathematics rendering and editing in the existing editor communities. There is no explicit mechanism to capture that experience and make use of it in planning, development or review. Such efforts as have been made to do so are limited in extent and driven by the user community rather than WMF. The role of Community Advocates in linking the editor community and WMF planning and developers in this context has not been effective.

Suggestions

 * General
 * WMF planning address the issue of development of mathematics and other complex rendering markup and editing components.
 * WMF liaise actively and effectively with existing editor and reader communities in (1).
 * WMF draw up roadmap for development of complex rendering and editing.
 * WMF liaise actively and effectively with volunteer developer communities to determine required frameworks and work packages.
 * WMF allocate funds and resources to support work packages.


 * Specific
 * Mathematics rendering to be based on MathJax as principal vehicle, with efficiency and resources issues resolved on a wide variety of platforms.
 * LaTeX markup retained as principal mode of editing mathematics text with concomitant option to directly edit at the wikitext markup level.
 * WMF establish a workflow for further development and deployment of the math extension, using the https://www.mediawiki.org/wiki/Extension:Math/Roadmap page to coordinate the development process.
 * WMF designate a fixed contact person at WMF that cares about math related questions and a brief to maintain regular and frequent contact with volunteer community.


 * Short-term
 * Fix MathSource mode is currently disabled: see  which resolves this issue.
 * Fix issues with experimental mathoid (MathML + SVG) support on the Beta Cluster.

discussion about texvccheck and texvc
Log of a private IRC discussion between cscott and physikerwelt on 25th July 2014 (01:17:16 PM) physikerwelt: Hi, sorry but I have doubts to change texvc. Maybe you can convince someone else to merge your changes. (01:17:16 PM) cscott  : I'm not here right now (04:39:04 PM) physikerwelt: I told the full story on github (04:39:29 PM) physikerwelt: so if you find time to further work on that it might be helpful (04:40:37 PM) cscott: did you mean 'gerrit' rather than 'github'? i'm confused. (04:41:38 PM) cscott: i think that gwicke does indeed want to use texvcjs for mathoid. it should already be 'good enough', i've tested it against texvccheck pretty extensively. (04:41:49 PM) physikerwelt: https://github.com/cscott/texvcjs/issues/1 (04:42:25 PM) physikerwelt: did you find this page? (04:42:27 PM) physikerwelt: http://www.formulasearchengine.com/Verify%20texvc%20light (04:47:46 PM) cscott: i think first I would like to import the test cases from MathTest (04:52:46 PM) cscott: wFormula-nocount.txt is very interesting, though! (04:53:24 PM) physikerwelt: Maybe I heard to many talks about automated theorem proofing, but I think it's strange to modify texvc and whatever other syntax at the same time, if the goal is to reproduce the current behavior (04:54:02 PM) physikerwelt: yes but that's definitly the second step. I think starting the the testpage makes more sense (04:54:09 PM) cscott: i don't think the goal is to reproduce the current behavior. i think the goal is to document and test *correct* behavior. (04:55:00 PM) cscott: verifying that the algorithm is idempotent is an important part to ensuring that texvc isn't destroying/corrupting the input. (04:55:27 PM) cscott: there are a number of bugs in texvc, i have no intention of duplicating those. (04:56:25 PM) physikerwelt: that's the reason why I stopped with the php port (04:57:34 PM) physikerwelt: if texvccheck is idempotent we can pass the output to texvc (04:57:55 PM) cscott: right, but we need to patch texvc in order to do that. ;) (04:58:00 PM) physikerwelt: the fact that it's currently not idempotent prevents us from ding that (04:58:02 PM) cscott: since texvc (for example) won't accept (04:58:09 PM) cscott: \mbox{\coppa} (04:58:13 PM) physikerwelt: I know that (04:59:09 PM) physikerwelt: but once we did the input check with texvccheck we don't need most parts of texvc (04:59:12 PM) cscott: i understand that changing the output of texvc, even in trivial ways, causes issues for production. but i don't think the solution is to never change texvc's output. (04:59:38 PM) physikerwelt: I think the solution is to get rid of texvc\ (04:59:50 PM) cscott: well, that's quite possible. (05:00:27 PM) physikerwelt: even today there is no need for texvc (05:00:48 PM) physikerwelt: mathoid could also generate png images for IE<9 (05:01:53 PM) physikerwelt: and texvcjs can be integrated to mathoid easily (05:04:00 PM) physikerwelt: to speed up the process we could merge the changes to texvccheck verify that against texvcjs and change texvc only once in the end if we really want to (05:05:10 PM) physikerwelt: changes in texvccheck have almost no influence on production. Only if texvccheck rejects input that was accepted by texvc this will cause a change (05:05:50 PM) physikerwelt: but this won't produce significant server load since texvc would not be called (05:06:46 PM) physikerwelt: for example the input $$\color$$ that might exist somewhere would be a syntax error afterwards (05:07:39 PM) cscott: i'd be surprised if i don't find a few such issues in en-wiki-formulae (05:08:01 PM) physikerwelt: You'll find a few. But only a few (05:14:43 PM) physikerwelt: I'd like to merge the changes for texvccheck i.e. https://gerrit.wikimedia.org/r/#/c/149097/5/texvccheck/texutil.ml https://gerrit.wikimedia.org/r/#/c/148677/8/texvccheck/lexer.mll https://gerrit.wikimedia.org/r/#/c/148677/8/texvccheck/texutil.ml https://gerrit.wikimedia.org/r/#/c/148677/8/tests/MathInputCheckTexvcTest.php (05:15:04 PM) physikerwelt: those changes are save to merge without ops interaction (05:20:33 PM) physikerwelt: are you open to change them separate or would you prefere to keep that in sink with texvc (05:21:04 PM) cscott: well, it's certainly harder to keep track of which changes i've only applied to one or the other if they are separate patches (05:21:42 PM) cscott: but i can probably manage; there aren't that many changes after all. (05:21:55 PM) physikerwelt: I can keep track of them;-) (05:22:08 PM) cscott: it would help if we could start a separate branch for the 'fixed texvc', so that we could commit corresponding patches to texvc on that branch at the same time (05:23:28 PM) physikerwelt: I've been spending a significant amount of time on reorganizing changes for the math extension in the last year so that is just a very minor issue (05:23:44 PM) physikerwelt: yes I like the idea of a new branch (05:24:26 PM) physikerwelt: however, I had the feeling that nobody wants to review changes to non master branches (05:24:50 PM) cscott: so long as you're willing to review the changes, we won't have a problem ;) (05:25:08 PM) physikerwelt: yes (06:07:50 PM) physikerwelt: can we publish this irc log? (06:08:09 PM) cscott: sure