Wikitext/2011-01 format discussion

This is an attempt to structure the many discussions from many threads on wikitech-l:
 * What would be a perfect wiki syntax? (Re: WYSIWYG)
 * WYSIWTF working demo
 * Big problem to solve: good WYSIWYG	on WMF wikis

Feel free to edit this in-place as if it were in the main namespace, or move this out of my user space if you'd like to collaborate on it. I wasn't sure where to put it, so I put it here for now.

1. Goals:  what are we trying to achieve?
 * Tool interoperability
 * Alternative parsers
 * GUIs
 * Real-time editing (ala Etherpad)
 * Ease of editing raw text
 * Ease of structuring the data
 * Template language with fewer squirrelly brackets
 * Performance
 * Security
 * What else?

2. Abstract format:  regardless of syntax, what are we trying to express?
 * Currently, we don't have an abstract format; markup just maps to a subset of HTML (so perhaps the HTML DOM is our abstract format)
 * What subset of HTML do we use?
 * What subset of HTML do we need?
 * What parts of HTML do we *not* want to allow in any form?
 * What parts of HTML do we only want to allow in limited form (e.g. only safely generated from some abstract format)
 * Is the HTML DOM sufficiently abstract, or do we want/need some intermediate conceptual format?
 * Is browser support for XML sufficiently useful to try to rely on that?
 * Will it be helpful to expose the abstract format in any way

3. Syntax:  what syntax should we store (and expose to users)?
 * Should we store some serialization of the abstract format instead of markup?
 * Is hand editing of markup a viable long term strategy?
 * How important is having something expressible with BNF?
 * Is XML viable as an editing format? JSON?  YAML?

4. Tools (e.g. WYSIWYG) ....list goes on...
 * Do our tool options get better if we fix up the abstract format and syntax?
 * Tools:
 * Wikia WYSIWYG editor
 * Magnus Manske's new thing
 * Line-by-line editing

5. Infrastructure: how would one support mucking around with the data?
 * Support for per-wiki data formats?
 * Support for per-page data formats?
 * Support for per-revision data formats?
 * Evolve existing syntax with no infrastructure changes?