User:Miranche/BiblioTex

This page helps me document the work on a bibliography-oriented MediaWiki extension.

Summary
This extension is aimed at users who use MediaWiki as a productivity tool in research, and is focused on integrating bibliography information with wikitext. Currently, it can:


 * Import bibliography information from several sources, including server-based BibTex bibliography databases, online services, and other web pages.
 * Generate numbered references integrated with the  and   tags of the Cite extension and used as a standard in Wikipedia.
 * Store bibliography information for single items in a separate MediaWiki namespace for easy import and annotation.
 * Display bibliography entries using a half dozen different citation styles.
 * Provide APIs for the user to define their own data sources and citation styles.

See below for a full description of features and the dirty details.

What's the Point?
There's no single one. This started as me playing with MediaWiki and the original two extensions to help me keep my notes on PhD readings. I went in to customize some details in the code, then one thing led to another. While there is as of yet no idea on, well, fitness for a particular purpose, I am developing with the following goals in mind:


 * 1) Configurability – Support many different styles of use.
 * 2) Extensibility – Design APIs for users to define their own citation display styles, bibliographic data sources, etc.
 * 3) Interoperability – Leverage off other bibliographic extensions the user may have installed.
 * 4) Integration – Make full use of MediaWiki's existing features, wherever I can recognize their existence.
 * 5) Incompleteness – Be a fox, not a hedgehog. Build a set of tools potential users can turn to their own purposes, and do NOT attempt to provide a neat, gleaming, unified solution.

The last one is, needless to say, my favorite.

Name
The working name of this extension, BiblioTeX, is a portmanteau of the names of two extensions, Biblio and BibTex, it is based on. I like the name, but there is a proprietary application that already uses it. Also, since I'm building up from Biblio & am fully backward compatible with it, this work can be seen as leading to a potential new Biblio version.

Main Features

 * Completed
 * Extend Biblio parser to recognize plain BibTeX syntax, and to search for and source BibTeX files on the file system.
 * Use a common BibTeX-based back end for all citation data.
 * Build a flexible citation renderer that supports user-defined citation styles.


 * In Progress
 * Integrate Biblio citations with the Cite extension, when present.
 * Emulate citation display formats used in Wikipedia and in the current versions of Biblio and BibTeX.
 * Fully support existing Biblio and Bibtex features.
 * Add a MediaWiki namespace, BibTeX, which serves to store BibTeX code of individual sources locally.
 * Support several standard scholarly citation styles.


 * Next up
 * Support page numbers and parenthetical ("Harvard") citations.
 * Support customizable decoration of citations with external and internal links.
 * Test the extension on all major browsers.
 * Fully document existing and new features.


 * Planned
 * Provide core functionality in older releases of MediaWiki back to 1.5, and full functionality in versions back to 1.11.
 * Provide and document an extensible API letting users add their own bibliographic data sources.
 * Support interoperability with other bibliographic MediaWiki extensions wherever feasible.
 * Consider interoperability with Zotero.

Completed and In Progress headings are self-explanatory. Next up comprises features I intend to complete for the first beta release, while Planned are ideas for further down the road. Details follow.

Data Sources and Formats

 * Completed
 * Extend PubMed requests to handle DOIs
 * Get rid of dependence on SOAP in favor of CGI requests
 * use SimpleXML to process Web query results rather than purpose-written XML parsers
 * Standardize representation of data retrieved by PubMed & ISBNdb.com requests, using data structures modeled on BibTeX
 * Parse author names correctly depending on data source, breaking them into first, last, "von" and "jr" parts as described in BibTeX documentation (Tame the BeaST)
 * Extend &lt;biblio&gt; tag parser to accept BibTeX entries (already largely implemented in the NMR version) and to recognize and handle plain BibTeX syntax
 * Provide a way to look for and source BibTeX databases from the file system


 * In Progress
 * Google Scholar bibtex compatibility ("hack to remove extra braces?")


 * Next up
 * Look for & eliminate possible opportunities for malicious HTML insertion through sourcing files & urls
 * Standardize the handling of special characters
 * Standardize the way source file URLs are generated (see function makeUrl)
 * Consider whether to provide support for some of the less-than-standard BibTeX entry types JabRef has @electronic, @patent, @periodical and @standard, of which @periodical is likely the most important


 * Planned
 * Get citations from other repositories, see Extension:Bibwiki for possibilities (Amazon, Arxiv etc)
 * Standardize the API for getting citations from external sources and/or files, provide a way for users to program in their own source types.
 * Output COinS: see http://ocoins.info/, Citation and Citation core templates in wp
 * If necessary, consider changing the underlying data structure from BibTeX-based to something more general.

Display of Citations

 * Completed
 * Write a general bibliography entry renderer expecting a BibTeX-like data structure
 * Unify rendering for BibTex, PubMed and ISBN &lt;biblio&gt; entries
 * Support different citation styles: Chicago author-date & humanities, APA, AMA, MLA
 * Create a citation style ("biblio-2006") that replicates the existing Biblio citation display
 * Create a citation style ("wikipedia") that renders citations using Cite templates. Need: ParserFunctions, templates listed on w:Template:Citation/core/doc


 * In Progress
 * Special characters seem to prevent BibTeX code popup
 * Complete the following standard citation types:
 * confirm the styles for Chicago, esp. for @techreport
 * MLA, APA & AMA don't have @inbook, @techreport, @manual
 * add Turabian
 * document limitations of the current way of defining citation styles


 * Next up
 * Generalize hypertext additions to textual citations into configurable "output decorator" classes
 * Handle bibtex attributes that can decorate the citation with links & popups, e.g. annot & abstract (link or popup), ps & pdf (link with icon), url (hyperlinked at the end of the citation)
 * Expand citation styles to meta-styles which include
 * Citation style (how to generate text for a single citation)
 * Footnote style (how to integrate citations into the Wiki page: numeric vs author-date labels, sort order)
 * Decorators (which bells & whistles to add to citation text before, while & after it's created: links, headers & footers)
 * Create meta-styles that fully mimic existing &lt;biblio&gt; and &lt;bibtex&gt; output


 * Planned
 * Display page numbers in in-text links
 * Provide options to display in-text links in parenthetical format and to sort bibliography by author name
 * Check out Citation Style Language

MediaWiki Integration

 * Completed
 * If Cite/Cite.php is present, provide an option to make Biblio interoperable with it:
 * Handle &lt;cite&gt; tags as named &lt;ref&gt;s
 * Allow the &lt;biblio&gt; section to appear within &lt;references&gt;, or to double as &lt;references&gt;
 * Handle &lt;nocite&gt; tags correctly: extend Cite to not complain about bib entries with no references in the text
 * In absence of Cite, make output look & function like Cite, with anchors links between &lt;biblio&gt; citation list and text
 * Add BibTeX namespace, storing individual BibTeX citations
 * When starting a new page in the BibTeX namespace, preload it with the appropriate BibTeX code and a redirect/link to a putative annotation page
 * Create a citation style ("wikititle") that defines the default title style of the annotation page
 * Provide an option to display citation titles as links to pages in the BibTeX namespace. Similar to RefHelper but without automagically creating pages.


 * In Progress
 * Investigate whether & how to unify Cite and Biblio error display
 * Handle consecutive & ranged numbered citations (in Biblio for sure, in Cite, perhaps)
 * How does Cite in Wikipedia highlight the citation the person clicks on?


 * Next up
 * Extend Cite class to create author-date in-text links for &lt;ref&gt;s that use appropriate citation templates
 * Investigate the use of "Harvard" citations in Wikipedia
 * Investigate the use of the (deprecated) ref/note citation style.
 * Provide option to automagically generate BibTeX namespace pages
 * Implement transcluding the BibTeX namespace page. Display the citation or the BibTeX code dependent on context/setting.
 * Make Biblio & Bibtex CSS consistent with those used in MediaWiki, Extension:Cite and Template:Citation
 * Where appropriate, use DOM, SimpleXml and/or MediaWiki Xml functions to generate XML content


 * Planned
 * Provide option to integrate with the Cite namespace citations generated by RefHelper, where present
 * Possibly integrate with Zotero?
 * If yes, how would integration with Zotero coordinate with BibTeX and Cite?
 * Decide what to do with &lt;bibtex&gt; tags. Here's a strategy:
 * Treat them as BibTeX-specific ways to display a citation anywhere, similar to templates
 * If full BibTeX syntax in it, the citation is displayed. If "automagical BibTeX namespace pages" is checked, the page is created.
 * If just a BibTeX ID is in it, the bib info is pulled from the given page
 * Provide templates that do the same things as the respective tags?

Compatibility

 * In Progress
 * Continually make sure original &lt;bibtex&gt; and &lt;biblio&gt; syntax work
 * &lt;biblio&gt;: sourcing URLs: wiki, interwiki & external
 * &lt;bibtex&gt;: in-text citation, help output, pasting bibtex templates into the edit window (probably broken in IE)


 * Next up
 * Test in key older MediaWiki versions, starting with 1.5 or with the oldest version used by existing Biblio users
 * Use parse instead of recursiveTagParse prior to 1.8
 * Test in other browsers, esp. IE7 and IE8


 * Planned
 * Install & test (Apache + PHP +) MediaWiki + this extension on Mac OS X and Windows

Links

 * Mediawiki


 * Extension:Biblio
 * Extension:Bibtex
 * Extension:Bibwiki
 * Extension:Cite/Cite.php
 * Extension:RefHelper
 * Extension:WikiCitation


 * Authors and Users


 * http://openwetware.org/wiki/Wikiomics:Biblio
 * http://gsc2.cemif.univ-evry.fr/index.php/Mediawiki_extensions
 * http://nmrwiki.org/wiki/index.php?title=Help:Biblio_Extension
 * http://nmrwiki.org/wiki/index.php?title=Using_BibTex_in_NMRWiki
 * http://www.hodgeslab.org/2010/02/php_mediawiki_biblio_and_nusoa.html (test case on mac)
 * http://neurov.is/on/Help:Creating_citations
 * http://neurov.is/on/Cite:Gray_Atkinson_2003
 * http://neurov.is/w/index.php?title=Special:RefSearch
 * http://neurov.is/w/index.php?title=Special:RefHelper


 * BibTex


 * BibTeX entry types
 * http://www.kfunigraz.ac.at/~binder/texhelp/bibtx-7.html
 * http://www.lsv.ens-cachan.fr/~markey/bibla.php
 * http://artis.imag.fr/~Xavier.Decoret/resources/xdkbibtex/bibtex_summary.html
 * http://www.math.utah.edu/~beebe/bibliographies.html
 * http://ftp.math.utah.edu/pub//emacs/bibtex-mods.el


 * Citation Style Guides


 * http://www2.liu.edu/cwis/cwp/library/workshop/citation.htm
 * http://library.hsu.edu/Tutorial/
 * http://www.chicagomanualofstyle.org/tools_citationguide.html
 * http://www.umuc.edu/library/guides/chicago.shtml