User:Badon

From MediaWiki.org
Jump to: navigation, search
Sean Astin 1.jpg
MediaWiki-extensions-icon.svg This user is a proud MediaWiki extension developer and participant in WikiProject Extensions.
Buggie.svg This user is an official member of the MediaWiki Bug Squad.
MediaWiki logo without tagline.png This user is an editor on MediaWiki.org. (verify)
Script error

I am an unpaid volunteer for the Wikimedia Foundation, and I do mostly software testing, bug hunting, and documentation writing. I have a particular interest in doing it for Semantic MediaWiki and related extensions, but I jump in anywhere I feel I can be helpful. I usually avoid documentation writing for SMW and related extensions, so instead I prefer to put useful bits of info I want to share on my user page here.

I have officially reported 75 bugs as of January 1, 2012 (and a few more unofficially). 56 of them are either FIXED or still open, and the rest are DUPLICATE, WONTFIX, LATER, or buried forever. That gives me a 75% "usefulness" rate[1] in my bug reports, with the remaining 25%[2] often resulting in improved documentation, especially when the bug[3] is actually an undocumented feature. I am always trying to improve my bug hunting skills and methods.

On 2012-07-03 04:56:24 UTC I submitted a patch for a critical bug that had the potential to bring down a server, in bugzilla:38136. My patch was applied and marked as fixed on 2012-10-17 16:27:59 UTC. That was my first patch, and I hope to find more opportunities to fix the bugs I find. Maybe that will help to avoid being "shot" for an unwelcome contribution of bug reports and workaround solutions.

I also sometimes do bug hunting for the Tor Project, PC-BSD, Corz checksum, MultiPar, HexChat, Notepad++, and many others.

I am the founder and project lead for the Coin Compendium (CC) and livebusinesschat.com (LBC).

TrueCrypt hidden message[edit | edit source]

There is a hidden message on the new sourceforge TrueCrypt site. The first line of the site is this:

WARNING: Using TrueCrypt is not secure as it may contain unfixed security issues

If you take just the first letter of each word, except the word "WARNING":

Using TrueCrypt is not secure as it may contain unfixed security issues

you get this:

uti nsa im cu si

It's Latin that roughly means:

Unless I want to use the NSA

So, the full message seems to be this:

WARNING: Using TrueCrypt is not secure as it may contain unfixed security issues, unless I want to use the NSA

Which is English that roughly means:

Don't use TrueCrypt because it is under the control of the NSA

I have been informed that I may be the first to publish this hidden message. I wrote up a more formal article at LBC that contains this information, which is on Reddit here: Hidden message on the new sourceforge TrueCrypt site.

Bug Hunter Hate[edit | edit source]

Why so many bugs?[edit | edit source]

Why does Mediawiki (Wikipedia) and extensions have so many bugs? It's because there are hundreds of developers working on the code base simultaneously using a continuous integration development model. Although there are always a lot of bugs, at least the software is always in a state where it works well enough that the bugs can be isolated and fixed. Ambitious software projects far smaller than MediaWiki tend to never deliver a functional product if they don't use a continuous integration development model. The problem with a continuous integration development model is that it moves forward so quickly that it's impractical to thoroughly test new code before putting it into service.

Programming is a fun thing to do, and the developers that do it not only enjoy the creative process in their software, they also receive recognition and accolades for their work. Very often, they receive monetary payment too. By not doing rigorous bug testing, it's a little less work, and a little more glory for the developers, which works out great for them in a continuous integration development model. But, somebody has to do the bug testing eventually, or the problems will mount until the developers start getting more complaints than glory. If the developers aren't doing the testing, then that work is left for bug hunters.

You're distracting ... go away![edit | edit source]

Bug hunters take on the most difficult and frustrating jobs, but they seldom get thanked for it, and they certainly don't get any recognition or accolades. Sometimes, they receive immediate hostility from the first mention that bug testing is being done, much like spammers - "You're distracting ... go away". Some developers have grown so accustomed to praise that they turn into violent primadonnas that can't respond properly to bug reports.

Partly because of my negative experiences with bug reporting in MediaWiki, I haven't done very much bug hunting for the Wikimedia Foundation since 2012, but I'd like to get back into it (2014). Getting into the "bug hunter mindset" means I've started keeping notes and reporting problems I've found with other software I like and use. One of my recent bug reports was for the PortableApps platform, specifically the updater. I was experiencing several problems, but I didn't know at first whether they were distinct bugs, or the same problem with multiple consequences, or something else, so I searched for previous bug reports. I found these:

  1. Updater hangs
  2. Updates fail and Downloads stop at 39.8Mb during attempt to update large app.

The first one described exactly the same symptoms I was experiencing that made the PortableApps Updater fail completely. The second one reminded me of other problems I experienced before the updater failed completely, and I suspected it might be related. Indeed, the person who initially reported the problem asked:

[I want] to know if I am alone or this is something others are experiencing.

I saw that no one had made any progress in helping the person, so I posted a response that I thought would be helpful (my most effective bug hunting technique). It said:

Others are experiencing similar issues, but there's not a consensus yet about what is happening, and whether they're related problems or not.

I was immediately chastised for my good deed, before anything more could be discussed:

Millions of people use the updater daily without issue. I realize it is frustrating having an issue on your end, but that's no reason to go around posting in unrelated bugs like this one.

And things just got worse from there...

I solved the problem[edit | edit source]

As it turned out, 5 out of 8 of the computers we tested experienced some other bug (discussion #1 above) with the PortableApps Updater that I was not able to isolate before they spontaneously resolved themselves. The one that didn't resolve had received an automated Microsoft Internet Explorer (IE) update that was unsupported, and some unknown interaction from that prevented the PortableApps Updater from succeeding somehow. All 8 computers had no internet connectivity problems, and all 8 had working installations of IE, which was trivial to test.

I also was able to determine that the problems on all 8 computers that were tested, in both of the 2 discussions I participated in, actually ARE related! They are all caused by a single design flaw in the updater.[4] That flaw uniquely prevented ONLY the updater from functioning, while all other software had no internet connectivity problems. The design flaw was bad enough that even for very different problems the updater wouldn't time-out or give an error message in about half of the cases I tested. After 2 days of bug hunting with no sleep, I had succeeded in isolating a very difficult bug, and then I posted a solution of how to fix it, which goes beyond mere bug hunting - I solved the problem - but none of those contributions were welcomed. Actually, they were spurned, like pearls before swine.

The silly-fied excerpts below are originally from Reddit, with an added link to connect the 2 parallel discussions:

John T. Haller:

Badon ... blah blah blah ... troubleshooting ... blah blah blah ... 32 hours ... blah blah blah ... IE didn't function ... blah blah blah ... complaining ... blah blah blah ... politely asked ... blah blah blah ... proved my point ... blah blah blah ... right?

badon:

blah blah blah ... I solved the problem ... blah blah blah ... Did you sleep well?

Note:

I'll admit, after 30+ hours working on these bugs without any sleep, I was a little excited about isolating the most difficult bug case AND producing a fix for it (bonus points!), when it wouldn't spontaneously resolve on its own. That was the bug triggered by a seemingly unrelated problem with an automated Windows Update of MS IE. It wasn't the only problem in the 2 discussions I participated in, but it was part of a class of several problems with the PortableApps Updater that resulted in poor reliability.

The updater was overly complicated, but it wasn't able to do basic things like time-out and give an error message if an update fails. Total reliance on Microsoft's Internet Explorer is unnecessary, and a bad design decision that resulted in many other problems that have nothing to do with IE. It's not often that I can both isolate a bug, and produce a fix for it, so forgive me for relishing that small success - After all, there is no other reward for bug hunters. Still, it's not a good reason for developers to hate unpaid volunteer bug testers like they're irritating bugs themselves.

...and your little bug too![edit | edit source]

John T. Haller finished up by giving me a publicly visible "Final Warning" that makes him look like a fair and well-meaning person. In reality, he had already non-publicly banned me so that I would be unable to respond - and you can bet I would have responded because I tend to indignantly resist injustice (especially when it hurts the innocent people I was trying to help by bug-hunting for them). He abused that advantage to dishonestly manipulate the public's perception of me, which goes beyond mere ambiguous rejection, to violent hate.

John T. Haller accused me of spamming "ads" in my forum signature, after he had already deleted my forum signature so that no one can see for themselves what was there (unlinked public service mentions of MultiPar, FreeArc, and TypeMatrix keyboards, which you can see in my signatures on most of the forums I participate in). He insulted me by accusing me of trolling, and claiming I did not following his troubleshooting instructions (which I did, and more). Once again, this isn't my first time doing bug testing, and such jabs clearly stem from irresponsible unrestrained hostility.

John T. Haller seems to hate bug hunters. Not "dislike" them, not "prefers groupies instead" - we're looking at immediate, unprovoked, raw hatred directed at somebody who would have been hailed a hero if I were the developer responding to those people's questions, instead of just a lowly unpaid volunteer bug tester. That hate manifested almost immediately when it became clear that a thorough investigation was being done.

There's no place like home[edit | edit source]

Fortunately, the Wikimedia Foundation is not a one-man show like the PortableApps platform is. I have experienced similar Bug Hunter Hate (BHH) when reporting bugs to MediaWiki developers, but there are enough other people involved that some of them might eventually step in to defend the lowly unpaid volunteer bug hunter if he's getting more than his fair share of BHH abuse. BHH is everywhere, so much so that I think we just have to accept it as an inherent fact that goes with the territory. From there we can do our part in reducing it as much as possible.

Other victims[edit | edit source]

Here are some doozies:

Some other things I care about[edit | edit source]

Mediawiki etc.[edit | edit source]

Support[edit | edit source]

Localization[edit | edit source]

Image annotation[edit | edit source]

Data integrity[edit | edit source]

Miscellaneous[edit | edit source]

Extension:ReplaceText known issues[edit | edit source]

Yaron Koren has been described as the "rock star" of MediaWiki developers due to the enormous quantity of code he writes, but he very often dislikes the contributions of others in his projects, especially when they address security issues or inadequate documentation. So, this improved documentation about the bug testing work several of us did for security issues in Extension:ReplaceText (version 0.9.5, November 2012) has to be made available here on my user page, instead of Yaron's extension page. One of us already submitted a modified SpecialReplaceText.php to fix the most serious of these issues, so this documentation could become outdated when Yaron releases a future version.

If you need something fixed immediately, please support Yaron's generous gifts to the MediaWiki community by hiring him and/or his team at his company WikiWorks. Your support will benefit all of us who use Yaron's extensions. Note that this should not be construed as an endorsement, because I have not used WikiWorks myself yet. For more information about supporting MediaWiki developers, see User:Badon#Giving gifts and User:Badon#Giving gifts to Project?, and a list of other commercial support options.

UPDATE 2013 January 22:

I tested and produced a patch that partially works for getting around these problems with Replace Text 0.9.6. I also uploaded my slightly edited version of Woozle's SpecialReplaceText.php for people who just want to replace the file and get back up and running until Replace Text can be more properly fixed.

Large replacements taking a long time[edit | edit source]

The replacement actions themselves are structured as MediaWiki "jobs", to ensure that the system is not overloaded if the user wants to do many at the same time. This means that a large set of replacements will not be done immediately, and may take minutes, hours or even longer to complete. Jobs get activated every time a page is viewed on the wiki; to speed up the process (or slow it down), you change the number of jobs run when a page is viewed; the default is 1. For information on how to change it, see the $wgJobRunRate page.

Large replacements error message[edit | edit source]

If you have a large number of replacements, some of them may not get done; or you may see an error message that reads "You must select at least one namespace". See: bugzilla:38170, bugzilla:43472. In both cases, that's due to security features of PHP and the Suhosin extension that limit the number of form inputs to reduce the effectiveness of some types of DoS attacks. This can be worked around by temporarily compromising those security features. Restore them immediately when you're done with using Replace Text. You do not need to wait for the jobs to complete. Here are the workarounds:

  • The PHP setting max_input_vars (available since PHP 5.3.9) typically defaults to limit the number of replacements to 1000. You can try setting this parameter to a number larger than the number of replacements you want to do, like 3000, for example.
  • If you have the Suhosin PHP extension installed on your server, you can try temporarily disabling it while you do the replacements.
  • If you have Suhosin installed but don't want to disable it, you can simply increase the value of the variables suhosin.post.max_vars and suhosin.request.max_vars.

Suppressing redirects when changing page names[edit | edit source]

In order for redirects not to be created on page moves, i.e. if the "Save the old titles as redirects to the new titles" checkbox is unchecked, you will have to give the relevant user the 'suppressredirect' permission. Assuming you've given the 'replacetext' permission only to sysops, you would need to add the following to LocalSettings.php:

$wgGroupPermissions['sysop']['suppressredirect'] = true;

Compressed revisions are not searchable[edit | edit source]

If your revisions are compressed (that is, if $wgCompressRevisions is enabled in LocalSettings.php) then ReplaceText will not work, because it makes use of SQL queries that can't search compressed text.

Widgets[edit | edit source]

Extension:Widgets is a very powerful extension that's simple to use, and thoroughly replaces the obsolete Extension:RawMsg. Here's a few widgets I made that you can use on your own wiki with the Widgets extension installed. It is very useful when you want to show someone how to do something on your wiki.

UPDATE 2013 Febuary 03:

It appears Extension:WidgetsFramework is well on its way to resolving most or all of the problems with Extension:Widgets. I particularly like the benevolent community-centered approach used by the friendly developers of that extension. I have never experienced any kind of snide arrogance or abuse from them, so I'm eager to give it a try. With their leadership, I'm hoping to be able to contribute to their project without having my work alienated to my user page anymore. Hallelujah!

Google translate widget[edit | edit source]

<noinclude>__NOTOC__
{{#ifeq: {{FULLPAGENAME}} | Widget:Google translate
        |
        | <span class="error" style="font-size:3em;">This code is on the wrong page! It's supposed to be on [[Widget:Google translate]]!</span><br /><br />
}}<nowiki />
This widget enables [http://translate.google.com/translate_tools Google translate tools]. You may use the widget like this:

<pre>
{{#widget: Google translate }}
</pre>

[[Category:Widgets]]
</noinclude><includeonly><div id="google_translate_element"></div><script>
function googleTranslateElementInit() {
  new google.translate.TranslateElement({
    pageLanguage: 'en',
    multilanguagePage: true,
    layout: google.translate.TranslateElement.InlineLayout.SIMPLE
  }, 'google_translate_element');
}
</script><script src="//translate.google.com/translate_a/element.js?cb=googleTranslateElementInit"></script></includeonly>

It's not required, but if you add the following to your MediaWiki:Common.css, your widget will display nicely on the right side of wiki pages, on the same line as the page H1 title header thing:

/* For the Widget:Google_translate in MediaWiki:Sitenotice */
#siteNotice { height:0; }
#localNotice { height:0; margin-bottom:0; }
#siteNotice p { margin:0; padding:0; } /* Clean easy-to-read wiki formatting while producing superfluous <p></p> tags WITHOUT extra spacing in the sitenotice */

/* For the Widget:Google_translate in MediaWiki:Sitenotice */
#google_translate_element { text-align:right; position:relative; top:-0.67em; }

Twiddla widget[edit | edit source]

<noinclude>__NOTOC__
This widget allows you to add a [http://www.twiddla.com/ Twiddla] meeting to a page. <span class="plainlinks">[{{fullurl:{{FULLPAGENAME}}|action=edit}} Edit the page]</span> to see the widget text.
You may use the widget like this:

<pre>
{{#widget: Twiddla }}
</pre>

[[Category:Widgets]]
</noinclude><includeonly>
<a href="http://www.twiddla.com/New.aspx?title=" title="Twiddle this page!" onclick="this.href+=document.title;"><img src="http://img.twiddla.com/images/buttons/twiddle_this_100x20.gif" border="0" alt="Twiddle this page!" /></a></includeonly>

HTML5 video and subtitles widget[edit | edit source]

Widgets are usually pretty easy to make, but this one was a challenge. If you use this widget, I would be delighted to hear about it. It features not only video, but also user-entered multi-lingual subtitling. It's annotation for video! A big "Thank You!" to everyone who made this widget possible. You know who you are.

UPDATE: This widget does not work as well as I had hoped. I have discovered several flaws in the subtitling that sometimes cause other things to not work. I recommend ditching this widget in favor of the YouTube widget for now.

<noinclude>__NOTOC__
{{#ifeq: {{FULLPAGENAME}} | Widget:Html5media
        |
        | <span class="error" style="font-size:3em;">This code is on the wrong page! It's supposed to be on [[Widget:Html5media]]!</span><br /><br />
}}<nowiki />
This widget enables HTML5 video and audio using the [http://html5media.info/ html5media] JavaScript component. This widget also optionally enables [http://www.universalsubtitles.org/en/ Universal Subtitles] using the [http://www.universalsubtitles.org/en/get-code/ US Amara Widgetizer]. You may use the widget like this:

<pre>
{{#widget:Html5media
| url=http://www.somewebsite.com/somevideo.webm
| width=640
| height=360
| poster=http://www.somewebsite.com/someimage.png
| controls=yes
| preload=none
| subtitlesforall=yes
}}
</pre>

* '''url''' is the URL of the video, and is the only required parameter.
* '''width''' and '''height''' are optional video size dimensions. '''width=640''' and '''height=360''' is the default.
* '''poster''' is an optional image URL to display while the video is loading. A video frame is ideal.
* '''controls''' is an optional parameter that will show controls (play, pause, volume, etc) immediately upon load, before playback begins, when the value is '''yes''', which is the the default value if not specified. Use the value '''no''' not show controls.
* '''preload''' is an optional parameter that will preload the video immediately upon load, before playback begins, when the value is '''auto'''. If not specified, the value '''none''' will be used to not preload. The value '''metadata''' will only preload metadata. Note that not preloading the video will cause the '''subtitlesforall''' parameter to not take effect until the user begins playback.
* '''subtitlesforall''' will enable subtitles for all videos on a page when the value is '''yes'''. Note that not preloading the video will cause the '''subtitlesforall''' parameter to not take effect until the user begins playback.
=== Recommended widths and heights ===
* For 16:9 wide screen format videos
** width=400<br /> height=215
** width=420<br /> height=236
** width=640<br /> height=360
** width=800<br /> height=450
** width=1024<br /> height=576
* For 4:3 monitor format videos
** width=400<br /> height=300
** width=420<br /> height=315
** width=640<br /> height=480
** width=800<br /> height=600
** width=1024<br /> height=768
=== Recommended software ===
* Screen recording
** http://camstudio.org/
** http://www.hyperionics.com/hc/downloads.asp
* Video format encoding
** http://www.mirovideoconverter.com/ - To convert to WebM format.
=== How to setup incompatible browsers to view WebM video ===
[http://en.wikipedia.org/wiki/HTML5_video#Browser_support Browser support for HTML5 WebM video] is now widespread. Only a few browsers cannot display HTML5 WebM video. The most notable browsers are Microsoft Internet Explorer (IE or MSIE, including mobile versions) and Apple Mac Safari (including iPad, iPhone, etc, versions). However, most of those web browsers can be made to work by [http://www.webmproject.org/users/ installing a plugin to enable HTML5 WebM video].
=== How to setup your HTTP server for the correct WebM MIME type ===
* http://www.webcoder.de/2010/06/04/how-to-configure-your-webserver-for-webm-video-and-audio-files/
=== Uploading ===
Upload your videos at the [[Special:Upload]] page.
=== Subtitling best practices ===
* http://blog.universalsubtitles.org/2011/08/09/best-practices-for-subtitling-videos/
=== Future ideas ===
* https://www.google.com/search?q=subtitles+html5+video - There are other subtitlings options that should be explored and further developed, that do not rely on third parties like Amara Universal Subtitles. They could allow the wiki to host the subtitles after they're created using US Amara tools (or other tools), and they could then be translated on-wiki using [http://www.mediawiki.org/wiki/Extension:Translate Extension:Translate].
[[Category:Widgets]]
</noinclude><includeonly><!--{if $subtitlesforall == 'yes'}--><script type="text/javascript" src="http://s3.www.universalsubtitles.org/js/unisubs-widgetizer.js"><!--{/if}--></script><script src="http://api.html5media.info/1.1.4/html5media.min.js"></script><video style="background-color:#000000;" src="<!--{$url|validate:url}-->" width="<!--{$width|default:400|validate:int|escape:'html'}-->" height="<!--{$height|default:300|validate:int|escape:'html'}-->" poster="<!--{$poster|validate:url}-->" <!--{if $controls == 'no'}--><!--{else}-->controls<!--{/if}--> preload="<!--{$preload|default:none|escape:'html'}-->" ></video></includeonly>

YouTube widget[edit | edit source]

<noinclude>__NOTOC__
{{#ifeq: {{FULLPAGENAME}} | Widget:YouTube
        |
        | <span class="error" style="font-size:3em;">This code is on the wrong page! It's supposed to be on [[Widget:YouTube]]!</span><br /><br />
}}<nowiki />
This widget allows you to add the YouTube video player to a page. <span class="plainlinks">[{{fullurl:{{FULLPAGENAME}}|action=edit}} Edit the page]</span> to see the widget text.
You may use the widget like this:

<pre>
{{#widget:YouTube
| id=DWef69ItVrU
| width=640
| height=360
| showmore=yes
| subtitlesforall=yes
}}
</pre>

* '''id''' is the YouTube video ID, taken from the last part of the YouTube URL, like this (colored <span style="color:#FF0000;">red</span>): http://www.youtube.com/watch?v=<span style="color:#FF0000;">DWef69ItVrU</span>
* '''width''' and '''height''' are optional video size dimensions. '''width=420''' and '''height=315''' is the default.
* '''showmore''' will suggest more videos if the value is '''yes'''. If not specified, the default is '''no'''.
* '''subtitlesforall''' will enable subtitles for all compatible videos on a page when the value is '''yes'''. YouTube videos will not be affected by other videos if the value is not given or is '''no''' (the default).
* https://developers.google.com/youtube/player_parameters
[[Category:Widgets]]
</noinclude><includeonly><!--{if $subtitlesforall == 'yes'}--><script type="text/javascript" src="http://s3.www.universalsubtitles.org/js/unisubs-widgetizer.js"></script><object width="<!--{$width|escape:'html'|default:'420'}-->" height="<!--{$height|escape:'html'|default:315}-->"><param name="movie" value="http://www.youtube.com/v/<!--{$id|escape:'urlpathinfo'}-->?version=3&hl=en_US<!--{if $showmore == 'yes'}-->&rel=0<!--{/if}-->"></param><param name="allowFullScreen" value="true"></param><param name="allowscriptaccess" value="always"></param><embed src="http://www.youtube.com/v/<!--{$id|escape:'urlpathinfo'}-->?version=3&hl=en_US<!--{if $showmore == 'yes'}-->&rel=0<!--{/if}-->" type="application/x-shockwave-flash" width="<!--{$width|escape:'html'|default:'420'}-->" height="<!--{$height|escape:'html'|default:315}-->" allowscriptaccess="always" allowfullscreen="true"></embed></object><!--{else}--><iframe width="<!--{$width|escape:'html'|default:'420'}-->" height="<!--{$height|escape:'html'|default:315}-->" src="http://www.youtube.com/embed/<!--{$id|escape:'urlpathinfo'}--><!--{if $showmore == 'yes'}-->?rel=0<!--{/if}-->" frameborder="0" allowfullscreen></iframe><!--{/if}--></includeonly>

Bug hunting tips[edit | edit source]

Buggie.png

This section may evolve into a full guide for debugging. For now, he's a few tips:

  • Check to make sure you haven't accidentally pasted the wrong text into a page. Semantic Forms pages and template pages getting swapped can produce errors that take down the wiki (until the code prevents those error conditions).
  • Check your clipboard manager's hotkey settings (if you have one) to make sure that you're not accidentally triggering hotkey functions that can be screwing up your pastes.
  • Use code inspectors like FireFox FireBug, Opera Dragonfly, or Chrome Inspector, to look at what HTML, CSS, JS, etc is being generated on a wiki page that isn't behaving like you think it should.
  • Use some code to automatically show an error message if you paste code into the wrong page by accident, like this (with Parser functions):
{{#ifeq: {{FULLPAGENAME}} | the page the code is supposed to be on
 |
 | <span class="error" style="font-size:3em;">This code is on the wrong page!<br />
   It's supposed to be on [[the page the code is supposed to be on]]!</span><br /><br />
}}<nowiki />
  • Strip down the problem code or circumstance until the bug goes away. That will be your bug demo.
  • If you're struggling with isolating the bug, try setting up a fresh MediaWiki install with the bare minimum of extensions and other "accessories" so you can reproduce the bug with minimum confusion.
    • Try running your test MediaWiki on a separate host from your usual host, so you can eliminate the possibility of a hosting quirk as being the source of your bug. I recommend No Support Linux Hosting for a really cheap hosting solution that makes it easy to set up as many as you want, as long as you're skilled enough to not need support for setting up a website.
    • Make sure common data corruption is not the problem by doing your production hosting on a ZFS system. I think nearly all mysterious failures that can't be traced to a bug are caused by normal data corruption. Data corruption is something that I hope is one day not considered normal anymore. Here are some hosting suggestions:
      1. No Support Linux Hosting virtualizes their Linux systems on top of a ZFS storage backbone, so you can use ZFS with Linux even though Linux doesn't support ZFS yet.
      2. Another recommendation for ZFS hosting is Entic.net, with very good support and Cartika, which I haven't used yet, but I would like to. All of my recommended hosts will not cause "bugs" to suddenly appear due to data corruption.
      3. You can also set up your own ZFS hosting with FreeBSD, which has software support that's almost as good as Linux, and ZFS support that's almost as good as Solaris.
      4. If you can't get ZFS, consider at least using MultiPar to protect your files. Read on Wikipedia about MultiPar's underlying Parchive technology.
  • Finally, if you're sure you've found a bug, report it.

Advanced MediaWiki programming toolkit[edit | edit source]

Other extensions[edit | edit source]

Advanced MediaWiki programming tips[edit | edit source]

  • Whitespace is the enemy. If you can't find it when it's screwing something up, it's because they're wascally whitespace wabbits.
  • Whitespace wabbits like this always bite me at first, but I hunt them down later:
[[Some invisible SMW property::Some value| ]] <!-- This is supposed to be invisible, but the space in front of this comment will cause a <pre/> tag to be inserted by the MediaWiki parser. -->
  • If you can't get rid of the whitespace that's screwing up your wiki pages (especially in tables), put this in your MediaWiki:Common.css to make most of the annoying whitespace "disappear":
.wikitable p { margin:0; padding:0; }
  • Since MediaWiki and web browsers do not show all whitespace characters, finding extra whitespace can be done with the urlencode magic word, like this:
{{urlencode: This line is perfect except for 2 spaces before the last  word }}
To get this, where you can see two %20 next to each other:
This%20line%20is%20perfect%20except%20for%202%20spaces%20before%20the%20last%20%20word }}
  • You can also persuade the web browser to preserve whitespace using CSS, so you can see where the problems are. Use the CSS white space properties to reveal them.
  • Use HTML lists and tables instead of wiki lists and tables. It makes formatting much easier, and less fragile.
  • Use <nowiki /> at the end of lines to prevent the buildup of empty wiki paragraphs (1 <p></p>paragraph for each 2 newlines), like this:
{{#if: something
| do this<nowiki />
| or do this<nowiki />
}}
{{#if: something
 | do this<nowiki />
 |<!-- Don't do anything. --><nowiki />
}}
  • Do not use <nowiki /> inside #vardefine if you are going to test it later with an #if, because <nowiki /> in your variables will cause them to always pass the test. Like this:
<!-- This is wrong. -->
{{#vardefine: MyVariable |
 {{#if: something
  | do this<nowiki />
  |<!-- Don't do anything. --><nowiki /><!-- This is a problem. -->
 }}<nowiki /><!-- This is a problem. -->
}}<nowiki />
{{#if: {{#var: MyVariable }} <!-- MyVariable will always pass the test because it always has 2 <nowiki /> in it. -->
 | It will always do this
 | It will never do this
}}<nowiki />

<!-- This is right. -->
{{#vardefine: MyVariable |
 {{#if: something
  | do this
  |<!-- Don't do anything. Notice that there is no space in front of this comment, which could also screw this up. -->
 }}
}}<nowiki />
{{#if: {{#var: MyVariable }}
 | It will sometimes do this
 | Or, it will sometimes do this
}}<nowiki />
  • <nowiki /> is not needed inside functions that do not directly or indirectly output anything, like in string manipulation code.
  • It is better to use many #vardefine's within nested #if tests than to place your #if statements inside a #vardefine. It makes debugging mysterious whitespace much easier. Like this:
<!-- This is wrong. -->
{{#vardefine: MyVariable |
 {{#if: something
  | do this<nowiki />
  |<!-- Don't do anything. --><nowiki />
 }}
}}
<!-- This is right. -->
{{#if: something
 | {{#vardefine: MyVariable | do this }}<nowiki />
 | {{#vardefine: MyVariable |}}<!-- Don't do anything. --><nowiki />
}}

Distributed template job queue management[edit | edit source]

When a high-usage template is edited, it can dominate the MediaWiki job queue for days or weeks, sometimes more. The simple solution to this problem is to create several identical versions of the same template, each with a different number. Then, when edits need to be made, the template changes can be rolled out gradually to all the identical templates. While the job queue is processing each numbered template, other jobs can get into the queue before the next template is edited. This also improves testing of template changes, because errors do not affect the entire site, and multiple changes to the same template do not further clog the job queue.

About 30 numbered, identical templates can be used for MediaWiki sites up to about the 100,000 page level. You can use Extension:ReplaceText to find and replace the "main" template with a randomly numbered identical template, like this:

  • Find: {{Some template}}
  • Replace: {{Some template {{subst:#rand: 1 | 30 }} }}

See also: Bug 37570: Enable parser functions and magic words in Semantic Forms template definition to allow distributed template job queue management.

I have some fancy distributed template management code that I might be willing to share if you ask me about it. I don't want to release it under an open source license yet, until I'm sure it will get used by someone other than direct competitors to my own sites. In other words, it shouldn't be hard to persuade me to release it, if you ask.

The ultimate solution to every difficult problem[edit | edit source]

If you're having no success in solving a difficult problem, you can keep pathetically hacking away at it, or you can put down your psychological torture devices and help someone else solve their problems. Every time I get stuck on something because:

  • I don't have enough skills.
  • I don't have enough experience.
  • I don't have enough knowledge.
  • I've spent too much time coding and my brain is fried.
  • All of the above.

I always find the solution after spending time helping other people solve their problems. Why? Well, for starters, even if I know exactly how to solve the problem, I usually still need to freshen my knowledge by re-reading documentation and doing some easy searches. After a few minutes to a few days of answering questions on the mailing lists, forums, talk pages, and IRC, most of the time I have read and written enough documentation (the answer to someone's question) to stumble upon a fresh idea for solving my own issues.

Whenever I can't sleep at night because some problem is challenging me, I find peace and inspiration by helping others. That works for me, and maybe it will work for you too.

Giving gifts[edit | edit source]

This is one of my favorite photos.

If you are an awesome MediaWiki helper, please put your Amazon wishlist on your user page. Note that you will not be able to stay anonymous if you do that. If you want to remain anonymous, "printable" eBay gift certificate codes can be emailed to you. If you're a recipient of help from an awesome MediWiki helper, here's where you go to get an eBay gift certificate code to show your appreciation:

https://www.paypal.com/cgi-bin/webscr?cmd=_ebay-gift-certificate

Be sure to select the option:

  • I will print out the gift certificate and deliver it myself.

And then select the amount you want to give. Don't enter in anything else.

Buying stuff for people on their Amazon wish-list, and sending "printable" eBay gift certificate codes, are both anonymous for the sender too! I recommend that you stay anonymous when sending people gifts for several reasons:

  1. Everyone loves a mystery.
  2. It does not create any expectations of future gift-giving.
  3. It encourages everyone to be polite and helpful to each other, since there's no way to know who's sending the gifts and who isn't.

That last one is the most important. Even if only 2 or 3 people are sending gifts, as long as they do it anonymously, it makes the MediaWiki community much more friendly for every one of the thousands of people involved.

See also: WMF Benefactors.

Notes and references[edit | edit source]

  1. "If you want to help, write decent bugs report like other people seem to manage."
  2. Sometimes valid bugs are marked as invalid, apparently for the purpose of humiliating bug hunters. No one would insist that a developer not mark a bug as fixed, but it's OK to do that to bug hunters, as in bugzilla:33479.
  3. bugzilla:32817#c7, bugzilla:30004#c6, bugzilla:29891#c3
  4. The flaw exists solely "to show download progress in NSIS". That feature isn't even used in the bug case where the updater hangs, in discussion #1. John T. Haller said a rewrite of the PortableApps updater is planned, so he is probably aware that it is unreliable. I don't know how that translates into immediate maltreatment behavior toward well-meaning bug hunters.