Extension talk:Quiz

Suggestions de David Crochet sur le café de la wikiversité
L'évolution du système pourrait permettre de pouvoir donner une réponse chiffrée. Ce qui permettrait de répondre à des question de type : " quel est le nombre de planètre du système solaire ? " sans que l'on ai besoin d'indiquer tous les choix possible imposés de réponses.
 * Autre évolution possible : lorsque l'on demande les résultats, indiquer seulement si la réponse à la question est (vrai|fausse|non répondu) (avec le trait vertical à gauche de la question) ce qui permet de se corriger et premettre de re-répondre à la question ; puisqu'avec la méthode actuelle, on voit directement la bonne réponse en demandant la correction.

Crochet.david 28 mars 2007.

Older comments about documentation
I was looking at some earlier (?) versions of the extension-writing documentation on other wikis, and it seemed to be longer. Is this the authoritative version here? McCormack 05:15, 6 April 2007 (UTC)
 * This is all known "help on syntax" written so far :

Lrbabe 08:09, 7 April 2007 (UTC)
 * a bad one in french on my own wiki
 * a good one from Michael Frey on the wikiversity
 * an old one in french and english (you can change the language at the bottom of the page) with outdated syntax.


 * Thanks :) McCormack 08:53, 7 April 2007 (UTC)

Documentation up to 0.8x
The English-language help page at v:Help:Quiz has been substantially extended - now at 20 kB. Some bits are still missing. McCormack 04:58, 21 April 2007 (UTC)

version 0.7
I see that the quiz extension is working well on DE.wikiversity.org, but it does not appear to work on www.mediawiki.org (which makes documentation difficult).

What is the status of implementation? How likely is it that the quiz extension will be accepted and implemented on EN.wikiversity.org? When will it be on www.mediawiki.org ?

McCormack 06:03, 6 April 2007 (UTC)


 * It already works on en.wikiversity.org, see v:Special:Version. But it will never be installed on mediawiki. We have to take screenshots of examples on the wikiversity and insert them in our help. Lrbabe 08:09, 7 April 2007 (UTC)


 * Ah. Is there any reason why it won't be installed on MediaWiki? McCormack 08:31, 7 April 2007 (UTC)


 * Because we don't need it? robchurch | talk 08:19, 11 April 2007 (UTC)

version 0.8x
What are the procedures and timescales for rolling out new versions on the various Wikiversities? Who does one ask and how does it happen? Perhaps we could also collect information in this section about when roll-outs occur? McCormack 05:42, 19 April 2007 (UTC)


 * Wikiversity have 0.8.2 :-)
 * http://de.wikiversity.org/wiki/Spezial:Version
 * http://en.wikiversity.org/wiki/Special:Version
 * I think, they are disbrubuted over the normall update procedere.
 * And the servers of the wikimedia foundation allways run on very new Mediawiki versions from svn.
 * -- MichaelFrey 15:50, 19 April 2007 (UTC)


 * Yes, thanks. But I'm curious through - how quickly do these updates generally go through? Is it always just a couple of days? McCormack 18:31, 19 April 2007 (UTC)
 * I'm answering my own question on the basis of an answer I received over at the English Wikiversity. Implementation of extension updates is anything between minutes and weeks, and asking a developer helps speed things up (so I was told). McCormack 05:01, 21 April 2007 (UTC)

Round 1

 * Possibility to exclude the options and colour key at the top - e.g. using &lt;quiz display=simple&gt;. This would enable people to insert miniature one-question quizzes into their articles without too much space being taken away by the options table.
 * Simple text Typed response (yes - I see you're already planning that one). Possible syntax:

What is the capital of France? {Paris}

(Answer marked by curly brackets and replaced with a textfield when first displayed to the learner.)
 * Cloze/gapfill, conceived as a multiplication of simple text typed responses within the same question. Possible syntax:

The capital of France is {Paris} and its people speak {French}. Its most famous emperor was {Napoleon}.


 * An important feature for simple typed response and gapfills is to support alternative answers. I suggest separating them with a pipe.

What language do people speak in Canada? {English|French}

McCormack 06:13, 6 April 2007 (UTC)


 * Similar to the simple Text input, i'd like to have a number input.
 * I mean with tolerance, min/max (all betwen is correct) or something like that.
 * A nice to have is then, support of different number systems like binary and hexadecimal.
 * -- MichaelFrey 13:47, 6 April 2007 (UTC)


 * Hi Michael. I'm curious about your numerical idea. I wonder whether this plays to the strengths of a wiki? Programmers love mathematical things, but wikis are very text-orientated things. On the other hand, why not? But if it makes things too complicated... McCormack 08:29, 7 April 2007 (UTC)


 * You both have very good ideas, I'll work on it as soon as possible. Lrbabe 08:09, 7 April 2007 (UTC)


 * I'm interested in the "display=simple" option for miniature questions. This would make a huge difference for new learners. I'm also interested in the "numeric" question type where a possible syntax could be:

Using the graph above, what is speed of the car at t=2.0 s? {Answer|Tol}


 * Kruhly 04:43, 11 April 2007 (UTC)


 * Hi, Kruhly :) Not quite sure I understand your answer syntax. What's "Tol"? And how does this differ from the "typed response" type suggested? For example, numerical responses could theoretically use formatted fields, spinners, sliders, comboboxes... Or do you want a text field with no preformatting? Are you thinking of something like Moodle? McCormack 04:59, 11 April 2007 (UTC)

I've changed "simple text response" to "typed response" in my previous posts, as I think this is a better name for the thing. "Text" suggested the answer could not be numerical. "Typed response" would, to my mind, be both numerical and text. Or do we need something more for numerical responses? McCormack 05:20, 11 April 2007 (UTC)

Suggested revised syntax:

 {How many spheres in the pyramid?} + There are {32|thirty-two} spheres in the pyramid. || The answer is 32.

 {Name ONE official language of Canada.} + {English|French} || English and French are the two official languages of Canada.

Comments: in the answer section, text outside the curly brackets is displayed first time round, but the {...} section is replaced with a textfield.

Modified for gapfill/cloze (here showing two text fields):

 {Complete the following sentence.} + The most widely spoken language in Canada is {English}, but in the province of Quebec the majority speak {French}. || Feedback: French is also an official language of the whole of Canada.

McCormack 05:32, 11 April 2007 (UTC)

Tol is short for Tolerance -- sorry old habits die hard. I have not looked at the javascript or PHP so I may be off track here... my thoughts are that a numeric question type would be slightly different than a typed question type. I'm thinking that there would initially be a blank text field ( no correct answers or distractors) I'm assuming a unformatted text field returns a string and that  {Answer|Tolerance} are both strings when processed. Therefore, there would need to be a function like compareWithInTolerance(Guess,Answer,Tolerance) where Guess is the value returned via the text field. All the strings would need to be converted to floating points and my guess is that it would be done inside the aforementioned function. So if an "N" question type does not already exist, a syntax could look like:

 {Using the graph above, what is speed of the car at t=2.0 s?} + {4|2} m/s || {Correct, the answer is 4.|Incorrect, hint given here... and try again}

where Tolerance=2 would be a percent value not an absolute value, and implementing the hint would involve some more coding. The aforementioned syntax assumes that the units are provided by the educator. Another question type would be with a numeric answer and a drop down menu for unit choices. I would have to think about a clear syntax though -- this is a wish list right ;-)

Kruhly 06:47, 11 April 2007 (UTC)


 * Ah! I understand now. What do you think, Lrbabe? (Lrbabe is programming it). Tolerance is what Moodle has in its numerical responses as well. McCormack 07:24, 11 April 2007 (UTC)
 * A different point: just picking over the telegraphic brilliance of Kruhly's post. He inserted an idea for a syntax which copes with the need of typed & numerical responses for correctness-conditional feedback. In the feedback section, he put the whole feedback inside {...} and used a pipe to separate the two types of feedback. I was thinking about this before, but I didn't come up with such a neat way of doing it. McCormack 07:24, 11 April 2007 (UTC)

Ok, after having read each advises, my ideas are as follows: { A text field question Some text parsed as wikitext where you could even insert links. Then we have a { text field }. And another line of { text } with { two } text fields. You can specify the width (in numbe of characters) of the text field like thi{ s _1 }. You can allow more than { one|two|three } good answer. Lrbabe 10:42, 12 April 2007 (UTC)
 * The object type could be name "Text field" (or Textfield ?).
 * The symbol could be "{}" : type="{}".
 * the syntax could be :
 * type="{}" }
 * Here wa can add some feedback.
 * The whole object is parsed line by line, as if it was in a tag.
 * The text field syntax require to put the text between two spaces : { this will be parsed as a text field } {this will not be parsed as a text field }.
 * For numerical answers :
 * I prefer allowing tolerance with a min-max syntax, example : The human DNA has { 20000-100000 } genes.
 * With ECMAscript we could allow only numerical input in such text fields : try this "exemple"

Here is a preview of textfield : on my own wiki. Correction doesn't work today, maybe tomorrow ; ) Lrbabe 16:30, 12 April 2007 (UTC)

Round 2
The new version is exciting. Some comments....
 * You've condensed the three types we discussed into one (typed response, gapfill, numerical). Very efficient. Neat solution.
 * I like your max-min solution for numerical responses. Easier for non-mathematical people.
 * Why are you using spaces as well as curly brackets to mark textfields? I assume there's a good reason, but you didn't say.
 * I am using spaces as well as curly brackets for textfields to allow using curlybrackets without creating systematically a textfield. Lrbabe 00:29, 14 April 2007 (UTC)


 * Why are you using underscore as the divider for textfield width? Perhaps one should not introduce too many reserved characters - ? How about { answer1|answer2||width } (i.e. use a double-pipe).
 * I am using underscore as the divider for textfield width because it is the symbol of a place where one could write a character. I think it's not a better idea to use double pipe since it's the symbol of feedback. And it is not realy a reserved characters since it is only matched if it's a the very end of the textfield ({ a_3b } will not be matched for example).Lrbabe 00:29, 14 April 2007 (UTC)

McCormack 18:58, 12 April 2007 (UTC)
 * Looked at your site and saw the display="simple" in there! Good stuff.
 * So how soon can we see the new version on Wikiversity ? :))

Hi Lrbabe and McCormack,

The new version is very exciting! I have some comments as well. I will start with ones that I think will help me clarify my understanding of the syntax:


 * In a question with more than one text field, can you have feedback for each text field? If so, must the || delimiter of the feedback occur before the subsequent text field? Or can they all occur at the end. Example of feedback occuring at the end:

Je m'appelle { Rob }. My nickname is { Kruhly } || Incorrect, Kruhly is my nickname. || Incorrect, mon prénom est Robert.
 * It will not be possible to have more than one feedback. If it is essential to separate feedback, then just split the text... Lrbabe 00:29, 14 April 2007 (UTC)


 * Can there be more than one feedback statement per textfield? For example: Correct feedback ( for numeric question -> when the answer is in the desired range, and Incorrect feedback (for numeric question -> when the answer is out of the desired range)? If not, and only incorrect feedback is given, I don't see how the user would get feedback on the "theoretical answer."
 * I'm not sure that I understand you, but : each "theoretical answer" is displayed. if the textfield was { red|green|white } no matter how the user answered, he must be informed of each correct answer. I'll discuss later about how it should be done. Lrbabe 00:29, 14 April 2007 (UTC)


 * Does each answer in the following situation ( { answer1|answer2|answer3 }) get its own feedback? (ie || Correct || Almost, you may have forgotten to square your result || Almost, you may have taken the square root instead of the square. ) If so this is fantastic! From my experience with numeric questions on online quizzes, there tends to be a couple simple mistakes that occur in the majority of incorrect answers. Multiple answers (some that are more correct than others) that have targeted feedback would be a huge step towards "instantly" correcting the simple mistakes that the majority of learners make.
 * Sorry again : It seems better for me, to write each advises in the same feedback, so that every user could see it : || If you found X, then you'll get two pieces of robot, if you found Y, then you may have taken the square root instead of the square, ... Lrbabe 00:29, 14 April 2007 (UTC)

I also looked at your site and noticed the display="simple" immediately. Merci beaucoup!

With regards to the min-max syntax: It is nice to have the flexibility to define an asymmetric range around the theoretical answer, however, I think it a less common approach than tolerances and not as easy to create a data table of "user guesses" versus "theoretical answers" because the theoretical answer is not specifically stated. I also think it would be more time consuming for an educator to calculate the min and max manually, than to specify a single number for the tolerance. For those not comfortable with tolerances, a default tolerance could be sufficient to start with and this would reduce the syntax back to the theoretical answer within { }. --

Kruhly 05:56, 13 April 2007 (UTC)


 * Great to read your opinions on this, Rob. I'd like to suggest a little caution at this stage, though. Complexity in quizzing can be a bottomless bucket, and addictive as well. There are always excellent reasons for the next layer of complexity. But I think one needs to also keep one's eyes on the benefits of simplicity. This is wiki-markup, after all. I think the great Jimmie Wales no less is often quoted for his vision of small, careful steps. McCormack 06:49, 13 April 2007 (UTC)
 * I agree at 100% with that Lrbabe 00:29, 14 April 2007 (UTC)
 * As regards feedback, I'd support your idea, Rob, about having different feedback for right/wrong solutions. But perhaps the feedback element should not get any more complicated than this? (At least, not yet).
 * If I understood Lrbabe's current syntax correctly, because the response works like a poem, a new feedback item can be entered on each new line, with intervening text and textfields. If a gapfill exercise restricts itself to one textfield per line, then the next line can be item-specific feedback. So I think that (to a certain extent) your wish for multiple feedback items is there.
 * I agree that the convention in quizzing has been for tolerance settings rather than min-max, but I also think this is a mistaken trend and is more complex. Suppose a humanities-educator wants to allow the answers 7, 8 and 9. It's quicker to write {7-9} than get out a calculator and work out that it's {8|13}. Or again, suppose the answer is the year 1789, but you want to allow a year less or more. Which is easier: {1788-1790} or {1789|0.056} ? McCormack 06:49, 13 April 2007 (UTC)
 * I can allow both syntax :) : {1789-1790} as well as {1024 2%}. Lrbabe 00:29, 14 April 2007 (UTC)
 * Wonderful! You read my mind ... -- Kruhly 02:19, 14 April 2007 (UTC)

Now I'm concerned about feedback :
 * I don't understand the utility of this syntax : || { Great! | Not so great :( } . The feedback is only here to add useful informations, it is the color that inform the user if his answer is correct or not. And I see no reasons to hide potentially useful informations to a user that answers right... Furthermore, it adds complexity to the syntax. Lrbabe 00:29, 14 April 2007 (UTC)
 * I think I'm having a hard time  with the feedback structure. I guess most of my questions stem from the fact that I'm thinking the feedback is returned based on the answer to the question, while in the examples I have seen it is returned based on the correction button being pressed. Then the as you mention above the color indicates correctness.
 * At WikiEducator, there are many authors and learners from the developing world that don't have the latest technology. This leads me to think of some potential problems with color as indicator for correctness: 1) Red/Green color blindness 2) black and white monitors (cell phones, pdas, etc) 3) black and white printers, 4) text only browsers(?).
 * To limit potential problems, the feedback syntax of {correct feedback|incorrect feedback} gives an additional clear indicator -- if the correct feedback can be given when the question is answered correctly, and the incorrect feedback when there is an incorrect answer. -- Kruhly 02:19, 14 April 2007 (UTC)


 * The big question is : how to display answers we were expecting in a textfield ? It seems quite hard to add it in a standard feedback... Therefore I see two alternatives :
 * with css, like I have tried today on my own wiki, but it looks cheap if we want it to be compatible with ie and ff... (and it's worst with opera). And I see no very user friendly way to inform the user that there is such a popup behind the textfield :( ...
 * The other solution is to transform the textfield into a form : example. I think I prefer this solution. Lrbabe 00:29, 14 April 2007 (UTC)
 * I'm not clear on the second solution but I think it is better than the first. Can you toggle the text field between editable and non-editable? -- Kruhly 03:42, 14 April 2007 (UTC)

Round 2a: feedback discussion
Hi, everyone. I've tried adding a bit of structure to our debate and putting some user names back next to their contributions, as this was getting slightly confusing. Hope that's OK with you all. McCormack 05:36, 14 April 2007 (UTC)

On feedback: McCormack 05:36, 14 April 2007 (UTC)
 * Unfortunately Rob and I aren't familiar with the options available in the PHP code itself. (I'll have a go at reading the PHP, but it looks very advanced to me!)
 * We both agree that it would be desirable to have correctness-conditional feedback, especially in the typed response, gapfill and numerical types (type="{}"). However Lrbabe has raised questions of complexity, syntax and possibly coding.
 * "Correctness-conditional" means: show feedback A if correct, show feedback B if not correct.
 * Reason 1: it's normal to have this in quizzes.
 * Reason 2: it provides a text alternative for people with colour-blind screens and/or eyes.
 * Reason 3: it allows learners to be sent to places they may need to study - for example, if a question is wrong, you can show them a hyperlink to page X.
 * However, the bottom line is that it isn't ultimately necessary to have this. We could use the documentation to explain to people that they have to word their feedback to cope with it not being correctness-conditional.

Possible coding problems:
 * If I understand how Lrbabe's code works, there is no logical or coded relationship between an interactive line (with a radio button or text field) and a feedback line that might follow it. A line beginning || is simply a line which does not appear before correction and appears after correction.
 * As a result of the above, correctness-conditional feedback would not be possible for gapfill or multiple-choice questions (i.e. questions with multiple interactive elements). However it might still be possible for a typed-response question with only one textfield.
 * Lrbabe, is the following possible and reasonably easy?

||+ Show this feedback if the whole question is right (colour green). ||- Show this feedback if the whole question is wrong (colour red). ||~ Show this feedback if the whole question is something else.

McCormack 05:36, 14 April 2007 (UTC)

Round 2b: other points
When "display=simple" is used, the colour key doesn't appear when the correction button is pressed. McCormack 05:40, 14 April 2007 (UTC)


 * I thought this is what you were expecting... If we use it only for examples. Lrbabe 23:37, 16 April 2007 (UTC)


 * The idea is not just for examples, but also for miniature quizzes with reduced complexity (but nevertheless proper correction so that the learner knows if they were correct or not). However I also understand Kruhly's point about colour blindness. If my suggestion about ||+ and ||- is compatible with your coding, then the quiz author could insert something like the following as an alternative to colour coding:

||+ ||-


 * Then the colour key could be kept out of the simple display. McCormack 04:29, 17 April 2007 (UTC)


 * Hi Lrbabe, I was thinking that the "display=simple" mode would be for miniature quizzes with proper correction also. -- Kruhly 05:33, 17 April 2007 (UTC)

Round 3
Ok, I think I'll not work on improving the feedback before the release of the 1.0 version. My priorities are to have a working extension with all basic features one can expect from a decent quiz tool, and to make it stable and cross browser (I mean compatible with ie6-7, ff and opera). If you think conditionnal feedbacks are not that important, I'll keep it in mind for 2.0, because it is a lot of work (you can't imagine how many hours I spent on creating a cross browser css for the textfields). I've uploaded the 0.8.1 on the SVN, if you could test it, especially with IE7 and report bugs ... Regards, Lrbabe 23:50, 16 April 2007 (UTC)


 * Many thanks for your hard work. Looking forward to trying it out. McCormack 04:24, 17 April 2007 (UTC)
 * Yes your hard work is greatly appreciated. -- Kruhly 05:34, 17 April 2007 (UTC)


 * Just starting trying it (with IE7).
 * The colour correction codes on the text fields are excellent.
 * I like the check on the text field size.
 * McCormack 04:33, 17 April 2007 (UTC)


 * What does V mean when you correct it? McCormack 04:37, 17 April 2007 (UTC)


 * I tried your previous example with the backwards words using FF 1.7 but I'm not sure if this is the latest. I agree the color correction on the text fields are great. I did encounter a couple things that were not as I expected.
 * When I purposely answered one of the backwards words incorrectly or left it blank, the result after the correction button has been pressed is an "Editable space" for lack of a better phrase (ie The "editable space" does not have the border around it like the original text field). This was confusing at first because I did not think there was a opportunity to "Try Again."
 * FF tries to fill in the text field with previous guesses (since I tried it a few days ago).
 * Any thoughts on a "Try Again/Reset Button" that clears the question?
 * I really like McCormack's idea about the correct/incorrect icons -- Maybe a small green check mark and a small red x that have the same "footprint"?
 * Cheers -- Kruhly 12:35, 17 April 2007 (UTC)


 * The icons idea is a purely user-defined thing. The idea would be to shift the responsibility for accessibility (i.e. visual deficits such as colour-blindness) from the programmer to the quiz author. Children might get sharks for errors and cherries for correct answers - but that would be up to the quiz author. McCormack 16:56, 17 April 2007 (UTC)

Round 4
Only minore changes in 0.8.2

I need to explain how the correction of a textfield is displayed : I spent a lot of hours thinking on the most intuitive and "compatible with most browsers" way to display it... And yet, it is displayed in a different way on each browser... What do you think about it?
 * On firefox, you have to click on the "&#9660;", and it is displayed until you click elsewhere.
 * On IE6, a bug that I didn't understand ( because it was working with a blank page ) forced me to use javascript. Therefore you need to update quiz.js. The correction is displayed if you click somewhere in the textfield until your cursor goes out of it.
 * On IE7... I don't know, but I hope that it works like on firefox !
 * On opera, it looks bad ! The css's active pseudo-class is interpreted in a strange way and the outline property seems to be buged... I will rely on the statistics of w3schools, where opera accounts for 1.6% of the Net surfers.

Other changes :
 * If you don't answer to a single textfield but make no mistake in this very question, it is marked as "Not answered" (in 0.8.1 you only need to answer right once to have to whole question right).
 * My idea of an alternative to colors in order to indicate the "state" of a corrected question, is to indicate it in the obect's title. When you hover it, the state is displayed as well as the points. Does that seem sufficient for the moment? (although that can be worked over again later)


 * I've spent very little time with CSS and javascript but it has been enough to show me how painful it can be to make something work with multiple browsers. You might find some .css and .js in the "Cloze iDevice" of eXe (an open source XHTML editor)  that might be helpful to your development.  The web export of the "Cloze iDevice" has similar color coding of the correct/incorrect answers but toggles the background color of the "text" input element. They also use autocomplete="off" in their "text" input element which eliminates the autocomplete feature of firefox (I have not tried it on other browsers).
 * I noticed that clicking on a textfield in the "backwards words" example (after it has been attempted) pops-up the answer while hovering on the "&#9660;" pops-up the result (marks).
 * I tried it in Opera and agree that it is very messy. -- Kruhly 05:11, 21 April 2007 (UTC)

Version 2.0 Wishlist
Improving the feedback is important for many of the use cases on WikiEducator so I thought I would start a wishlist for version 2.0. Kruhly 12:35, 17 April 2007 (UTC)


 * 1) "Correctness-conditional feedback": show feedback A if correct, show feedback B if not correct for each question.


 * Not possible to blank out the question itself (between { and |type=). Using a non-breaking space works, but otherwise you get a syntax error.
 * When the test is corrected, the correction button should change to "reset". On clicking the rest button, the quiz should return to its initial state so that the user can retry it.
 * The colour key still doesn't seem to appear when the the simple display is chosen and the correction button is pressed.
 * The purple element of the colour key should probably display only if there is a syntax error.
 * What's the status of case-sensitivity with textfield responses?
 * McCormack 10:12, 20 April 2007 (UTC)
 * Some of your ideas are good and simple enough to be included in 1.0. take a look at : my own wiki Lrbabe 00:45, 21 April 2007 (UTC)
 * Thanks - I had a look. Definitely prettier :) McCormack 04:54, 21 April 2007 (UTC)

Another suggestion: it should be possible to suppress question numbering. Rather than adding an attribute to the quiz tag, I suggest that if there is only 1 question, it should not be numbered. McCormack 05:32, 21 April 2007 (UTC)

Terminology
I created a page on terminology for making quizzes, as I noticed that we are using critical words differently! Comment requested. McCormack 08:47, 7 April 2007 (UTC)

Didactic considerations
I've added a section at the top of http://en.wikiversity.org/wiki/Help:Quiz on didactic considerations for the use of the quiz extension. Comment is invited. One of the things which emerges from my analysis is that development of the quiz extension should pay less attention to points systems (marking, assessment) and more attention to layout and feedback (visuals, information flow). Any thoughts, anyone? McCormack 06:36, 8 April 2007 (UTC)


 * I agree that more attention to layout and feedback is appropriate. Kruhly 04:24, 11 April 2007 (UTC)