Topic on Project:Support desk

Open source exam and quiz banks: An idea for a new wiki

4
Guy vandegrift (talkcontribs)

I am a college professor of physics who is concerned about the high cost of education. In the technical fields (math and science) there in more than enough information on the internet-at-large for a self-learner to acquire more than 50% of a four year college degree. Although the wikis (e.g. Wikipedia and Wikiversity) are still underdeveloped compared to the internet-at-large, we are approaching the point where this knowledge will also be available in a convenient form that is open source.

What is missing is a means by which a self-learner can easily document expertise in these subjects. Test banks are proprietary and carefully guarded by the publishers of textbooks.

Wikiversity:Quizbank represents my effort to fill this need. The idea is to post these questions publicly and use software to first store the questions on a personal computer, and then to create wikitext that can be uploaded to a wiki and printed out as pdf files. I have already done that with a Matlab code that randomly selects questions and renders them in wikitext with pagebreaks such that all the multiple choice options are on the same page as the question. I am not a computer programmer and my software is hard to use.

Here are some challenges that need to be addressed in the future, if this idea is to take off:

  1. The Matlab software that retrieves and randomly selects questions from a copy of the quizbank that resides on a personal computer creates textfiles that must be uploaded to a wiki and then rendered as wikitext that can be printed. At the moment this is a public action. We need to arrange for these exams to be written and printed out in secret. I am able to do this because I am a Wikiversity custodian who can hide edits soon after they are made. And I make my exams many months in advance, and have only a small number of students. It is highly unlikely they would monitor my efforts and attempt to capture an exam as I write it. (Also, why cheat if the questions are already available?)
  2. Teachers need to write test questions differently. One issue is the reluctance of students to ready anything. I tried to write tests based on Wikipedia articles, but found that all but the most dedicated students preferred to simply study the testbank.
    1. One remedy is to select two or three questions from each exam and ask students to explain them in a short paragrah. ** Another remedy is to include written passages as part of the question. A large number of questions would be associated with that passage. As students prepare for the exams by reading the passage, they might find it easier to understand the passage than to attempt to memorize all the answers.
    2. Another remedy is to require that students propose questions. The grading of such efforts is quite useful for any instructor who is a contributor to this bank. A related homework exercise for students is to verify that questions written for a specific Wikipedia article are still valid after the article has undergone a series of edits.
  3. Another relatively trivial challenge is making the open source exam bank compatible with the conventional "secret" banks already available. The expectation is that colleges and universities would use the open source testbank as only a portion of the grade assessment and certification. There will always be room for those "secret" questions that only the instructor knows about.

There is no great rush to solve these problems. I am still struggling to get a small version of the quizbank prepared for the four courses that I teach. Serious work needs to be performed after I have established that this is a good way to teach. But if anybody has any suggestions, please let me know. I am only vaguely aware of what sort of extensions can be written because I am not really a programmer.

88.130.105.132 (talkcontribs)

Hi Guy,

for that you write that you are not a programmer, you obviously have already made up big part of the software needed. You are writing much, but you do not have clear questions (yet) - so let me give you some thoughts:

You say that certain questions (and their answers obviously) need to be secret. Hiding a revision from view is one way to do that, and it might not even be the worst. If you set up an own wiki for that stuff, kind of your own one, then you could fully control what is going on there. E.g. you could take people's read access away from certain pages. However, MediaWiki is not made for that and you cannot regard that as "secure". There will most likely always be a way around such limitation. Making the complete wiki private would however be possible (and secure if you ask me).

If I understand correctly you or some co-workers will prepare tests. This should also continue that way - you do not want a computer algorithm to randomly pick questions for you. The Matlab software creates text files, which you are uploading to the wiki. Since I guess you know (or can control) the exact format, in which questions and answers are presented in these files, it should be rather easy to process them further. But if I understand you correctly, your software is already able to produce wikitext out of them.

Guy vandegrift (talkcontribs)

Thanks for responding. Let me give you two explanations and ask you a question:

The explanations:

1. I chose a program that randomly selects from the quizbank because that was easier to write. It works OK for the time being.

2. The "secret" question bank I refer to are those proprietary banks that professors keep on their hard drives. These questions will always play a role in grade assessment.

My question is about "secret" exams from a "public" (open source) question bank that need to be rendered by first uploading to Wikiversity, and then printing out as a pdf file. At the moment this is a "public" act that anybody can see.

I don't need to do this now, but someday a larger open source organization (such as openStax College) might want to create tests "off-wiki", meaning that it is not necessary to go through Wikiversity or Wikipedia to render the wikitext into a pdf file. The solution would be to maintain a wikitext server with access that is by invitation only. I need to know if this feasible.

I have noticed that two by-invitation-only wikis are at UC Davis with Chemwiki and the OpenStax College textbooks (affiliated with Rice University). These environments are much different than wikimedia "wikitext" syntax, using mathjax for equations, for example.

Someday (maybe in a decade or so) I would like to see it possible for professors to render exams in wikitext in a way that nobody can see it until the day of the test. The wikitext would be created on personal computers, and ultimately rendered as pdf for paper-copy printout. If such "private" rendering of wikitext is feasible, I will continue to write exam questions on Wikiversity with the understanding that the technology will eventually catch up.

On the other hand, if Wikimedia is adamantly against supporting any wiki that permits "secret" or "private" rendering of wikitext, and if a private server cannot be created that renders equivalent Wikipedia-style wikitext, then I need to think about working with the syntax used by OpenStax College or Chemwiki. I have contacted both organizations, and they are both giving me moral support. But neither organization seems to have the resources to create the software I hope will happen sometime in the future.

Let me put the question simply with a hypothetical scenario: Suppose I were not an individual, but a consortium of dozens of professors who need a way to render wikitext in a fashion that nobody can see it until the day of the test. Would Wikimedia (or somebody else) be able to provide that support?

I believe the answer is yes.

MarkAHershberger (talkcontribs)
Suppose I were not an individual, but a consortium of dozens of professors who need a way to render wikitext in a fashion that nobody can see it until the day of the test. Would Wikimedia (or somebody else) be able to provide that support?

It would be a SMOP, yes. You can set up your own copy of Wikiversity and and individual like myself or someone on the list of people on professional development and consulting would be able to help you.

Reply to "Open source exam and quiz banks: An idea for a new wiki"