Extension talk:OAIRepository

Wikimedia OAIRepository configuration
From CommonSettings.php:

(Just guessing: probably to be included into LocalSettings.php rather than CommonSettings.php???) --Vigilius 20:41, 16 July 2008 (UTC)

Installation Instructions
An installation instruction is very much needed. The readme speaks only of adding one table, but several sql scripts are present. I could not get this to work. Please help. --Vigilius 22:23, 17 July 2008 (UTC)

This sounds like a great extension - it would be great if somebody was willing to put down a bit more information about how to use it. Bjoern 14:42, 19 June 2009 (UTC)


 * Please refer to the installation instructions on the extension's page itself. They may not work for everyone, but they have a lot of notes about all of the "gotcha"s that I ran into when doing an (eventually) successful installation of this extension.
 * Hope it helps!
 * -SColombo 15:17, 19 June 2009 (UTC)


 * Thanks. I would really like to know which scripts go on the server, and which go onto the client. I guess the instructions describe wiki-lucene configuration, rather than a wiki-wikimirror configuration.
 * The README mentions "A harvester script forms the client half. Apply oaiharvest_table.sql to clients to allow saving a checkpointing record; this ensures consistent update ordering.". Would be good to have some details there on what else needs doing on the client. Bjoern 17:35, 19 June 2009 (UTC)

IIS
Proposing the following:

Enabling OAI Authentication with IIS:

To get OAI auth. working with IIS, you need to do the following:
 * 1) Add the $oaiAuth = true; line to your LocalSettings.php file (as documented above)
 * 2) Create the Audit/User database (use oaiuser_table.sql, included in the download) and set up access to the OAI audit/security database -- see main page
 * 3) Add one (or more) OAI users to the oaiuser table (see main page)
 * 4) Most important step: Open IIS Manager, select your wiki site (or directory) and right-click, select Properties. Under the Directory Security tab, click the Edit button and uncheck everything but anonymous access -- you will probably need to un-check the 'Integrated Windows Authentication' box.

Tested with IIS7, MW 1.13.3, OAIRepository r39772.

Status 500 when running IncrementalUpdater
At the time of this writing, the OAIRepo_body.php code does not use. There is now a bugzilla bug for this. If you need to fix it and the bug isn't resolved yet, just go into the code and find all instances where one of the new table names is called and make sure to prepend the $wgDBprefix value to it (you will have to put "GLOBAL $wgDBprefix" at the top of the same function).

Redirecting too many times
This is actually an open-ended question for anyone who could figure it out. When attempting to run the IncrementalUpdater, I get the following error: [...] 8626 [main] INFO org.wikimedia.lsearch.oai.IncrementalUpdater  - Authenticating ... 8732 [main] INFO  org.wikimedia.lsearch.oai.IncrementalUpdater  - Authenticating ... 8809 [main] INFO  org.wikimedia.lsearch.oai.IncrementalUpdater  - Authenticating ... 8942 [main] INFO  org.wikimedia.lsearch.oai.IncrementalUpdater  - Authenticating ... 9020 [main] INFO  org.wikimedia.lsearch.oai.IncrementalUpdater  - Authenticating ... 9088 [main] INFO  org.wikimedia.lsearch.oai.IncrementalUpdater  - Authenticating ... 9152 [main] INFO  org.wikimedia.lsearch.oai.IncrementalUpdater  - Authenticating ... 9287 [main] INFO  org.wikimedia.lsearch.oai.IncrementalUpdater  - Authenticating ... 9429 [main] INFO  org.wikimedia.lsearch.oai.IncrementalUpdater  - Authenticating ... 9600 [main] INFO  org.wikimedia.lsearch.oai.IncrementalUpdater  - Authenticating ... 9601 [main] WARN  org.wikimedia.lsearch.oai.OAIHarvester  - I/O exception listing records: Server redirected too many  times (20) java.lang.NullPointerException at org.wikimedia.lsearch.oai.IncrementalUpdater.main(IncrementalUpdater.java:181) 9601 [main] WARN org.wikimedia.lsearch.oai.IncrementalUpdater  - Retry later: error while processing update for lyricwiki : null 9601 [main] INFO org.wikimedia.lsearch.oai.IncrementalUpdater  - Sleeping for 60000 ms [...] Going directly to the url which the updater accesses (of the form ) and successfully shows a page though, not infinite redirects. Any ideas? -SColombo 00:03, 18 January 2009 (UTC)


 * "Server redirected too many times" is associated with authentication failures, and Special:OAIRepository is password-protected, so the updater is probably repeatedly failing to authenticate (which looks to it like a redirect loop). —Emufarmers(T 11:16, 18 January 2009 (UTC)


 * Thanks Emufarmers! That was exactly the problem. There was one more section of lsearch.conf that needed some authentication vars.  I've fixed the docs again (and fixed some parts that I had wrong).  It's still not "working" yet, but that particular problem is subdued.  Almost there!
 * -SColombo 19:57, 18 January 2009 (UTC)


 * Got everything working now (I think)! I updated the docs as I went along and made notes about any problems that I ran into that I think might be common, so I think the installation instructions should be fairly complete/accurate now at least as a good starting point. *sleeps*
 * -SColombo 21:26, 18 January 2009 (UTC)

classnotfound exception
[root@testwiki lucene-search-2.1]# ls -l *.jar -rwxrwxrwx 1 root root 6575971 Mar 23 14:12 LuceneSearch.jar [root@testwiki lucene-search-2.1]# java -Xmx256m -cp LuceneSearch.jar org.wikimedia.lsearch.ranks.RankBuilder wikidb.xml wikidb Exception in thread "main" java.lang.NoClassDefFoundError: org/wikimedia/lsearch/ranks/RankBuilder Caused by: java.lang.ClassNotFoundException: org.wikimedia.lsearch.ranks.RankBuilder at java.net.URLClassLoader$1.run(URLClassLoader.java:217) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:205) at java.lang.ClassLoader.loadClass(ClassLoader.java:323) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) at java.lang.ClassLoader.loadClass(ClassLoader.java:268) at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:336) Could not find the main class: org.wikimedia.lsearch.ranks.RankBuilder. Program will exit. [root@testwiki lucene-search-2.1]# I am getting the problem above, what is wrong?