Talk:Offline content generator

Previous page history was archived for backup purposes at Talk:Offline content generator/LQT Archive 1 on 2015-06-25.

Moved discussion to other thread.

It looks like this is the way to go. These instructions however fail as cheerful as these on a debian 8. I am a bit lost? Example when trying to run ./mw-ocg-service.js -c localsettings.js :

/.../w/extensions/Collection/OfflineContentGenerator/lib/cli.js:33
		config = require(configFile)(config) || config;
		                            ^
TypeError: object is not a function
    at Object.exports.parseConfig (/.../w/extensions/Collection/OfflineContentGenerator/lib/cli.js:33:31)
    at Object.<anonymous> (/.../w/extensions/Collection/OfflineContentGenerator/mw-ocg-service.js:46:18)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:497:10)
    at startup (node.js:119:16)
    at node.js:906:3

I guess it will be nice to use this instead of mwlib since I believe that a lot of effort has been put into this software.

I have added a more verbose version on how I got here at this spot.

Could you paste the contents of your localsettings.js? Something is probably wrong there.

I really should refrain from cross-posting and it probably was the result of this utter disappointment I encountered when trying to follow the current instructions. I figured to install OCG should not be worse to achieve that mwlib but the opposite seems to be the case. This post is basically about getting some human readable instructions. To cut it short, I answered on the linked thread.

Ok, I will respond at the other thread.

Hi. I installed the OCG bundle, and configurate my wiki, When I try to use it, the mw-ocg-service.js server fetching stays for a while and ends saying: "Status: Bundling process died with non zero code: 1" The process is the next:

first it says:

Progress: 0 % Status: Fetching wiki configuration (wiki page: http://my.domain.org) After few seconds:

Progress: 2.5 % Status: Fetching wiki configuration (wiki page: https://commons.wikimedia.org/w ) and there it stays for a while...

After several minutes say:

Generation of the document file has failed.

Status: Bundling process died with non zero code: 1

What's going on?

Maybe it has releated with the need of using proxy to connect to external sites. What it is not supposed that the server should not use anything outside (OFFLINE Content Generator)?

What I'm doing wrong?

Excuse my bad grammar.

Does your wiki use Manual:$wgUseInstantCommons ? It seems like the fetch is failing when trying to connect to commons. If you don't use instant commons, it shouldn't be trying to connect.

(The "OFFLINE" refers to the *result*, not the *process*.)

Status:

• PDF renderer should work again on beta.wmflabs.org as of June 9.
• More content requested: bugzilla:66402.
• Will be enabled as secondary method in production at the end of July.

What's the URL of the current test instance? Both and are offline. I'd like to organise a Wikibooks/Wikisource bug day to retest old mwlib bugs from http://web.archive.org/web/20111201152942/http://code.pediapress.com/wiki/report/1 but I don't know where. In the meanwhile I pasted a list at etherpad:BugTriage-mwlib.

I'll have to set a server back up again -- I failed to move the server during the labs migration. It's on my list of things to do (very high up); there's just been a boatload of fundraising duties getting in the way. I think this week those've settled. My goal has this done by tomorrow in beta labs; and if not then, by the end of the week.

Thank you very much for the answer. Let us know here when it's ready for testing. :)

Nemo, I'm an idiot... It took me till today to setup a wmlib server in labs; only to realize I didn't actually need one. Production PDF rendering is available from labs... (So use en.wikipedia.beta.wmflabs.org or an equivalent.) I'm blocked on offering the new renderer for testing in labs because I don't have an Ubuntu 14.04 image (and there's other deployment troubles I'm fighting.)

Basically; you can test collection / mwlib to your hearts content right now anywhere in betalabs. I'll have to get back to you when I have everything setup for the new renderer.

Ouch, sorry for the confusion. I mentioned the known mwlib bugs, but I want to test them against the new PDF renderer (I see this was only explicitly said in the etherpad but not in this thread).

It's useful to know that labs can be used to test mwlib, but I'll keep waiting for the new test instance.

Nemo; I'm sorry it took this long -- but after a week and a half of fighting with deployments and ubuntu 14.04; the test rendering instance is up and running on beta labs. It's the WMF PDF option in Special:Collection and in the side pane.

Wee. :) Now we only need a good content sample. Can we get a biggish Wikipedia and a Wikisource fully imported on some domain? Otherwise let's hope giving import rights to all willing testers works. :)

Hello,

Current PDF rederer does not handle HTML5 tag like <time> or <data> so I hope this new one will come quickly.

I've tried to test it on en.wikipedia.beta.wmflabs.org (random page, tested several time), but

• I don't see WMF PDF on Special:Book (= Special:Collection)
• When I click on "Download as WMF PDF" in the side pane, I end-up on a "Book rendering failed" page, with the message "There was an error while attempting to render your book.

What am I doing wrong ?

Same here when trying a random article. Something must have happened last week, it used to work.

Pages on http://en.wikipedia.beta.wmflabs.org/ offer "Print/export > Download as WMF PDF" for me, but would be awesome to also test this on complex and RTL pages...

I accidentally broke the renderer last week by applying a faulty firewall rule and then not having enough time to test it... I fixed that last night.

That works better, thanks.

The HTML 5 tags are correctly printed.

But it seams tables are not on the pdf.

Tested on this page, mix of real French Wikipedia content and some simple exemples.

Also noticed these minor issues :

• Generated pdf do not have a .pdf extension, so firefox offer me to download it instead of displaying it as pdf.
• The name in the collection tool seams to be <coll-format-rdf2latex>, with is not user friendly.

I went through all bugs in etherpad:BugTriage-mwlib. About half were triaged or postponed, on the other half I'd use help because they're harder to assess or testing is currently blocked on other bugs. Time for a bug day for real now, I guess.

I've created Bug management/Triage/201410, please signup and remove dates which are not ok for you.

The date was set (2014-10-08) and announced to 10 mailing lists and 54 talk pages.

The labs test-instance seems to be down.

Production seems to work, though. Please test. ;-) https://etherpad.wikimedia.org/p/BugTriage-mwlib

What was the final result here?

The first sprint resulted in code, and there's a second 2-week (I think) sprint starting today to try to get it deployed.

Correct; 2nd sprint is from January 6 and end on January 17 (according to ops ml).

Tracked in Phabricator
Task T73808

The page has no mention of the fact that tables are unsupported. Is there any page where the topic is covered? Information on the new PDF rendering is scattered all over the place so I don't know if this is the best page still.

In any case, here are some things to watch for when adding tables support (taken from the mwlib tracker):

• http://web.archive.org/web/20110800000000/http://code.pediapress.com/wiki/ticket/833
  • Empty cells are ignored
  • assigned – volker – normal – major – 18 months

• http://web.archive.org/web/20110800000000/http://code.pediapress.com/wiki/ticket/845
  • Find a way to force a "vertical no break"
  • assigned – volker – normal – tweak – 18 months

• • "Some small tables should never be split"
• http://web.archive.org/web/20110800000000/http://code.pediapress.com/wiki/ticket/504
  • Rendering of Infoboxes and other tables with specific width or nested tables
  • accepted – volker – normal – major – 2 years

• http://web.archive.org/web/20110800000000/http://code.pediapress.com/wiki/ticket/870
  • Table is centered
  • assigned – volker – none – feature – 13 months

• http://web.archive.org/web/20110800000000/http://code.pediapress.com/wiki/ticket/919
  • Table titles unrecognized / displayed improperly
  • assigned – volker – urgent – major – 11 months

• http://web.archive.org/web/20110800000000/http://code.pediapress.com/wiki/ticket/933
  • Pipes in images found in tables
  • new – ralf – urgent – major – 8 months

• http://web.archive.org/web/20110800000000/http://code.pediapress.com/wiki/ticket/939
  • Do not automatically exclude tables/divs with class="navbox"
  • assigned – volker – urgent – major – 8 months

• http://web.archive.org/web/20110800000000/http://code.pediapress.com/wiki/ticket/753
  • table caption parsing fails
  • new – ralf – normal – major – 23 months

• http://web.archive.org/web/20110800000000/http://code.pediapress.com/wiki/ticket/774
  • HTML tables not parsed if opening table tag missing
  • new – ralf – normal – major – 21 months

• http://web.archive.org/web/20110800000000/http://code.pediapress.com/wiki/ticket/889
  • sole &nbsp: in table cell dropped
  • new – ralf – normal – block – 13 months

• http://web.archive.org/web/20110800000000/http://code.pediapress.com/wiki/ticket/932
  • Table background not properly rendered
  • new – volker – normal – major – 8 months

• http://web.archive.org/web/20110800000000/http://code.pediapress.com/wiki/ticket/442
  • huge tables can not be rendered --> treecleaner?!
  • new – volker – low – feature – 3 years

• http://web.archive.org/web/20110800000000/http://code.pediapress.com/wiki/ticket/744
  • Gaps in tables for no apparent reason
  • assigned – ralf – low – tweak – 23 months

• http://web.archive.org/web/20110800000000/http://code.pediapress.com/wiki/ticket/817
  • tables unnecessarily removed by treecleaner
  • new – volker – low – tweak – 19 months

• http://web.archive.org/web/20110800000000/http://code.pediapress.com/wiki/ticket/462
  • tables with cells exceeding page height fail
  • new – volker – normal – feature – 3 years

• http://web.archive.org/web/20110800000000/http://code.pediapress.com/wiki/ticket/649
  • improve border rendering for tables
  • new – volker – normal – feature – 2 years