Topic on Talk:Reading/Web/PDF Functionality/Flow

I posted this to the project support desk page and did not get a reply so I'm trying here.

I am upgrading my MediaWiki to

MediaWiki  1.31.1

PHP        7.2.10 (apache2handler)

MariaDB    10.3.11-MariaDB

When I try and start the book creator I get a page that says:

"Book Creator is undergoing changes"

However, that page links to (here):

https://www.mediawiki.org/wiki/Reading/Web/PDF_Functionality

for details.  This page seems to indicate that the book generator is supposed to be operational, but I cannot tell so I don't know what should work and what does not.

What is the current status?  Is this issue that the "Download as PDF" is not working?  Rather creating a PDF via PediaPress should work?   Neither is working for me, but I am familiar with the process as I have extensively used book creation via "Download as PDF" in the past.

Thank you.  Brent

Uh, good question. I don't think the book-to-PDF creation should have been included in 1.31.1 but normal PDF creation should, but I'm guessing here. Do you know, @OVasileva (WMF)?

The information in the article does seem to be unclear. PediaPress are currently involved in two different ways:

1. You can create a book, upload it to the PediaPress web site and order print-on-demand physical copies.

2. PediaPress are also rewriting Wikipedia's own PDF book renderer, and while they are doing this it is not possible to create or print a PDF softcopy wikibook. This is the main subject of the recent update posts.

But I do not know what functionality is included in the 1.31.1 build.

Hope this helps a little.

The "Download as PDF" is greyed out (it shows on the page but unavailable for use).

The "Preview with PediaPress" is "available". So I gather it is supposed to work?

If there is a way for me to get on the inside track for enabling/testing an alpha or beta "Download as PDF", please let me know.

Thank you for your replies. Brent

p.s. I might suggest that the www.mediawiki.org/wiki/Reading/Web/PDF_Functionality be more clear about what users can expect to work/not-work in the application as-of a specific MediaWiki release.

The "Preview with PediaPress" is the proprietary print-on-demand option, which is fully working and so is not greyed out.

There is an "alpha"-ish test build of the open-source "Download as PDF" book creator, though it does not yet have the Chapter headings wrapper or anything, at https://pediapress.com/collector

That's very much aimed at users of the Wikimedia wikis, yes. I'll see what we can figure out.

I see there is more discussion about this at Topic:Uqod5bg3xaswxjn9. (www.mediawiki.org/wiki/Topic:Uqod5bg3xaswxjn9)

I'm just pasting this here so if someone is reading this thread in the future, hopefully, it will save them time.

with http://mediawiki2latex.wmflabs.org/ you can get a PDF

Yes I found it! Thank you Dirk :)

Just to close the loop on this thread in case someone else stumbles across this. My solution ultimately landed on getting MediaWiki 1.27 up with a version of Collection extension supporting book creation operational. Versions of MediaWiki after 1.27 do not appear to be compatible with the Collection extension that supports book creation.

Hi Brent, nice to hear that you found a solution. But consider that 1.27 will be end of life in june 2019. Which is in four month from now. See https://www.mediawiki.org/wiki/Version_lifecycle Yours Dirk

Hi all, apologies for the late reply. Unfortunately due to low usage, we will only be supporting PDF book creation via Pediapress for Wikimedia projects in the future, which is the renderer that Pediapress are currently working on. That said, the book creation process of the collections extension (everything outside the actual PDF download) are still supported and functional, as is the PDF creation from individual articles. It was a difficult decision for us to make, but fixing the book creator after retiring the OCG rendering service proved to be very complex and to require a lot of technical support in the future. As the usage of the feature was very low, we decided to continue providing the functionality on a smaller scale, and focus on rendering individual PDFs of articles instead.

Hi, thanks for the update. I feel a bit honoured that I do provide a feature ( http://mediawiki2latex.wmflabs.org/ ) in my free time which needs so much programmer time that the WMF decided that they cannot afford to implement it, especially since WMF got a budget of more than $90 million. Furthermore I implemented significant parts of the software when I was shaving cows in a cowshed for approx 5 EUR / hour and could not find any other job.

As a physicist this makes me laugh like looking a measured data that does surely not resemble reality. If you see something like that, something with the way you set up the experiment or with the way you analysed the data has gone terribly wrong. So yes the prophecy of the sect of purely functional programmers is true.

Quoting from the paper http://www.cse.chalmers.se/~rjmh/Papers/whyfp.pdf form 1984 "Functional programmers argue that there are great material benefits - that a functional programmer is an order of magnitude more productive than his conventional counterpart, because functional programs are an order of magnitude shorter." Yours Dirk

When I learn that a book conversion which used to take five minutes on Wikipedia takes Dirk's system a time measured in hours, I wonder how much of that is down to hardware, how much to the chosen programming language, and how much to the code design. Maybe one day Dirk can install the forthcoming Wikimedia solution and see how fast it runs?

I think its because I am using http to get the data and I only do one request at once since the servers might not respond otherwise. So you could get a significant speedup if you connected to the database directly. But there is one large amount of costs which is the compilation with LaTeX, which has to be done 4 time in order to get the references right and that takes a least 20% of the runtime, likely more, so it will still be hours for large books, independent of the programming language and hardware used, in particular you can not parallelize the LaTeX run. You can get the LaTeX source from the mediawiki2latex web server and do your own measurements. You could do it without LaTeX, but you will not get the typographic quality that way.

so I consulted some documentation. The larges book I compiled was roughly 5000 pages it is here

https://de.wikipedia.org/wiki/Benutzer:M2k~dewiki/B%C3%BCcher/Ausgew%C3%A4hlte_Beitr%C3%A4ge_und_Bearbeitungen

and here

https://drive.google.com/file/d/1SA6TEKWrdpXAxDyHZe-umBa2cJ5Ya77X/view?usp=sharing

it took about 9 hours to compile. From other documentation I found that 31% of the runtime are due to the latex compile step on averige.

So such a book will take about 3 hours to compile at minimum, independent of any software use to prepare the LaTeX source.

Perhaps that LaTeX processing overhead is why the Proton project tried to do it with html/mathml via headless Chrome? That was the mistake - it proved the wrong core to build books on. Even the single-article renderer is still warning of compositing problems. With hindsight, refreshing OCG would have been a faster and lower-risk strategy (evolution not revolution, as they say). It will be interesting to see how the PediaPress code performs.