Extension talk:Collection

Previous page history was archived for backup purposes at Extension talk:Collection/LQT Archive 1 on 2015-07-10.

Archives

/Archive 1

Im using mediawiki 1.28 with the PdfBook and Collection. Im running 4.4.0-67-generic #88-Ubuntu SMP Installed the mw-render and used this tutorial: https://www.mediawiki.org/wiki/Setup_a_render_server_on_Ubuntu_12.04_LTS Following error after pdf-download action:

/***************************************** Startof log.txt *********************************************************/
/data/mwcache/log.txt                                                                                                                                                                                           179535/175K              100%
have 0 jobs
count: 18
all channels idle


error finish: bab34137f302840d:makezip: 'RuntimeError: command failed with returncode 256: [\'mw-zip\', \'-o\', \'/data/mwcache/ba/bab34137f302840d/collection.zip\', \'-m\', \'/data/mwcache/ba/bab34137f302840d/metabook.json\', \'--status
\', \'qserve://localhost:14311/bab34137f302840d:makezip\', \'--config\', \'https://wiki.senedo.de\', \'--username\', \'senedo\', \'--password\', \'{OMITTED}\']\nLast Output:\n    1%  creating nuwiki in u\'/data/mwcache/ba/bab34137f302840
d/tmpqb3ZCc/nuwiki\'\n    removing tmpdir u\'/data/mwcache/ba/bab34137f302840d/tmpqb3ZCc\'\n    memory used: res=17.1 virt=124.3\n    1% error Traceback (most recent call last):\n      File "/usr/local/bin/mw-zip", line 11, in <module>\n
        sys.exit(main())\n      File "/usr/local/lib/python2.7/dist-packages/mwlib/apps/buildzip.py", line 155, in main\n        make_zip(output, options, env.metabook, podclient=podclient, status=status)\n      File "/usr/local/lib/pyth
on2.7/dist-packages/mwlib/apps/buildzip.py", line 49, in make_zip\n        from mwlib.apps.make_nuwiki import make_nuwiki\n      File "/usr/local/lib/python2.7/dist-packages/gevent/builtins.py", line 93, in __import__\n        result = _
import(*args, **kwargs)\n      File "/usr/local/lib/python2.7/dist-packages/mwlib/apps/make_nuwiki.py", line 6, in <module>\n        from mwlib.net import fetch, sapi as mwapi\n      File "/usr/local/lib/python2.7/dist-packages/gevent/bu
iltins.py", line 93, in __import__\n        result = _import(*args, **kwargs)\n      File "/usr/local/lib/python2.7/dist-packages/mwlib/net/fetch.py", line 7, in <module>\n        import gevent, gevent.pool, gevent.coros, gevent.event\n
     File "/usr/local/lib/python2.7/dist-packages/gevent/builtins.py", line 93, in __import__\n        result = _import(*args, **kwargs)\n    ImportError: No module named coros\n     in function system, file /usr/local/lib/python2.7/dist
-packages/mwlib/nslave.py, line 64'
error finish: bab34137f302840d:render-rl: 'RuntimeError: RuntimeError: command failed with returncode 256: [\'mw-zip\', \'-o\', \'/data/mwcache/ba/bab34137f302840d/collection.zip\', \'-m\', \'/data/mwcache/ba/bab34137f302840d/metabook.js
on\', \'--status\', \'qserve://localhost:14311/bab34137f302840d:makezip\', \'--config\', \'https://wiki.senedo.de\', \'--username\', \'senedo\', \'--password\', \'{OMITTED}\']\nLast Output:\n    1%  creating nuwiki in u\'/data/mwcache/ba
/bab34137f302840d/tmpqb3ZCc/nuwiki\'\n    removing tmpdir u\'/data/mwcache/ba/bab34137f302840d/tmpqb3ZCc\'\n    memory used: res=17.1 virt=124.3\n    1% error Traceback (most recent call last):\n      File "/usr/local/bin/mw-zip", line 1
1, in <module>\n        sys.exit(main())\n      File "/usr/local/lib/python2.7/dist-packages/mwlib/apps/buildzip.py", line 155, in main\n        make_zip(output, options, env.metabook, podclient=podclient, status=status)\n      File "/us
r/local/lib/python2.7/dist-packages/mwlib/apps/buildzip.py", line 49, in make_zip\n        from mwlib.apps.make_nuwiki import make_nuwiki\n      File "/usr/local/lib/python2.7/dist-packages/gevent/builtins.py", line 93, in __import__\n
      result = _import(*args, **kwargs)\n      File "/usr/local/lib/python2.7/dist-packages/mwlib/apps/make_nuwiki.py", line 6, in <module>\n        from mwlib.net import fetch, sapi as mwapi\n      File "/usr/local/lib/python2.7/dist-Tr
aceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/qs/slave.py", line 150, in handle_one_job
    result = workhandler(qs).dispatch(job)
  File "/usr/local/lib/python2.7/dist-packages/qs/slave.py", line 50, in dispatch
    return m(**tmp)
  File "/usr/local/lib/python2.7/dist-packages/mwlib/nslave.py", line 171, in rpc_render
    return doit(**params)
  File "/usr/local/lib/python2.7/dist-packages/mwlib/nslave.py", line 158, in doit
    self.qaddw(channel="makezip", payload=dict(params=params), jobid="%s:makezip" % (collection_id, ), timeout=20 * 60)
  File "/usr/local/lib/python2.7/dist-packages/qs/slave.py", line 66, in qaddw
    raise RuntimeError(error)
RuntimeError: RuntimeError: command failed with returncode 256: ['mw-zip', '-o', '/data/mwcache/ba/bab34137f302840d/collection.zip', '-m', '/data/mwcache/ba/bab34137f302840d/metabook.json', '--status', 'qserve://localhost:14311/bab34137f
302840d:makezip', '--config', 'https://wiki.senedo.de', '--username', 'senedo', '--password', '{OMITTED}']
Last Output:
    1%  creating nuwiki in u'/data/mwcache/ba/bab34137f302840d/tmpqb3ZCc/nuwiki'
    removing tmpdir u'/data/mwcache/ba/bab34137f302840d/tmpqb3ZCc'
    memory used: res=17.1 virt=124.3
    1% error Traceback (most recent call last):
      File "/usr/local/bin/mw-zip", line 11, in <module>
        sys.exit(main())
      File "/usr/local/lib/python2.7/dist-packages/mwlib/apps/buildzip.py", line 155, in main
        make_zip(output, options, env.metabook, podclient=podclient, status=status)
      File "/usr/local/lib/python2.7/dist-packages/mwlib/apps/buildzip.py", line 49, in make_zip
        from mwlib.apps.make_nuwiki import make_nuwiki
      File "/usr/local/lib/python2.7/dist-packages/gevent/builtins.py", line 93, in __import__
        result = _import(*args, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/mwlib/apps/make_nuwiki.py", line 6, in <module>
        from mwlib.net import fetch, sapi as mwapi
      File "/usr/local/lib/python2.7/dist-packages/gevent/builtins.py", line 93, in __import__
        result = _import(*args, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/mwlib/net/fetch.py", line 7, in <module>
        import gevent, gevent.pool, gevent.coros, gevent.event
      File "/usr/local/lib/python2.7/dist-packages/gevent/builtins.py", line 93, in __import__
        result = _import(*args, **kwargs)
    ImportError: No module named coros
     in function system, file /usr/local/lib/python2.7/dist-packages/mwlib/nslave.py, line 64
127.0.0.1 - - [2017-04-12 13:25:52] "POST / HTTP/1.0" 200 2187 0.005049
127.0.0.1 - - [2017-04-12 13:25:53] "POST / HTTP/1.0" 200 2187 0.004951
/***************************************** End of log.txt *********************************************************/

Hi, I have the same problem. Did you find any solution?

It's a problem with gevent. Coros was deprecated and is now removed.

Downgrade to a previous version with

pip install gevent==1.0

Thanks a lot, it seems to work now, at least for simple articles.

I had an older version of the render server running on Ubuntu 14.04 and it seemed to work better:

- The rendering page updated automatically. Now I have to reload the page manually.

- It seemed to work for more complex pages. Now I very often get the error "WARNING: Article could not be rendered - ouputting plain text.

Potential causes of the problem are: (a) a bug in the pdf-writer software (b) problematic Mediawiki markup (c) table

is too wide"

Okay, I hacked Image.py of Pillow to make tostring available again. Now more pages can be rendered properly.

I wish the render server components would be upgraded as they seem to be quite outdated.

I ran into the same problem. I found out that gevent.coros was renamed to gevent.lock. Therefore you could also update the code of mwlib by doing:

find . -type f -exec sed -i 's/coros/lock/g' {} \;

I also got the error KeyError: 'revisions', and solved it by replacing revs = e["revisions"] with revs = e.get("revisions","") in pp/local/lib/python2.7/site-packages/mwlib/net/sapi.py:311.

Running newest mwlib and collection...

Not a reel fix but see: https://www.mediawiki.org/wiki/Topic:Rokm611ol9px8vcg

Can't get the print/export menu in the sidebar to show up in custom namespaces. It shows everywhere else. Any ideas of how to fix this?

I believe you will have to add this namespace to the $wgCollectionArticleNamespaces configuration parameter.

Thanks. I had already done that but rechecked and had a typo. All good now.

I'm running the latest version of the extensions on MW1.25alpha and I'm seeing that when I click 'Download as PDF', the licensing information shows License: unknown. I have the following variables configured for the extension, as well as the default for the whole wiki:

$wgLicenseName = "Creative Commons Attribution-Share Alike 3.0 license"; $wgLicenseURL = "http://en.wikipedia.org/wiki/Wikipedia:Text_of_Creative_Commons_Attribution-ShareAlike_3.0_Unported_License";

Shouldn't the licensing info appear on the PDFs as configured?

Thanks

I tried setting:

$wgLicenseName = null;
$wgLicenseURL = null;

As I have the following configured:

$wgRightsUrl = "http://creativecommons.org/licenses/by-sa/3.0/";
$wgRightsText = "Creative Commons Attribution-ShareAlike";

According to the ReadMe in the Collections source tree, the license should default to the Rights text in this case. It doesn't. Rendered PDFs and Books still have a License: unknown.

It's probably time to report an issue at phabricator.

You're right.

https://phabricator.wikimedia.org/T91262

Thank you for doing this. This issue is actually a bit worrisome since licensing is an integral part of providing content. So the correct attribution is something that should not be missed.

Indeed, I've already argued that the extension should not output at all any content which can't be attributed.

I'm very curious how this is working on WP and not my wiki? Can anyone else reproduce this?

Same here, I'm on MW 1.27 and Collection 1.7

License for all the images is Unknown even though it is clearly mentioned as CC-BY-SA 4.0 and this is also same for images of commons used with InstantCommons

Hello guys, I would like to generate the collection_id from set of article and use this collection_id to create the pdf file from Wikipedia

Now

How to get https://gerrit.wikimedia.org/r/#/c/336342/ merged? Are there any active maintainers of this extension? Is there any deployment step besides the merge in Gerrit itself? ~~~~

see community consensus.

Such requests are piped in via Phabricator. I suggest to create a task for this.

Is there's any way to change the design of the rendered pdf? i.e - add some css, fonts, etc.?

Hi! Yeah! I planned to do that and obviously, to start making some kind of structure. Unfortunately I have had a lot of work and it couldn't have done. Would you like to join me and make a contribution or building this together?

Hi. we have a project that is due in a month, so we're in to it. Lets talk and proceed somehow.

Is mwlib working with collection 1.26? I just can't make it work, always get the "Book rendering failed: An error occurred while trying to render your book" error. I use the public render server ('http://tools.pediapress.com/mw-serve/').

Hi, I'm using the same version... I can't make it work .. I've tried to build a rendered server on Ubuntu or use the public render server you said.. no matter what I do, I always get the same error

Hi again, Better to use a own render service

Follow the instructions on http://mwlib.readthedocs.org/en/latest/installation.html#mwlib-install and launch the listener services such as nserve, etc through console or better on startup as a script

Now, all of this is working in Ubuntu 64 bits with the last release of Collection extension in mediawiki 1.26

our organization is using windows only.So is there anway we can create our render server for windows.

Hi All,

Setting up this Extension seemed relatively simple. I've done everything I needed in terms of configuration:

require_once "$IP/extensions/Collection/Collection.php";

$wgCollectionPODPartners = false;

$wgEnableWriteAPI = true;

$wgCollectionMWServeURL = "http://tools.pediapress.com/mw-serve/";

$wgCollectionMWServeCredentials = "user:userpass";

$wgCollectionFormats = array(
'rl' => 'PDF', # enabled by default
    'odf' => 'ODT',
    'docbook' => 'DocBook XML',
    'xhtml' => 'XHTML 1.0 Transitional',
    'epub' => 'e-book (EPUB)',
    'zim' => 'Kiwix (OpenZIM)',

);
$wgGroupPermissions['sysop']['collectionsaveascommunitypage'] = true;
$wgGroupPermissions['sysop']['collectionsaveasuserpage']      = true;

However all I get after clicking Download as PDF is "There was an error while attempting to render your book.". No details to help me debug it. is there any way to enable some more detailed error reports or does anyone know if there's anything wrong with my config?

This post was posted by Edga~mediawikiwiki, but signed as Edga.

I am having the same problem and it's driving me crazy , been sitting at my computer for more than 10 hours without any success !

the test (mw-zip and mw-render) works fine on the render server's console. but my wiki won't connect to the render server. it doesn't even show any activity on the render server's log. it's just the plain "There was an error while attempting to render your book." ...

I even installed a new virtual machine and set the render server up again from scratch, same results.

appreciate any help / idea

(wiki site : www.fidicterms.org)

(temporary render server : 107.170.1.247)

do you solved the probelm?

Could this be related to "The Book Creator software has been broken since it was rebuilt in 2014, the management process is inactive, and no fixes are in sight." from https://en.wikipedia.org/wiki/Help:Books?

Hi,

If you use your own renderer, take care to specify port number used by nserve (8899 by default) in LocalSettings.php :

$wgCollectionMWServeURL = 'http://mywebsite:8899'

And don't use 'localhost' or IP like 192.168... It's necessary to set a "real" name :

$wgServer = "http://mywebsite";

I was getting the same error upon setting up this extension and then realized access to my wiki was restricted to logged in users only and the public http://tools.pediapress.com/mw-serve/ needed to access the page in order to render it.

So on a private wiki, you can use the following hack to give access to tools.pediapress.com (78.46.16.76). Keep in mind that an ip address is easily spoofed so use at your own risk.

if(isset($_SERVER['REMOTE_ADDR']) && $_SERVER['REMOTE_ADDR'] == '78.46.16.76')
{
  $wgGroupPermissions['*']['read'] = true;
}

Did not see this before but there is also the "$wgCollectionMWServeCredentials" configuration parameter used like $wgCollectionMWServeCredentials = 'user:pass';