Talk:Citoid/2015

Previous archives are at /Archive 1

Auto-archive

I would like to suggest that Citoid automatically archives pages when they are used in a reference. Yeah, I know, you have a very long to-do list, but put this on your eventually list. Oiyarbepsy (talk) 03:06, 3 January 2015 (UTC)

Do you mean "automatically finds the archive URL from the Internet Archive and makes it available" or something else? Jdforrester (WMF) (talk) 19:19, 6 January 2015 (UTC)

Yes, or, automatically creates the archive would be even better. We could potentially eliminate the scourge of dead links. Oiyarbepsy (talk) 05:35, 8 January 2015 (UTC)

"Creates the archive" meaning "asks the Internet Archive to archive the URL"? Or are you asking for WMF to archive the Web (in which case I believe the answer is a very clear "no" from Legal, as repeatedly discussed over the last few years). Jdforrester (WMF) (talk) 16:14, 8 January 2015 (UTC)

The internet Archive is already "crawling all new external links, citations and embeds made on Wikipedia pages within a few hours of their creation / update." [1] Only enWP, it seems (and caveat robots.txt).

The french WP even automatically adds an archive-link to wikiwix on all reference links.

The rest is rotting away.

PS:

Both cite_web and cite_news have parameters to use "archiveurl=" and "archivedate= ". Atlasowa (talk) 11:37, 9 January 2015 (UTC)

Open up a task on phabricator! :) https://phabricator.wikimedia.org/project/board/62/ Mvolz (WMF) (talk) 22:55, 17 January 2015 (UTC)

Done: https://phabricator.wikimedia.org/T89438 Daniel Mietchen (talk) 09:40, 13 February 2015 (UTC)

Citoid on Wikidata?

I'd like to be able to use Citoid's "cite from ID" functionality to fill in statements on Wikidata items about references with Citoid-covered IDs. Is something like this on the horizon somewhere, possibly through a gadget? Daniel Mietchen (talk) 23:38, 14 March 2015 (UTC)

It's certainly something that could be built. I'm not sure it's a great area to focus on, until we have actual structured citations on WIkidata; right now you're meant to just add a reference with type=URI and the link – there's not really anywhere for the content that Citoid fetches to go… Jdforrester (WMF) (talk) 18:23, 15 March 2015 (UTC)

Actually, that's not exactly true; it's convention to just add the URI because that's the easiest way to do it. But there is support for structured citations, see claim "original language of work" in wikidata:Q43361. There you can see that the reference is "in" a wikidata item, a book, which has its own metadata, and then there are other properties you can annotate the citation with, like page number and quote, etc.

That said, it is easily more complicated than what is currently being done with citoid/VE or citoid/wikitext because of course, it requires you to both a) search wikidata and b) create new items on wikidata which is already more complicated than VE/wikitext which essentially amounts to just inserting text. And it also is a more complicated language, because each reference has to be annotated with potentially specific fields depending on the kind of citation; we have to know which fields go in the wikidata item the reference is "in" (such as volume) and which fields get annotated directly onto the reference (such as page number), and we have to do that with each citoid type (which are just zotero types- see incomplete map between zotero and wikidata types here: wikidata:Wikidata:WikiProject_Source_MetaData/Source_types).

Although I agree with James that it's probably not *immediately* in the pipeline since there is more low-hanging fruit before we get there. Mvolz (talk) 13:27, 16 March 2015 (UTC)

Oh, sure, you can do that in Wikidata, but only if the item you're referencing already exists; there doesn't appear to be community appetite just yet for auto-creating items just to serve as reference points. Jdforrester (WMF) (talk) 15:45, 16 March 2015 (UTC)

Doesn't? Helder 00:59, 30 March 2015 (UTC)

Happy to be proven wrong if you have a link. :-) Jdforrester (WMF) (talk) 01:05, 30 March 2015 (UTC)

https://www.wikidata.org/wiki/Help:Sources#Adding_a_source_to_a_statement

"Add the source as an item if: i) it's not in Wikidata already and ii) it is not a webpage"

So if citoid retrieves information about a book, newspaper article, or journal article, both the journal article itself and the journal it's published in should be added as items in wikidata, as see: https://www.wikidata.org/wiki/Help:Sources#Scientific.2C_newspaper_or_magazine_article

If the item is a webpage, then only the reference URL should be inserted, but an item for the particular website (publisher) itself should not be unless it is already in wikidata. Date and access date can be added: https://www.wikidata.org/wiki/Help:Sources#Web_page

I can see an issue arising where if citoid miscategorises a blog as a newspaper, this could conceivably contribute to "junk" being in wikidata. Right now though it's very "cautious" in this regard; currently only results coming from zotero are classified as newspapers or journals, where a human has assessed this. We currently assign urls where no zotero translators exist that have the open graph type "article" to itemType "blogPost" even though plausibly these could be news or journal articles as well. Mvolz (WMF) (talk) 11:07, 30 March 2015 (UTC)

No. https://www.wikidata.org/wiki/Help:Sources#Web_page is about the system in Wikidata for references for claims in Wikidata. It's not something that can work for external claims as it currently is. Jdforrester (WMF) (talk) 14:57, 30 March 2015 (UTC)

I believe User: Daniel Mietchen's question was about using citoid to create references for claims in Wikidata, which is what I was responding to. Is that not the case? Mvolz (WMF) (talk) 16:12, 30 March 2015 (UTC)

Sorry, you're right, I lost track of the parallel conversations. :-) Jdforrester (WMF) (talk) 17:26, 30 March 2015 (UTC)

Could you clarify what you mean by "reference points"? Helder 01:46, 30 March 2015 (UTC)

Interesting discussion. I would like Citoid to support Wikidata and creating new items based on identifiers they don't exist already. So9q (talk) 20:08, 10 February 2022 (UTC)

Flat author list

Currently, the mappings for the authors and editors require an array of template paramaters for first names and last names. However, in the German wikipedia the cite templates do not have such parameters, but just a single "author" and "editor" parameter were all authors and editors are listed. Is it possible to get a flat author and editor list? Mps (talk) 01:11, 23 April 2015 (UTC)

I've filed a bug for this: https://phabricator.wikimedia.org/T97044

We won't probably provide a flat field in the citoid service. Any flattening should occur on the front-end. Each wiki has different templates and when deciding on the format we tried to make it as flexible as possible, but unfortunately there will always be conflicts like this.

That said, we might be able to do something on the extension-level. Mvolz (talk) 16:54, 23 April 2015 (UTC)

Thanks for opening the bug report. Mps (talk) 18:40, 27 April 2015 (UTC)

Polish Wikipedia?

Hi, I'd like to know if there is any ETA on enabling this excellent extension on Polish Wikipedia. I've been using Zotero for years now and it works wonders, I'm glad there's an easier way to use its potential on wiki. Or can we install it ourselves? Halibutt (talk) 22:32, 1 May 2015 (UTC)

We don't have a specific timeline for rolling it out to each project yet. Technically, any admin can install it, although it's not very simple. You have to make a list like this one at en.wp, and you must have good TemplateData on the citation templates, or it won't work.

But before you try to turn it on, I strongly recommend thinking about what sources are commonly used at pl.wp, and testing them to see whether Zotero supports them. This will be easier for you since you're familiar with Zotero already. The service is rather Anglo-centric, and if Polish is poorly supported, then it might be an exercise in frustration for most editors.

Do you know how to write translators for Zotero? When this starts spreading, we will probably have a lot of requests. Whatamidoing (WMF) (talk) 03:34, 4 May 2015 (UTC)

A quick note to let you know that as of next week, I should be able to generate the list of most common domains used in references for any wiki. I can generate that list for pl.wikipedia then, if it helps. Guillaume (WMF) (talk) 16:08, 7 May 2015 (UTC)

@Whatamidoing (WMF): I contacted one of our more tech-savvy admins, we'll try to do it ourselves and tell you of the results. TemplateData are already in place, we even have a single Szablon:Cytuj that could be used for all fields (reminiscent of Template:Citation at en.wiki).

As to sources, I guess the most natural would be the National Library of Poland, it works great with Zotero (and has been for ages). The Federation of Digital Libraries and all the affiliates work out of the box as well. *Some* university libraries also work just fine (note to self: UP Cracow, Kielce University, Łódź University, Poznań University - to name a few). So, all in all we're ready as far as sources are concerned.

Finally, as to translators, I did some work on some basic translators a couple of years back, but it was mostly trial and error. However, there already is a decent translator for English Wikipedia, so mapping Polish field names to English parameters should not be a difficult task. Or am I missing something here?

@Guillaume (WMF): Yeah, it would really be interesting. I know what I use, but what is the statistics behind the entire wiki - I'm not sure. Halibutt (talk) 00:29, 20 May 2015 (UTC)

I'm generating the data set and I should be able to post the list of top domains later today if all goes well. Guillaume (WMF) (talk) 16:25, 21 May 2015 (UTC)

The list is now at https://phabricator.wikimedia.org/P669 Guillaume (WMF) (talk) 21:03, 21 May 2015 (UTC)

Please let me know how it goes, Halibutt. I'll be interested in seeing how you do and also in getting your ideas about how to explain the process for other projects. Whatamidoing (WMF) (talk) 00:03, 21 May 2015 (UTC)

Thanks for your assistance, we'll give it a try later today and see how it goes. I'll be sure to let you know what's the result. Halibutt (talk) 08:52, 21 May 2015 (UTC)

@Whatamidoing (WMF) @Guillaume (WMF) Hi guys, I forgot to let you know how great Citoid works (well, worked) at Polish wiki, I was apparently too busy using it :) We implemented it without any problems and it worked precisely as planned. Sure, not all popular Polish pages have correspondent Zotero plugins, but the extension worked pretty well for most book sources and did a decent job for web pages. Thanks again for your assistance.

We (me and @Tar Lócesilion) were thinking of switching the MediaWiki:Citoid-template-type-map.json table to point at Szablon:Cytuj (it's a one-size-fits-all citation template, quite similar to Template:Citation at English Wikipedia), but this might be a tad complicated I guess (unlike traditional citation templates with name1, name2 and so on, this one uses a single author field with basic syntax analysis for instance). Does any wiki use similar templates with Citoid?

For some weird reason Citoid stopped responding today (it used to be one of the windows in VE's "Przypis" field (Polish for "Citaton"), but today it disappeared. It works at English wiki, so I guess it's a local problem, but I have no clue where to look for possible changes that could've broken it. Any ideas? Halibutt (talk) 00:27, 3 August 2015 (UTC)

Turning name1, name2, and so in into a single field is phab:T97044, which should be working now. I've never written any TemplateData to do that, though. Whatamidoing (WMF) (talk) 19:12, 4 August 2015 (UTC)

@Halibutt re: Citoid not working, I was able to use it there today (although it fails with a couple of Polish sources - I think that generally speaking your map should not feature templates which do not exist locally though!). Please report outages at https://phabricator.wikimedia.org/T93800 . I followed instructions provided here, which are meant as a temporary solution though. Elitre (WMF) (talk) 06:26, 3 August 2015 (UTC)

@Elitre (WMF) I didn't notice that problem before, we fixed it now (all the instances that failed in your test page should work now, although in the most basic way as there's no specific Zotero plugin for those sources).

And Citoid works again, I have no idea what went wrong. Thanks, I'll remember that for the future. Halibutt (talk) 10:54, 3 August 2015 (UTC)

A loosely-related note: I was thinking of writing a translator for Poland's National Library new Encore OPAC catalogue but it turns out the page works perfectly well with Zotero. It doesn't work with Citoid though. I reported it as a bug @ Phabricator. Halibutt (talk) 00:34, 12 August 2015 (UTC)

Thanks a lot for doing it, Halibutt! Elitre (WMF) (talk) 09:20, 12 August 2015 (UTC)

Is it possible to provide some filtering for language field? Recently I spot an incorrect language code used in the citations (example). Although the the code itself is not incorrect, however, it is not allowed in templates, because it is not recognized. The specific code pl-PL should be replaced with pl or not used at all, since such language is default in plwiki and info about it is redundant for readers. Paweł Ziemian (talk) 05:48, 15 October 2015 (UTC)

I added your comment here. Thanks. Elitre (WMF) (talk) 10:44, 15 October 2015 (UTC)

Spanish wikipedia

Hi

In spanish wikipedia we have created the templatedata blocks for cite templates, the citoid maps and the mediawiki page.

After creating the mediawiki page the "Cite" menu in VisualEditor turned into the citoid automatic/manual menu, but it doesn't work quite well. For random websites it generates a correct citation, but not for DOIs or other mre complex websites.

Do you know what os going on? Ignacio Rodríguez (talk) 17:15, 22 May 2015 (UTC)

Ninovolador, does this section answer your question? I had tried Spanish sources and published the results in Phabricator. Also, here is the list of the most referenced domains on your wiki. See also the thread below for more information about what can be done with this list.

Please point to pages where tests are being made so we can check what happens. Thank you! Elitre (WMF) (talk) 17:31, 22 May 2015 (UTC)

I think that is possible that it is due to lack of zotero translators.

I ran several tests on es:Usuario:Ninovolador/Taller3. Scielo is one of the most popular scientific websites in spanish and portuguese, and citoid does OK.

But, some english popular websites, like Wiley or sciencedirect are not OK.

I will continue to test other sites, and put the results here if you like. Ignacio Rodríguez (talk) 21:39, 22 May 2015 (UTC)

i conduced more test, at :es:Usuario:Ninovolador/Taller4 with some of the websites on your list.

Again, some well known English websites doesn't work quite well. I check the zotero github site for translators for, example, BBC, and there is one, but it doesn't seem to be used. Ignacio Rodríguez (talk) 22:10, 22 May 2015 (UTC)

Ninovolador, not all of the translators that are in the list on the github repo can be used/are used at any given time, unfortunately.

If you want a better list/explanation, see this: Citoid/Determining_if_a_URL_has_a_translator_in_Zotero

(I've basically just a copied an email in there, so it's a bit rambling) Mvolz (WMF) (talk) 10:52, 29 May 2015 (UTC)

@Mvolz (WMF): Is there something we or I can do to help implementing some of the most important translators? (such as JSTOR, sciencedirect, etc) I am not an expert in programming, but i am disposed to learn Ignacio Rodríguez (talk) 14:31, 1 June 2015 (UTC)

Maybe Halibutt knows how to do that? Hallibut knows a lot about Zotero. Whatamidoing (WMF) (talk) 22:41, 1 June 2015 (UTC)

There's a guide on the Zotero site. It states, "Who is this guide for? Anyone! No previous experience required!", which is encouraging. Elitre (WMF) (talk) 14:05, 3 June 2015 (UTC)

The tutorial is good, but it doesn't include testing for the translators using the "server" option or 'v' flag, which is what we need.

To test with translation-server, download and install https://github.com/zotero/translation-server

And your translator will have to have the 'v' flag enabled for 'browserSupport'. More here on that: https://www.zotero.org/support/dev/translators

I.e. see https://github.com/zotero/translators/blob/master/3news.co.nz.js you will see there are a bunch of letters, one of which is v, which corresponds to translation-server. If server support is not enabled by testing it/ then adding the 'v' flag, we won't be able to use the translator. Mvolz (WMF) (talk) 10:22, 4 June 2015 (UTC)

Capitalisation

What's the correct capitalisation, 'citoid' or 'Citoid'? The article uses both seemingly interchangeably. Matěj Grabovský (talk) 21:50, 28 May 2015 (UTC)

The service is 'citoid' and the the extension is 'Citoid'. Mostly due to capitalisation conventions for node.js libraries versus mediawiki extensions. Mvolz (WMF) (talk) 10:10, 29 May 2015 (UTC)

So the introductory sentence is, in fact, incorrect – “Citoid is a node.js service”? Matěj Grabovský (talk) 13:50, 29 May 2015 (UTC)

Not necessarily. Common sentence case rules say that even if a word is normally lowercase, then you can capitalize it at the start of a sentence. Thus "IPhones are expensive" and "An iPhone is expensive" are both correct. Whatamidoing (WMF) (talk) 22:56, 1 June 2015 (UTC)

I've modified the wording on the page. Hopefully it's clearer? Jdforrester (WMF) (talk) 20:07, 3 June 2015 (UTC)

How to use Citoid?

Hello there, I just tried to add a reference using Visual Editor editing a page here on mediawiki.org and in the reflist I only got the URL

[[2]] is my experiment. I have added it using the "Cite" menu.

What have I done wrong? It doesn't add any of the needed information: publication date, author name or title.

Thanks. —  Ark25  (talk) 19:22, 3 June 2015 (UTC)

Citoid is only configured for use on (I think) the English, French, Italian, Russian and Polish Wikipedias. It doesn't make sense to enable it on this wiki, which doesn't use content citations. Jdforrester (WMF) (talk) 20:01, 3 June 2015 (UTC)

I've noted this on the page. Hopefully it's clearer? Jdforrester (WMF) (talk) 20:05, 3 June 2015 (UTC)

Citoid for users not using VisualEditor

Hello again, I have another question: Some (many?) editors (like me) do not enjoy using VE. Is there any plan to use the citoid code in a tool that can generate references automatically for those like me?

Thanks —  Ark25  (talk) 19:31, 3 June 2015 (UTC)

The phabricator task for this is here:

https://phabricator.wikimedia.org/T94223 Mvolz (WMF) (talk) 10:08, 4 June 2015 (UTC)

Yes, it's one of the things in our backlog, but we're not currently working on it. I don't have a good estimate of when you should expect it, sorry. Jdforrester (WMF) (talk) 20:01, 3 June 2015 (UTC)

Access date

It has been told to me on the Spanish WIkipedia, that the access date is formated on an unusual way for spanish speakers: aaaa/mm/dd, and it seems to be that way by default. In spanish we use either «day de month de year» (08 de junio de 2015) or dd/mm/yyyy.

It is there a way to change that? Ignacio Rodríguez (talk) 22:26, 8 June 2015 (UTC)

The dates are in ISO format, which is an international standard; it's not really the typical way of writing the date in any language, really :). The short answer is that on the back-end, we're sticking to ISO and in the future all dates will all be in ISO, not just access date. This is because it is an unambiguous way to present the date in any language.

If the community doesn't like the way this looks to the user (understandable!), it is possible to edit the citation template to format the ISO dates to something that is standard in your language. For instance, you can add logic to the template such that if the date is detected to be in ISO yyyy/mm/dd format, the date is reformatted *to appear* to be dd/mm/yyyy on the page. However, if you do this, the underlying data (i.e. when you edit the wikitext, or the form in VisualEditor) will still remain the same. Mvolz (WMF) (talk) 07:57, 9 June 2015 (UTC)

Citoid installing help

Hey, I will ask the Finnish Wikipedia community to enable citoid on fiwiki after I can first understand how to set things up.

1) I added the following things to Template:Cite web/doc (https://fi.wikipedia.org/w/index.php?title=Malline:Verkkoviite/ohje&diff=prev&oldid=15001734)

2) I created MediaWiki:Citoid-template-type-map.json (now deleted by me). After that there was a new "cite" button on the VE navigation bar. I tried to add a web site link there but it told me then that "Määritä osoite!" (="give a URL") and generated only something like "<ref>{{Cite web}}</ref>"

What did I do wrong? Or should I even do the step 2 before than citoid is requested on Phabricator? Stryn (talk) 19:22, 15 June 2015 (UTC)

I reverted that change temporarily because it looks like you deleted the 'paramOrder' key when you added maps; I'm not sure if that was accidental or not, but paramOrder is different from maps and should be left in. Overall it looked mostly good except that 'author' is currently required to be a 2D Array (i.e. [[]] ); this is somewhat temporary as once https://phabricator.wikimedia.org/T97044 is resolved (hopefully soon) you'll be able use the template data as you've written it. For the moment I've removed the author field.

One possible reason for the empty template appearing is if you didn't do a null edit (edit the page without changing anything) on the Malline:Verkkoviite . There is a long standing bug with transcluded templateData (i.e. templateData coming from Malline:Verkkoviite/ohje instead of directly from Malline:Verkkoviite). You can check to know for sure if the maps is available to citoid by visiting the api page: https://fi.wikipedia.org/w/api.php?action=templatedata&titles=Malline:Verkkoviite&format=jsonfm

On that page, you should see your values reflected in the 'maps' field. If it is an empty object, citoid won't be able to read it because it reads the values from the API. Mvolz (WMF) (talk) 07:27, 16 June 2015 (UTC)

I deleted the 'paramOrder" intentionally as I wanted to try the maps thing without it first, because I just copypasted the maps code from another Wikipedia, and I was lazy to look at where's the correct place to add the code so that there's not additional brackets.

Now I tried it again and it's working, yay! Stryn (talk) 15:33, 16 June 2015 (UTC)

Also, I'm confused now, what means "If you would like to request the Citoid automatic reference feature for your wiki, please post a request in the Citoid project on Phabricator" as said on https://meta.wikimedia.org/wiki/VisualEditor/Newsletter/2015/June? It's already done on fiwiki or is it something else? Stryn (talk) 16:31, 16 June 2015 (UTC)

This was just so that people who aren't as a keen as you to just try things out can get help. :-) You're now done, it looks like. Jdforrester (WMF) (talk) 16:40, 16 June 2015 (UTC)

No description of the Citoid extension

Could someone please add a description of what the Citoid extension actually does? Right now it just says the extension "aims to provide use of the citoid service to VisualEditor". What does that mean? Does it just provide an API? Does it provide UI? Kaldari (talk) 17:02, 16 July 2015 (UTC)

@KaldariI'm working on a user-friendly tutorial/help page on Citoid at Polish Wikipedia. It's in Polish, but you could try using some automatic translation - or simply copy the structure and graphics. I know it's not much, but it's a starting point at least. Check w:pl:Wikipedysta:Halibutt/Citoid. Halibutt (talk) 00:02, 12 August 2015 (UTC)

Translator tutorial

I'd like to work on a translator for many of the reliable sources I use at enwp, but I'm really lost at where to begin. I can find my way around the code, but is there any tutorial on making a translator for any given webpage (start to finish)? czar 04:58, 31 July 2015 (UTC)

On that site, there are a couple of guides (1 - 2). Elitre (WMF) (talk) 11:03, 3 August 2015 (UTC)

Adding OCLC number to populate citation template

We're at the Wikiconference USA session on OCLC, and there's a request to add the OCLC number to the citation generation template in the source editor editing bar. One person here thinks this task might be part of the Citoid team's portfolio, and says if it's not, you'd know who does this.

We're all really enthusiastic about how the ISBN can generate quick citations, and hoping the OCLC number could be made to do the same thing. Djembayz (talk) 20:01, 9 October 2015 (UTC)

maybe should be at https://en.wikipedia.org/wiki/Wikipedia_talk:RefToolbar Bawolff (talk) 20:22, 10 October 2015 (UTC)

Thanks for your note. This is on the list for possible future improvements.

Tracked in Phabricator
Task T115248

Whatamidoing (WMF) (talk) 17:39, 27 October 2015 (UTC)

Problem with "publisher" parameter

Citoid appears to be using the rel="publisher"code on web pages to generate a value for the "publisher" parameter in a citation. That's wrong - such code on the web page is providing a URL, not a publisher name; the URL is for a plus.google.com page. Further discussion is here: https://en.wikipedia.org/wiki/Wikipedia:VisualEditor/Feedback#cs1.7C2_.7Cpublisher.3D_parameter_and_google.2B_links John Broughton (talk) 21:45, 29 November 2015 (UTC)

Hi,

We have a phabricator task for that here: https://phabricator.wikimedia.org/T118773 Mvolz (WMF) (talk) 08:36, 30 November 2015 (UTC)

Potential webinar on Zotero translator coding

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

Behind Citoid and likely any of our future citation parsing services is are a series of Zotero "translators". I recently wrote one and it had a bit of a learning curve. However, Sebastian from the Zotero project has offered to give a webinar on translator coding if there is interest. It would include basic scraping by manual code and the streamlined "Framework" (less coding). I'm trying to gauge interest, so please leave a note on my enwp talk page with the nature of your interest (e.g., are you just curious or do you plan to write translators for specific sites?) Please share with other language Wikipedias and let me know? czar 16:58, 6 December 2015 (UTC)

Hey @Czar. My colleagues and I were wondering whether Sebastian could be interested in turning the "webinar" into a Tech talk? We could make it happen around February or March, if so. Thanks for your help! Elitre (WMF) (talk) 16:01, 14 December 2015 (UTC)

Here is Sebastian's webpage and Twitter (not sure if the Northwestern address still works). Let's try Twitter czar 18:24, 18 December 2015 (UTC)

I confirmed that Sebastian's email in the linked CV is the right one, and he said Feb/March is fine czar 16:39, 19 December 2015 (UTC)

Thanks! Elitre (WMF) (talk) 23:03, 4 January 2016 (UTC)

Learn how to write a translator! A related Tech Talk is happening on Feb 29th. Join us! Elitre (WMF) (talk) 10:29, 18 February 2016 (UTC)

The discussion above is closed. Please do not modify it. No further edits should be made to this discussion.

Formatting ISO in enwp's CS1

Thought you might be interested in this discussion at enwp, which will discuss how we implement custom date formatting based on Citoid's choice to use ISO ymd czar 16:45, 19 December 2015 (UTC)

Would it be possible to get an official answer to some of the questions raised there?

Date ranges in ISO 8601-like format are not currently supported by WP:DATESNO so are not supported by cs1|2. If Citoid needs to specify a date range will it use the ISO 8601 format? If Citoid needs to specify a season, will it use ISO 8601 format? Is there an ISO 8601 format for seasons? What about dates outside of the Gregorian calendar; does Citoid support those? In what format? In a conversion, how do we specify use of abbreviated month names?

czar 23:40, 19 December 2015 (UTC)

@Mvolz may be able to tell you what the current and/or future plans are for date ranges (e.g., May–June 2015). DATESNO doesn't appear to address ranges at all. I'm not sure what "not supported by cs1" means; I hope that it doesn't mean "emits error messages if given an accurate publication date for a bi-monthly magazine".

On the question of Julian calendars, are you aware of any URLs that report publication dates that pre-date the creation of the world wide web by several centuries? Whatamidoing (WMF) (talk) 02:03, 24 December 2015 (UTC)

cs1|2 supports a subset of date formats allowed by the various sections of Wikipedia:Manual_of_Style/Dates_and_numbers#Dates.2C_months_and_years of which Wikipedia:Manual of Style/Dates and numbers#Ranges is a portion. DATESNO is a term often (mis)used to refer to the dates portion of that MOS page because it is a redirect to en:Wikipedia:Manual of Style/Dates and numbers#Unacceptable date formats. cs1|2 compliance with MOS is specified at en:Help:Citation_Style_1#CS1_compliance_with_Wikipedia.27s_Manual_of_Style.

cs1|2 do not support ISO 8601-like date-range-formats because MOS only supports YYYY-MM-DD single-date style. At present, cs1|2 will emit an error message if given a YYYY-MM-DD/YYYY-MM-DD range.

It is very common for periodicals to date issues according to season. How does Citoid report those dates? What about dates that are proper names? cs1|2 allows Christmas YYYY as a date because there are periodicals that use that date. How does Citoid report that kind of date?

I don't know how Citoid gets its dates. If Citoid only works with on-line publications where the date of the cited material is guaranteed to be Gregorian, then there is no issue. But, if Citoid can report dates in the Julian calendar then that may or may not be a problem. Trappist the monk (talk) 10:29, 31 December 2015 (UTC)

(I've fixed the links above. I believe that problem was that you had used [[en:foo]] instead of [[:en:foo]] - i.e. interlanguage links instead of interwiki links. :-) For future reference, you can edit your posts (and do other actions) by clicking on the "..." icon at the top corner of each post (and the topic as a whole). You can also flip between wikitext and visualeditor, by clicking the icon in the bottom corner. Hope that helps.) Quiddity (WMF) (talk) 21:44, 14 January 2016 (UTC)

That didn't come out right. Apparently I can't edit it to fix it either. Flow is, I guess, not ready for prime time. What I really wrote is at:

Help_talk:Citation_Style_1#Automatic_ISO_conversion

on the English Wikipedia

Hope this works. Trappist the monk (talk) 10:42, 31 December 2015 (UTC)

@Trappist the monk

[3] czar 03:41, 31 December 2015 (UTC)

Confusion with older dates

The project page indicates all dates will be in what it calls the "ISO" format, that is, yyyy-mm-dd. I infer this is the ISO 8601 format. This format requires all dates be in the Gregorian calendar, and that any use of the format before the year 1583 be agreed to by data exchange partners. Use of ISO 8601 format precludes pre-1583 sources. Also, any citation to a daily newspaper published in a country that retained the Julian calendar longer than Roman Catholic countries [such as the United Kingom (1752) or Greece (1923)] would have to be converted by the editor from the Julian date stated in the newspaper to the Gregorian calendar for entry into Citoid, and then converted again by the citation in a template in the Wikipedia in which the citation occurs (such as the English Wikipedia, which requires the use of the Julian calendar in articles set at a time and place where that calendar was in force). Jc3s5h (talk) 18:37, 21 December 2015 (UTC)

This is a pretty complex problem! At some point (I believe) we are going to store citations in a backend which will probably be wikidata- this is a relatively solved problem in wikidata which allows you to specify which calendar the date is in. Currently we don't really know for sure what calendar the date provided is in anyway because they're often scraped from data providers with unknown rules. So for now I think we will table this until then.

Feel free to add a bug report for it in phabricator! https://phabricator.wikimedia.org/project/view/62/ Mvolz (WMF) (talk) 19:08, 21 December 2015 (UTC)

Mvoilz, thanks for your response. I have created a bug report: T122071: Guarantee valid dates.

I agree the Wikidata format could accept dates for year 1 and later. Although 1 BC and earlier are still a mess, editors will hardly ever cite a work that early; most likely they will cite a modern scholarly edition of any work that old.

A fault with the Wikidata user interface, but not with the data structure, is that the user interface does not provide input or output of time zones. For web-based sources involving current events, the time zone associated with a date is significant. This is hard to detect through automation, and few editors have the skill and/or motivation to figure it out. Jc3s5h (talk) 20:06, 21 December 2015 (UTC)

@Jc3s5h, can you post an example of a URL that actually has this problem? Whatamidoing (WMF) (talk) 01:43, 24 December 2015 (UTC)

I do not have a specific URL in mind, but I will give you a plausible fictitious scenario:

A major terorrist event happens in London at 02:00 March 7, 2017 UTC. A television station website in New York covers it, and dates the story in local time, March 6.

A Wikipedia editor cites the story using the software we are discussing, giving the date just as March 6, 2017, without stating a time zone. The citation is stored in WikiData, which is incapable of storing a date without a time zone, so it stores the date as 2017-03-06T00:00Z with precision set to 1 day. This is interpreted as the story was published between 00:00 hours and 24:00 hours March 6, 2016. Therefore the New York television reporter must be psychic (or worse, involved in the terrorism) because the television station published the story before the event happened, according to Wikipedia and WikiData. Jc3s5h (talk) 21:23, 7 March 2016 (UTC)

I meant a URL that has a problem with a date from, e.g., 1655 CE.

For the timezone problem, (a) breaking news reports are always primary sources, so let's hope that it isn't cited for very long, and (b) this problem would exist equally with an accurate hand-written citation. (Perhaps you want to proposed a new "time of publication" parameter for CS1.) Whatamidoing (WMF) (talk) 16:39, 14 March 2016 (UTC)

When I started the thread, it wasn't clear to me that Citoid would only be used to aid in obtaining publication dates from URLs. So long as that there is no effort to expand it to printed sources, URLs with publication dates in the Julian calendar will seldom be a problem (perhaps a few religious websites). As for hand-written vs. Citoid dates of publication for breaking news, the hand-written citation will ordinarily have just a date, with no timezone indication. The natural interpretation will be the local time in the place of publication. Dates stored in WikiData, at present, must have a time zone (and if entered through the user interface, the time zone must be UT). If you use WikiData you can't help but make an assertion about the time zone, and that assertion has a good chance of being false.

Of course, you could ask the WikiData folks to provide a local time data type for you, but they don't seem in any big rush to fix their date problems. Jc3s5h (talk) 21:51, 19 March 2016 (UTC)