Topic on Talk:Content translation

As far as I know, the language team uses Wikimedia Phabricator team to handle active tasks. However, the translation tool links here for any comment, suggestion, etc. That means there is two sources of input to watch for a team with already heavy workload. Additionally, the one input that gets fresher feedback and bug reports likely is less watched since it is not the one actually used in their workflow.

Example: there were > 10 users here commenting yesterday that there has been a regression with the tool failing to load stored translation under process The first reference in this page tracked back to April 3rd morning. However, last night it did not appeared in the phabricator's tasks, despite having sections for that. It seems it was later reported in https://phabricator.wikimedia.org/T249400 (Sat, Apr 4, 5:30 AM) and seems to have been fortunate, since a user already familiar with phabricator also run into the bug.

I guess phabricator may not be appealing for regular users but some mirroring/triaging of threads here seems to be necessary. Several bugs (this one, categories, etc) appears to be regressions and there is already discussion about the need for more systematic regression tests for changes. I think that unless the info on this talk page is integrated in Phabricator, the technical team will lack relevant info to prevent regressions or unintended effects of changes.

How could it be done?

idem

Phabricator is the way to declare directly internal bugs of the tool under a ticket number: 'I can no longer do this', 'Option XXX is not working' 'I loose my translations' 'I got unexpected error XXX' .... and is rather used to reach developpers on behind (regressions or unintended effects of changes as you say). Talk pages here are rather to discuss on 'how do I do that' or 'Should it not be ....' etc. Of course discussion readers will not correct the bogues but they can create a task forward in Phab according to the severity they judge. I think product-maintainers should be designated to anwser to these urgent issues. On the other side we cannot forward each topic on Phabricator since they need a triage.

I am aware volunteers are grouping same problems as we had a rough week or so, so appreciating your commitment. While I am not sure if increased number of CX2 users overflew the comfort zone for the system/database.

I thought the purpose of this thread is to discuss if technical or bot work could filter to-dos to Phab. We have two issues here: (a) In short, is this a CX2 specific Village Pump or not; (b) CX2 users are not guided properly how to _report_ issues on the documentation page I am afraid. Or general basics of editing as well as editorship.

(1) Suppose we have passed the phase any feedback is wanted, but I hope at the same time, we give our input in seeking for some magic remedy. FAQ could be expanded so that it be triggered to guide similar new feedback to FAQ, instead of no answers for weeks.

(2) There should be an automated notice if a slack/backlog is causing such problem as users don’t find their work saved, or CX2 hicks up and tries to load forever. Is it very complicated to put out a notice of overload?

(3) We better start an educational Dashborad and support CX2 users master the system itself and their editorship. Filtering CX2 user input to Phab requires understanding of both worlds. On the other hand, we encourage learning stage novice editors to apply CX2, if they are not comfortable writing from scratch. They need to learn how editing works, while themselves use CX2 as translators. Is the support threshold functioning then?

(4) FYI, more and more new pages have been created via CX2, great news. But sucking up those translators’ time (correction continues to the end of input by Omotecho 00:32, 14 April 2020 (UTC)) who care to support novice CX2 users esp on jawp, trying to communicate CX2 users that they need to understand and share responsibility to make an article, not just throw in _Cx2 fabricated paragraphs_ and move on to next target. It’s not a arcade game you trophy numbers of articles you created. Negativeness had piled up to the extent to RfC if boycott CX2 output above certain percentage of direct MT output by filtering tags.

The problem involves new editors active on producing articles via CX2 are not guided how an editor must behave onwiki projects. Who is the instructions provider or coach? A Community wishlist 2020 matter?

Thanks, @Omotecho for your understanding. Obviously, we all now the people working on software are very few for the workload, so this thread was aimed to brainstorm how the process could be streamlined. I'm leaving a long message so I would underline the specific proposal.

My concern is that a tool user is often not a software engineer. They are users and as such they work unconsciously as beta testers: they use the tool both right and wrong and find a lot of outcomes. They are also "free" compared to hired QA people. However, they are often not knowledgeable enough about how to go from "it is broken, help" to a useful bug report.

But I feel the interface and work flow does not help with that. A user that finds a problem usually don't know what to do. After some cursory checking, they find a "leave a message" that, by elimination, is their only resource. So, first, the interface is no intuitive for a bug report to be placed.

This way, they go to a discussion page which is also used for other purposes, which does gives some advice. Reporting a bug is just one of the thing somebody can do here and is usually do anarchically. There is a lack of guiding, either by manuals of how to properly report a bug or by any telemetry (error report generation from a given translation gone wrong). If you are familiar with phabricator, there is a suggestion to go there but for most reporters it will be a too complex mechanism. So, secondly I think there is not a lot of assistance for a non-technical user to leave a technical description of its problem. Telemetry for bug reports may not be easy to implement but it may be helpful.

Then, I note the language team uses phabricator in their daily routine, whilst this forum may take a few days to be reviewed. I obviously understand reviewing tens of "It does not load" "Same" "+1" is not productive from them, specially if the alternative is a professional to-do list of task. However, this is a system design issue: having two platforms means you need to check to two inputs and have additional work integrating them. With work overload, it should be avoided when possible

My personal opinion is that interface and organization should be aimed at simplifying the manpower needed. Bugs/problem should be reported apart and the process should guide as much as possible the reporter to provide the critical information. Technical staff should not have to navigate in a non-curated forum to find information that may be useful to debug problems.

Additionally, it also generates poor user experience. Users are led to a forum to report issues and do not see it solved, which does not incentive constructive discussion but piling into that "me too". Here I point @Chronus did register in phabricator to give feedback. But it shows a bottleneck: from more than a dozen people reporting a bug, just one keep going up to the platform used by the team.

The past event got me thinking: the team in phabricator may be getting around 5-10% of the potential info here (albeit with higher mean quality there). Having had 10-20 reports for 1-2 days with no answer and getting the same issue triaged as critical and solved in a very short notice, I felt there is a bottleneck in bug discovery compared with bug solving. There was a problem in a production environment that took as much to get identified as to get solved.

That feeling aligned with comments in phabricator about recent regressions or discussion about QA testing. The more widespread the tool is deployed, the higher the pressure and workload be. Getting information flowing smoothly may prove helpful for the staff to work better. Increasing the tool users until they flood this board and useful information is lost there means the need for either more manpower to curate the message or better reporting to navigate the threads here.

Thank you so much you made it much readable for both people with beginner and advanced tech knowledge. Could you point me to where I can learn about telemetry? I wish, too, no tech staff waste their time on shifting through raw user input, and that’s as you suggested imho we have a hole in loop.

I imagine a flowchart which will deliver we user’s “It doesn’t work” to solutions or somewhere. But all of that flowchart making needs statistics what type of bottleneck we users face here. Stages on the flow could have things like:

• Either to invite us for user training like Dashboard and they will upgrade themselves with CX2 how-tos;
• Link to FAQ where we users find options how we modify our translation or sentences in local language that the system could swallow.

As I am a translator, sometimes on mechanical documents all very unfamiliar to me, it is OK if I learn something or some rule how things work. Like which button to push on a new remote controller and make the machine operate the task the users’ manual lists I can choose, I wish to maneuver by pushing a button. Maybe it sucks that I feel like I am a button to be pressed, but I am not sure what performance is expected to happen. (;

What I would advise (this discussion was aimed to check other users think ):

• Translation interface should have clear, distinct "report a problem" link (for example, after leave a comment)
• That link should automatically capture relevant info, in addition to the message left by the reporter. By such info I think: Page being translated, from which language into which langue, translator used (apertium...). I'll bet that Mediawiki, being in php if I recall correctly, could use some get_browser() to get the browser/OS version from the user. Console logs may not be possible to capture that way, but any way to facilitate its capture will be valuable. Guides for user to report better, as you suggest, will be very helpful.
• Such reports should be published in a "bug dashboard" to be triaged. That board should be separated from comments, discussions, etc. I like your idea of a bot to mirror comments left here and in phabricator. This can be used as a more "user friendly" environment whilst the phabricator board may be more effective. What I find important is the information flow. Developers should not loss time checking this to create items in https://phabricator.wikimedia.org/project/board/776/ and reporters should not have to be familiar with phabricator to find if the problem is being addressed.

By telemetry I meant this: https://en.wikipedia.org/wiki/Telemetry#Software I guess it should be done carefully for privacy but I think the people working on this will need some proper feedback to keep the environment bug-free.

About telemetry, that would surely be genius. That was the goal of Sentry (phab:T91649, ping @Tgr: fyi about the good usecase of Content translation errors). It looks like deploying logstash for client-side (already used for server-side errors) is the next goal (phab:T217142).