Selenium/Ruby/Browser testing user satisfaction survey

18 people participated in the survey. Questions got from 8 to 16 answers.

Jenkins
Participants of the survey are pretty happy about stability of mwext-mw-selenium job. Most of them think they are fast enough. That was not a surprise to me, since I share that view.

What surprised me is that they are also mostly happy with stability and speed of selenium* jobs, since some jobs are failing with hard to reproduce failures, and some jobs take hours to run.

Most of the people also know how to use continuous integration entry points for Ruby (Rake) and JavaScript (Grunt).



Jobs are stable enough (n=16).

Jobs are fast enough (n=16).



Jobs are stable enough (n=16).

Jobs are fast enough (n=16).



Rake (Ruby) (n=16).

Grunt (JavaScript) (n=16).

Ruby
I was surprised to see that (slightly) more people are happy with Ruby testing tools (mediawiki_selenium, Cucumber, RSpec). The majority of the people did not have strong feelings. I was expecting more people being unhappy with the Ruby testing tools.



Selenium+Ruby framework mediawiki_selenium (n=15).

Cucumber (n=14).

RSpec (n=15).

JavaScript
JavaScript testing framework Malu is in it's early development and this survey was probably the first time some of the participants have heard about it. That might explain the highest &lt;nowiki&gt;:|&lt;/nowiki&gt; (or 3) response about Malu. 11 participants (69%) gave that response, the highest number of votes that any other question got. Participants were in general positive towards Mocha and QUnit.



Selenium+JavaScript framework (Malu) (n=16).

Mocha (n=15).

QUnit (n=15).

Selenium


Writing tests (n=16).

Fixing failed tests (n=15).

Getting help


mediawiki.org (n=16).

IRC (n=16).

Phabricator (n=16).

Mailing list (n=16).

Documentation


Getting started documentation (n=16).

Documentation on how to do testing (n=16).

I need help


I need help with getting started with testing (n=9).



I need help with fixing failed tests (n=8).