Browser Test Automation for VisualEditor (Outreach Program for Women - Round 6)
Name: Rachel Thomas
Email: rachelqa99 [at] gmail [dot] com
IRC handle: rachel99
Location: Boston, MA (UTC -4:00 )
Timezone: EDT (UTC -4:00)
Typical working hours: Flexible, but most likely would work 8:30am-4:30pm EDT
As the major issues in VisualEditor have been fixed, the VisualEditor has recently been deployed to 14 non-English Wikipedias so that users can become familiar with the editor. It was already been deployed to the English Wiki in December 2012. VisualEditor will eventually become the default editor for Wikipedia, and thus it is important that its current features are tested thoroughly. These tests can then be run as regression tests to make sure nothing breaks as more new features are deployed to VisualEditor in future phases. As the editor is finally stable enough to have automated tests written and run against it, now is a good time to set up test cases that confirm VisualEditor is working the way it should.
To construct a battery of web automation tests for the thorough testing of VisualEditor. These would be Accepted Test Driven Development tests written in Cucumber and falling under the same framework as the currently running web automation tests. The Cucumber tests would be coded using Ruby Page Object Design, and then run under Jenkins. The tests should cover all the current features of VisualEditor.
A top-level overview of what needs to be done:
- Come up with the features needed to be tested for VE functionality
- Add these VE features to current QA/Backlog page under VE section
- Write Cucumber test cases for each feature
- Run each test case, and write Ruby code to execute it properly
- Check into Gerrit, and have it approved by Chris or Zeljko
- Add it to Jenkins
Some examples of features needing to be tested:
Test that VE works in User and Article namespaces
Test that VE doesn't work for talk pages, templates, categories, etc.
Test Vector skin to make sure VisualEditor is working
Test Monobook skin to make sure VisualEditor is working
VisualEditor works in the most modern versions of Firefox, Chrome and Safari
Test that editing references or templates that cannot be edited yet, will be cross-hatched, have a tooltip and can be deleted
- multiple character sets
- right-to-left languages like Hebrew
- Wikimedia projects that may deploy VE in the future (i.e. Wikisource)
Consider open bugs https://bugzilla.wikimedia.org/buglist.cgi?list_id=179710&product=VisualEditor&query_format=advanced&resolution=---&resolution=LATER&resolution=DUPLICATE&order=bug_id%20DESC&query_based_on=
This a tentative timeline, and is subject to change. Note that I believe I overestimated how much time it takes to code Cucumber tests.
|May 8- June 6||Continue to code features for QA web automation tests in the current backlog, in order gain more familiarity with Ruby and Selenium. Will seek help from Zeljko when needed.|
|June 7- June 14||Work with Visual Editor to develop expertise in using VE.|
|June 17 - June 24||Come up with other features to test in VE as I look at VE more closely. Also add feature testing based on critical bugs found in Bugzilla. Add the complete list of VE features to be tested to current QA/Backlog page under VE section. Sequence the tests from most important to test to least important to test, getting input from Zeljko and Chris. Divide these tests into chunks, and prepare to complete one chunk before doing the next chunk. Start with the most important chunk first.|
|June 25 - July 2||Take the first chunk of tests and code them into Cucumber language. I suggest chunks should be about 4 -6 tests, depending on complexity.|
|July 3- July 31||Run each test case in Cucumber, and write the corresponding code in Ruby. Check code into Gerrit for code review after each Ruby sequence for the test script is written. When each test case is complete, and successfully passed code review, add it into Jenkins to be deployed continuously as a automatic test.|
|August 1 - August 10||Take the second chunk of tests and code them into Cucumber language.|
|August 11 - Sept 2||Write the corresponding code in Ruby. Deploy in Gerrit for code review and Jenkins.|
|Sept 3 - Sept 23||Clean-up period for any remaining tests to be written and coded, including important bugs added to Bugzilla in past 2 months. Buffer period.|
Past Open Source Experience
I have been volunteering at Wikipedia for the past 3 months in the Browser web automation area to gain more experience in QA automation, as well as to engage in working on Open Source. I have always been a supporter of free and open software, so I am really excited to be working with the Wikimedia Foundation. I have really enjoyed my time volunteering here. I have been mentored by Chris Mcmahon and Zeljko Fillipin as I have gone through the process of learning the environment at Mediawiki, and then learning Cucumber and Ruby to write and code the test cases. I have done the test case for the SpecialNewsFeed feature and search feature in Cucumber, and have also coded the corresponding search.feature script in Ruby. Zeljko and I paired up to work on the search.feature script, and using IRC, etherpad, and our own local running environments we were able to complete it successfully
I have also learned how to use git and gerrit in order to contribute tests for Search that we are running right now.
Browser automation skills and experience are not specific to any given test target, so my contributions need not be limited to VisualEditor or Search or any other particular target, but VisualEditor is just a particularly good match for browser tests, and those tests do not exist right now. As VisualEditor testing is now becoming a higher priority, it seems like a good time for me to start writing tests for it.
I have a B.S in Mathematics, Minor Computer Science from SUNY Albany and have over 20 years experience working in IT at various positions including Linux System Administration, Release Engineer, QA tester and Technical Support Engineer. As the lead QA tester for a small startup, I developed test cases, performed functional, regression and integration tests, and filed bug reports in Bugzilla.
I have always been interested in Quality Assurance, particularly in QA automation. I really enjoy coding and have done coding in Shell scripting, Perl, and Java throughout my career. I am adept at learning new coding languages quickly.
I hope to pursue a career in QA automation in the future, while still staying involved as a volunteer in Wikipedia. I would like to continue to help the Wikipedia QA automation group by adding test cases to their backlog in the future, or by doing other volunteer work of interest.
I am also interested in community building, particularly among women in technology, and I was a member of Society of Women Engineers for many years. During that time, I managed a committee of volunteers at SWE-Boston that were responsible for re-doing the SWE Boston website. I received the "Outstanding SWE-Boston Member Award" in 2003 for my work in getting the new website constructed.
Other community organizations that I am currently involved in are my temple, where I am on the Board of Directors, and the Software Quality Group of New England, where I am a member.
- completed my Visual Editor Charter which lists the areas of Visual Editor that I will be writing test cases for
- familiarized myself with new features in Visual Editor
- Found 2 Visual Editor bugs, and reported them on Bugzilla
- am working on automation scripts to test the references feature of Visual Editor.
- have completed the VE_references feature file that sets up the tests for VE references
- worked in Cucumber, Ruby and with Page Objects to translate the cucumber language to Ruby for several cases.
- Pair programmed with Chris McMahon several times to learn more about Cucumber/RubyPage Objects.
- Reported a problem running my tests on test2 wiki
- continue doing exploratory testing on VE to look for bugs
- participate on QA mailing list
- attended OPW kickoff meeting, 2 Engineering Community Team Meetings, and GSOC/OPW all hands meeting.
- make improvements to my Visual Editor Test plan, and add new features to be tested
- write Visual Editor links test (first scenario), and have it merged with master branch
- write test for testing VE for logged-in user and anonymous user
- add to VE links test a test for internal links, and links that don't exist
- start work on VE test for headings
- test Mediawiki-vagrant Windows install
- help out with QA automation workshop, and participate on QA mailing list
- attend 4 ECT meetings
- file several Bugzilla bugs on Visual Editor
- polish my Visual Editor Test plan
- write a VE headings script to test the different VE headings, and get it merged to master
- start work on VE script to test bullets, numbering, indents, outdents
- train a QA volunteer on writing cucumber/ruby/selenium in a pairing session
- participate on QA mailing list
- after VE scripts are moved to a new repo, copy the new repo to my computer, and set it up to work with Gerrit
- create login on Sauce Labs to view instances of failed tests
- attend 3 ECT meetings
- write a blog post on the tool Cucumber
- finish work on VE script to test bullets, numbering, indents, and outdents
- participate on QA mailing list
- attend wrap up OPW meeting on IRC
- attend 2 ECT meetings
- write my wrap up blog post