Test Automation - Tools and Practices

05 May 2014

Test Automation - Tools and Practices

Ok after stumbling upon various IRC channels, having talk to some top most QA testers, Automation geeks, and API developers this week I am here with a few thoughts. So this week was a part of my research period, My main goals for this week were :

Goals for this week

First Task

This week I will mainly concentrate on exploring and researching about the model of the testing that we are going to use for this application. Whether a change is required or not ? If yes are we in a position to make that change ? If no , explore more about other methods that can tackle the problem for database object modelling.

Explore more about the number of tests that require modelling of database objects to make sure that do we really need FactoryBoy and how will it be useful.
If we need FactoryBoy then can we merge the code for tests with the application code ? Explore advantages and disadvantages for merging the code.
If we cannot merge the code are there any other alternatives that will allow better modelling of Database objects ?

After answering these questions with concrete arguments and proofs I would like to discuss it with the team and may be have a poll ?
Some of the references I am exploring:
http://sqa.stackexchange.com/questions/2143/keeping-automation-code-away-from-application-source-code/
http://sqa.stackexchange.com/questions/1260/whats-the-recommended-practice-for-separating-unit-tests-from-integration-tests
https://groups.google.com/forum/#!topic/selenium-users/2KH8v7vVBs4

I will also like to explore the problems and limitations(if any) that might arise if we are ready to use FactoryBoy using past examples from Rails applications testing using FactoryGirl and see if we have a solution to them.

Second Task
The second task I will like to take up this week is to discuss with lizzard, pragmatic, rbillings, marcia and team to come up with a more specific design for template for task creation as discussed in meeting yesterday.

The template would later help to autofill the tasks after scraping bugs data from other API’s,
inturn helping in Task creation automation that I proposed in my proposal.

Along with this I will use this time to become familiar with OneandDone codebase.

So I have roughly achieved most of the tasks on the list, and have explored many resources related to Test Automation.

Here are some resources that I went through and found really very useful

1 Talk on testing django applications using pytest by Andreas Pelme(who himself is an author of pytest-django) Link to Video
2 Talk on Integration and Functional Testing using LiveServerTestCase and Selenium Link to Video
3 A Presentation on Testing and Django Link to Slides

I also came across some of talks from very experienced Python developers who highlighted the advantages of pytest for easier and better testing.

1 Holger Krekel: Improving automated testing with py.test Link to Video

2 Andy Todd: Why I use py.test and maybe you should too Link to Video

Also while talking to a one of the pytest developers I came across a completely new framework for automated testing : Robot Framework

I believe this is as far the most advance tool for acceptance testing that I came across which integrates with Selenium to provide better test coverage and test outputs as shown in one of videos Link to Video

But our existing test suites are quite different from the Robot Framework and we are more concerned about Integration + Functional testing. So in our case it might not be of much use, but really it was a fun to get to know more about this framework.

Exploring these tools and talking to so many developers, I am in an opinion to keep the test suite with application itself so that we can use FactoryBoy for better data population along with pytest fixtures but since this time the tests will be with the application itself we have an option for using LiveServerTestCase or the original pytest-mozwebqa plugin.

I am still not sure about how pytest-mozwebqa will integrate with Fixtures and LiveServerTestCase if we use it, but I am looking forward to talk with my mentor and team about it and finalize the tools by next week.