Keyword-driven testing, also known as table-driven testing or action-word testing, is a software testing methodology for automated testing that separates the test creation process into two distinct stages: a Planning Stage, and an Implementation Stage.
Suggested Approach1:
1) Identify the list of objects of the Application and most frequently used set of operations performed on them
2) Spy those objects to identify the native class of the tool and also find out native methods of the tool for those objects
3) List out most frequently used operations/methods performed on different classes of the objects
4) Classify the methods listed in step3 to the keywords
5) Maintain 3 excel sheets as below
i) Testdriver-> an excel sheet which contains list of test cases to execute between start and end keywords
ii) Testcase-> an excel sheet which contains keywords/actions as well as object name(the alias defined in step iii)
iii) Objects-> an excel sheet which has list of objects and properties which can be arrived by spying on the application, for each row of an object given an alias, so that it can be referenced by the test case excel sheet
6) Write a main function which iterates through the Testdriver excel sheet, which has reference to testcase excel sheet and objects excel sheet .For each test case it places calls to functions as below
func1: iterates through Objects excel sheet and builds a hashmap/dictonary of the object and its properties.Basically buids a keyvalue pair
func2: iterates through which TestCase excel sheet which has switch case on keywords, each case statements intern calls a method defined for the keyword
7) Each method of keywords should have a detailed switch case on the object types, which in turn has a code to perform operations on AUT
Advantages of Suggested Approach1:
1) No need to have advanced knowledge in Automation tool
2) Consistency of the script development is achieved across the project
3) Easier to maintain scripts
4) Easier to maintain test data
5) Faster development of scripts
6) Easier to schedule the scripts with out any test management tool
7) Can be used across all applications,with little modifications to keywords. Provided the automation tool supports record and play back on the Applications
DisAdvantages:
1) It restricts individual learning on tool
2) Little bit tough to debug as there is no code except keywords in it
3) Need to remind keywords and there usage
4) Little bit tough to handle conditional execution of scripts,alternative is write code in a functions and call the function in test case excel sheet using predefined keyword
Sunday 15 August 2010
Saturday 14 August 2010
Data Driven Framework
Overview
Data-driven testing is the creation of interacting test scripts together with their related data results in a framework used for the methodology. In this framework, variables are used for both input values and output verification values: navigation through the program, reading of the data sources, and logging of test status and information are all coded in the test script. The logic executed in the script is dictated by the data values.
When to use?
If an application has too many screens which needs lot of input from user go for data driven framework.
Suggested Approach :
Step 1) Maintain test data for each screen of the ui in separate excel sheet, maintain a separate row of data for each test case and data for each test case should refer to the same row of all the sheets.
Step 2) Write a separate function for each screen which retrieves values from excel sheet and inputs data in to Application
Step3) In Main script, insert a calls to all the functions written in step2, initializing the rownumber of the excel sheet so that all functions gets data from the same row of different sheets
Demonstration of Suggested Approach with example:
If there is a manual test case which has validation on ui or calculation at the screen 7 for example
1) Write a new main script, initialize row number to the row in excel sheet which maintains data needed for execution of test case.
2) Assuming that you have written individual function for each screen which gets data from different sheets, put the function calls as below
call func_screen1
call func_screen2
call func_screen3
call func_screen4
call func_screen5
call func_screen6
Write code for the screen 7 to perform verification on screen 7.
Note:Don't put any validations in the functions apart from getting data from excel and inputting in to screens.
Advantages of using the Approach:
1) Automation team need not know the functionality of complete application
2) Even a starter in the team who has experience in tool, can quickly start working on the scripts
3) Faster development of scripts
Data-driven testing is the creation of interacting test scripts together with their related data results in a framework used for the methodology. In this framework, variables are used for both input values and output verification values: navigation through the program, reading of the data sources, and logging of test status and information are all coded in the test script. The logic executed in the script is dictated by the data values.
When to use?
If an application has too many screens which needs lot of input from user go for data driven framework.
Suggested Approach :
Step 1) Maintain test data for each screen of the ui in separate excel sheet, maintain a separate row of data for each test case and data for each test case should refer to the same row of all the sheets.
Step 2) Write a separate function for each screen which retrieves values from excel sheet and inputs data in to Application
Step3) In Main script, insert a calls to all the functions written in step2, initializing the rownumber of the excel sheet so that all functions gets data from the same row of different sheets
Demonstration of Suggested Approach with example:
If there is a manual test case which has validation on ui or calculation at the screen 7 for example
1) Write a new main script, initialize row number to the row in excel sheet which maintains data needed for execution of test case.
2) Assuming that you have written individual function for each screen which gets data from different sheets, put the function calls as below
call func_screen1
call func_screen2
call func_screen3
call func_screen4
call func_screen5
call func_screen6
Write code for the screen 7 to perform verification on screen 7.
Note:Don't put any validations in the functions apart from getting data from excel and inputting in to screens.
Advantages of using the Approach:
1) Automation team need not know the functionality of complete application
2) Even a starter in the team who has experience in tool, can quickly start working on the scripts
3) Faster development of scripts
Automation Basics
Test automation is the use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions[1]. Commonly, test automation involves automating a manual process already in place that uses a formalized testing process.
There are two general approaches to test automation:
One way to generate test cases automatically is model-based testing through use of a model of the system for test case generation, but research continues into a variety of alternative methodologies for doing so.[citation needed]
What to automate, when to automate, or even whether one really needs automation are crucial decisions which the testing (or development) team must make. Selecting the correct features of the product for automation largely determines the success of the automation. Automating unstable features or features that are undergoing changes should be avoided.[2]
Code driven test automation is a key feature of Agile software development, where it is known as Test-driven development (TDD). Unit tests are written to define the functionality before the code is written. Only when all tests pass is the code considered complete. Proponents argue that it produces software that is both more reliable and less costly than code that is tested by manual exploration. It is considered more reliable because the code coverage is better, and because it is run constantly during development rather than once at the end of a waterfall development cycle. The developer discovers defects immediately upon making a change, when it is least expensive to fix. Finally, code refactoring is safer; transforming the code into a simpler form with less code duplication, but equivalent behavior, is much less likely to introduce new defects.
A variation on this type of tool is for testing of web sites. Here, the "interface" is the web page. This type of tool also requires little or no software development. However, such a framework utilizes entirely different techniques because it is reading html instead of observing window events.[citation needed]
Another variation is scriptless test automation that does not use record and playback, but instead builds a model of the application under test and then enables the tester to create test cases by simply editing in test parameters and conditions. This requires no scripting skills, but has all the power and flexibility of a scripted approach. Test-case maintenance is easy, as there is no code to maintain and as the application under test changes the software objects can simply be re-learned or added. It can be applied to any GUI-based software application.
One must keep satisfying popular requirements when thinking of test automation:
There are various types of frameworks. They are categorized on the basis of the automation component they leverage. These are:
Overview
Although manual tests may find many defects in a software application, it is a laborious and time consuming process. In addition it may not be effective in finding certain classes of defects. Test automation is a process of writing a computer program to do testing that would otherwise need to be done manually. Once tests have been automated, they can be run quickly. This is often the most cost effective method for software products that have a long maintenance life, because even minor patches over the lifetime of the application can cause features to break which were working at an earlier point in time.There are two general approaches to test automation:
- Code-driven testing. The public (usually) interfaces to classes, modules, or libraries are tested with a variety of input arguments to validate that the results that are returned are correct.
- Graphical user interface testing. A testing framework generates user interface events such as keystrokes and mouse clicks, and observes the changes that result in the user interface, to validate that the observable behavior of the program is correct.
One way to generate test cases automatically is model-based testing through use of a model of the system for test case generation, but research continues into a variety of alternative methodologies for doing so.[citation needed]
What to automate, when to automate, or even whether one really needs automation are crucial decisions which the testing (or development) team must make. Selecting the correct features of the product for automation largely determines the success of the automation. Automating unstable features or features that are undergoing changes should be avoided.[2]
[edit] Code-driven testing
A growing trend in software development is the use of testing frameworks such as the xUnit frameworks (for example, JUnit and NUnit) that allow the execution of unit tests to determine whether various sections of the code are acting as expected under various circumstances. Test cases describe tests that need to be run on the program to verify that the program runs as expected.Code driven test automation is a key feature of Agile software development, where it is known as Test-driven development (TDD). Unit tests are written to define the functionality before the code is written. Only when all tests pass is the code considered complete. Proponents argue that it produces software that is both more reliable and less costly than code that is tested by manual exploration. It is considered more reliable because the code coverage is better, and because it is run constantly during development rather than once at the end of a waterfall development cycle. The developer discovers defects immediately upon making a change, when it is least expensive to fix. Finally, code refactoring is safer; transforming the code into a simpler form with less code duplication, but equivalent behavior, is much less likely to introduce new defects.
[edit] Graphical User Interface (GUI) testing
Many test automation tools provide record and playback features that allow users to interactively record user actions and replay them back any number of times, comparing actual results to those expected. The advantage of this approach is that it requires little or no software development. This approach can be applied to any application that has a graphical user interface. However, reliance on these features poses major reliability and maintainability problems. Relabelling a button or moving it to another part of the window may require the test to be re-recorded. Record and playback also often adds irrelevant activities or incorrectly records some activities.A variation on this type of tool is for testing of web sites. Here, the "interface" is the web page. This type of tool also requires little or no software development. However, such a framework utilizes entirely different techniques because it is reading html instead of observing window events.[citation needed]
Another variation is scriptless test automation that does not use record and playback, but instead builds a model of the application under test and then enables the tester to create test cases by simply editing in test parameters and conditions. This requires no scripting skills, but has all the power and flexibility of a scripted approach. Test-case maintenance is easy, as there is no code to maintain and as the application under test changes the software objects can simply be re-learned or added. It can be applied to any GUI-based software application.
[edit] What to test
Testing tools can help automate tasks such as product installation, test data creation, GUI interaction, problem detection (consider parsing or polling agents equipped with oracles), defect logging, etc., without necessarily automating tests in an end-to-end fashion.One must keep satisfying popular requirements when thinking of test automation:
- Platform and OS independence
- Data driven capability (Input Data, Output Data, Meta Data)
- Customizable Reporting (DB Access, crystal reports)
- Easy debugging and logging
- Version control friendly – minimal binary files
- Extensible & Customizable (Open APIs to be able to integrate with other tools)
- Common Driver (For example, in the Java development ecosystem, that means Ant or Maven and the popular IDEs). This enables tests to integrate with the developers' workflows.
- Support unattended test runs for integration with build processes and batch runs. Continuous Integration servers require this.
- Email Notifications (automated notification on failure or threshold levels). This may be the test runner or tooling[clarification needed] that executes it.
- Support distributed execution environment (distributed test bed)
- Distributed application support (distributed SUT)
[edit] Framework approach in automation
A framework is an integrated system that sets the rules of Automation of a specific product. This system integrates the function libraries, test data sources, object details and various reusable modules. These components act as small building blocks which need to be assembled to represent a business process. The framework provides the basis of test automation and simplifies the automation effort.There are various types of frameworks. They are categorized on the basis of the automation component they leverage. These are:
- Data-driven testing
- Modularity-driven testing
- Keyword-driven testing
- Hybrid testing
- Model-based testing
[edit] Popular Test Automation Tools
| Tool Name | Company Name | Latest Version |
|---|---|---|
| HP QuickTest Professional | HP | 10.5 |
| IBM Rational Functional Tester | IBM Rational | 8.1.0.3 |
| Rational robot | IBM Rational | 2001 |
| Selenium | OpenSource Tool | 1.0.6 |
| SilkTest | Micro Focus | 2009 |
| TestComplete | SmartBear Software | 8.0 |
| TestPartner | Micro Focus | 6.3 |
| WATIR | OpenSource Tool | 1.6.5 |
Subscribe to:
Posts (Atom)