Functional
Test Automation using QTP:
Test automation: It is the use of software to control the execution of tests, the comparison of actual outcomes to predicted
outcomes, the setting up of test preconditions, and other test control and test
reporting functions. Commonly, test automation involves automating a manual
process already in place that uses a formalized testing process.
Automated Testing Advantages,
Disadvantages and Guidelines
§ Detailed test cases, including predictable
"expected results", which have been developed from Business
Functional Specifications and Design documentation
§ A standalone Test
Environment, including a Test Database that is restorable to a known constant,
such that the test cases are able to be repeated each time there are
modifications made to the application.
The following types of testing can be automated
§ Functional - testing that
operations perform as expected.
§ Regression - testing that the
behavior of the system has not changed.
§ Exception or Negative - forcing error
conditions in the system.
§ Stress - determining the
absolute capacities of the application and operational infrastructure.
§ Performance - providing
assurance that the performance of the system will be adequate for both batch
runs and online transactions in relation to business projections and
requirements.
§ Load - determining the
points at which the capacity and performance of the system become degraded to
the situation that hardware or software upgrades would be required.
Benefits of Automated Testing
Reliable: Tests perform
precisely the same operations each time they are run, thereby eliminating human
error
Repeatable: You can test how the
software reacts under repeated execution of the same operations.
Programmable: You can program sophisticated tests that bring out hidden information from the application.
Comprehensive: You can build a suite of tests that covers every feature in your application.
Programmable: You can program sophisticated tests that bring out hidden information from the application.
Comprehensive: You can build a suite of tests that covers every feature in your application.
Reusable: You can reuse tests
on different versions of an application, even if the user interface changes.
Better Quality Software: Because you can run more tests in less time with fewer resources
Fast: Automated Tools run
tests significantly faster than human users.
Cost Reduction: As the number of
resources for regression test are reduced.
Choosing
the right tools for the job and targeting the right areas of the organization
to deploy them can only realize these benefits. The right areas where the
automation fit must be chosen.
The following areas must be automated first
1.
Highly redundant tasks or scenarios
2. Repetitive tasks that are boring or tend to cause human error
3. Well-developed and well-understood use cases or scenarios first
4. Relatively stable areas of the application over volatile ones must be automated.
2. Repetitive tasks that are boring or tend to cause human error
3. Well-developed and well-understood use cases or scenarios first
4. Relatively stable areas of the application over volatile ones must be automated.
Automated testers must follow the following guidelines to get the
benefits of automation:
• Concise: As simple as possible
and no simpler.
• Self-Checking: Test reports its own
results; needs no human interpretation.
• Repeatable: Test can be run many
times in a row without human intervention.
• Robust: Test produces same
result now and forever. Tests are not affected by changes in the external
environment.
• Sufficient: Tests verify all the
requirements of the software being tested.
• Necessary: Everything in each
test contributes to the specification of desired behavior.
• Clear: Every statement is
easy to understand.
• Efficient: Tests run in a
reasonable amount of time.
• Specific: Each test failure
points to a specific piece of broken functionality; unit test failures provide
"defect triangulation".
• Independent: Each test can be run
by itself or in a suite with an arbitrary set of other tests in any order.
• Maintainable: Tests should be easy
to understand and modify and extend.
• Traceable: To and from the code
it tests and to and from the requirements.
Disadvantages of Automation Testing
Though
the automation testing has many advantages, it has its own disadvantages too.
Some of the disadvantages are:
• Proficiency is required to write the automation test scripts.
• Debugging the test script is major issue. If any error is present in the test script, sometimes it may lead to deadly consequences.
• Test maintenance is costly in case of playback methods. Even though a minor change occurs in the GUI, the test script has to be rerecorded or replaced by a new test script.
• Maintenance of test data files is difficult, if the test script tests more screens.
Some of the above disadvantages often cause damage to the benefit gained from the automated scripts. Though the automation testing has pros and corns, it is adapted widely all over the world.
• Proficiency is required to write the automation test scripts.
• Debugging the test script is major issue. If any error is present in the test script, sometimes it may lead to deadly consequences.
• Test maintenance is costly in case of playback methods. Even though a minor change occurs in the GUI, the test script has to be rerecorded or replaced by a new test script.
• Maintenance of test data files is difficult, if the test script tests more screens.
Some of the above disadvantages often cause damage to the benefit gained from the automated scripts. Though the automation testing has pros and corns, it is adapted widely all over the world.
Automated vs Manual testing
|
ADVANTAGES
|
|
|
Automated Testing
|
Manual Testing
|
|
• If you have to run
a set of tests repeatedly automation is a huge gain
|
•
If Test Cases have to be run a small number of times it's more likely to
perform manual testing
|
|
•
Helps performing "compatibility testing" - testing the software on
different configurations
|
•
It allows the tester to perform more ad-hoc (random testing)
|
|
•
It gives you the ability to run automation scenariosto perform regressions in
a shorter time
|
•
Short term costs are reduced
|
|
•
It gives you the ability to run regressions on a code that is continously
changing
|
•
The more time tester spends testing a module the grater the odds to find real
user bugs
|
|
•
Can be run simultaneously on different machines thus decreasing testing time
|
|
|
•
Long term costs are reduced
|
|
|
DISADVANTAGES
|
|
|
Automated Testing
|
Manual Testing
|
|
•
It's more expensive to automate. Initial investments are bigger than manual
testing
|
•
Manual tests can be very time consuming
|
|
•
You cannot automate everything, some tests still have to be done manually
|
•
For every release you must rerun the same set of tests which can be tiresome
|
|
OTHER FACTORS
|
|
|
•
The performance of yout test tools
|
|
|
•
The knowledge level of your testing team
|
|
|
•
The continous growth of software to be tested
|
|
|
•
Number of necessary regressions
|
|
Manual and Automation testing Challenges
Software
Testing has lot of challenges both in manual as well as in automation.Generally
in manual testing scenario developers through the build to test team assuming
the responsible test team or tester will pick the build and will come to ask
what the build is about? This is the case in organizations not following
so-called ‘processes’. Tester is the middleman between developing team and the
customers, handling the pressure from both the sides. And I assume most of our
readers are smart enough to handle this pressure. Aren’t you?
This is not the case always.
Some times testers may add complications in testing process due to their
unskilled way of working.In this post I have added
most of the testing challenges created due to testing staff, developing staff,
testing processes and wrong management decisions.
So here we go with the top challenges:
So here we go with the top challenges:
1)
Testing the complete application:
Is it possible? I think impossible. There are millions of test combinations. It’s not possible to test each and every combination both in manual as well as in automation testing. If you try all these combinations you will never ship the product
Is it possible? I think impossible. There are millions of test combinations. It’s not possible to test each and every combination both in manual as well as in automation testing. If you try all these combinations you will never ship the product
2)
Misunderstanding of company processes:
Some times you just don’t pay proper attention what the company-defined processes are and these are for what purposes. There are some myths in testers that they should only go with company processes even these processes are not applicable for their current testing scenario. This results in incomplete and inappropriate application testing.
Some times you just don’t pay proper attention what the company-defined processes are and these are for what purposes. There are some myths in testers that they should only go with company processes even these processes are not applicable for their current testing scenario. This results in incomplete and inappropriate application testing.
3)
Relationship with developers:
Big challenge. Requires very skilled tester to handle this relation positively and even by completing the work in testers way. There are simply hundreds of excuses developers or testers can make when they are not agree with some points. For this tester also requiresgood communication, troubleshooting and analyzing skill.
Big challenge. Requires very skilled tester to handle this relation positively and even by completing the work in testers way. There are simply hundreds of excuses developers or testers can make when they are not agree with some points. For this tester also requiresgood communication, troubleshooting and analyzing skill.
4) Regression testing:
When project goes on expanding the regression testing work simply becomes uncontrolled. Pressure to handle the current functionality changes, previous working functionality checks and bug tracking.
When project goes on expanding the regression testing work simply becomes uncontrolled. Pressure to handle the current functionality changes, previous working functionality checks and bug tracking.
5) Lack
of skilled testers:
I will call this as ‘wrong management decision’ while selecting or training testers for their project task in hand. These unskilled fellows may add more chaos than simplifying the testing work. This results into incomplete, insufficient and ad-hoc testing throughout thetesting life cycle.
I will call this as ‘wrong management decision’ while selecting or training testers for their project task in hand. These unskilled fellows may add more chaos than simplifying the testing work. This results into incomplete, insufficient and ad-hoc testing throughout thetesting life cycle.
6) Testing always under time constraint:
Hey tester, we want to ship this product by this weekend, are you ready for completion? When this order comes from boss, tester simply focuses on task completion and not on the test coverage and quality of work. There is huge list of tasks that you need to complete within specified time. This includes writing, executing, automating and reviewing the test cases.
Hey tester, we want to ship this product by this weekend, are you ready for completion? When this order comes from boss, tester simply focuses on task completion and not on the test coverage and quality of work. There is huge list of tasks that you need to complete within specified time. This includes writing, executing, automating and reviewing the test cases.
7) Which
tests to execute first?
If you are facing the challenge stated in point no 6, then how will you take decision which test cases should be executed and with what priority? Which tests are important over others? This requires good experience to work under pressure.
If you are facing the challenge stated in point no 6, then how will you take decision which test cases should be executed and with what priority? Which tests are important over others? This requires good experience to work under pressure.
8 )
Understanding the requirements:
Some times testers are responsible for communicating with customers for understanding the requirements. What if tester fails to understand the requirements? Will he be able to test the application properly? Definitely No! Testers require good listening and understanding capabilities.
Some times testers are responsible for communicating with customers for understanding the requirements. What if tester fails to understand the requirements? Will he be able to test the application properly? Definitely No! Testers require good listening and understanding capabilities.
9) Automation testing:
Many sub challenges – Should automate the testing work? Till what level automation should be done? Do you have sufficient and skilled resources for automation? Is time permissible for automating the test cases? Decision of automation or manual testing will need to address the pros and cons of each process.
Many sub challenges – Should automate the testing work? Till what level automation should be done? Do you have sufficient and skilled resources for automation? Is time permissible for automating the test cases? Decision of automation or manual testing will need to address the pros and cons of each process.
10)
Decision to stop the testing:
When to stop testing? Very difficult decision. Requires core judgment of testing processes and importance of each process. Also requires ‘on the fly’ decision ability.
When to stop testing? Very difficult decision. Requires core judgment of testing processes and importance of each process. Also requires ‘on the fly’ decision ability.
11) One
test team under multiple projects:
Challenging to keep track of each task. Communication challenges. Many times results in failure of one or both the projects.
Challenging to keep track of each task. Communication challenges. Many times results in failure of one or both the projects.
12) Reuse
of Test scripts:
Application development methods are changing rapidly, making it difficult to manage the test tools and test scripts. Test script migration or reuse is very essential but difficult task.
Application development methods are changing rapidly, making it difficult to manage the test tools and test scripts. Test script migration or reuse is very essential but difficult task.
13)
Testers focusing on finding easy bugs:
If organization is rewarding testers based on number of bugs (very bad approach to judge testers performance) then some testers only concentrate on finding easy bugs those don’t require deep understanding and testing. A hard or subtle bug remains unnoticed in such testing approach.
If organization is rewarding testers based on number of bugs (very bad approach to judge testers performance) then some testers only concentrate on finding easy bugs those don’t require deep understanding and testing. A hard or subtle bug remains unnoticed in such testing approach.
14) To
cope with attrition:
Increasing salaries and benefits making many employees leave the company at very short career intervals. Managements are facing hard problems to cope with attrition rate. Challenges – New testers require project training from the beginning, complex projects are difficult to understand, delay in shipping date!
Increasing salaries and benefits making many employees leave the company at very short career intervals. Managements are facing hard problems to cope with attrition rate. Challenges – New testers require project training from the beginning, complex projects are difficult to understand, delay in shipping date!
No comments:
Post a Comment