Manual Test

Manual testing

Manual testing is the process of manually testing software for defects. It requires a tester to play the role of an end user, and use most of all features of the application to ensure correct behavior. To ensure completeness of testing, the tester often follows a written test plan that leads them through a set of important test cases.

A simple tool to manage manual testing

Introducing IBM Rational Manual Tester

Although manual testing is an important part of software quality assurance efforts, it's largely been overlooked by tool companies in the past. Finally, IBM® Rational® Manual Tester gives testing staffs a tool to manage creation and maintenance of manual test scripts simply and straightforwardly. The tool balances a full feature set with ease of learning and implementation. This introduction to Manual Tester will give you a preview of its intuitive interface and comprehensive features. We start with a discussion of why manual testing is important and what problems can plague it when it's managed using Excel spreadsheets or Word tables, as it commonly is.

Editor's note: IBM Rational Manual Tester is available both separately and as part of IBM Rational Functional Tester version 6.1 and later. Both versions of the tool offer the same functionality.

Why do manual testing?

Even in this age of short development cycles and automated-test-driven development, manual testing contributes vitally to the software development process. Here are a number of good reasons to do manual testing:

• By giving end users repeatable sets of instructions for using prototype software, manual testing allows them to be involved early in each development cycle and draws invaluable feedback from them that can prevent "surprise" application builds that fail to meet real-world usability requirements.
• Manual test scripts gives testers something to use while awaiting the construction and debugging of automated scripts.
• Manual test scripts can be used to provide feedback to development teams in the form of a set of repeatable steps that lead to bugs or usability problems.
• If done thoroughly, manual test scripts can also form the basis for help or tutorial files for the application under test.
• Finally, in a nod toward test-driven development, manual test scripts can be given to the development staff to provide a clear description of the way the application should flow across use cases.

Problems with using Excel or Word to manage manual testing

Because manual test script creation is, well, manual, organizations may not realize all of the benefits of manual test scripting in today's short development cycles. Typically, testing staffs use Excel spreadsheets (or, less typically, Word tables) to record testing steps, expected results, and pass/fail state in the required timeframes. They compile the results of testing either manually or with homegrown programs that import the Excel spreadsheets or Word tables, process the test results, and then produce reports.

But anyone attempting to use Excel spreadsheets or Word tables to manage manual test scripts will run into these problems:

• The need to constantly scroll horizontally and vertically makes these methods inefficient and difficult to use.
• The typically wide formats of Excel spreadsheets or Word tables and the need to reproduce headings on each page make it difficult to print reports.
• It's hard to organize scripts by grouping testing steps.
• There's no standard way to identify expected results within test scripts.
• There's no standard way to report test results.
• It's hard or impossible to reuse test script lines or groups of lines in different test scripts.

A tool that solves these problems would enable test departments to take full advantage of their enlightened decision to incorporate manual test scripts into the application development process.

Problems with using Excel or Word to manage manual testing

Because manual test script creation is, well, manual, organizations may not realize all of the benefits of manual test scripting in today's short development cycles. Typically, testing staffs use Excel spreadsheets (or, less typically, Word tables) to record testing steps, expected results, and pass/fail state in the required timeframes. They compile the results of testing either manually or with homegrown programs that import the Excel spreadsheets or Word tables, process the test results, and then produce reports.

But anyone attempting to use Excel spreadsheets or Word tables to manage manual test scripts will run into these problems:

• The need to constantly scroll horizontally and vertically makes these methods inefficient and difficult to use.
• The typically wide formats of Excel spreadsheets or Word tables and the need to reproduce headings on each page make it difficult to print reports.
• It's hard to organize scripts by grouping testing steps.
• There's no standard way to identify expected results within test scripts.
• There's no standard way to report test results.
• It's hard or impossible to reuse test script lines or groups of lines in different test scripts.

A tool that solves these problems would enable test departments to take full advantage of their enlightened decision to incorporate manual test scripts into the application development process.

Manual Tester to the rescue

IBM Rational Manual Tester is the tool that testing staffs have been waiting for to address the problems listed above and make management of manual test scripts simple and straightforward. It doesn't take much hands-on experience with Manual Tester to prompt a tester to ask, "Why use Excel any longer for manual test scripts?" Manual Tester is easy to learn (allow about an hour and a half) due to the detailed tutorials that come with the tool, the tool's smart design, and the intelligence of its user interface.

The best thing about using this tool to manage manual testing is that the test scripts created by multiple testers will all follow the same standards. Here are the capabilities of Manual Tester that lead to standardization in scripts:

• Displays and prints standard icons identifying script lines by type (script step, verification point, reportable point, or group of steps).
• Allows the reuse of script steps or groups of steps among different scripts.
• Allows easy grouping of steps, which makes grouping an easily enforceable organizational standard.
• Associates script lines with standard sets of tester-defined properties, such as Name, Compare Data, and links to attachments (which can opened within Manual Tester).
• Provides an interface that enforces standardized expected results (for example, "inconclusive," "pass," "fail," "error") during execution of scripts.
• Allows for standardized comparisons of data during execution of scripts.
• Allows the test team to select font and color for the text of script steps.

·         A look at Manual Tester's features

·         As you can see in Figure 1, Manual Tester's well-formatted script-editing screen minimizes the need for scrolling. Multiple scripts can be opened and accessed via tabs at the top of the window. A toolbar provides icon shortcuts to major functions.

The Manual Tester user interface also makes it easy to group and reuse script steps, and to track the results of executing scripts. And Manual Tester's wizards make it easy to import and export test scripts. We'll take a look at each of these features.

Grouping script steps

Grouping of script steps is accomplished via an Outline panel (think Windows Explorer) that allows intuitive dragging and dropping onto folder icons that represent groups (note the Logon folder in Figure 2).

Reusing script steps

Reuse of script steps is also accomplished in Manual Tester via drag-and-drop functionality. First the tester drags the script step or group (folder) of steps from the Outline panel to the Reuse panel (see Figure 3 for an example of dragging the Logon folder to the Reuse panel).

Then the tester drags the script step or group of steps from the Reuse panel to the desired location in the same or in a different script. The reused script steps are automatically referenced (see Figure 4 for an example of the Logon folder being referenced in a different script).

Although the reused steps are displayed in the new script as though they were a physical part of the script, these script lines are actually only references to a single (true) base instance of the steps. This fact endows Manual Tester with the nifty ability to automatically refresh all referenced script lines whenever the base script lines are modified. Thus, reuse of automatically generated and maintained script steps is a workable reality in Manual Tester that has the potential to save significant amounts of usually scarce test department resources.

Tracking script execution

Manual Tester executes scripts by displaying a user interface for each step in succession (see Figure 5). The tester selects standard results (for example, "pass," "fail," "error," "inconclusive") from a drop-down list. Custom result types can also be added to the list in order to enforce existing organization standards. Comparison values and any related attachments are displayed on the same screen in the Properties panel. The Properties panel also allows the tester to annotate the execution step with free-form comments. The execution window can be made translucent so that the application under test can be seen behind it.

Exporting and importing scripts

The Manual Tester Export Wizard makes it easy to export scripts from Manual Tester (see Figure 6), in case an organization still needs to rely on in-house reporting or other management of test scripts.

The Import Wizard makes it easy to migrate from legacy manual scripting environments, such as Excel or Word, to Manual Tester (see Figure 7). Importing such legacy scripts benefits individuals outside of the test department, such as end users, use case analysts, or project managers who have the need to create manual test scripts for validation or communication purposes. The tester can choose from either Excel or Word and then bring these scripts into Manual Tester format. The tester can select whether to keep imported scripts in their original files or merge them into a single Manual Tester script.

The next dialog window allows the tester to select which formatter Manual Tester should use when importing the legacy script. (There's currently only one, for Excel -- see Figure 8.)

Curtain drawn back
As this article has described, Rational Manual Tester nicely balances a full feature set with ease of learning and implementation. The tool accomplishes this by leveraging an intuitive interface that manages to keep all aspects of manual test script creation and maintenance simple and straightforward. Now if it could just generate all of the script steps automatically...

Manual and Automation testing Challenges

Software Testing has lot of challenges both in manual as well as in automation. Generally in manual testing scenario developers through the build to test team assuming the responsible test team or tester will pick the build and will come to ask what the build is about? This is the case in organizations not following so-called ‘processes’. Tester is the middleman between developing team and the customers, handling the pressure from both the sides. And I assume most of our readers are smart enough to handle this pressure. Aren’t you?
This is not the case always. Some times testers may add complications in testing process due to their unskilled way of working. In this post I have added most of the testing challenges created due to testing staff, developing staff, testing processes and wrong management decisions.

So here we go with the top challenges:
1) Testing the complete application:
Is it possible? I think impossible. There are millions of test combinations. It’s not possible to test each and every combination both in manual as well as in automation testing. If you try all these combinations you will never ship the product
2) Misunderstanding of company processes:
Some times you just don’t pay proper attention what the company-defined processes are and these are for what purposes. There are some myths in testers that they should only go with company processes even these processes are not applicable for their current testing scenario. This results in incomplete and inappropriate application testing.
3) Relationship with developers:
Big challenge. Requires very skilled tester to handle this relation positively and even by completing the work in testers way. There are simply hundreds of excuses developers or testers can make when they are not agree with some points. For this tester also requires good communication, troubleshooting and analyzing skill.
4) Regression testing:
When project goes on expanding the regression testing work simply becomes uncontrolled. Pressure to handle the current functionality changes, previous working functionality checks and bug tracking.
5) Lack of skilled testers:
I will call this as ‘wrong management decision’ while selecting or training testers for their project task in hand. These unskilled fellows may add more chaos than simplifying the testing work. This results into incomplete, insufficient and ad-hoc testing throughout the testing life cycle.
6) Testing always under time constraint:
Hey tester, we want to ship this product by this weekend, are you ready for completion? When this order comes from boss, tester simply focuses on task completion and not on the test coverage and quality of work. There is huge list of tasks that you need to complete within specified time. This includes writing, executing, automating and reviewing the test cases.
7) Which tests to execute first?
If you are facing the challenge stated in point no 6, then how will you take decision which test cases should be executed and with what priority? Which tests are important over others? This requires good experience to work under pressure.
8 ) Understanding the requirements:
Some times testers are responsible for communicating with customers for understanding the requirements. What if tester fails to understand the requirements? Will he be able to test the application properly? Definitely No! Testers require good listening and understanding capabilities.
9) Automation testing:
Many sub challenges – Should automate the testing work? Till what level automation should be done? Do you have sufficient and skilled resources for automation? Is time permissible for automating the test cases? Decision of automation or manual testing will need to address the pros and cons of each process.
10) Decision to stop the testing:
When to stop testing? Very difficult decision. Requires core judgment of testing processes and importance of each process. Also requires ‘on the fly’ decision ability.
11) One test team under multiple projects:
Challenging to keep track of each task. Communication challenges. Many times results in failure of one or both the projects.
12) Reuse of Test scripts:
Application development methods are changing rapidly, making it difficult to manage the test tools and test scripts. Test script migration or reuse is very essential but difficult task.
13) Testers focusing on finding easy bugs:
If organization is rewarding testers based on number of bugs (very bad approach to judge testers performance) then some testers only concentrate on finding easy bugs those don’t require deep understanding and testing. A hard or subtle bug remains unnoticed in such testing approach.
14) To cope with attrition:
Increasing salaries and benefits making many employees leave the company at very short career intervals. Managements are facing hard problems to cope with attrition rate. Challenges – New testers require project training from the beginning, complex projects are difficult to understand, delay in shipping date!
These are some top software testing challenges we face daily. Project success or failure depends largely on how you address these basic issues.
For further reference and detailed solutions on these challenges refer book “Surviving the Top Ten challenges of Software Testing” written by William E. Perry and Randall W. Rice.

How to Improve Tester Performance?

Many Companies don’t have resources or can’t afford to hire the required number of testers on the project. So what could be the solution in this case?
The answer is simple. Companies will prefer to have skilled testers instead of a army of testers!
So how can build skilled testers on any project?
You can improve testers performance by assigning him/her to the single project.
Due to this the tester will get the detail knowledge of the project domain, Can concentrate well on that project, can do the R&D work during the early development phase of the project.
This not only build his/her functional testing knowledge but also the project Domain knowledge.
Company can use following methods to Improve the Testers performance:
1) Assign one tester to one project for long duration or to the entire project. Doing this will build testers domain knowledge, He/She can write better test cases, Can cover most of the test cases, and eventually can find the problem faster.
2) Most of the testers can do the functional testing, BV analysis but they may not know how to measure test coverage,How to test a complete application, How to perform load testing. Company can provide the training to their employees in those areas.
3) Involve them in all the project meetings, discussions, project design so that they can understand the project well and can write the test cases well.
4) Encourage them to do the extra activities other than the regular testing activities. Such activities can include Inter team talk on their project experience, Different exploratory talks on project topics.
Most important is to give them freedom to think outside the box so that they can take better decision on Testing activities like test plan, test execution, test coverage.

How to be a good tester?

It’s a every testers question. How to be a good tester? Apart from the technical knowledge, testing skills, tester should have some personal level skills which will help them to build a good rapport in the testing team.
What are these abilities , skills which make a tester as a good tester? Well, I was reading Dave Whalen’s article “Ugly Baby Syndrome!” and found it very interesting. Dave compared software developers with the parents who deliver a baby (software) with countless efforts. Naturally the product managers, architectures, developers spent their countless time on developing application for the customer. Then they show it to us (testers) and asks: “ How is the baby (Application)? “ And testers tell them often that they have and ugly baby. (Application with Bugs!)
Testers don’t want to tell them that they have ugly baby, but unfortunately its our job. So effectively tester can convey the message to the developers without hurting them. How can be this done? Ya that is the skill of a good tester!
Here are the tips sated by Dave to handle such a delicate situation:
Be honest and Responsive:
Tell developers what are your plans to attack their application.
Be open and available:
If any dev ask you to have a look at the application developed by him before the release, then politely give feedback on it and report any extra efforts needed. Don’t log the bug’s for these notes.
Let them review your tests:
If you have designed or wrote some test cases from the requirement specifications then just show them those test cases. Let them know your stuff as you are going to critic on developers work!
Use of Bug tracker:
Some testers have habit to report each and everything publicly. This attitude hurts the developers. So if you have logged any bug then let the bug tracking system report it to respective developers and managers. Also don’t each time rely on bug tracker, talk personally to developers what you logged and why you logged?
Finally some good personal points:
Don’t take it personally:
Do the job of messenger. You could be a close target always. So build a thick skin!
Be prepared:
A good message in the end, Be prepared for everything! If worst things might not happened till now but they can happen at any moment in your career. So be ready to face them.
[Thougt of the Day: When a virtually flawless application is delivered to a customer, no one says how well tested it was. Development teams will always get the credit. However, if it is delivered with bugs, everyone will wonder who tested it! - - Dave Whalen]

Developers are not good testers. What you say?

This can be a big debate. Developers testing their own code – what will be the testing output? All happy endings! Yes, the person who develops the code generally sees only happy paths of the product and don’t want to go in much details.
The main concern of developer testing is – misunderstanding of requirements. If requirements are misunderstood by developer then no matter at what depth developer test the application, he will never find the error. The first place where the bug gets introduced will remain till end, as developer will see it as functionality.
Optimistic developers – Yes, I wrote the code and I am confident it’s working properly. No need to test this path, no need to test that path, as I know it’s working properly. And right here developers skip the bugs.
Developer vs Tester: Developer always wants to see his code working properly. So he will test it to check if it’s working correctly. But you know why tester will test the application? To make it fail in any way, and tester surely will test how application is not working correctly. This is the main difference in developer testing and tester testing.
Should developers test their own work?
I personally don’t mind developers testing their own code. After all it’s there baby They know their code very well. They know what are the traps in their codes. Where it can fail, where to concentrate more, which is important path of the application. Developer can do unit testing very well and can effectively identify boundary cases. (Image credit)
This is all applicable to a developer who is a good tester! But most of the developers consider testing as painful job, even they know the system well, due to their negligence they tend to skip many testing  paths, as it’s a very painful experience for them. If developers find any errors in their code in unit testing then it’s comparatively easier to fix, as the code is fresh to them, rather than getting the bug from testers after two-three days. But this only possible if the developer is interested in doing that much testing.
It’s testers responsibility to make sure each and every path is tested or not. Testers should ideally give importance to all small possible details to verify application is not breaking anywhere.
Developers, please don’t review your own code. Generally you will overlook the issues in your code. So give it to others for review.
Everyone is having specialization in particular subject. Developers generally think how to develop the application on the other hand testers think how the end user is going to use the application.
Conclusion
So in short there is no problem if developers are doing the basic unit testing and basic verification testing. Developers can test few exceptional conditions they know are critical and should not be missed. But there are some great testers out there. Through the build to test team. Don’t waste your time as well. For success of any project there should be independent testing team validating your applications. After all it’s our (testers) responsibility to make the ‘baby’ smarter!!

How Domain knowledge is Important for testers?

Looking at the current scenario from the industry it is seen that the testers are expected to have both technical testing skills as well either need to be from the domain background or have gathered domain knowledge mainly for BFSI is commonly seen.
I would like to know why and when is this domain knowledge imparted to the tester during the testing cycle?”

First of all I would like to introduce three dimensional testing career mentioned by Danny R. Faught. There are three categories of skill that need to be judged before hiring any software tester. What are those three skill categories?
1) Testing skill
2) Domain knowledge
3) Technical expertise.

No doubt that any tester should have the basic testing skills like Manual testing and Automation testing. Tester having the common sense can even find most of the obvious bugs in the software. Then would you say that this much testing is sufficient? Would you release the product on the basis of this much testing done? Certainly not. You will certainly have a product look by the domain expert before the product goes into the market.

While testing any application you should think like a end-user. But every human being has the limitations and one can’t be the expert in all of the three dimensions mentioned above. (If you are the experts in all of the above skills then please let me know ) So you can’t assure that you can think 100% like how the end-user going to use your application. User who is going to use your application may be having a good understanding of the domain he is working on. You need to balance all these skill activities so that all product aspects will get addressed.

Nowadays you can see the professional being hired in different companies are more domain experts than having technical skills. Current software industry is also seeing a good trend that many professional developers and domain experts are moving into software testing.

We can observe one more reason why domain experts are most wanted! When you hire fresh engineers who are just out of college you cannot expect them to compete with the experienced professionals. Why? Because experienced professional certainly have the advantage of domain and testing experience and they have better understandings of different issues and can deliver the application better and faster.

Here are some of the examples where you can see the distinct edge of domain knowledge:
1) Mobile application testing.
2) Wireless application testing
3) VoIP applications
4) Protocol testing
5) Banking applications
6) Network testing

How will you test such applications without knowledge of specific domain? Are you going to test the BFSI applications (Banking, Financial Services and Insurance) just for UI or functionality or security or load or stress? You should know what are the user requirements in banking, working procedures, commerce background, exposure to brokerage etc and should test application accordingly, then only you can say that your testing is enough – Here comes the need of subject-matter experts.

Let’s take example of my current project: I am currently working on search engine application. Where I need to know the basic of search engine terminologies and concepts. Many times I see some other team tester’s asking me questions like what is ‘publishers’ and ‘advertisers’, what is the difference and what they do? Do you think they can test the application based on current online advertising and SEO? Certainly not. Unless and until they get well familiar with these terminologies and functionalities.

When I know the functional domain better I can better write and execute more test cases and can effectively simulate the end user actions which is distinctly a big advantage.

Here is the big list of the required testing knowledge:

• Testing skill
• Bug hunting skill
• Technical skill
• Domain knowledge
• Communication skill
• Automation skill
• Some programming skill
• Quick grasping
• Ability to Work under pressure …

That is going to be a huge list. So you will certainly say, do I need to have these many skills? Its’ depends on you. You can stick to one skill or can be expert in one skill and have good understanding of other skills or balanced approach of all the skills. This is the competitive market and you should definitely take advantage of it. Make sure to be expert in at least one domain before making any move.

What if you don’t have enough domain knowledge?
You will be posted on any project and company can assign any work to you. Then what if you don’t have enough domain knowledge of that project? You need to quickly grasp as many concepts as you can. Try to understand the product as if you are the customer and what customer will do with application. Visit the customer site if possible know how they work with the product, Read online resources about the domain you want to test the application, participate in events addressing on such domain, meet the domain experts. Or either company will provide all this in-house training before assigning any domain specific task to testers.

There is no specific stage where you need this domain knowledge. You need to apply your domain knowledge in each and every software testing life cycle.

If you are reading this article till this point then I would like to hear on which domain you are working on? So that our readers can get better idea of different domains and projects. Comment your domain below.

Unix basics for software testers

No matter whether you are working on stand-alone project or web project, operating systems and networking knowledge is must for testers. Many testing activities like installation testing, performance testing are dependent on operating system knowledge. Now days most of the web servers are Unix based. So Unix knowledge is mandatory for tester.

For the beginners in Unix, learning basic Unix commands is a good start. This article explains basic Unix commands and in next article we will move on to the Unix operating system basics and advanced Unix commands.

The best way to learn following commands is to read and simultaneously practice them on Unix operating system.

These are the Unix commands that are mostly used while interacting with Unix servers. Most of the time you might be interacting with Unix OS through remote windows machines using softwares like ‘Putty’. In such testing environment this article is very useful.

You can download the article written by Meenakshi on Unix basics for testers on below link:

Download: Unix basics for testers article (Doc file)

I will continue this series of “soft skills required for testers”. Here is list of skills on which I will concentrate in coming posts:

• Operating systems skill for testers.
• Networking skill.
• Database ( Mainly SQL) knowledge for testers.
• Basic programming skill.

All these skills are required to become a successful software tester.

If you think any other skill should be also included in this “soft skills for testers” article series then please let me know in below comment section. I will work on those subjects to include them in my future articles.

No matter whether you are working on stand-alone project or web project, operating systems and networking knowledge is must for testers. Many testing activities like installation testing, performance testing are dependent on operating system knowledge. Now days most of the web servers are Unix based. So Unix knowledge is mandatory for tester.

For the beginners in Unix, learning basic Unix commands is a good start. This article explains basic Unix commands and in next article we will move on to the Unix operating system basics and advanced Unix commands.

The best way to learn following commands is to read and simultaneously practice them on Unix operating system.

These are the Unix commands that are mostly used while interacting with Unix servers. Most of the time you might be interacting with Unix OS through remote windows machines using softwares like ‘Putty’. In such testing environment this article is very useful.

You can download the article written by Meenakshi on Unix basics for testers on below link:

Download: Unix basics for testers article (Doc file)

I will continue this series of “soft skills required for testers”. Here is list of skills on which I will concentrate in coming posts:

• Operating systems skill for testers.
• Networking skill.
• Database ( Mainly SQL) knowledge for testers.
• Basic programming skill.

All these skills are required to become a successful software tester.

If you think any other skill should be also included in this “soft skills for testers” article series then please let me know in below comment section. I will work on those subjects to include them in my future articles.

 

 

Software testing book for preparing testing interviews and learning basics of software testing [download]

I am in process to compile a list of good books on software testing. Soon I will share this list with you. But lately I am getting too many requests to share any book on software testing for preparing software testing interviews. So here is a quick post to share an online testing book I found “A Software Testing Primer” by Nick Jenkins.

Basically this book is an introduction to software testing. So those who are new to software testing field can start their preparation by reading this book. You will get basic idea of manual and automation testing.

Here is a summary of what this book is covering:

• What is the need of software testing?
• Different software development models
• Testing in the software development life cycle
• How to develop testing mindset?
• Regression Vs. Retesting
• White box Vs. Black box testing
• Verification and validation
• Alpha and beta testing
• Unit, Integration and System testing
• Acceptance testing
• Automation testing – Basics
• Testing the design
• Usability testing
• Performance testing
• Test planning
• Test estimation
• Test cases and elements of test cases
• Test tracking, Test planning and Test plan review
• How to manage defects and defect reports?
• Test metrics for testers
• Product release control

In all this book is a nice introduction to software testing. Author explained some key software testing concepts like Regression and Retesting difference, Alpha and beta testing etc. where many testers get confused.

Download “Testing Primer” book: downloaded - testingPrimer.pdf , see in rktesting book