White Box Testing AND Black Box Testing

What is WHITE BOX TESTING

White box testing strategy deals with the internal logic and structure of the code. White box testing is also called as glass, structural, open box or clear box testing. The tests written based on the white box testing strategy incorporate coverage of the code written, branches, paths, statements and internal logic of the code etc.

In order to implement white box testing, the tester has to deal with the code and hence is needed to possess knowledge of coding and logic i.e. internal working of the code. White box test also needs the tester to look into the code and find out which unit/statement/chunk of the code is malfunctioning

Advantages of White box testing :

• As the knowledge of internal coding structure is prerequisite, it becomes very easy to find out which type of input/data can help in testing the application effectively.
• The other advantage of white box testing is that it helps in optimizing the code
• It helps in removing the extra lines of code, which can bring in hidden defects.

Disadvantages of white box testing:

• As knowledge of code and internal structure is a prerequisite, a skilled tester is needed to carry out this type of testing, which increases the cost.
• It is nearly impossible to look into every bit of code to find out hidden errors, which may create problems, resulting in failure of the application.

What is Black Box Testing

Black Box Testing is testing without knowledge of the internal workings of the item being tested. For example, when black box testing is applied to software engineering, the tester would only know the “legal” inputs and what the expected outputs should be, but not how the program actually arrives at those outputs.

It is because of this that black box testing can be considered testing with respect to the specifications, no other knowledge of the program is necessary. For this reason, the tester and the programmer can be independent of one another, avoiding programmer bias toward his own work. For this testing, test groups are often used, “Test groups are sometimes called professional idiots…people who are good at designing incorrect data.” 1 Also, do to the nature of black box testing, the test planning can begin as soon as the specifications are written. The opposite of this would be glass box testing, where test data are derived from direct examination of the code to be tested. For glass box testing, the test cases cannot be determined until the code has actually been written. Both of these testing techniques have advantages and disadvantages, but when combined, they help to ensure thorough testing of the product.
Synonyms for black-box include: behavioral, functional, opaque-box, and closed-box.

Advantages of Black Box Testing

• more effective on larger units of code than glass box testing
• tester needs no knowledge of implementation, including specific programming languages
• tester and programmer are independent of each other
• tests are done from a user’s point of view
• will help to expose any ambiguities or inconsistencies in the specifications
• test cases can be designed as soon as the specifications are complete

Disadvantages of Black Box Testing

• only a small number of possible inputs can actually be tested, to test every possible input stream would take nearly forever
• without clear and concise specifications, test cases are hard to design
• there may be unnecessary repetition of test inputs if the tester is not informed of test cases the programmer has already tried
• may leave many program paths untested
• cannot be directed toward specific segments of code which may be very complex (and therefore more error prone)
• most testing related research has been directed toward glass box testing

Testing Strategies/Techniques

• black box testing should make use of randomly generated inputs (only a test range should be specified by the tester), to eliminate any guess work by the tester as to the methods of the function
• data outside of the specified input range should be tested to check the robustness of  the program
• boundary cases should be tested (top and bottom of specified range) to make sure the highest and lowest allowable inputs produce proper output
• the number zero should be tested when numerical data is to be input
• stress testing should be performed (try to overload the program with inputs to see where it reaches its maximum capacity), especially with real time systems
• crash testing should be performed to see what it takes to bring the system down
• test monitoring tools should be used whenever possible to track which tests have already been performed and the outputs of these tests to avoid repetition and to aid in the software maintenance
• other functional testing techniques include: transaction testing, syntax testing, domain testing, logic testing, and state testing.
• finite state machine models can be used as a guide to design functional tests
• According to Beizer 2 the following is a general order by which tests should be designed:
  1. Clean tests against requirements.
  2. Additional structural tests for branch coverage, as needed.
  3. Additional tests for data-flow coverage as needed.
  4. Domain tests not covered by the above.
  5. Special techniques as appropriate–syntax, loop, state, etc.
  6. Any dirty tests not covered by the above.

Read more »

MUTATION TESTING

A kind of testing in which, the application is tested for the code that was modified after fixing a particular bug/defect. It also helps in finding out which code and which strategy of coding can help in developing the functionality effectively.

Mutation testing ensure the accuracy of our unit tests.In some cases our unit tests will be successful and won’t detect any bugs or defects in our source code, but that may be not true. In this situations, we will do some alteration in source code (mutating) for making errors and again will run those unit test.Now if our unit tests are perfect then it will catch the defect in mutated code.Otherwise we can understand that our unit tests are not proper. So simply we can say that Mutation Testing is a testing of our tests.

Some more facts about Mutation testing

• Richard Lipton is the man who introduced this testing first.
• Timothy Budd implemented the first Mutation testing tool.
• Mutation testing will be very expensive for large applications without using any tools.
• While mutating the source code, normally small changes on variables and operands are performing.

Read more »

Types of software Testing

Software Testing Types: 

Black box testing – Internal system design is not considered in this type of testing. Tests are based on requirements and functionality.

White box testing – This testing is based on knowledge of the internal logic of an application’s code. Also known as Glass box Testing. Internal software and code working should be known for this type of testing. Tests are based on coverage of code statements, branches, paths, conditions.

Unit testing – Testing of individual software components or modules. Typically done by the programmer and not by testers, as it requires detailed knowledge of the internal program design and code. may require developing test driver modules or test harnesses.

Incremental integration testing – Bottom up approach for testing i.e continuous testing of an application as new functionality is added; Application functionality and modules should be independent enough to test separately. done by programmers or by testers.

Integration testing – Testing of integrated modules to verify combined functionality after integration. Modules are typically code modules, individual applications, client and server applications on a network, etc. This type of testing is especially relevant to client/server and distributed systems.

Functional testing – This type of testing ignores the internal parts and focus on the output is as per requirement or not. Black-box type testing geared to functional requirements of an application.

System testing – Entire system is tested as per the requirements. Black-box type testing that is based on overall requirements specifications, covers all combined parts of a system.

End-to-end testing – Similar to system testing, involves testing of a complete application environment in a situation that mimics real-world use, such as interacting with a database, using network communications, or interacting with other hardware, applications, or systems if appropriate.

Sanity testing - Testing to determine if a new software version is performing well enough to accept it for a major testing effort. If application is crashing for initial use then system is not stable enough for further testing and build or application is assigned to fix.

Regression testing – Testing the application as a whole for the modification in any module or functionality. Difficult to cover all the system in regression testing so typically automation tools are used for these testing types.

Acceptance testing -Normally this type of testing is done to verify if system meets the customer specified requirements. User or customer do this testing to determine whether to accept application.

Load testing – Its a performance testing to check system behavior under load. Testing an application under heavy loads, such as testing of a web site under a range of loads to determine at what point the system’s response time degrades or fails.

Stress testing – System is stressed beyond its specifications to check how and when it fails. Performed under heavy load like putting large number beyond storage capacity, complex database queries, continuous input to system or database load.

Performance testing – Term often used interchangeably with ‘stress’ and ‘load’ testing. To check whether system meets performance requirements. Used different performance and load tools to do this.

Usability testing – User-friendliness check. Application flow is tested, Can new user understand the application easily, Proper help documented whenever user stuck at any point. Basically system navigation is checked in this testing.

Install/uninstall testing - Tested for full, partial, or upgrade install/uninstall processes on different operating systems under different hardware, software environment.

Recovery testing – Testing how well a system recovers from crashes, hardware failures, or other catastrophic problems.

Security testing – Can system be penetrated by any hacking way. Testing how well the system protects against unauthorized internal or external access. Checked if system, database is safe from external attacks.

Compatibility testing – Testing how well software performs in a particular hardware/software/operating system/network environment and different combination s of above.

Comparison testing – Comparison of product strengths and weaknesses with previous versions or other similar products.

Alpha testing – In house virtual user environment can be created for this type of testing. Testing is done at the end of development. Still minor design changes may be made as a result of such testing.

Beta testing – Testing typically done by end-users or others. Final testing before releasing application for commercial purpose.

Read more »

Difference between QTP8.2 and QTP9.0/QTP9.2

Over and above features provided with QTP 9.0 , QTP 9.2 provides following features:

Mercury Screen Recorder :
Capture your entire run session in a movie clip or capture only the segments with errors, and then view your movie from the Test Results window.
Dynamic Management of Object Repositories:
QuickTest now has a new RepositoriesCollection reserved object that you can use to programmatically manage the set of object repositories that are associated with an action during a run session.

Over and above features provided with QTP 8.2 , QTP 9.0 provides following features:

Object Repository Manager:
You can use the Object Repository Manager to manage all of the shared object repositories in your organization from one, central location. This includes adding and defining objects, modifying objects and their descriptions, parameterizing test object property values, maintaining and organizing repositories, and importing and exporting repositories in XML format.
You can open multiple object repositories at the same time. Each object repository opens in its own resizable document window. This enables you to compare the content of the repositories, to copy or move objects from one object repository to another, and so forth.
Object Repository Merge Tool:
You can use the Object Repository Merge Tool to merge the objects from two shared object repositories into a single shared object repository. You can also use the Object Repository Merge Tool to merge objects from the local object repository of one or more actions or components into a shared object repository.

When you merge objects from two source object repositories, the content is copied to a new, target object repository, ensuring that the information in the source repositories remains unchanged.

If any conflicts occur during the merge, for example, if two objects have the same name and test object class, but different test object descriptions, the relevant objects are highlighted in the source repositories, and the Resolution Options pane details the conflict and possible resolutions.
Multiple Object Repositories per Action or Component:
QuickTest provides several options for storing and accessing test objects. You can store the test objects for each action or component in its corresponding local object repository, which is unique for each action and component. You can also store test objects in one or more shared object repositories that can be used in multiple actions and components. Alternatively, you can use a combination of objects from the local object repository and one or more shared object repositories. You choose the combination that matches your testing needs.
XML Object Repository Format:

QuickTest now enables you to import and export object repositories from and to XML format. This enables you to modify object repositories using the XML editor of your choice and then import them back into QuickTest. You can import and export files either from and to the file system or a Quality Center project (if QuickTest is connected to Quality Center).
Function Library Editor:

QuickTest now has a built-in function library editor, which enables you to create and edit function libraries containing VBScript functions, subroutines, modules, and so forth, and then call their functions from your test or component.
Handling Missing Actions and Resources:
Whenever a testing document (test, component, or application area) contains a resource that cannot be found, QuickTest opens the Missing Resources pane and lists the missing resource(s). For example, a test may contain an action or a call to an action that cannot be found; a testing document may use a shared object repository that cannot be found; or a testing document may use a object repository parameter that does not have a default value. In all of these cases, QuickTest indicates this in the Missing Resources pane, enabling you to map a missing resource to an existing one, or remove it from the testing document, as required.

Read more »

Enhancements in QTP 9.5

HP has made some major enhancements in QTP 9.5 over the previous versions. [For earlier versions, Please refer difference between QTP 8.2 and 9.2]
Environment Support:

1. QTP 9.5 supports MS Vista 64 bit edition also, in comparison to QTP 9.2's support for 32 bit edition only.
2. Support for omnipresent tabbed browsing is also introduced with this version.

Installation and Add-ins:

1. QTP 9.5 offers all add-ins integrated with the core installation package. Though you still need to buy add-ins separately to get them activated.
2. For installing QTP 9.5 you have to do a clean uninstall (as mentioned in the HP KB) of previous versions. That means, now on, you would not be able to upgrade from the previous versions.

QTP IDE:
A refreshing enhancement in QTP IDE GUI I liked this time was the introduction of the ability to see functions related to current tests. Whether you have an external file containing functions or they are inside a reusable actions, you would be able to see all of them at one place. Now no need to dig and scroll through the endless list of functions in your notepad!
Bitmap checkpoint: Earlier using bitmap checkpoint was a pain in a sense that a minor change in pixel and your test would fail. To increase the tolerance we have to go inside windows registry and change the required tolerance value. With QTP 9.5 HP has introduced support for tolerance level direct from the GUI. You can define tolerance in terms of RGB and/or pixels.
Web Add-in Extensibility: Using this feature you can configure and extend the support to those 3rd party custom web controls ( and new technologies like AJAX) which were not supported with the earlier versions. You need to possess fair amount of JavaScript knowledge to handle this. How much help this feature really provide is yet to be seen!
Download 14days QTP 9.5 evaluation

Read more »

Step by step guide, HOW to learn QTP

1) Start with basics of Software Testing.

2) Learn what is regression testing.

3) learn about when should automation testing come into picture.

Refer When should we automate using QTP?

Now the real thing:

4) Download QTP tutorial. Preferably for the latest version. Go through the the tutorial word by word.

There are some topics that I consider of utmost important for beginners

• Understanding how QTP recognizes objects
• GetToProperty vs GetRoProperty

5) Download QTP demo version. This is valid for 14 days, which - if you work hard - should be enough for you to get a good hold on basics.[Remember to put "quick test professional" as your search string not "qtp" on the HP site]

6) Download QTP Student handbook. It takes you through the step by step sample case study. Practice with the demo applications provided. One is web based and the other is windows based.

7) Next learn VBScript, the scripting language used with QTP. Refer basics of VB Script Part1, Part2 and Part3.

Also learn how to manage the data table with QTP and Excel Part1, Part2, Part3

8) Once you're comfortable with this much, you are ready to take on a real-world project. If you can't get hold of any project, practice with the demo applications provided.

9) I am sure by now you must have started hearing about descriptive programming.

Learn the basics of Descriptive Programming and when it should be preferred over normal record/replay.

10) Once you have mastered the basics, go to Testing Tools Forum . Try answering questions. Remember, questions asked in the forums are from the real-time scenarios - you will get to learn a lot by sharing and reading different approaches to solve a particular problem.

Read more »

Application Testing – Into the Basics of Software Testing!

Topics we will cover in this article:
- Application Testing
- Categories of Applications
- Application Testing Methodologies
- Application Testing Tools
- Software Test Plan
- Application Testing Cycles
- Application Testing – Best Practices

Application Testing is an activity that every software tester performs daily in his career. These two words are extremely broad in practical aspect. However, only the core and most important areas will be discussed here. The purpose of this article is to touch all the primary areas so that the readers will get all the basic briefing at a single place.

Categories of Applications

Whether it is small calculator software with only the basic arithmetic operations, or an online enterprise solution; there are two categories of applications.
a. Desktop
b. Web

For desktop applications, testing should take into account the UI, business logic, database, reports, roles and rights, integrity, usability and data flow. For web applications, along with all these major areas; testers should give sufficient importance to performance, load and security of the application. So AUT is either desktop software or a website.

Application Testing Methodologies

This is a well-known and well discussed aspect; there are only 3 universally accepted methodologies;

a. Black Box: In black-box testing, the AUT is validated against its requirements considering the inputs and expected outputs, regardless of how the inputs are transformed into outputs. Testers are least concerned with internal structure or code that implements the business logic of the application. There are four primary techniques to design test cases for black box testing;

i. BVA (Boundary value Analysis)
ii. EP (Equivalence Partitioning)
iii. Decision Tables
iv. State Transition Tables (and diagrams)

a. White Box: Primary focus of this methodology is to validate, how the business logic of application is implemented by code. Internal structure of the application is tested and the techniques available to do so are;

i. Code Coverage
ii. Path Coverage

Both the above listed techniques contain several other strategies that may be discussed in some other article. Some techniques are discussed in ‘Test Case Design Techniques’ topic.

a. Grey Box: Practically speaking, this is a mixture of black box and white box. In this methodology, mainly the tester tests the application as in black box. But, for some business critical or vulnerable modules of application; testing is done as white box.

Application Testing Tools

According to the best of my knowledge, there are at least 50 testing tools available in market today. These include both paid and open source tools. Moreover, some tools are purpose specific e.g. UI testing, Functional Testing, DB Testing, Load Testing, Performance, Security Testing and Link validation testing etc. However, some tools are strong and provide the facility of testing several major aspects of an application. The general concept of ‘Application Testing’ is its functional testing. So, our focus will be on functional testing tools.

Here is the list of some most important and fundamental features that are provided by almost all of the ‘Functional Testing’ tools.

a. Record and Play
b. Parametrize the Values
c. Script Editor
d. Run (the test or script, with debug and update modes)
c. Report of Run session

Different vendors provide some specific features that make their product unique to other competitor products. But the five features listed above are the most common and can be found in almost all the functional testing tools.

Following is the list of few widely used Functional Testing tools.

1) HP QTP (Quick Test Professional)
2) Selenium
3) IBM Rational Robot
4) Test Complete
5) Push to Test
6) Telerik

Software Test Plan (STP)

For any activity, some planning is always required and same is true for software testing. Without proper plan there is always high risk of getting distracted during the testing. If this risk becomes a fact, the results could be horrible.

Following are the 5 main parts of a good Test Plan:

 
a. Scope
i. Overview of AUT
ii. Features (or areas) to be tested
iii. Exclusions (features or areas not to be tested) with reason
iv. Dependencies (of testing activities on each other, if any)

b. Objectives: This section describes the goals of testing activity e.g. validation of bug fixes, new features added or revamp of AUT etc.

c. Focus: This section describes what aspect of application will be included in the testing e.g. security, functionality, usability, reliability, performance or efficiency etc.

d. Approach: This section describes what testing methodology will be adopted for which areas of AUT. For example, in the STP of an ERP application; the approach section may contain the information that black box testing will be approach for payroll. On the other hand, for reports the approach will be grey box testing.

e. Schedule: This sections describes that who will be doing what and where on the AUT, when and how. Schedule section is, in fact, a ’4Ws and H’ of the STP. Normally it is a simple table, but every organization may have its own customized format according to their own needs. Once the test plan is ready and application is under development; testers design and document the test cases. In the “Application Testing – Methodologies” section above, I have listed the TC design techniques.

Application Testing Cycles

Once the AUT is ready for testing, the practical phase of testing cycle starts in which testers actually execute the test cases on AUT. Keep in mind that here the testing cycle is discussed regardless of Testing Levels (Unit, Module, Integration, System and User Acceptance) and Testing Environments (Dev, QA, Client’s Replica, Live).

a. Smoke Testing: The very first testing cycle that is wide and shallow in approach. The purpose of smoke testing is to verify that there are no crashes in the application and it is suitable for further testing.

b. Sanity Testing: The second testing cycle that is narrow and deep in its approach. Its purpose is to verify that a specific module is working properly and is suitable for complete testing.

Tip: Usually there is not ample amount of time available to run these two cycles separately. So, a mixture of both these cycles is adopted in practical.

c. Functional Testing: The proper and full fledged testing of application is performed in this cycle. The primary focus of this activity is to verify the business logic of the application.

d. Regression Testing: This is the final cycle of testing in which the bug-fixes and/or updates are verified. Moreover, regression testing also ensures that there is no malfunctioning in other areas of AUT due to fixes and changes.

Bugs are logged in every testing cycle. There is no distinct border line between the testing cycles. For example, in Regression the Functionality is also verified and it may also require smoke, sanity or their merger first.

Application Testing – Best Practices

I think, hundreds of articles are available about this on internet. Every article suggests different number of best practices ranging from 7 to 30 (that I have seen so far). However, I have just 5 tips for readers.

1. Plan Properly
2. Test Keenly
3. Log the bugs Clearly
4. Do Regression Test Efficiently
5. Improve above four skills Continuously

Conclusion: Application Testing is a vast subject and the primary activity of any software tester. In this article, I have provided the overview of some most fundamental and necessary areas that fall under this topic. Application Testing involves strategies, phenomena, approaches, tools, technologies and guidelines. However, I have addressed the conceptual and practical insight of its salient concerns.

Read more »

Basic QTP interview questions

1. What are the features and benefits of Quick Test Pro(QTP)?
1. Key word driven testing
2. Suitable for both client server and web based application
3. VB script as the script language
4. Better error handling mechanism
5. Excellent data driven testing features
2. How to handle the exceptions using recovery scenario manager in QTP?
You can instruct QTP to recover unexpected events or errors that occurred in your testing environment during test run. Recovery scenario manager provides a wizard that guides you through the defining recovery scenario. Recovery scenario has three steps
1. Triggered Events
2. Recovery steps
3. Post Recovery Test-Run
3. What is the use of Text output value in QTP?
Output values enable to view the values that the application talks during run time. When parameterized, the values change for each iteration. Thus by creating output values, we can capture the values that the application takes for each run and output them to the data table.
4. How to use the Object spy in QTP 8.0 version?
There are two ways to Spy the objects in QTP
1) Thru file toolbar: In the File ToolBar click on the last toolbar button (an icon showing a person with hat).
2) Thru Object repository Dialog: In Objectrepository dialog click on the button “object spy…” In the Object spy Dialog click on the button showing hand symbol. The pointer now changes in to a hand symbol and we have to point out the object to spy the state of the object. If at all the object is not visible or window is minimized then hold the Ctrl button and activate the required window to and release the Ctrl button.
5. What is the file extension of the code file and object repository file in QTP?
File extension of
Per test object rep: filename.mtr
Shared Object rep: filename.tsr
Code file extension id: script.mts
6. Explain the concept of object repository and how QTP recognizes objects?
Object Repository: displays a tree of all objects in the current component or in the current action or entire test( depending on the object repository mode you selected).
we can view or modify the test object description of any test object in the repository or to add new objects to the repository.
Quicktest learns the default property values and determines in which test object class it fits. If it is not enough it adds assistive properties, one by one to the description until it has compiled the unique description. If no assistive properties are available, then it adds a special Ordianl identifier such as objects location on the page or in the source code.
7. What are the properties you would use for identifying a browser and page when using descriptive programming?
“name” would be another property apart from “title” that we can use. OR
We can also use the property “micClass”.
ex: Browser(“micClass:=browser”).page(“micClass:=page”)
8. What are the different scripting languages you could use when working with QTP?
You can write scripts using following languages:
Visual Basic (VB), XML, JavaScript, Java, HTML
9. Tell some commonly used Excel VBA functions.
Common functions are:
Coloring the cell, Auto fit cell, setting navigation from link in one cell to other saving
10. Explain the keyword createobject with an example.
Creates and returns a reference to an Automation object
syntax: CreateObject(servername.typename [, location])
Arguments
servername:Required. The name of the application providing the object.
typename : Required. The type or class of the object to create.
location : Optional. The name of the network server where the object is to be created.
11. Explain in brief about the QTP Automation Object Model.
Essentially all configuration and run functionality provided via the QuickTest interface is in some way represented in the QuickTest automation object model via objects, methods, and properties. Although a one-on-one comparison cannot always be made, most dialog boxes in QuickTest have a corresponding automation object, most options in dialog boxes can be set and/or retrieved using the corresponding object property, and most menu commands and other operations have corresponding automation methods. You can use the objects, methods, and properties exposed by the QuickTest automation object model, along with standard programming elements such as loops and conditional statements to design your program.
12. How to handle dynamic objects in QTP?
QTP has a unique feature called Smart Object Identification/recognition. QTP generally identifies an object by matching its test object and run time object properties. QTP may fail to recognize the dynamic objects whose properties change during run time. Hence it has an option of enabling Smart Identification, wherein it can identify the objects even if their properties changes during run time.
Check out this:
If QuickTest is unable to find any object that matches the recorded object description, or if it finds more than one object that fits the description, then QuickTest ignores the recorded description, and uses the Smart Identification mechanism to try to identify the object.
While the Smart Identification mechanism is more complex, it is more flexible, and thus, if configured logically, a Smart Identification definition can probably help QuickTest identify an object, if it is present, even when the recorded description fails.
The Smart Identification mechanism uses two types of properties:
Base filter properties – The most fundamental properties of a particular test object class; those whose values cannot be changed without changing the essence of the original object. For example, if a Web link’s tag was changed from to any other value, you could no longer call it the same object. Optional filter properties – Other properties that can help identify objects of a particular class as they are unlikely to change on a regular basis, but which can be ignored if they are no longer applicable.
13. What is a Run-Time Data Table? Where can I find and view this table?
In QTP, there is data table used, which is used at runtime.
-In QTP, select the option View->Data table.
-This is basically an excel file, which is stored in the folder of the test created, its name is Default.xls by default.
14. How does Parameterization and Data-Driving relate to each other in QTP?
To data driven we have to parameterize. i.e. we have to make the constant value as parameter, so that in each interaction(cycle) it takes a value that is supplied in run-time data table. Through parameterization only we can drive a transaction (action) with different sets of data. You know running the script with the same set of data several times is not suggested, and it’s also of no use.
15. What is the difference between Call to Action and Copy Action.?
Call to Action: The changes made in Call to Action, will be reflected in the original action (from where the script is called). But where as in Copy Action , the changes made in the script ,will not effect the original script(Action)
16. Explain the concept of how QTP identifies object.
During recording qtp looks at the object and stores it as test object. For each test object QT learns a set of default properties called mandatory properties, and look at the rest of the objects to check whether this properties are enough to uniquely identify the object. During test run, QTP searches for the run time objects that matches with the test object it learned while recording.
17. Differentiate the two Object Repository Types of QTP.
Object repository is used to store all the objects in the application being tested.
Types of object repository: Per action and shared repository.
In shared repository only one centralized repository for all the tests. where as in per action for each test a separate per action repository is created.
18. What the differences are and best practical application of Object Repository?
Per Action: For Each Action, one Object Repository is created.
Shared: One Object Repository is used by entire application
19. Explain what the difference between Shared Repository and Per Action Repository
Shared Repository: Entire application uses one Object Repository , that similar to Global GUI Map file in WinRunner
Per Action: For each Action, one Object Repository is created, like GUI map file per test in WinRunner
20. Have you ever written a compiled module? If yes tell me about some of the functions that you wrote.
Sample answer (You can tell about modules you worked on. If your answer is Yes then You should expect more questions and should be able to explain those modules in later questions): I Used the functions for Capturing the dynamic data during runtime. Function used for Capturing Desktop, browser and pages.
21. Can you do more than just capture and playback?
Sample answer (Say Yes only if you worked on): I have done Dynamically capturing the objects during runtime in which no recording, no playback and no use of repository is done AT ALL.
-It was done by the windows scripting using the DOM(Document Object Model) of the windows.
22. How to do the scripting. Are there any inbuilt functions in QTP? What is the difference between them? How to handle script issues?
Yes, there’s an in-built functionality called “Step Generator” in Insert->Step->Step Generator -F7, which will generate the scripts as you enter the appropriate steps.
23. What is the difference between check point and output value?
An output value is a value captured during the test run and entered in the run-time but to a specified location.
EX:-Location in Data Table[Global sheet / local sheet]
24. How many types of Actions are there in QTP?
There are three kinds of actions:
Non-reusable action – An action that can be called only in the test with which it is stored, and can be called only once.
Reusable action – An action that can be called multiple times by the test with which it is stored (the local test) as well as by other tests.
External action – A reusable action stored with another test. External actions are read-only in the calling test, but you can choose to use a local, editable copy of the Data Table information for the external action.
25. I want to open a Notepad window without recording a test and I do not want to use System utility Run command as well. How do I do this?
You can still make the notepad open without using the record or System utility script, just by mentioning the path of the notepad “( i.e. where the notepad.exe is stored in the system) in the “Windows Applications Tab” of the “Record and Run Settings window.

Read more »

Features of QTP:

QTP is widely used test automation tool mainly for functional testing. QTP has many more advanced options and HP recommends that all existing and new users should begin with QTP instead of WinRunner.
Features of QTP:

• Ease of use.
• Simple interface.
• Presents the test case as a business workflow to the tester (simpler to understand).
• Uses a real programming language (Microsoft’s VBScript) with numerous resources available.
• QuickTest Pro is significantly easier for a non-technical person to adapt to and create working test cases, compared to WinRunner.
• Data table integration better and easier to use than WinRunner.
• Test Run Iterations/Data driving a test is easier and better implement with QuickTest.
• Parameterization easier than WinRunner.
• Can enhance existing QuickTest scripts without the “Application Under Test” being available; by using the ActiveScreen.
• Can create and implement the Microsoft Object Model (Outlook objects, ADO objects, FileSystem objects, supports DOM, WSH, etc.).
• Better object identification mechanism.
• Numerous existing functions available for implementation – both from within QuickTest Pro and VBScript.
• QTP supports .NET development environment
• XML support
• The Test Report is more robust in QuickTest compared to WinRunner.
• Integrates with TestDirector and WinRunner (can kick off WinRunner scripts from QuickTest).

Read more »

Organizing the Test Team

The people component includes human resource allocations and the required skill sets. The test team should comprise the highest-caliber personnel possible. They are usually extremely busy because their talents put them in great demand, and it therefore becomes vital to build the best case possible for using these individuals for test purposes. A test team leader and test team need to have the right skills and experience, and be motivated to work on the project. Ideally, they should be professional quality assurance specialists but can represent the executive sponsor, users, technical operations, database administration, computer center, independent parties, etc. The latter is particularly useful during final system and acceptance testing. In any event, they should not represent the development team, for they may not be as unbiased as an outside party. This is not to say that developers shouldn’t test. For they should unit and function test their code extensively before handing it over to the test team.

There are two areas of responsibility in testing:

1. Testing the application, which is the responsibility of the test team
2. The overall testing processes, which is handled by the test manager.

The test manager directs one or more testers, is the interface between quality assurance and the development organization, and manages the overall testing effort. Responsibilities include:

• Setting up the test objectives
• Defining test resources
• Creating test procedures
• Developing and maintaining the test plan
• Designing test cases
• Designing and executing automated testing tool scripts
• Test case development
• Providing test status
• Writing reports
• Defining the roles of the team members
• Managing the test resources
• Defining standards and procedures
• Ensuring quality of the test process
• Training the team members
• Maintaining test statistics and metrics

The test team must be a set of team players and have the following responsibilities:

• Execute test cases according to the plan
• Evaluate the test results
• Report errors
• Design and execute automated testing tool scripts
• Recommend application improvements
• Record defects

The main function of a team member is to test the application and report defects to the development team by documenting them in a defect tracking system. Once the development team corrects the defects, the test team reexecutes the tests which discovered the original defects.

It should be pointed out that the roles of the test manager and team members are not mutually exclusive. Some of the team leader’s responsibilities are shared with the team member and visa versa.

The basis for allocating dedicated testing resources is the scope of the functionality and the development time frame, e.g., a medium development project will require more testing resources than a small one. If project A of medium complexity requires a testing team of 5, project B with twice the scope would require 10 testers (given the same resources).

Another rule of thumb is that the testing costs approach 25% of the total budget. Since the total project cost is known, the testing effort can be calculated and translated to tester headcount.

The best estimate is a combination of the project scope, test team skill levels, and project history. A good measure of required testing resources for a particular project is the histories of multiple projects, i.e., testing resource levels and performance compared to similar projects.

Read more »

Software Test Planning

The purpose of test planning is to provide the basis for accomplishing testing in an organized manner. From a managerial point of view it is the most important document, because it helps manage the test project. If a test plan is comprehensive and carefully thought out, test execution and analysis should proceed smoothly.

The test plan is an ongoing document, particularly in the spiral environment since the system is constantly changing. As the system changes, so does it. A good test plan is one which:

• Has a good chance of detecting a majority of the defects
• Provides test coverage for most of the code
• Is flexible
• Is executed easily, repeatably, and automatically
• Defines the types of tests to be performed
• Clearly documents the expected results
• Provides for defect reconciliation when a defect is discovered
• Clearly defines the test objectives
• Clarifies the test strategy
• Clearly defines the test exit criteria
• Is not redundant
• Identifies the risks
• Documents the test requirements
• Defines the test deliverables

The planning test methodology includes three steps:

1. Building the test plan
2. defining the metrics
3. reviewing/approving the test plan.

Step 1: Build a Test Plan

- Prepare an Introduction
- Define the High-Level Functional Requirements (In Scope)
- Identify Manual / Automated Test Types
- Identify the Test Exit Criteria
- Establish Regression Test Strategy
- Define the Test Deliverables
- Organize the Test Team
- Establish a Test Environment
- Define the Dependencies
- Create a Test Schedule
- Select the Test Tools
- Establish Defect Recording / Tracking Procedures
- Establish Change Request Procedures
- Establish Version Control Procedures
- Define Configuration Build Procedures
- Define Project Issue Resolution Procedures
- Establish Reporting Procedures
- Define Approval Procedures

Step 2: Define the Metric Objectives

- Define the Metrics
- Define the Metric Points

Step 3: Review/Approve the Plan

- Schedule / Conduct the Review
- Obtain Approvals

Read more »

Software Testing Techniques and Levels

In this post, Techniques cover different ways testing can be accomplished. Testing techniques can be defined in three ways: Preparation, Execution and Approach.

Preparation: From preparation point of view there are two testing techniques: Formal Testing and Informal Testing.

Formal Testing: Testing performed with a plan, documented set of test cases, etc that outline the methodology and test objectives. Test documentation can be developed from requirements, design, equivalence partitioning, domain coverage, error guessing, etc. The level of formality and thoroughness of test cases will depend upon the needs of the project. Some projects can have rather informal ‘formal test cases’, while others will require a highly refined test process. Some projects will require light testing of nominal paths while others will need rigorous testing of exceptional cases.

Informal Testing: Ad hoc testing performed without a documented set of objectives or plans. Informal testing relies on the intuition and skills of the individual performing the testing. Experienced engineers can be productive in this mode by mentally performing test cases for the scenarios being exercised.

From the execution point of view, the two testings types are: Manual Testing and Automated Testing.

Manual Testing: Manual testing involves direct human interaction to exercise software functionality and note behavior and deviations from expected behavior.

Automated Testing: Testing that relies on a tool, built-in test harness, test framework, or other automatic mechanism to exercise software functionality, record output, and possibly detect deviations. The test cases performed by automated testing are usually defined as software code or script that drives the automatic execution.

From the testing approach point of view, the two testings types are: Structural Testing and Functional Testing.

Structural Testing: Structural testing depends upon knowledge of the internal structure of the software. Structural testing is also referred to as white-box testing.

Data-flow Coverage: Data-flow coverage tests paths from the definition of a variable to its use.



Control-flow Coverage

Statement Coverage: Statement coverage requires that every statement in the code under test has been executed.

Branch Coverage: Branch coverage requires that every point of entry and exit in the program has been executed at least once, and every decision in the program has taken all possible outcomes at least one.

Condition Coverage: Condition coverage is branch coverage with the additional requirement that “every condition in a decision in the program has taken all possible outcomes at least once.” Multiple condition coverage requires that all possible combinations of the possible outcomes of each condition have been tested. Modified condition coverage requires that each condition has been tested independently.

Functional Testing: Functional testing compares the behavior of the test item to its specification without knowledge of the item’s internal structure. Functional testing is also referred to as black box testing.

Requirements Coverage: Requirements coverage requires at least one test case for each specified requirement. A traceability matrix can be used to insure that requirements coverage has been satisfied.

Input Domain Coverage: Input domain coverage executes a function with a sufficient set of input values from the function’s input domain. The notion of a sufficient set is not completely definable, and complete coverage of the input domain is typically impossible. Therefore the input domain is broken into subsets, or equivalence classes, such that all values within a subset are likely to reveal the same defects. Any one value within an equivalence class can be used to represent the whole equivalence class. In addition to a generic representative, each extreme value within an equivalence class should be covered by a test case. Testing the extreme values of the equivalence classes is referred to as boundary value testing.

Output Domain Coverage: Output domain coverage executes a function in such a way that a sufficient set of output values from the function’s output domain is produced. Equivalence classes and boundary values are used to provide coverage of the output domain. A set of test cases that “reach” the boundary values and a typical value for each equivalence class is considered to have achieved output domain coverage.

Various Software Testing Levels

Although many testing levels tend to be combined with certain techniques, there are no hard and fast rules. Some types of testing imply certain lifecycle stages, software deliverables, or other project context. Other types of testing are general enough to be done almost any time on any part of the system. Some require a particular methodology. When appropriate common utilizations of a particular testing type will be described. The project’s test plan will normally define the types of testing that will be used on the project, when they will be used, and the strategies they will be used with. Test cases are then created for each testing type.

Unit Testing: A unit is an abstract term for the smallest thing that can be conveniently tested. This will vary based on the nature of a project and its technology but usually focuses at the subroutine level. Unit testing is the testing of these units. Unit testing is often automated and may require creation of a harness, stubs, or drivers.

Component Testing: A component is an aggregate of one or more components. Component testing expands unit testing to include called components and data types. Component testing is often automated and may require creation of harness, stubs, or drivers.

Single Step Testing: Single step testing is performed by stepping through new or modified statements of code with a debugger. Single step testing is normally manual and informal.

Bench Testing: Bench testing is functional testing of a component after the system has been built in a local environment. Bench testing is often manual and informal.

Developer Integration Testing: Developer integration testing is functional testing of a component after the component has been released and the system has been deployed in a standard testing environment. Special attention is given to the flow of data between the new component and the rest of the system.

Smoke Testing: Smoke testing determines whether the system is sufficiently stable and functional to warrant the cost of further, more rigorous testing. Smoke testing may also communicate the general disposition of the current code base to the project team. Specific standards for the scope or format of smoke test cases and for their success criteria may vary widely among projects.

Feature Testing: Feature testing is functional testing directed at a specific feature of the system. The feature is tested for correctness and proper integration into the system. Feature testing occurs after all components of a feature have been completed and released by development.

Integration Testing: Integration testing focuses on verifying the functionality and stability of the overall system when it is integrated with external systems, subsystems, third party components, or other external interfaces.

System Testing: System testing occurs when all necessary components have been released internally and the system has been deployed onto a standard environment. System testing is concerned with the behavior of the whole system. When appropriate, system testing encompasses all external software, hardware, operating environments, etc. that will make up the final system.

Release Testing: Release tests ensure that interim builds can successfully deployed by the customer. This includes product deployment, installation, and a pass through the primary functionality. This test is done immediately before releasing to the customer.

Beta Testing: Beta testing consists of deploying the system to many external users who have agreed to provide feedback about the system. Beta testing may also provide the opportunity to explore release and deployment issues.

Acceptance Testing: Acceptance testing compares the system to a predefined set of acceptance criteria. If the acceptance criteria are satisfied by the system, the customer will accept delivery of the system.

Regression Testing: Exercises functionality that has stabilized. Once high confidence has been established for certain parts of the system, it is generally wasted effort to continue rigorous, detailed testing of those parts. However, it is possible that continued evolution of the system will have negative effects on previously stable and reliable parts of the system. Regression testing offers a low-cost method of detecting such side effects. Regression testing is often automated and focused on critical functionality.

Performance Testing: Performance testing measures the efficiency with respect to time and hardware resources of the test item under typical usage. This assumes that a set of non-functional requirements regarding performance exist in the item’s specification.

Stress Testing: Stress testing evaluates the performance of the test item during extreme usage patterns. Typical examples of “extreme usage patterns” are large data sets, complex calculations, extended operation, limited system resources, etc.

Configuration Testing: Configuration testing evaluates the performance of the test item under a range of system configurations. Relevant configuration issues depend upon the particular product and may include peripherals, network patterns, operating systems, hardware devices and drivers, user settings.

Read more »

Database Testing Overview: Different type of Keys used in DBMS

A key is an attribute (also known as column or field) or a combination of attribute that is used to identify records.
There is different type of keys available in database management system:

1. Primary Key
2. Foreign Key
3. Candidate Key
4. Composite Key
5. Alternate Key
6. Secondary Key
7. Super Key

1) Primary Key: The primary key of a relational table uniquely identifies each record in the table. A Primary Key can consist of one or more attributes of a table.
2) Foreign Key: A foreign key is an attribute or combination of attribute in one base table that points to the candidate key (generally it is the primary key) of another table. The purpose of the foreign key is to ensure referential integrity of the data i.e. only values that are supposed to appear in the database are permitted.
3) Candidate Key: A candidate key is a field or combination of fields that can act as a primary key field for that table to uniquely identify each record in that table. Each table may have one or more candidate keys.
4) Composite Key: If we use multiple attributes to create a Primary Key then that Primary Key is called Composite Key (also called a Compound Key or Concatenated Key).
5) Alternate Key: Alternate Key can be any of the Candidate Keys except for the Primary Key.
6) Secondary Key: An attribute or set of attributes that is basis for retrieval is known as secondary key.
For example, address in student table can be used to display all students who are living in a particular city. In this case, address will be used as secondary key.
7) Super Key: An attribute or a combination of attribute that is used to identify the records uniquely is known as Super Key.

Read more »

RTM, VV, Static and Dynamic Testing

1. Requirements Traceability Matrix (RTM)

A document showing the relationship/mapping between Test Requirements and Test Cases.

Elements of RTM:

a. Requirements ID

b. Requirements Description

c. Test Case ID

d. Status [Open, Closed, Defer (Later), On hold]

2. Verification and Validation (VandV)

Verification is the process confirming that something software meets its specification. Validation is the process confirming that it meets the user's requirements.

Difference between Verification and Validation:

Suppose, you are going to buy a pair of shoes having number 9 for you. You have chosen a pair and seen the tag with 9 written on it. This is verification, because your requirement was to buy a pair of shoes with 9 number.

But when you tried to wear it and found that shoe is not fitted into your feet. After inquery, you have found that company has tagged it 9 number by mistake. Actually it was 7 number shoe. This process is called Validation.

Example of Verification: Creating Traceability Matrix

Example of Validation: Executing Test Cases

3. Static and Dynamic Testing

Static black-box testing: Testing the specification is static black-box testing.

Two Types of Static black-box testing:

1. High-level review techniques

a. Research Existing Standards and Guidelines

b. Review and Test Similar Software

2. Low-level techniques

a. Specification Attributes Checklist (e.g. Spec must be complete, accurate, precise, consistent etc)

b. Specification Terminology Checklist (e.g. focus on the terms in Spec like "If…Then…(but missing Else)." or "Etc., And So Forth, And So On" etc)

Dynamic Black-Box Testing: Testing software without knowledge of code is dynamic black-box testing.

Static White-Box Testing: Static white-box testing is the process of carefully reviewing the software design, architecture, or code for bugs without executing it.

Three Types of Static White-Box Testing:

a. Peer Reviews: Peer Reviews are the least formal method. Peer reviews are often held with just the programmer who wrote the code and one or two other programmers or testers acting as reviewers.

b. Walkthroughs: In a Walkthrough, the programmer who wrote the code formally presents it to a small group of five or so other programmers and testers. The presenter reads through the code line by line, or function by function, explaining what the code does and why. The reviewers listen and question anything that looks suspicious.

c. Inspections: Inspections are the most formal type of reviews and more formalized than a 'walkthrough', typically with 3-8 people including a moderator, reader, and a recorder to take notes. The other participants are called inspectors.

Walkthrough:

1. It's a type of Semi Formal Review.

2. 2 to 7 People ate attaining it.

3. Author is Presenter.

4. Lead by Author only.

5. Reviewers are not aware of the subject/topic.

Inspection:

1. It's totally a Formal Review.

2. 2 to 10 or more People attaining it.

3. Author is not presenter. Some one else is giving presentation.

4. Lead by Moderator.

5. Reviewers are aware & well prepared for the subject/topic.

6. Recorder is noting down everything. Like defects, changes, improvements etc.

Dynamic White-Box Testing: is a method of testing software that tests internal structures or workings of an application.

Difference between Dynamic White-Box Testing and Debugging:

The goal of dynamic white-box testing is to find bugs. The goal of debugging is to fix them.

Read more »

Types of Software Testing

We have come across so many types of software testing. The two major approaches of software testing are manual software testing and automated software testing. Manual software testing means it is being done by a man. That means a person, ie a tester runs the software for errors. We would discuss each testing in detail later. Since this is being an introduction we would just mention each of them here.

Manual software testing has different stages. The following are the stages:

1. Unit testing
2. Integration testing
3. System testing
4. User acceptance testing

Different techniques are used for software techniques. The commonly used software testing techniques are the following:

• White box testing
• Black box testing
• Grey box testing

White box software testing is the testing of the working of the software and its internal structures. It can detect errors of the implemented parts, but the unimplemented parts goes undeteted. Black box testing is the testing of the functionality of the software as opposed to its internal structure. It can be done at all levels of software testing. Now automatically you will be eager to know what is grey box software testing, right? Well, the answer is too simple because it is the combination of white box as well as black box testing.

Read more »

Tips for Writing Test Cases

One of the most frequent and major activity of a Software Tester (SQA/SQC person) is to write Test Cases. First of all, kindly keep in mind that all this discussion is about ‘Writing Test Cases’ not about designing/defining/identifying TCs.

There are some important and critical factors related to this major activity.Let us have a bird’s eye view of those factors first.

a. Test Cases are prone to regular revision and update:

We live in a continuously changing world, software are also not immune to changes. Same holds good for requirements and this directly impacts the test cases. Whenever, requirements are altered, TCs need to be updated. Yet, it is not only the change in requirement that may cause revision and update to TCs.

During the execution of TCs, many ideas arise in the mind, many sub-conditions of a single TC cause update and even addition of TCs. Moreover, during regression testing several fixes and/or ripples demand revised or new TCs.

b. Test Cases are prone to distribution among the testers who will execute these:

Of course there is hardly the case that a single tester executes all the TCs. Normally there are several testers who test different modules of a single application. So the TCs are divided among them according to their owned areas of application under test. Some TCs related to integration of application, may be executed by multiple testers while some may be executed only by a single tester.

c. Test Cases are prone to clustering and batching:

It is normal and common that TCs belonging to a single test scenario usually demand their execution in some specific sequence or in the form of group. There may be some TCs pre-requisite of other TCs. Similarly, according to the business logic of AUT, a single TC may contribute in several test conditions and a single test condition may consist of multiple TCs.

d. Test Cases have tendency of inter-dependence:

This is also an interesting and important behavior of TCs that those may be interdependent on each other. In medium to large applications with complex business logic, this tendency is more visible.

The clearest area of any application where this behavior can definitely be observed is the interoperability between different modules of same or even different applications. Simply speaking, wherever the different modules or applications are interdependent, the same behavior is reflected in the TCs.

e. Test Cases are prone to distribution among developers (especially in TC driven development environment):

An important fact about TCs is that these are not only to be utilized by the testers. In normal case, when a bug is under fix by the developers, they are indirectly using the TC to fix the issue. Similarly, where the TCD development is followed, TCs are directly used by the developers to build their logic and cover all scenarios, addressed by TCs, in their code.

So, keeping the above 5 factors in mind, here are some tips to write test cases:

1. Keep it simple but not too simple; make it complex but not too complex:

This statement seems a paradox, but I promise it is not so. Keep all the steps of TCs atomic, precise with correct sequence and with correct mapping to expected results, this is what I mean to make it simple.

Now, making it complex in fact means to make it integrated with the Test Plan and other TCs. Refer to other TCs, relevant artifacts, GUIs etc. where and when required. But do this in balanced way, do not make tester to move to and fro in the pile of documents for completing single test scenario. On the other hand do not even let the tester wish you had documented these TCs in some compact manner. While writing TCs, always remember that you or someone else will have to revise and update these.

2. After documenting Test cases, review once as Tester:

Never think that the job is done once you have written the last TC of the test scenario. Go to the start and review all the TCs once, but not with the mind of TC writer or Testing Planner. Review all TCs with the mind of a tester. Think rationally and try to dry run your TCs. Evaluate that all the Steps you have mentioned are clearly understandable, and expected results are in harmony with those steps.

The test data specified in TCs is feasible not only for actual testers but is according to real time environment too. Ensure that there is no dependency conflict among TCs and also verify that all references to other TCs/artifacts/GUIs are accurate because, testers may be in great trouble otherwise.

3. Bound as well as ease the testers:

Do not leave test data on testers, give them range of inputs especially where calculations are to be performed or application’s behavior is dependent on inputs. You may divide the test item values among them, but never give them liberty to choose the test data items themselves. Because, intentionally or unintentionally, they may use same test data and some important test data may be ignored during the execution of TCs.

Keep the testers eased by organizing TCs according to the testing categories and related areas of application. Clearly instruct and mention which TCs are inter-dependent and/or batched. Similarly, explicitly indicate which TCs are independent and isolated so that tester may manage his overall activity at his or her own will.

4. Be a Contributor:

Never accept the FS or Design Document as it is. Your job is not just to go through the FS and identifying the Test Scenarios. Being a quality related resource, never hesitate to contribute. Suggest to developers too, especially in TC-driven development environment. Suggest the drop-down-lists, calendar controls, selection-list, group radio buttons, more meaningful messages, cautions, prompts, improvements related to usability etc.

5. Never Forget the End User

The most important stakeholder is the ‘End User’ who will actually use the AUT. So, never forget him at any stage of TCs writing. In fact, End User should not be ignored at any stage throughout the SDLC, yet my emphasis so far is just related to my topic. So, during the identification of test scenarios, never overlook those cases which will be mostly used by the user or are business critical even of less frequent use. Imagine yourself as End User, once go through all the TCs and judge the practical value of executing all your documented TCs.

Conclusion:

Test Case Writing is an activity which has solid impact on the whole testing phase. This fact makes the task of documenting TCs, very critical and subtle. So, it should be properly planned first and must be done in well-organized manner. The person who is documenting the TCs must keep in mind that, this activity is not for him or her only, but a whole team including other testers and developers, as well as the customer will be directly and indirectly affected by this work.

So, the due attention must be paid during this activity. “Test Case Document” must be understandable for all of its users, in an unambiguous way and should be easily maintainable. Moreover, TC document must address all important features and should cover all important logical flows of the AUT, with real time and practically acceptable inputs.

Read more »