Testing Strategies
Strategy is a general approach rather than a method of devising particular systems for component tests.
Different strategies may be adopted depending on the type of system to be tested and the development process used. The testing strategies are
{mosgoogle left}
Top-Down Testing
Bottom - Up Testing
Thread Testing
Stress Testing
Back- to Back Testing
1. Top-down testing
Where testing starts with the most abstract component and works downwards.
2. Bottom-up testing
Where testing starts with the fundamental components and works upwards.
3. Thread testing
Which is used for systems with multiple processes where the processing of a transaction threads its way through these processes.
4. Stress testing
Which relies on stressing the system by going beyond its specified limits and hence testing how well the system can cope with over-load situations.
5. Back-to-back testing
Which is used when versions of a system are available. The systems are tested together and their outputs are compared. 6. Performance testing.
This is used to test the run-time performance of software.
7. Security testing.
This attempts to verify that protection mechanisms built into system will protect it from improper penetration.
8. Recovery testing.
This forces software to fail in a variety ways and verifies that recovery is properly performed.
Large systems are usually tested using a mixture of these strategies rather than any single approach. Different strategies may be needed for different parts of the system and at different stages in the testing process.
Whatever testing strategy is adopted, it is always sensible to adopt an incremental approach to sub-system and system testing. Rather than integrate all components into a system and then start testing, the system should be tested incrementally. Each increment should be tested before the next increment is added to the system. This process should continue until all modules have been incorporated into the system.
When a module is introduced at some stage in this process, tests, which were previously unsuccessful, may now, detect defects. These defects are probably due to interactions with the new module. The source of the problem is localized to some extent, thus simplifying defect location and repai
Debugging
Brute force, backtracking, cause elimination.
|
Unit Testing
|
Coding
|
Focuses on each module and whether it works properly. Makes heavy use of white box testing
|
|
Integration Testing
|
Design
|
Centered on making sure that each module works with another module. |
|
Validation Testing
|
Analysis
|
Ensuring conformity with requirements
|
|
Systems Testing
|
Systems Engineering
|
Making sure that the software product works with the external environment, e.g., computer system, other software products.
|
Driver and Stubs {mosgoogle left}
Driver: dummy main program
Stub: dummy sub-program
This is because the modules are not yet stand-alone programs therefore drive and or stubs have to be developed to test each unit.
When do we prepare a Test Plan?
[Always prepared a Test Plan for every new version or release of the product? ]
For four or five features at once, a single plan is fine. Write new test cases rather than new test plans. Write test plans for two very different purposes. Sometimes the test plan is a product; sometimes it's a tool.
What is boundary value analysis?
Boundary value analysis is a technique for test data selection. A test engineer chooses values that lie along data extremes. Boundary values include maximum, minimum, just inside boundaries, just outside boundaries, typical values, and error values. The expectation is that, if a systems works correctly for these extreme or special values, then it will work correctly for all values in between. An effective way to test code is to exercise it at its natural boundaries.
Boundary Value Analysis is a method of testing that complements equivalence partitioning. In this case, data input as well as data output are tested. The rationale behind BVA is that the errors typically occur at the boundaries of the data. The boundaries refer to the upper limit and the lower limit of a range of values or more commonly known as the "edges" of the boundary.
Describe methods to determine if you are testing an application too much?
Answer1:
While testing, you need to keep in mind following two things always:
-- Percentage of requirements coverage
-- Number of Bugs present + Rate of fall of bugs
-- Firstly, There may be a case where requirement is covered quite adequately but number of bugs do not fall. This indicates over testing.
--- Secondly, There may be a case where those parts of application are also being tested which are not affected by a CHANGE or BUG FIXTURE. This is again a case of over testing.
-- Third is the case as you have suggested, with slight modification, i.e bug has sufficiently dropped off but still testing is being at SAME levels as before.
Methods to determine if an application is being over-tested are--
1. Comparison of 'Rate of Drop in number of Bugs' & 'Effort Invested in Testing' (With all Requirements been met) That is, if bug rate is falling (as it generally happens in all applications), but effort invested in man hours does not fall, this implies Over testing.
2. Comparison of 'Achievment of bug rate threshold' & 'Effort Invested in Testing' (With all Requirements been met) That is, if bug rate has already achieved the agreed-upon value with business and still the testing efforts are being invested with no/little reduction.
3. Verifying if the 'Impact Analysis' for 'Change Requests' has been done properly and being implemented correctly. That is, to check and verify that the components of AUT which have got impacted by the new change are being tested only and no other unrequired component is being tested unneccessarily. If unaffected components are being tested, this implies Over testing.
Answer2:
If the bug find rate has dropped off considerably, the test group should shift its testing strategy. One of the key problems with heavy reliance on regression testing is that the bug find rate drops off even though there are plenty of bugs not yet found. To find new bugs, you have to run new tests.
Every test technique is stronger for some types of bugs and weaker for others. Many test groups use only a few techniques. In our consulting, James Bach and I repeatedly worked with companies that relied on only one or two main techniques.
When one technique, any one test technique, yields few bugs, shifting to new technique(s) is likely to expose new problems.
At some point, you can use a measure that is only partially statistical -- if your bug find rate is low AND you can't think of any new testing approaches that look promising, THEN you are at the limit of your effectiveness and you should ship the product. That still doesn't mean that the application is overtested. It just means that YOU'RE not going to find many new bugs.
Answer3:
Best way is to monitor the test defects over the period of time
Refer williams perry book, where he has mentioned the concept of 'under test' and 'over test', in fact the data can be plotted to see the criteria.
Yes one of the criteria is to monitor the defect rate and see if it is almost zero second method would be using test coverage when it reach 100% (or 100% requirement coverage)
Procedural Software Testing Issues
Software testing in the traditional sense can miss a large number of errors if used alone. That is why processes like Software Inspections and Software Quality Assurance (SQA) have been developed. However, even testing all by itself is very time consuming and very costly. It also ties up resources that could be used otherwise. When combined with inspections and/or SQA or when formalized, it also becomes a project of its own requiring analysis, design and implementation and supportive communications infrastructure. With it interpersonal problems arise and need managing. On the other hand, when testing is conducted by the developers, it will most likely be very subjective. Another problem is that developers are trained to avoid errors. As a result they may conduct tests that prove the product is working as intended (i.e. proving there are no errors) instead of creating test cases that tend to uncover as many errors as possible.
How do I start with testing?
Think twice (or may be more) times before you choose a career. Are you interested in it or do u just want to jump on the bandwagon?
Prerequisite
You can join a software development company as a tester if you can convince the interviewer
1. You have a knack for breaking software
2. You are aware of basic Quality concepts and belive in them
3. You want to pursue Testing as a career and not just to try it
OO Software Testing Issues
A common way of testing OO software testing-by-poking-around (Binder, 1995). In this case the developer's goal is to show that the product can do something useful without crashing. Attempts are made to "break" the product. If and when it breaks, the errors are fixed and the product is then deemed "tested".
Testing-by-poking-around method of testing OO software is, in my opinion, as unsuccessful as random testing of procedural code or design. It leaves the finding of errors up to a chance.
Another common problem in OO testing is the idea that since a superclass has been tested, any subclasses inheriting from it don't need to be.
This is not true because by defining a subclass we define a new context for the inherited attributes. Because of interaction between objects, we have to design test cases to test each new context and re-test the superclass as well to ensure proper working order of those objects.
Yet another misconception in OO is that if you do proper analysis and design (using the class interface or specification), you don't need to test or you can just perform black-box testing only.
However, function tests only try the "normal" paths or states of the class. In order to test the other paths or states, we need code instrumentation. Also it is often difficult to exercise exception and error handling without examination of the source code.
What is the purpose of black box testing?
Answer1:
The main purpose of BB Testing is to validate that the application works as the user will be operating it and in the environments of their systems. How do you do system testing and integration testing?
You may lose time and money but you may also lose Quality and eventually Customers!
Answer2:
"What is the purpose of black box testing?"
Black-box testing checks that the user interface and user inputs and outputs all work correctly. Part of this is that error handling must work correctly. It's used in functional and system testing.
"We do everything in white box testing: - we check each module's function in the unit testing"
Who is "we"? Are you programmers or quality assurance testers? Usually, unit testing is done by programmers, and white-box testing would be how they'd do it.
"- once unit test result is ok, means that modules work correctly (according to the requirement documemts)"
Not quite. It means that on a stand-alone basis, each module is okay. White box testing only tests the internal structure of the program, the code paths. Functional testing is needed to test how the individual components work together, and this is best done from an external perspective, meaning by using the software the way an end user would, without reference to the code (which is what black-box testing is).
if we doing testing again in black box will we lose time and money?"
No, the opposite: You'll lose money from having to repair errors you didn't catch with the white-box testing if you don't do some black-box testing. It's far more expensive to fix errors after release than to test for them and fix them early on.
But again, who is "we"? The black box testers should not be the people who did the programming; they should be the QA team -- also some end users for the usability testing.
Now that I've said that, good programmers will run some basic black-box tests before handing the application to QA for testing. This isn't a substitute for having QA do the tests, but it's a lot quicker for the programmer to find and fix an error right away than to have to go through the whole process of reporting a bug, then fixing and releasing a new build, then retesting.
How do you create a test plan/design?
Test scenarios and/or cases are prepared by reviewing functional requirements of the release and preparing logical groups of functions that can be further broken into test procedures. Test procedures define test conditions, data to be used for testing and expected results, including database updates, file outputs, report results. Generally speaking...
* Test cases and scenarios are designed to represent both typical and unusual situations that may occur in the application.
* Test engineers define unit test requirements and unit test cases. Test engineers also execute unit test cases.
* It is the test team that, with assistance of developers and clients, develops test cases and scenarios for integration and system testing.
* Test scenarios are executed through the use of test procedures or scripts.
* Test procedures or scripts define a series of steps necessary to perform one or more test scenarios.
* Test procedures or scripts include the specific data that will be used for testing the process or transaction.
* Test procedures or scripts may cover multiple test scenarios.
* Test scripts are mapped back to the requirements and traceability matrices are used to ensure each test is within scope.
* Test data is captured and base lined, prior to testing. This data serves as the foundation for unit and system testing and used to exercise system functionality in a controlled environment.
* Some output data is also base-lined for future comparison. Base-lined data is used to support future application maintenance via regression testing.
* A pretest meeting is held to assess the readiness of the application and the environment and data to be tested. A test readiness document is created to indicate the status of the entrance criteria of the release.
Inputs for this process:
* Approved Test Strategy Document.
* Test tools, or automated test tools, if applicable.
* Previously developed scripts, if applicable.
* Test documentation problems uncovered as a result of testing.
* A good understanding of software complexity and module path coverage, derived from general and detailed design documents, e.g. software design document, source code, and software complexity data.
Outputs for this process:
* Approved documents of test scenarios, test cases, test conditions, and test data.
* Reports of software design issues, given to software developers for correction.
What is the purpose of a test plan?
Reason number 1: We create a test plan because preparing it helps us to think through the efforts needed to validate the acceptability of a software product.
Reason number 2: We create a test plan because it can and will help people outside the test group to understand the why and how of product validation.
Reason number 3: We create a test plan because, in regulated environments, we have to have a written test plan.
Reason number 4: We create a test plan because the general testing process includes the creation of a test plan.
Reason number 5: We create a test plan because we want a document that describes the objectives, scope, approach and focus of the software testing effort.
Reason number 6: We create a test plan because it includes test cases, conditions, the test environment, a list of related tasks, pass/fail criteria, and risk assessment.
Reason number 7: We create test plan because one of the outputs for creating a test strategy is an approved and signed off test plan document.
Reason number 8: We create a test plan because the software testing methodology a three step process, and one of the steps is the creation of a test plan.
Reason number 9: We create a test plan because we want an opportunity to review the test plan with the project team.
Reason number 10: We create a test plan document because test plans should be documented, so that they are repeatable.
Can we prepare Test Plan without SRS?
It is not always mandatory that you should have SRS document to prepare a Test Plan. This kind of Documents Hierarchy is maintained to maintain Organizational standards and also to have clear understanding of the things.
Yes you can Prepare a Test plan directly without SRS, When the Requirements are clear with your clients,and when your URD(User Requirement Document ) is supportive enough to clarify the issues.
Though we don't have SRS clients will be giving some information SRS only contains mainly Product information
But we will not know the Testing effort if we don't have SRS.
SRS contains How many cycles we are testing, and on the platforms we are testing , etc.
Actually there won't be any harm in doing so, becoz, ultimately you will send your Test plan document to your client and after getting approval from him only you start Testing.
(Note:- SRS is the document which you get in the Analysis phase of your Software Development. Test plan is the document , which contains the details of Product interms of , Tset strategy , Scope of testing, Types of tests to be conducted,Risk Managemnet , Mention of Automation Tool ,About Bug tracking Tool, etc..,)
How do test plan templates look like?
The test plan document template helps to generate test plan documents that describe the objectives, scope, approach and focus of a software testing effort. Test document templates are often in the form of documents that are divided into sections and subsections. One example of a template is a 4-section document where section 1 is the description of the "Test Objective", section 2 is the the description of "Scope of Testing", section 3 is the the description of the "Test Approach", and section 4 is the "Focus of the Testing Effort".
All documents should be written to a certain standard and template. Standards and templates maintain document uniformity. They also help in learning where information is located, making it easier for a user to find what they want. With standards and templates, information will not be accidentally omitted from a document. Once Rob Davis has learned and reviewed your standards and templates, he will use them. He will also recommend improvements and/or additions.
A software project test plan is a document that describes the objectives, scope, approach and focus of a software testing effort. The process of preparing a test plan is a useful way to think through the efforts needed to validate the acceptability of a software product. The completed document will help people outside the test group understand the why and how of product validation.
How to Test a desktop systems ?
You will likely have to use a programming or scripting language to interact with the service directly. You will have more control over the raw information that way.
You will have to determine what the service is supposed to do and how it is supposed to interact with other applications and services. A data dictionary likely exists. It may not be called that however. What this document does is explain what commands the service will respond to and what sort of data should be sent. You will have to use this document to do your testing. Get close to the person or people who created the document or the service and expect them to keep you in the loop when changes take place (it doesn't help anyone if you report a defect and it's really only reflecting an expected change in the operation of the service).
Desktop applications are generally designed to run and quit. You have to be concerned with memory leaks and system usage.
How do you create a test strategy?
The test strategy is a formal description of how a software product will be tested. A test strategy is developed for all levels of testing, as required. The test team analyzes the requirements, writes the test strategy and reviews the plan with the project team. The test plan may include test cases, conditions, the test environment, a list of related tasks, pass/fail criteria and risk assessment.
Inputs for this process:
* A description of the required hardware and software components, including test tools. This information comes from the test environment, including test tool data.
* A description of roles and responsibilities of the resources required for the test and schedule constraints. This information comes from man-hours and schedules.
* Testing methodology. This is based on known standards.
* Functional and technical requirements of the application. This information comes from requirements, change request, technical and functional design documents.
* Requirements that the system can not provide, e.g. system limitations.
Outputs for this process:
* An approved and signed off test strategy document, test plan, including test cases.
* Testing issues requiring resolution. Usually this requires additional negotiation at the project management level.
How to do Estimating Testing effort ?
Time Estimation method for Testing Process
Note : folloing method is based on use case driven specification.
Step 1 : count number of use cases (NUC) of system
step 2 : Set Avg Time Test Cases(ATTC) as per test plan
step 3 : Estimate total number of test cases (NTC)
Total number of test cases = Number of usecases X Avg testcases per a use case
Step 4 : Set Avg Execution Time (AET) per a test case (idelly 15 min depends on your system)
Step 5 : Calculate Total Execution Time (TET)
TET = Total number of test cases * AET
Step 6 : Calculate Test Case Creation Time (TCCT)
useually we will take 1.5 times of TET as TCCT
TCCT = 1.5 * TET
Step 7 : Time for ReTest Case Execution (RTCE) this is for retesting
useually we take 0.5 times of TET
RTCE = 0.5 * TET
Step 8 : Set Report generation Time (RGT
usually we take 0.2 times of TET
RGT = 0.2 * TET
Step 9 : Set Test Environment Setup Time (TEST)
it also depends on test plan
Step 10 : Total Estimation time = TET + TCCT+ RTCE + RGT + TEST + some buffer...;)
Example
Total No of use cases (NUC) : 227
Average test cases per Use cases(AET) : 10
Estimated Test cases(NTC) : 227 * 10 = 2270
Time estimation execution (TET) : 2270/4 = 567.5 hr
Time for creating testcases (TCCT) : 567.5*4/3 = 756.6 hr
Time for retesting (RTCE) : 567.5/2 = 283.75 hr
Report Generation(RGT) = 100 hr
Test Environment Setup Time(TEST) = 20 hr.
-------------------
Total Hrs 1727.85 + buffer
-------------------
here 4 means Number of testcases executed per hour
i.e 15 min will take for execution of each test case
What is the purpose of test strategy?
Reason number 1: The number one reason of writing a test strategy document is to "have" a signed, sealed, and delivered, FDA (or FAA) approved document, where the document includes a written testing methodology, test plan, and test cases.
Reason number 2: Having a test strategy does satisfy one important step in the software testing process.
Reason number 3: The test strategy document tells us how the software product will be tested.
Reason number 4: The creation of a test strategy document presents an opportunity to review the test plan with the project team.
Reason number 5: The test strategy document describes the roles, responsibilities, and the resources required for the test and schedule constraints.
Reason number 6: When we create a test strategy document, we have to put into writing any testing issues requiring resolution (and usually this means additional negotiation at the project management level).
Reason number 7: The test strategy is decided first, before lower level decisions are made on the test plan, test design, and other testing issues.
What's Quality Approach document? what should be the contents and things like that...
Answer1:
you should start thinking from your company business type, and according to it define different processes for your organization. like procurment, CM etc
Then think over different matrices you will be calculating for each process, and define them with formula, the kind of analysis will be doing and when shall the red flag to be raised,
Decide on your audit policies frequencies etc. Think on the change control board if any process needs modification.
Answer2:
By defining the process i mean the structured collection of practices that describe the characteristics of the work and its quality. writting process means creating a system with which every one will work, the benefits of it are like common language and a shared vision across organization, its will be a frame work for prioritizing actions.
From implementation point of view first you need to break the complete life cycle of your product in diffrent meaningful steps, and setting the goals for each phase.
you can create different document templates which every one shall follow, Define the dependencies among different groups for each project, Define risks for each project and what is mitigation plan for each risk. etc
You can read the CMMI model, customize that as per your organization goal. for a start up company As per my personal opinion, its better to define and reach at the process for Level 3 First and then go for level 5.
What does a test strategy document contain?
The test strategy document contains test cases, conditions, the test environment, a list of related tasks, pass/fail criteria and risk assessment. The test strategy document is a formal description of how a software product will be tested. What is the test strategy document developed for? It is developed for all levels of testing, as required. How is it written, and who writes it? It is the test team that analyzes the requirements, writes the test strategy, and reviews the plan with the project team.
Why Q/A should not report to development?
Based on research from the Quality Assurance Institute, the percent of quality groups in each location is noted,
50% - reports to Senior IT Manager - This is the best positioning because it gives the Quality Manager immediate access to the IT Manager to discuss and promote Quality issues, when the quality manager reports elsewhere, quality issues may not be raised to the appropriate level or receive the necessary action.
25% - reports to Manager of systems/programming
15 % reports to Manger oprerations.
10 % outside IT function.
Which of the following statements about Regression statements are true?
1---Regression testing must consist of a fixed set of tests to create a base line
2---Regression tests should be used to detect defects in new feature
3---Regression testing can be run on every build
4--- Regression testing should be targeted areas of high risk and known code change
5---Regression testing when automated, is highly effective in preventing defects.
Answer1:
1---Regression testing must consist of a fixed set of tests to create a base line
Don't think it is true as a "must" -- it
depends on whether your regression testing style involves repeating identical tests or redoing testing in previously tested areas with similar tests or tests that address the same risks. For example, some people do regression testing with tests whose specific parameters are determined randomly. They broaden the set of values they test while achieving essentially the same testing. Second example--some regression test suites include random stringing together of test cases (they include load testing and duration testing in their regression series, reporting their results as part of the assessment of each build). Depending on your theory of the _point_ of regression testing, these may or may not be entirely valid regression tests.
2---Regression tests should be used to detect defects in new feature
How do you create new regression tests? Should you design new tests as standalone, or should you develop a strategy in which the tests you use for bug-hunting are designed to be reusable as regression tests? If the latter, and I have certainly heard some skilled testers argue that the latter approach worked well in their sistuation, then #2 is sometimes true.
3---Regression testing can be run on every build
This is true, though it might be silly and a big waste of time.
4--- Regression testing should be targeted areas of high risk and known code change
Hmmm, there's a area of computer science called program slicing and one of the objectives of this class of work is to figure out how to restrict the regression test suite to a smaller number of tests, which test only those things that might have been impacted by a change. Bob Glass has criticized the results of some of this work, but if #4 is false, some Ph.D.'s and big research grants should be retracted.
5---Regression testing when automated, is highly effective in preventing defects.
Unit-level automated regression testing is highly effective in preventing defects--read up on test-driven development.
Answer2:
Let me explain why I think 2 & 5 are false
2---Regression tests should be used to detect defects in new feature
Since regression tests only address existing features and functionality, it can't find defects in new features. It can only find where existing features and functionality have been broken by changes.
5---Regression testing when automated, is highly effective in preventing defects.
Since no tests prevent defects, they only find them, it's impossible to prevent defects with a regression test. I will add, however, that if a developer can use an automated regression test to test their own code before submitting it to the code repository (say in the form a series of unit tests coupled to a library, etc.) then you could in some way prevent defects with a regression test.
I also don't like 1- and 4. 1- since a regression test suite grows as the product does. Therefore the tests are not fixed. 4- because a regression test tests the whole application, not just a targeted area. In the past, I have used the concept of test depth (level 1 being the basic regression tests--higher number reflect additional functionality) so you could run a level one regression on the whole program but do level three on the transport layer "because we've updated the library". T
an automated set of tests would be the most likely way to make 3- a possibility. It is unlikely that with daily builds, as many companies run their build process, that anything short of an automated regression test suite would be able to be run daily with any efficacy. if the builds were weekly, then a manual regression test would be likely.
Answer3:
As per the difinition of regression testing and actual workaround if you have to have answer this question then option 3 & 4 is the best choice among all.The reason behind it is :
3---Regression testing can be run on every build It is a normal phenomenon if there is build coming on weekly basis or it is a RC build.Since,there is nothing mention about daily build ,only thing mention is every build so it can be correct.
4---Regression testing should be targeted areas of high risk and known code change This is also true in most of the situation,it is not universally true but in certain condition where there is code change and the related modules are only tested in regression automation rather than whole code.
5 is not true coz in regression we detect the defect not prevent normally.
How do you execute tests?
Execution of tests is completed by following the test documents in a methodical manner. As each test procedure is performed, an entry is recorded in a test execution log to note the execution of the procedure and whether or not the test procedure uncovered any defects. Checkpoint meetings are held throughout the execution phase. Checkpoint meetings are held daily, if required, to address and discuss testing issues, status and activities.
* The output from the execution of test procedures is known as test results. Test results are evaluated by test engineers to determine whether the expected results have been obtained. All discrepancies/anomalies are logged and discussed with the software team lead, hardware test lead, programmers, software engineers and documented for further investigation and resolution. Every company has a different process for logging and reporting bugs/defects uncovered during testing.
* A pass/fail criteria is used to determine the severity of a problem, and results are recorded in a test summary report. The severity of a problem, found during system testing, is defined in accordance to the customer's risk assessment and recorded in their selected tracking tool.
* Proposed fixes are delivered to the testing environment, based on the severity of the problem. Fixes are regression tested and flawless fixes are migrated to a new baseline. Following completion of the test, members of the test team prepare a summary report. The summary report is reviewed by the Project Manager, Software QA Manager and/or Test Team Lead.
* After a particular level of testing has been certified, it is the responsibility of the Configuration Manager to coordinate the migration of the release software components to the next test level, as documented in the Configuration Management Plan. The software is only migrated to the production environment after the Project Manager's formal acceptance.
* The test team reviews test document problems identified during testing, and update documents where appropriate.
Inputs for this process:
* Approved test documents, e.g. Test Plan, Test Cases, Test Procedures.
* Test tools, including automated test tools, if applicable.
* Developed scripts.
* Changes to the design, i.e. Change Request Documents.
* Test data.
* Availability of the test team and project team.
* General and Detailed Design Documents, i.e. Requirements Document, Software Design Document.
* A software that has been migrated to the test environment, i.e. unit tested code, via the Configuration/Build Manager.
* Test Readiness Document.
* Document Updates.
Outputs for this process:
* Log and summary of the test results. Usually this is part of the Test Report. This needs to be approved and signed-off with revised testing deliverables.
* Changes to the code, also known as test fixes.
* Test document problems uncovered as a result of testing. Examples are Requirements document and Design Document problems.
* Reports on software design issues, given to software developers for correction. Examples are bug reports on code issues.
* Formal record of test incidents, usually part of problem tracking.
* Base-lined package, also known as tested source and object code, ready for migration to the next level.
What is a requirements test matrix?
The requirements test matrix is a project management tool for tracking and managing testing efforts, based on requirements, throughout the project's life cycle.
The requirements test matrix is a table, where requirement descriptions are put in the rows of the table, and the descriptions of testing efforts are put in the column headers of the same table.
The requirements test matrix is similar to the requirements traceability matrix, which is a representation of user requirements aligned against system functionality. The requirements traceability matrix ensures that all user requirements are addressed by the system integration team and implemented in the system integration effort.
The requirements test matrix is a representation of user requirements aligned against system testing. Similarly to the requirements traceability matrix, the requirements test matrix ensures that all user requirements are addressed by the system test team and implemented in the system testing effort.
Can you give me a requirements test matrix template?
For a requirements test matrix template, you want to visualize a simple, basic table that you create for cross-referencing purposes.
Step 1: Find out how many requirements you have.
Step 2: Find out how many test cases you have.
Step 3: Based on these numbers, create a basic table. If you have a list of 90 requirements and 360 test cases, you want to create a table of 91 rows and 361 columns.
Step 4: Focus on the the first column of your table. One by one, copy all your 90 requirement numbers, and paste them into rows 2 through 91 of the table.
Step 5: Now switch your attention to the the first row of the table. One by one, copy all your 360 test case numbers, and paste them into columns 2 through 361 of the table.
Step 6: Examine each of your 360 test cases, and, one by one, determine which of the 90 requirements they satisfy. If, for the sake of this example, test case number 64 satisfies requirement number 12, then put a large "X" into cell 13-65 of your table... and then you have it; you have just created a requirements test matrix template that you can use for cross-referencing purposes.
What metrics are used for bug tracking?
Metrics that can be used for bug tracking include the followings: the total number of bugs, total number of bugs that have been fixed, number of new bugs per week, and the number of fixes per week. Metrics for bug tracking can be used to determine when to stop testing, for example, when bug rate falls below a certain level. You CAN learn to use defect tracking software.
1. In QA team, everyone talks about process. What exactly they are taking about?
2. Are there any different type of process?
Answer1:
When you talk about "process" you are generally talking about the actions used to accomplish a task.
Here's an example: How do you solve a jigsaw puzzle?
You start with a box full of oddly shaped pieces. In your mind you come up with a strategy for matching two pieces together (or no strategy at all and simply grab random pieces until you find a match), and continue on until the puzzle is completed.
If you were to describe the *way* that you go about solving the puzzle you would be describing the process.
Some follow-up questions you might think about include things like:
- How much time did it take you to solve the puzzle?
- Do you know of any skills, tricks or practices that might help you solve the puzzle quicker?
- What if you try to solve the puzzle with someone else? Does that help you go faster, or slower? (why or why not?) Can you have *too* many people on this one task?
- To answer your second question, I'll ask *you* the question: Are there different ways that people can solve a jigsaw puzzle?
There are many interesting process-related questions, ideas and theories in Quality Assurance. Generally the identification of workplace processes lead to the questions of improvement in efficiency and productivity. The motivation behind that is to try and make the processes as efficient as possible so as to incur the least amount of time and expense, while providing a general sense of repeatability, visibility and predictability in the way tasks are performed and completed.
The idea behind this is generally good, but the execution is often flawed. That is what makes QA so interesting. You see, when you work with people and processes, it is very different than working with the processes performed by machines. Some people in QA forget that distinction and often become disillusioned with the whole thing.
If you always remember to approach processes in the workplace with a people-centric view, you should do fine.
Answer2:
There is:
* Waterfall
* Spiral
* Rapid prototype
* Clean room
* Agile (XP, Scrum, ...)
What metrics are used for test report generation?
Metrics that can be used for test report generation include...
McCabe metrics: cyclomatic complexity metric (v(G)), actual complexity metric (AC), module design complexity metric (iv(G)), essential complexity metric (ev(G)), pathological complexity metric (pv(G)), design complexity metric (S0), integration complexity metric (S1), object integration complexity metric (OS1), global data complexity metric (gdv(G)), data complexity metric (DV), tested data complexity metric (TDV), data reference metric (DR), tested data reference metric (TDR), maintenance severity metric (maint_severity), data reference severity metric (DR_severity), data complexity severity metric (DV_severity), global data severity metric (gdv_severity).
McCabe object-oriented software metrics: encapsulation percent public data (PCTPUB), access to public data (PUBDATA), polymorphism percent of unoverloaded calls (PCTCALL), number of roots (ROOTCNT), fan-in (FANIN), quality maximum v(G) (MAXV), maximum ev(G) (MAXEV), and hierarchy quality (QUAL).
Other object-oriented software metrics: depth (DEPTH), lack of cohesion of methods (LOCM), number of children (NOC), response for a class (RFC), weighted methods per class (WMC), Halstead software metrics program length, program volume, program level and program difficulty, intelligent content, programming effort, error estimate, and programming time.
Line count software metrics: lines of code, lines of comment, lines of mixed code and comments, and lines left blank.
What is quality plan?
Answer1:
the test plan is the document created before starting the testing process, it includes that types of testing that will be performed, high level scope of the project, the envirnmental requirements of the testing process, what automated testing tools will be used (If available), the schedule of each test, when it will start and end.
Answer2:
you should not only understand what a Quality Plan is, but you should understand why you're making it. I don't beleieve that "because I was told to do so" is a good enough reason. If the person who told you to create it can't tell you 1) what it is, and 2) how to create it, I don't think that they actually know why it's needed. That breaks the primary rule of all plans used in testing:
We write quality plans for two very different purposes. Sometimes the quality plan is a product; sometimes it's a tool. It's too easy, but also too expensive, to confuse these goals.
If it's not being used as a tool, don't waste your time (and your company's money) doing this.
What is the difference between verification and validation?
Verification takes place before validation, and not vice versa.
Verification evaluates documents, plans, code, requirements, and specifications. Validation, on the other hand, evaluates the product itself.
The inputs of verification are checklists, issues lists, walkthroughs and inspection meetings, reviews and meetings. The input of validation, on the other hand, is the actual testing of an actual product.
The output of verification is a nearly perfect set of documents, plans, specifications, and requirements document. The output of validation, on the other hand, is a nearly perfect, actual product.
What is the difference between efficient and effective?
"Efficient" means having a high ratio of output to input; which means working or producing with a minimum of waste. For example, "An efficient engine saves gas." Or, "An efficient test engineer saves time".
"Effective", on the other hand, means producing or capable of producing an intended result, or having a striking effect. For example, "For rapid long-distance transportation, the jet engine is more effective than a witch's broomstick". Or, "For developing software test procedures, engineers specializing in software testing are more effective than engineers who are generalists".
How effective can we implement six sigma principles in a very large software services organization?
Answer1:
Effective way of implementing sixsigma.
there are quite a few things one needs
1. management buyin
2. dedicated team both drivers as well as adopters
3. training
4. culture building - if you have a pro process culture, life is easy
5. sustained effort over a period towards transforming, people, thoughts and actions Personally technical content is never a challenge, but adoption is a challenge.
Answer2:
"Six sigma" is a combination of process recommendations and mathematical model. The name "six sigma" reflects the notion of reducing variation so much that errors -- events out of tolerance -- are six standard deviations from a desired mean. The mathematics are at the core of the process implementation.
The problem is that software is not hardware. Software defects are designed in, not the result of manufacturing variation.
The other side of six sigma is the drive for continuous improvement. You don't need the six sigma math for this and the concept has been around long before the six sigma movement.
To improve anything, you need some type of indicator of its current state and a way to tell that it is improved. Plus determination to improve it. Management support helps.
Answer3:
There are different methodologies adopted in sixsigma. However, it is commonly referenced from the variance based approach. If you are trying to look at sixsigma from that, for software services, fundamentally the measurement system should be reliable - industry has not reached the maturity level of manufacturing industry where it fits to a T. The differences between SW and HW/manufacturing industry is slightly difficult to address.
There are some areas you can adopt sixsigma in its full statistical form(eg in-process error rate, productivity improvements etc), some areas are difficult.
The narrower the problem area is, the better it gets even in software services to address adopting the statistical method.
There are methodologies that have a bundle of tools,along with statistical techniques, are used on the full SDLC.
A generic observation is ,SS helps if we look for proper fitment of methodology for the purpose. Else doubts creep in
What stage of bug fixing is the most cost effective?
Bug prevention techniques (i.e. inspections, peer design reviews, and walk-throughs) are more cost effective than bug detection.
What is Defect Life Cycle.?
Answer1:
Defect life cycle is....different stages after a defect is identified.
New (When defect is identified)
Accepted (when Development team and QA team accepts it's a Bug)
In Progress (when a person is working to resolve the issue-defect)
Resolved (once the defect resolved)
Completed (Some one who can take up the responsibly Team lead)
Closed/reopened (Retested by TE and he will update the Status of the bug)
Answer2:
Defect Life Cycle is nothing but the various phases a Bug undergoes after it is raised or reported.
A general Interview answer can be given as:
1. New or Opened
2. Assinged
3. Fixed
4. Tested
5. Closed.
What is the difference between a software bug and software defect?
"Software bug" is nonspecific; it means an inexplicable defect, error, flaw, mistake, failure, fault, or unwanted behavior of a computer program. Other terms, e.g. "software defect", or "software failure", are more specific.
While the word "bug" has been a part of engineering jargon for many-many decades; many-many decades ago even Thomas Edison, the great inventor, wrote about a "bug" - today there are many who believe the word "bug" is a reference to insects that caused malfunctions in early electromechanical computers.
What is the difference between a software bug and software defect?
In software testing, the difference between "bug" and "defect" is small, and also depends on the end client. For some clients, bug and defect are synonymous, while others believe bugs are subsets of defects.
Difference number one: In bug reports, the defects are easier to describe.
Difference number two: In my bug reports, it is easier to write descriptions as to how to replicate defects. In other words, defects tend to require only brief explanations.
Commonality number one: We, software test engineers, discover both bugs and defects, before bugs and defects damage the reputation of our company.
Commonality number two: We, software QA engineers, use the software much like real users would, to find both bugs and defects, to find ways to replicate both bugs and defects, to submit bug reports to the developers, and to provide feedback to the developers, i.e. tell them if they've achieved the desired level of quality.
Commonality number three: We, software QA engineers, do not differentiate between bugs and defects. In our reports, we include both bugs and defects that are the results of software testing.
Are developers smarter than tester? Any suggestion about the future prospects and technicality involvedin the testing job?
Answer1:
QA & Testing are thankless jobs. In a software development company developer is a core person. As you are a fresh graduate, it would be good for you to work as a developer. From development you can always move to testing or QA or other admin/support tasks. But from Testing or QA it is little difficult to go back to development, though not impossible(as u are BE comp)
Seeing the job market, it is not possible for each & every fresher to get into development. But you can keep searching for it.
Some big company's have seperate Verifiction & Validation groups where only testing projects are executed. Those teams have TLs, PLs who are testing experts. They earn good salary same as development people.
In technical projects the testing team does lot of technical work. You can do certifications to improve your technical skills & market value.
It all depends on your way of handling things & interpersonal, communication and leadership skills. If it is difficult for you to get a job in developement or you really like testing, just go ahead. Try to achieve excellence as a testing professional. You will never have a job problem .Also you will always get onsite opportunities too!! Yuo might have to struggle for initial few years like all other freshers.
Answer2:
QA and Testing are thankless only in some companies.
Testing is part of development. Rather than distinguish between testing and development,distinguish between testing and programming.
Programming is also thankless in some companies.
Not suggesting that anyone should or should not go into testing. It depends on your skills and interests. Some people are better at programming and worse at testing, some better at testing and worse at programming, some are not suited for either role. You should decide what you are good at and what fascinates you. What type of work would make you WANT to stay at work for 60-80 hours a week for a few years because it is so interesting?
Suggesting that there are excellent testing jobs out there, but there are bad ones too (in testing and in programming, both).
Have not seen any certification in software testing that improves the technical skill of anyone. Apparently, testing certification improves a tester's market value in some markets.
Most companies mean testing when they say "QA". Or they mean Testing plus Metrics, where the metrics tasks are low-skill data collection and basic data analysis rather than thinking up and justifying measurement systems appropriate to the questions at hand. In terms of skill, salary, intellectual challenge and value to the company, testing+metrics is the same as testing. Some companies see QA more strategically, and hire more senior people into their groups. Here is a hint--if you can get a job in a group called QA with less than 5 years of experience, it's a testing group or something equivalent to it.
Answer3:
Nothing is considered as great or a mean job. As long as you like and love to do, everything in that seems to be interesting.
I started as a developer and slowly moved to Testing. I find testing to be more challenging and interesting. I have solid 6 years of testing experience alone and many sernior people are there in my team, who are professional testers.
Answer4:
testing is low-skill work in many companies.
Scripted testing of the kind pushed by ISEB, ISTQB, and the other certifiers is low skill, low prestige, offers little return value to the company that pays for it, and is often pushed to offsite contracting firms because it isn't worth doing in-house. In many cases, it is just a process of "going through the motions" -- pretending to do testing (and spending a lot of money in the pretense) but without really looking for any important information and without creating any artifacts that will be useful to the project team.
The only reason to take a job doing this kind of work is to get paid for it. Doing it for too long is bad for your career.
There are much higher-skill ways to do testing. Some of them involve partial automation (writing or using programs to help you investigate the program more effectively), but automation tools are just tools. They are often used just as mind-numbingly and valuelessly as scripted manual testing. When you're offered this kind of position, try to find out how much judgment you will have to exercise in the analysis of the product under test and the ways that it provides value to the users and other stakeholders, in the design of tests to check that value and to check for other threats to value (security failures, performance failures, usability failures, etc.)--and how much this position will help you develop your judgment. If you will become a more skilled and more creative investigator who has a better collection of tools to investigate with, that might be interesting. If not, you will be marking time (making money but learning little) while the rest of the technical world learns new ideas and skills.
What's the difference between priority and severity?
The word "priority" is associated with scheduling, and the word "severity" is associated with standards. "Priority" means something is afforded or deserves prior attention; a precedence established by urgency or order of or importance.
Severity is the state or quality of being severe; severe implies adherence to rigorous standards or high principles and often suggests harshness; severe is marked by or requires strict adherence to rigorous standards or high principles. For example, a severe code of behavior.
The words priority and severity do come up in bug tracking. A variety of commercial, problem-tracking / management software tools are available. These tools, with the detailed input of software test engineers, give the team complete information so developers can understand the bug, get an idea of its severity, reproduce it and fix it. The fixes are based on project priorities and severity of bugs. The severity of a problem is defined in accordance to the end client's risk assessment, and recorded in their selected tracking tool. A buggy software can severely affect schedules, which, in turn can lead to a reassessment and renegotiation of priorities.
How to test a web based application that has recently been modified to give support for Double Byte Character Sets?
Answer1:
should apply black box testing techniques (boundary analysis, equivalence partioning)
Answer2:
The Japanese and other East Asian Customers are very particular of the look and feel of the UI. So please make sure, there is no truncation at any place.
One Major difference between Japanese and English is that there is no concept of spaces between the words in Japanese. The line breaks in English usually happens whenever there is a Space. In Japanese this leads to a lot of problem with the wrapping on the text and if you have a table with defined column length, you might see text appearing vertical.
On the functionality side:
1. Check for the date format and Number format. (it should be in the native locale)
2. Check that your system accepts 2-byte numerals and characters.
3. If there is any fields with a boundary value of 100 characters, the field should accept, the same number of 2-byte character as well.
4. The application should work on a Native (Chinese, Japanese, Korean) OS as well as on an English OS with the language pack installed.
Writing a high level test plan for 2-byte support will require some knowledge of the application and its architecture.
What is the difference between software fault and software failure?
Software failure occurs when the software does not do what the user expects to see. Software fault, on the other hand, is a hidden programming error.
A software fault becomes a software failure only when the exact computation conditions are met, and the faulty portion of the code is executed on the CPU. This can occur during normal usage. Or, when the software is ported to a different hardware platform. Or, when the software is ported to a different complier. Or, when the software gets extended.
before creating test cases to "break the system", a few principles have to be observed:
Testing should be based on user requirements. This is in order to uncover any defects that might cause the program or system to fail to meet the client's requirements.
Testing time and resources are limited. Avoid redundant tests.
It is impossible to test everything. Exhaustive tests of all possible scenarios are impossible, simple because of the many different variables affecting the system and the number of paths a program flow might take.
Use effective resources to test. This represents use of the most suitable tools, procedures and individuals to conduct the tests. The test team should use tools that they are confident and familiar with. Testing procedures should be clearly defined. Testing personnel may be a technical group of people independent of the developers.
Test planning should be done early. This is because test planning can begin independently of coding and as soon as the client requirements are set.
Testing should begin at the module. The focus of testing should be concentrated on the smallest programming units first and then expand to other parts of the system.
We look at software testing in the traditional (procedural) sense and then describe some testing strategies and methods used in Object Oriented environment. We also introduce some issues with software testing in both environments.
Would like to know whether Black Box testing techniques like Boundary Value Analysis and Equivalence Partitioning - during which phases of testing are they used,if possible with examples ?
Answer1:
Also Boundary Value Analysis and Equivalence Partitioning can be used in unit or component testing, and generally is used in system testing
Example, you have a module designed to work out the tax to be paid:
An employee has £4000 of salary tax free. The next £1500 is taxed at 10%
The next £28000 is taxed at 22%
Any further amount is taxed at 40%
You must define test cases that exercise valid and invalid equivalence classes:
Any value lower than 4000 is tax free
Any value between 4000 and 5500 must paid 10%
Any value between 5501 and 33500 must paid 22%
Any value bigger than 33500 must paid 40%
And the boundary values are: 4000, 4001, 5501, 33501
Answer2:
This Boundary value analysis and Equivalence partitioning is used to prepare the positive and negative type test cases.
Equivalence partitioning: If you want to validate the text box which accepts the value between 2000 to 10000 , then the test case input is partitioned as the following way
1. <=2000
2. >=2000 and <=10000
3. >10000
The boundary Values analysis is checking the input values on boundaries. IN the above case it can checked with whether the input values is on the boundary or above the boundary or in low boundary.
Test Case Design
Test cases should be designed in such a way as to uncover quickly and easily as many errors as possible. They should "exercise" the program by using and producing inputs and outputs that are both correct and incorrect. Variables should be tested using all possible values (for small ranges) or typical and out-of-bound values (for larger ranges). They should also be tested using valid and invalid types and conditions. Arithmetical and logical comparisons should be examined as well, again using both correct and incorrect parameters. The objective is to test all modules and then the whole system as completely as possible using a reasonably wide range of conditions.
How to use methods/techniques to test the bandwidth usage of a client/server application?
Bandwidth Utilization:
Basically at the client-server model you will be most concerned about the bandwidth usage if your application is a web based one. It surely is a part of concern when the throughput and the data transfer comes into the picture.
I suggest you to use the Radview's Webload for the Load and Stress testing tool for the same.
Available at the demoware.. you can record the scenarios of the normal user over the variable connection speed and then run it for hours to know about the bandwidth utilisation and the throughput and data trasfer rate, hits per sec, etc... there is a huge list of parameters which can be tested over a n no of combinations..
How do test case templates look like?
Software test case templates are blank documents that describe inputs, actions, or events, and their expected results, in order to determine if a feature of an application is working correctly. Test case templates contain all particulars of test cases. For example, one test case template is in the form of a 6-column table, where column 1 is the "test case ID number", column 2 is the "test case name", column 3 is the "test objective", column 4 is the "test conditions/setup", column 5 is the "input data requirements/steps", and column 6 is the "expected results".
All documents should be written to a certain standard and template. Why? Because standards and templates do help to maintain document uniformity. Also because they help you to learn where information is located, making it easier for users to find what they want. Also because, with standards and templates, information is not be accidentally omitted from documents.
How to insert a check point to a image to check enable property in QTP?
Answer1:
AS you are saying that the all images are as push button than you can check the property enabled or disabled. If you are not able to find that property than go to object repository for that objecct and click on add remove to add the available properties to that object. Let me know if that works. And if you take it as image than you need to check visible or invisible property tht also might help you are there are no enable or disable properties for the image object.
Answer2:
The Image Checkpoint does not have any property to verify the enable/disable property.
One thing you need to check is:
* Find out form the Developer if he is showing different images for activating/deactiving i.e greyed out image. That is the only way a developer can show deactivate/activate if he is using an "image". Else he might be using a button having a headsup with an image.
* If it is a button used to display with the headsup as an image you woudl need to use the object Properties as a checkpoint.
How do you write test cases?
When I write test cases, I concentrate on one requirement at a time. Then, based on that one requirement, I come up with several real life scenarios that are likely to occur in the use of the application by an end user.
When I write test cases, I describe the inputs, action, or event, and their expected results, in order to determine if a feature of an application is working correctly. To make the test case complete, I also add particulars e.g. test case identifiers, test case names, objectives, test conditions (or setups), input data requirements (or steps), and expected results.
Additionally, if I have a choice, I like writing test cases as early as possible in the development life cycle. Why? Because, as a side benefit of writing test cases, many times I am able to find problems in the requirements or design of an application. And, because the process of developing test cases makes me completely think through the operation of the application.
