Checklist to do Negative Testing

Written by Raj

HP Mercury QTP

A single fault can trigger many failures, particularly when processing a large amount of repeating data. Example; a switch re-configuration causes all records of phone calls to be improperly formatted – now the system will pass all calls through non-optimised error handling routines, and write details to a log. Handling the volume of problems can cause the system to slow, and cause its storage to fill. This may, in turn, cause further errors, and the

The following examples will help you to identify the test cases for negative testing;

• · A single fault can trigger many failures, particularly when processing a large amount of repeating data. Example; a switch re-configuration causes all records of phone calls to be improperly formatted – now the system will pass all calls through non-optimised error handling routines, and write details to a log. Handling the volume of problems can cause the system to slow, and cause its storage to fill. This may, in turn, cause further errors, and the
• problem can accelerate.
• · If an exception is not detected, the system will carry on as if no problem has been found. This can often lead to problems at an Operating System level, as requests are made that cannot be fulfilled.
• · If an exception is detected, but the message detailing that information is lost or delayed, the system may carry on as if no problem has been found. This can often lead to worse problems.
• · When asking a user for a corrected input, the system may block further activity. Systems can get stuck in this state, and may also restrict valid activity, such as screen re-draws and other background tasks. This problem can also be influenced by handling control to another process (a screen-saver, say) while in a restricted state.
• · Simple logging mechanisms may not scale, introducing problems of concurrency, disk space or resource hogging.
• · If a great many messages are sent to an observer or log, they may be ignored (so many that the problem becomes expected) or obscured (one important message is hard to spot in many irrelevant ones).
• · Coders often write code to act on known error conditions. An unrecognised error condition may slip through undetected. With input validation, if the code excludes only known input errors, an invalid entry that is not recognised may slip through. It is better practice to exclude everything that does not fit the expectations of valid input than to exclude everything that fits
• expectations of invalid input.
• · Poor messaging is a common flaw. Obscure, incomplete, duplicate or ambiguous messages not only cause problems to users, but also make negative testing harder – and may signify misunderstandings in the design/coding team or other systematic errors.
• · Localisation / internationalisation will require the text of user-facing error messages to be chosen from an appropriate language set. This may introduce new functionality, and will certainly be problematic on systems that have hard-coded error messages.
• · Generic messaging systems typically store their messages centrally, referring to them by an identifier. This can lead to problems where the wrong message is sent; either because the data is corrupt or badly set up, or because the coder has used the wrong identifier. It may be necessary to test the data itself.
• · Resuming control flow from the correct point is not necessarily trivial. It involves passing complex information, and returning control to a state that can cope, but is not functionally different from the state that threw the initial error. It may be necessary to carefully check the state of the system before executing more tests.
• · Tidying up after an error is also not trivial. A system may need to resubmit requests, tidy screen and memory, rollback transactions, restart processes and return control to the right point, in the right state. If the error has been caused by a bug elsewhere, inconsistent information may have been passed into storage – and later tests may be compromised.
• · Exceptions that are handled uniquely, outside an existing exception-handling process, may be less reliable, and less tested, than those handles by a centralised, generic process.
• · Error messages can have multiple or ill-defined meanings (i.e. poorly maintained legacy system). A definitive list of errors can reduce the impact of this problem on testing, and including an unambiguous numeric code can hep testing and support.
• · External resource: unresponsive, slow response, unintelligible / inappropriate response
• · Co-operative process failure: particularly intermittent processes, multi-tasking and recursion
• · Concurrent use: resource locked, request for lock refused, deadlock, lock response delayed
• · Sacrificial processes: processes allowed to fail and recover in a controlled manner
• · File system: File cannot be found, opened, read, written to, permissions changed, file system
• recognises media error, media removed, media full
• · Network: Network down, network busy/slow, transmission segment lost/corrupt/out of
• sequence, dialogue between processes interrupted
• · Memory: not enough for requested allocation, fragmented
• · Limits reached: Queues, licences, threads, connections, array size, resource allocation

Ordered and classified lists of faults at:
Bug Taxonomy and Statistics (2001), B. Beizer, O. Vinter
http://inet.uni2.dk/~vinter/bugtaxst.doc
Testing Computer Software (Second Edition): Appendix: Common Software Errors: Cem Kaner, Jack
Falk, Hung Quoc Nguyen
http://qacity.logigear.com/nr/qacity/documents/common_errors.html