Security Testing

As more and more vital data is stored in web applications and the number of transactions on the web increases, proper security testing of web applications is becoming very important. Security testing is the process that determines that confidential data stays confidential (i.e. it is not exposed to individuals/ entities for which it is not meant) and users can perform only those tasks that they are authorized to perform (e.g. a user should not be able to deny the functionality of the web site to other users, a user should not be able to change the functionality of the web application in an unintended way etc.). 

Some key terms used in security testing 

Before we go further, it will be useful to be aware of a few terms that are frequently used in web application security testing: 

What is “Vulnerability”? 

This is a weakness in the web application. The cause of such a “weakness” can be bugs in the application, an injection (SQL/ script code) or the presence of viruses.

What is “URL manipulation”? 

Some web applications communicate additional information between the client (browser) and the server in the URL. Changing some information in the URL may sometimes lead to unintended behavior by the server. 

What is “SQL injection”? 

This is the process of inserting SQL statements through the web application user interface into some query that is then executed by the server. 

What is “XSS (Cross Site Scripting)”? 

When a user inserts HTML/ client-side script in the user interface of a web application and this insertion is visible to other users, it is called XSS. 

What is “Spoofing”? 

The creation of hoax look-alike websites or emails is called spoofing.

Security testing approach: 

In order to perform a useful security test of a web application, the security tester should have good knowledge of the HTTP protocol. It is important to have an understanding of how the client (browser) and the server communicate using HTTP. Additionally, the tester should at least know the basics of SQL injection and XSS. Hopefully, the number of security defects present in the web application will not be high. However, being able to accurately describe the security defects with all the required details to all concerned will definitely help. 

Password cracking

The security testing on a web application can be kicked off by “password cracking”. In order to log in to the private areas of the application, one can either guess a username/ password or use some password cracker tool for the same. Lists of common usernames and passwords are available along with open source password crackers. If the web application does not enforce a complex password (e.g. with alphabets, number and special characters, with at least a required number of characters), it may not take very long to crack the username and password. If username or password is stored in cookies without encrypting, attacker can use different methods to steal the cookies and then information stored in the cookies like username and password. 

URL manipulation through HTTP GET methods

The tester should check if the application passes important information in the querystring. This happens when the application uses the HTTP GET method to pass information between the client and the server. The information is passed in parameters in the querystring. The tester can modify a parameter value in the querystring to check if the server accepts it. Via HTTP GET request user information is passed to server for authentication or fetching data. Attacker can manipulate every input variable passed from this GET request to server in order to get the required information or to corrupt the data. In such conditions any unusual behavior by application or web server is the doorway for the attacker to get into the application. 

SQL Injection

The next thing that should be checked is SQL injection. Entering a single quote (‘) in any textbox should be rejected by the application. Instead, if the tester encounters a database error, it means that the user input is inserted in some query which is then executed by the application. In such a case, the application is vulnerable to SQL injection. SQL injection attacks are very critical as attacker can get vital information from server database. To check SQL injection entry points into your web application, find out code from your code base where direct MySQL queries are executed on database by accepting some user inputs. If user input data is crafted in SQL queries to query the database, attacker can inject SQL statements or part of SQL statements as user inputs to extract vital information from database. Even if attacker is successful to crash the application, from the SQL query error shown on browser, attacker can get the information they are looking for. Special characters from user inputs should be handled/escaped properly in such cases. 

Cross Site Scripting (XSS)

The tester should additionally check the web application for XSS (Cross site scripting). Any HTML e.g. <HTML> or any script e.g. <SCRIPT> should not be accepted by the application. If it is, the application can be prone to an attack by Cross Site Scripting. Attacker can use this method to execute malicious script or URL on victim’s browser. Using cross-site scripting, attacker can use scripts like JavaScript to steal user cookies and information stored in the cookies. Many web applications get some user information and pass this information in some variables from different pages. Attacker can easily pass some malicious input or <script> as a ‘&query’ parameter which can explore important user/server data on browser. 

Important: During security testing, the tester should be very careful not to modify any of the following: 

· Configuration of the application or the server 

· Services running on the server 

· Existing user or customer data hosted by the application 

Additionally, a security test should be avoided on a production system. 

The purpose of the security test is to discover the vulnerabilities of the web application so that the developers can then remove these vulnerabilities from the application and make the web application and data safe from unauthorized actions.

SQL Injection

Many applications use some type of a database. An application under test might have a user interface that accepts user input that is used to perform the following tasks:

1. Show the relevant stored data to the user e.g. the application checks the credentials of the user using the log in information entered by the user and exposes only the relevant functionality and data to the user

2. Save the data entered by the user to the database e.g. once the user fills up a form and submits it, the application proceeds to save the data to the database; this data is then made available to the user in the same session as well as in subsequent sessions

Some of the user inputs might be used in framing SQL statements that are then executed by the application on the database. It is possible for an application NOT to handle the inputs given by the user properly. If this is the case, a malicious user could provide unexpected inputs to the application that are then used to frame and execute SQL statements on the database. This is called SQL injection. The consequences of such an action could be alarming.

The following things might result from SQL injection:

1. The user could log in to the application as another user, even as an administrator.

2. The user could view private information belonging to other users e.g. details of other users’ profiles, their transaction details etc.

3. The user could change application configuration information and the data of the other users.

4. The user could modify the structure of the database; even delete tables in the application database.

5. The user could take control of the database server and execute commands on it at will.

Since the consequences of allowing the SQL injection technique could be severe, it follows that SQL injection should be tested during the security testing of an application. Now with an overview of the SQL injection technique, let us understand a few practical examples of SQL injection.

Important: The SQL injection problem should be tested only in the test environment.

If the application has a log in page, it is possible that the application uses a dynamic SQL such as statement below. This statement is expected to return at least a single row with the user details from the Users table as the result set when there is a row with the user name and password entered in the SQL statement.

SELECT * FROM Users WHERE User_Name = ” & strUserName & “ AND Password = ” & strPassword & “;

If the tester would enter John as the strUserName (in the textbox for user name) and Smith as strPassword (in the textbox for password), the above SQL statement would become:

SELECT * FROM Users WHERE User_Name = ‘John’ AND Password = ‘Smith’;

If the tester would enter John’– as strUserName and no strPassword, the SQL statement would become:

SELECT * FROM Users WHERE User_Name = ‘John’– AND Password = ‘Smith’;

Note that the part of the SQL statement after John is turned into a comment. If there were any user with the user name of John in the Users table, the application could allow the tester to log in as the user John. The tester could now view the private information of the user John.

What if the tester does not know the name of any existing user of the application? In such a case, the tester could try common user names like admin, administrator and sysadmin. If none of these users exist in the database, the tester could enter John’ or ‘x’=’x as strUserName and Smith’ or ‘x’=’x as strPassword. This would cause the SQL statement to become like the one below.

SELECT * FROM Users WHERE User_Name = ‘John’ or ‘x’='x’ AND Password = ‘Smith’ or ‘x’=’x’;

Since ‘x’=’x’ condition is always true, the result set would consist of all the rows in the Users table. The application could allow the tester to log in as the first user in the Users table.

Important: The tester should request the database administrator or the developer to copy the table in question before attempting the following SQL injection.

If the tester would enter John’; DROP table users_details;’—as strUserName and anything as strPassword, the SQL statement would become like the one below.

SELECT * FROM Users WHERE User_Name = ‘John’; DROP table users_details;’ –‘ AND Password = ‘Smith’;

This statement could cause the table “users_details” to be permanently deleted from the database.

Though the above examples deal with using the SQL injection technique only the log in page, the tester should test this technique on all the pages of the application that accept user input in textual format e.g. search pages, feedback pages etc.

SQL injection might be possible in applications that use SSL. Even a firewall might not be able to protect the application against the SQL injection technique.

I have tried to explain the SQL injection technique in a simple form. I would like to re-iterate that 

SQL injection should be tested only in a test environment and not in the development environment, production environment or any other environment. Instead of manually testing whether the application is vulnerable to SQL injection or not, one could use a web vulnerability scanner that checks for SQL injection.

Letter to a Programmer

Dear Programmer,

My job is to help you look good. My job is to support you as you create quality; to ease that burden instead of adding to it.

In that spirit, I make the following commitments to you-

• I provide a service. You are an important client of that service. I am not satisfied unless you are satisfied
• I am just the gatekeeper of quality. I don’t “own” quality. Shipping a good product is a goal shared by all of us
• I will test your code as soon as I can after you deliver it to me. I know that you need my test results quickly (especially for fixes and new features)
• I will strive to test in a way that allows you to be fully productive. I will not be a bottleneck
• I will make every reasonable effort to test, even if I have only partial information about the product
• I will learn the product quickly, and make use of that knowledge to test more cleverly
• I will test important things first, and try to find important problems. (I will also report things you might consider unimportant, just in case they turn out to be important after all, but I will spend less time on those)
• I will strive to test in the interests of everyone whose opinions matter, including you, so that you can make better decisions about the product
• I will write clear, concise, thoughtful, and respectful problem reports. (I may make suggestions about design, but I will never presume to be the designer)
• I will let you know how I’m testing, and invite your comments. And I will confer with you about little things you can do to make the product much easier to test
• I invite your special requests, such as if you need me to spot check something for you, help you document something, or run a special kind of test
• I will not carelessly waste your time. Or if I do, I will learn from that mistake.

Regards,

TESTER

Why Mobile App Testing is needed

There are many vendors with variety of mobile handset in the market which supports various mobile applications in it. As there are many options available to the user it is very important that mobile handset in conjunction to its various application must be tested before going into market to ensure Right first time (RFT) theory. Any leftover fault may create wrong perception in user’s mind which will results into business loss, no matter how good applications are.

In the below section we will try to understand different issues which may left in the application if proper mobile application testing will not be performed.

Usability issue

After designing and developing a mobile app it must be tested by a group of eager mobile users. It might be possible that application is functionally working fine however it is very difficult to use and so feedback from few selected mobile users on complete user experience must be recorded and acted accordingly if required.

Navigation issue

As we all know there are many navigations required in any mobile application which needs to be tested thoroughly. It includes all menu functions and internal/external links.

Unexpected behavior in negative environment

There can be unexpected e apps reactions against system changes such as low memory or low battery and so beforehand this negative testing is required. This should also be checked against negative challenges such as malicious attacks.

Hidden defects

If all is well with the general user experience of any app, there could still be hidden issues that could cause sporadic performance or later problems. These defects are found through both software and hardware tests and are only completely detectable through professional services.

Connectivity issue

Many mobile apps rely on internet connectivity in some form or another after original download (even if just for updates). Monitoring how a mobile app functions in conditions of low internet connectivity or mobile signal is a very important stage in mobile app testing and will ensure that any problems formed during app development can be corrected before release.

Audio performance

Another area which needs to be tested is the apps ability to interact with various audio settings on different handsets. App details including audio and vibrate feedback (when a sound or buzz plays on a touch) also need to be thoroughly checked to eliminate any future glitches.

Multiple mobile device platforms

There are many mobile platforms that are in use currently and any mobile application must support this. To ensure mobile application working with multi platform prior testing is needed. Though it is not possible to test application against each and every platforms in the market however we can perform this against most widely used platforms in the market.

Diversity of mobile devices and its hardware configuration

As we all know there are varieties of mobile devices and so there are varieties of their corresponding mobile hardware. Any mobile application must be tested against few commonly used mobile devices to ensure it’s compatibility with various mobile devices and its hardware.

Network Diversity

There are many ways to carry the traffic. Mobile application performance may vary depending on different network gateways for carrying out traffics and hence corresponding testing against various network gateways must be performed before launching any product.

Cross Browser Testing

Here are few guidelines which can be used to come up with an approach for Cross Browser Testing of a web application.  

• List out all the stable releases of browsers with version mostly used in market and the platforms also  
• Collect all the installers and required applications at one common place
• Ask for the stable release of the application which has been tested (functional at least) and identified issues have been resolved up to an extent
• Ask for a baseline browser and platform and create a test environment
• Using the baseline platform try to investigate the maximum area of application and record the evidences if time permits
• While performing the baseline focus should be on Layout, GUI, Formatting consistency etc.
• Compare Text/Image wrapping across the website on different Web Browsers against baseline browser, and check for any discrepancy.
• Compare Alignment of text and images across the website on different Web Browsers against baseline browser, and check for any discrepancy.
• Compare formatting, font and size of text across the website on different Web Browsers against baseline browser, and check for any discrepancy.
• Compare formatting, font and size of Buttons/Links across the website on different Web Browsers against baseline browser, and check for any discrepancy.
• Compare scrolling on the web pages (Horizontal and vertical) across the website on different Web Browsers against baseline browser, and check for any discrepancy.
• Compare the Header across the application on different Web Browsers against baseline browser, and check for any discrepancy.
• Compare the footer across the application on different Web Browsers against baseline browser, and check for any discrepancy.
• Compare the pages content across the application on different Web Browsers against baseline browser, and check for any discrepancy.
• Compare scrolling on the web pages (Horizontal and vertical) across the website on different Web Browsers against baseline browser, and check for any discrepancy.
• If applicable, check for Flash videos, Audio across the website. Generic Functions (Play, Pause, Full Screen, maintain Volume etc)
• If applicable, check for you tube Videos across the website. Generic Functions (Play, Pause, Full Screen, maintain Volume etc)
• If applicable check for the advertisements across the application on multiple browsers.
• Check for the static and dynamic both type for contents on multiple browsers
• If applicable, check for navigation of third party applications/URL/links and compare the results with the baseline
• Check for firewall settings and issues on multiple browsers and platforms

Who A Great Tester Is

If you ask me, I'll tell you a great tester

Is devious

A great tester has a streak of deviousness.  Anyone can follow the lists of test cases that abundantly fill most books on testing.  A great tester can move beyond these lists and dream up an endless series of gnarly methods for attacking the program.  A great tester is described by developers as "sick" and "demented".

Is curious

A great tester is interested by everything.  A great tester wants to understand why everything works that way it does.  The best (or worst, depending on your point of view) bugs are a result of interaction between two pieces of software (applications, modules, components, whatever).  A great tester knows that understanding how something works leads directly to understanding how that something interacts with another something, which interaction leads directly to bugs.  A great tester manifests this curiosity in every aspect of life:  how does marketing work?  How are construction cranes built?  Why do they add rebar to concrete?  How are crayons made?  A great tester's curiosity knows no bounds.

Is excited by bugs

A great tester thinks bugs are cool.  A great tester shows up in a developer's office on a regular basis with a big grin eager to show off the latest nifty keen horridly awful bug that the tester found in the developer's code.  A great tester boasts about bugs to other testers and eagerly listens to other testers' exploits.

Knows there are always more bugs

A great tester knows that no application is ever bug free.  A great tester knows that an application that seems to be bug free is really full of bugs they haven't thought to look for.  A great tester is always on the lookout for new types of bugs.  A great tester views every bug found by a customer as a sign they missed an entire class of bugs.

Stays on track

A great tester knows that finding and isolating bugs to their root cause requires focus.  A great tester doesn't ignore bugs found along the way, but postpones investigating them until the current bug is nailed.  (And, of course, gleefully told to the corresponding developer.  And boasted about to other testers.)

Scopes appropriately

A great tester knows that they will not have sufficient time to run every test case they would like to run.  A great tester prioritizes and scopes their tests so that the tests most likely to find the bugs most likely to affect the customer are executed first.

Investigates weird behavior

A great tester watches for odd occurrences.  Icons that display one position off from where they should and radio buttons that don't stay set may be a simple programming error, but a great tester knows that such oddities are just as likely to be but the tip of a nasty bug.  A great tester goes beyond "That's weird but that's life" to "A-ha!  That's what's going on!"

Writes precise bugs

A great tester takes the time to narrow a bug down to the minimum number of steps necessary to reproduce a bug.  A great tester tests around a bug to understand what the bug actually is.  A great tester writes bugs that state the bug exactly and clearly distinguish between what is proven fact and what is conjecture on the part of the tester.

Has passion for the customer

A great tester knows that they are the last defense against the customer receiving a product that doesn't serve the customer's needs.  A great tester understands every aspect of the customer.  A great tester understands what the customer needs to do and how the customer wants to use the product.  A great tester looks beyond the customer's needs to see how the product can revolutionize the customer's tasks.  A great tester promotes the customer's point of view throughout the product cycle, from the first nascent product vision through specifying and implementing features to cutting features and triaging bugs to product release and ongoing maintenance.  A great tester helps the rest of the product team understand the customer as well as they do.

Is a specializing generalist

A great tester is completely familiar with every detail of their feature.  A great tester also understands how their feature fits into and affects the entire product.  A great tester is willing to change or even cut their feature in order to make the product as a whole better.

Picks their fights

A great tester recognizes that fixing every bug is often not worth the resources that would be required.  A great tester balances each bug against each other bug and allows some bugs to be shipped so that other bugs can be fixed.

Stands their ground

A great tester knows that some bugs just have to be fixed.  A great tester is willing to be obstinate and obdurate and to ruffle feathers if necessary in order to ensure that a must fix bug is in fact fixed.  A great tester calmly shows why the bug must be fixed and convinces the rest of the team that it indeed can't be shipped.

Can ask developers where the bathroom is

Visitors to a foreign country are well advised to become familiar with the language and customs of the country, enough so that they can get a cab back to their hotel, ask for directions to the bathroom, and know whether shaking hands is the height of civility or completely gauche.  Likewise, a great tester is familiar with the language and customs of developers.  A great tester understands UML well enough to get the gist of UML diagrams and to draw a class or sequence diagram without making developers laugh too hard.  A great tester can write code at least as well as first year programming college students.  A great tester understands design concepts sufficiently to participate in design discussions and reviews without being asked to leave the room.

Knows testability is just one of many concerns

A great tester knows the only way to truly test an application is to build testability in to every aspect of the product.  A great tester analyzes the product's architecture, design, and features, and develops a plethora of ideas for ensuring the product can be tested.  A great tester, however, knows that testability is not the only factor affecting the architecture, design and features.  A great tester balances testability against the other factors and helps the team create the right mix.

Knows when to ask for help

A great tester takes pleasure in a challenge.  A great tester, then, enjoys banging up against a brick wall and slowly breaking through it.  Some walls are thicker than others, however, and sometimes the wall has a tester-size hole that the tester continually manages to miss.  A great tester realizes when it's time to ask for help and does so.  A great tester knows who to ask for help.  A great tester knows there isn't any shame in asking for help.

Makes time for training

A great tester knows that the only way to continue to be a great tester is to never stop learning.  A great tester doesn't limit this education to testing, either, but also researches programming, program management, marketing, and anything else that is remotely related to the process of creating software.

Never stops testing

A great tester goes beyond feature boundaries and tests throughout the product.  A great tester tests other products. A great tester tests books, refrigerators, lights, doors...anything in any part of their life that makes them go "That's not right".

Bugs with Emotions

Acceptance - A bug that accepts it will be found. Typically a bug that is obvious as soon as you login (or even one that prevents you from logging in)

Affection - A bug that loves other bugs. Some bugs are gregarious and like to live together, these are generally found all in one place

Alertness - A bug that is alert to being found. It shifts its position so that just as you think you have it nailed down, it moves somewhere else. Memory leaks are good examples

Ambivalence - A bug that doesn’t care whether it is found. Typically a bug that you come across by accident

Anger - A bug that hits you whenever you get near it, e.g. the blue screen of death

Annoyance - A bug that gets cross when it is found and causes another bug to be created once fixed

Anticipation - A bug that expects to be found and looks forward to it. Typically a bug that keeps showing up throughout the application

Anxiety - A bug that worries about being found. These bugs hide in the corner and have to be winkled out with specially designed tests

Awe - A bug that is amazed you have found it. It pops up with unusual error messages like “Error: This should never happen!”

Boredom - A bug that just gets fed up with being there. These bugs usually go away of their own volition in the next release

Calmness - A bug that can’t get you excited. Typically, a cosmetic bug, such as a mis-spelling

Compassion - A bug that cares about you. This type of bug gives you an indication of what the problem is with a sensible error message

Confusion - A bug that is not sure whether it exists or not. Typically this type of bug can cause endless arguments with developers

Contempt - A bug that really thinks testers are lowly creatures. Usually manifests itself with incomprehensible techno-speak in the error message

Contentment - A bug that is happy with its lot, it likes being there. Usually these bugs are difficult to eradicate as they just like being there

Curiosity - A bug that looks for different parts of the application to get into to see what sort of damage it can do there. This type of bug is often due to a continual bad coding practice throughout the application

Depression - A bug that throws itself at you suicidally. Normally found early on in the testing cycle

Desire - A bug that finds testers incredibly attractive. Every tester on the project keeps finding this bug, usually finds its way into the bug tracking system multiple times

Disappointment - A bug that is disappointed you didn’t find it. Usually shows itself in Live

Disgust - A bug that finds testers disgusting. Generally gives a supercilious error message

Doubt - A bug that doubts its cause. Could be one of a number of different reasons why this bug exists

Ecstasy - A bug that is high on illegal substances

Embarrassment - A bug that is embarrassed to be found. Generally has error messages like “Sorry, you can’t do that”

Empathy - A bug that feels for the tester. Typically shows itself after you have had your morning coffee and never last thing at night

Emptiness - A bug that has no content. This type of bug is where functionality is missing that should be there

Enthusiasm - A bug that has to show itself spectacularly, typically a system crash

Envy - A bug that wishes it could be like other bugs. Typically this is a bug that looks serious, but in reality is only a minor problem

Fanaticism - A bug that goes round telling other bugs how to create mayhem. Typically this is a bug that goes round multiple systems

Fear - A bug that fears being found. A particularly difficult bug to find and has to be tempted out into the open

Frustration - A bug that wishes it could be a working system Tries hard to work, so only shows itself intermittently

Gratification - A bug not found until system in Live. This is a happy bug has defeated all attempts to find it and has finally achieved its aim of thwarting you

Gratitude - A bug that thanks you for finding it. When it is found it shows you the position of other bugs, so you get multiple bugs shown on one screen

Grief - A bug that is inconsolable at the death of one of its fellow bugs and gives up trying to hide. Found when a previous bug, now fixed, was hiding this bug

Guilt - A bug that regrets being there. hese bugs are keen to atone for their sins and are, therefore, easily fixed

Happiness - Found during “Happy path” testing

Hatred - A bug that hates you is one that you think you have found, but keeps coming back to haunt you in every release

Hope - A bug that hopes not to be found. This type of bug just sits there and waits. It may be an easy bug to find or a difficult one, but is always obvious

Hostility - A bug that shouts at you. Usually shows itself with an error message ALL IN CAPS

Humiliation - A bug that is so ashamed at being found that this part of the system never displays any bugs ever again

Hysteria - A bug that shouts and screams all over the application, causing all around to react with panic. Usually shows as a system crash

Inspiration - A bug that is as a result of some really clever coding brought about through an inspired developer….who got it wrong

Jealousy - A bug that shows the symptoms of another type of bug which is superior to it

Kindness - A bug that is kind to the developer by being easy to fix

Loneliness - A bug that is a hermit. This type of bug is isolated from the rest of the application in an area that has no other bugs

Love - Another gregarious bug. Not only does this bug congregate with other bugs, it actively hangs on to them, so you find two, three or more all doing the same thing

Lust - This is a bug that spawns other bugs. A prolific breeder which means both tester and developer are forever chasing its children

Panic - This bug causes all the parts of the system to go into meltdown. Typically this is a tight loop that consumes all of the CPU

Patience - This bug just takes its time. Often seen as a Performance bug

Pride - A bug that thinks it is the best bug in the whole system. Often surfaces before a fall-over

Repentance - A bug that feels bad about existing and once fixed, fixes a number of other bugs as a side effect

Resentment - A bug that does not like to be found, such that once fixed it re-appears again and again

Shyness - A bug that is really hard to find and only surfaces in rare circumstances

Suffering - A bug that causes the entire system to suffer. Often found as a result of security testing

Surprise - A bug that does not expect to exist. Typically a simple bug that should have been found in unit test

Wonder - A bug that is amazed to exist and cannot understand how a developer could possibly have coded it that way

Be Proud to be a TESTER

A real good article and worth reading!!

Once while at work I was getting my test cases reviewed about a project/domain which I was new to and one of my colleague gave a strong comment "Mr.X, please think of more cases".

I was a bit disappointed because already there were enough test cases that it will take 3 days to complete a cycle and I thought "should I further shoot at my own legs?" (The more cases I write, more I have to test, it's painful, isn't it?) ..

I tried to defend that testing would become complex if more cases are there than beyond the objective of the release/product requirements.

Ha! Thanks to what I told, he shared with me a great story which happened in his previous company and it is one of the best lessons I have had as a tester!

This (could be a) is True Story that happened in some company.

A digital camera was released in the market and after sometime a customer came back complaining that........

Problem: The camera's software was crashing continuously while he tried to capture images.

FIR: They first asked some basic information from the customer and made an attempt to reproduce the issue but it was not reproducible.

Analysis:

1) Software is crashing - so they listed out all possible scenarios where the software could crash.

2) Tried putting the camera to all listed possibilities.

3) Giving a report to the management that it was unable to be reproduced.

Further Analysis:

Taking this as a challenge, a great tester had the following concern –

1) He was concerned what festival was it?

2) He was concerned what was the customer was trying to capture?

3) He was concerned what is the frequency of the crash the customer noticed?

4) He was concerned about all other surrounding factors like temperature, place where the festival was celebrated ... etc?

Outcome of Analysis:

1) The festival was Diwali or Deepavali!

2) The customer was trying to capture fire crackers, explosion, fireworks display in an open ground at around 22 00 hours (night).

What a fantastic tester was he! He identified the problem and told the developers

"The camera software is crashing when the viewfinder is black (because of dark night) and a sudden gush of light through the firecracker explosion is putting the DSP (digital signal processor) to a load/stress beyond its boundaries to display the image. And at this point of time when the user tries to capture it simply crashes" What he said was right!

Finally they fixed it and now today it's a robust digital camera!

********End of a probable true story*********

Summary of lessons

• Collect real time data for testing a product/application
• When you are unable to reproduce an issue, think out of the box, and think out of the black box
• When you are unable to reproduce an issue call someone who can dig deeper than you
• Don't assign test case draft to the person who is going to test it; he may omit complicated cases to bring down his/her complexity in executing it
• Review the cases with a non team member/customers/domain specialist/tester network
• When you release a product: anticipate and be prepared to face such situation
• Hate the product while testing and love the product after its release Emotional testing (new concept)
• Good Testers not only report bugs, they suggest a solution!
• Keep learning to be a good tester!
• Test all your products in India, its The Testing Bowl of the world!
• Don't terminate an issue just because you are unable to reproduce, it could kill a company!

Re-Testing vs Regression Testing

You must retest fixes to ensure that issues have been resolved before development can progress.

So, retesting is the act of repeating a test to verify that a found defect has been correctly fixed.

Regression testing on the other hand is the act of repeating other tests in 'parallel' areas to ensure that the applied fix or a change of code has not introduced other errors or unexpected behaviour.

For example, if an error is detected in a particular file handling routine then it might be corrected by a simple change of code. If that code, however, is utilised in a number of different places throughout the software, the effects of such a change could be difficult to anticipate. What appears to be a minor detail could affect a separate module of code elsewhere in the program. A bug fix could in fact be introducing bugs elsewhere.

You would be surprised to learn how common this actually is. In empirical studies it has been estimated that up to 50% of bug fixes actually introduce additional errors in the code. Given this, it's a wonder that any software project makes its delivery on time.

Better processes will reduce this ratio but will never eliminate it. Programmers risk introducing casual errors every time they place their hands on the keyboard. An inadvertent slip of a key that replaces a full stop with a comma might not be detected for weeks but could have serious repercussions.

Regression testing attempts to mitigate this problem by assessing the ‘area of impact’ affected by a change or a bug fix to see if it has unintended consequences. It verifies known good behaviour after a change.