Wednesday, December 22, 2010

Random issues

 Have u faced random issues during testing? Testers look at the exceptions logged and try to find out the root cause of the issue. If they cannot find the root cause, they call in the developers. What if the devs are not able to find the root cause of the issue from the exception logs?

Here is the typical flow for such scenarios

Testers log the bug mentioning that the bug is random.
Two or three days down the line, a Dev team picks up the defect.Since he is not able to find the root cause from the logs, he marks the bug as not reproducible and moves it into QA plate.

QA tests it for two consecutive builds. Since he cannot reproduce the issue, he is forced to close the issue.

After a few months, the build is delivered to production, where the bug is reproduced.

Major showdown happens with client.

Few points

1. Neither the Dev or the QA is able to escape from this vicious cycle. Each is doing his best, but in the end - when bug occurs in production - its the proverbial mess -

2. No QA is able to stand up and say that since x number of bugs are critical + random, and since no code has been checked in to fix the issue, we should not ship the product.

So taking mass consensus before shipping the product ( a key characteristic I have seen in several agile projects ) is not useful here.

Thursday, July 22, 2010

Is the question stupid?

Questions which dont make sense.


Question 1. Which of the following columns is greater?

Column A   The number of integersthat are less than -10                                                        
Column B   The number of integers that are greater than 9
        

The answer they provide goes something like this.

First, you'll need to know that there are as many negative integers as there are positive integers, because for every positive integer,

there is an equivalent negative integer. However, the set of numbers in Column B is greater than the set of numbers in Column A because

Column B also includes positive 10, while Column A does not include negative 10.

I have a counter argument for this.

For the time being, lets map -11 to +10.
Now I would have to map -12 to +11.
-13 to +12 etc.
If I represent -11 as x and +10 as y (x and y being constants), the mapping becomes like this.

X -> Y
x-1 -> y+1
x-2 -> Y+2.

Now I denote the numbers added to x and y as z (z is a variable which increases by one always).

X-Z -> Y+Z . Note that Z is the same for both L.H.S and R.H.S.

The mapping holds true irrespective of the value of Z, even when  z-> infinity ( basically we say that Z tends to infinity). . For any value of Z from 1 to infinity, there is a number X-Z which can map to X+Z.

Hence by the law of induction, both are equal.

Column A is infinity and column B is infinity. This is the reason we cannot compare one infinity to another.

Do let me know your views. :)

Wednesday, July 21, 2010

Similarity between bug report and a case on trial before the court!

Sometimes logging a bug is like putting a case before the court.
In court, we talk about "air tight" cases. Similar is the case with bugs. In cases put on trial before the court, if there is any ambiguity in the facts, the case is summarily rejected. Similarly, we cannot log defects bases on "hunches". Hunches can be the "basis" of bugs. That is when we need to do our homework and make the case (the bug report) as tight as possible.

If I say that, in a particular scenario, the application is not working, then I should do my homework and find out what other scenarios it works and what all scenarios it wont work. This helps devs ( in fixing the issue) also .
The major advantage is that all the manifestations of the bug are fixed in one shot and no more time is spent by devs in fixing the same bug.

I have seen time and again, when a bug is logged, it is fixed for a particular scenario.Later on, the same bug is reopened or a new bug logged for another manifestation of the same issue in another scenario.

Thursday, June 10, 2010

Selenium architecture

Today somebody asked me about Selenium architecture and I had only a vague idea about it. One year back ( after I joined thoughtworks), I read about it and now i have forgotten it completely. :)

Looking for documents on selenium

How network connectivity affects performance of the team - in office

When we consider network downtime, we consider only the hours of network disconnectivity.

Say, network was down for 3 hours, we consider loss of 3 hours of productivity. But what i have observed is the slow erotion of "the spirit to work" when the team is subjected to frequents outages.

Friday, April 30, 2010

Problem with nant and .net 3.5

Today when i ran nant, i faced the following problem.


[loadtasks] Scanning assembly "NAntCollections" for extensions.
[loadtasks] Scanning assembly "NantTasks" for extensions.
[loadtasks] Scanning assembly "Newtonsoft.Json" for extensions.

BUILD FAILED

C:\ServicesTests\tests\ServicesTests.build(9,4):
Could not include build file 'C:\ServicesTests\tests\env.build'.
Object reference not set to an instance of an object.


The reason is a bug in the older nant.
It has been fixed in 0.90-alpha1 (April 1, 2010)

From the release notes

.NET Framework 3.5
•Modified version (as returned by framework::get-version()) to "3.5".

Wednesday, April 14, 2010

Using selenium with weird id's

One problem i faced today was with weird id's.

A button had the id "Seat_Outward_558694241". Each time, the number "558694241" would change. Using xpath with complete dom could create brittle tests. Since xpath is basically relative position, if the button is moved outside the table ( in which it was currently residing), the test would have failed.

Solution

//input[@type='button' and @value='Seat Plan']

This is also xpath, but it would not fail as long as there is a button with caption "Seat plan".

Saturday, April 03, 2010

Software testing adequacy

In 1972, Dijkstra claimed that “program testing can be used to show the presence of bugs, but never their absence”

Has software testing evolved after this?

Terms which are related to software testing adequacy

Statement coverage.

Branch coverage.

Path coverage.

Mutation adequacy.