Still Life

A Series of Mental Snapshots

What’s in a Name?

Posted by Steve on December 19, 2007

Why does there need to be a name for all different kinds of testing. Why do you need to identify if one case is a boundary case and another is not. There are clearly a class of test areas that could be considered ‘boundary tests’ but why need the name? I it as a convention that just exists for the sake of cross communication between people. IE so when you say boundary testing, I know what you are talking about, just as in we can identify that a shoe is a shoe and communicate that to someone else.

Other then for that reason I do not see much use for it; what’s the point of having names for things? To me having a name limits what you see and limits creativity. If you feel that certain things are not to be considered boundary tests, then maybe you won’t do them. Maybe you are pigeon holing yourself into constraints that do not need to be there. Furthermore it seems that everyone has a different idea of what a ‘boundary’ test is. IF that is the case then why even have a name for it? I looked online to see if I could get some definitions and here is what I hit:
“Boundary testing has definition A testing strategy based on testing at the boundaries of equivalence classes – since this is where most errors occur.”

So from that you need to look up equivalence classes:
“Equivalence Class: A testing strategy based on determining the possible equivalence classes and creating a test case for each”

Again this begs the question of what they define as an equivalence class. Found another website that defines things as such:
• Case #1: domain of data is an interval
– Class(es) of valid values
– Class of invalid values (out of boundary, inf. and sup.)
– Class of non member (for numerical values, characters are non members)

So from this it is a little clearer as to what a formal definition of what a ‘boundary’ test can be considered. Its a way to test such that elements that ‘would’ have the same outcome are not retested, thereby identifying equivalence classes and tests the elements that would yield only different results Here is an example to illustrate a simple case of equivalence cases and boundary conditions.

Example: There is a field that is only suppose to be able to take in Unsigned short ints (programmed in c++ MIN: 0 Max: 65535).
In this case I see the equivalence class here to be {1 – 65534}. It would be expected that these values would yield ok results. This would leave the ‘boundary’ cases to test, in this case {0, 65535} I see those as the working boundaries… and then there are the non working boundaries {-1, 65536, ‘A’, ü}. That class covers the min, the max, a letter and a symbol. All of the rest of the possible entries could be lumped into another equivalence case of improper values {The Rest}

So from this example we can see two equivalence cases and 6 entries for boundary testing:

Equivalence cases:
————————-
Valid Values {1 to 65534}
Invalid Values {-Infity to -2, 65537 to infinity, all leters (but one), all symbols(but one)}

Boundary cases:
———————
Valid Boundary Values {0},{65535}
Invalid Boundary Values {-1}, {65536}, {A}, {ü}
So from this example there is now a better definition and hopefully leads to better understanding of boundaries and equivalence cases. So not that we have a sort of loose definition for boundary cases we can look at, well maybe a ‘simple definition’ is a better term for that.

Going with this example I can already see places where I could have made a mistake, maybe something happens at -65536, maybe the numbers switch back over to 1? That kind of behaviour is not unheard of, but to me does not come up as what I would consider a ‘boundary’. This is what I was bringing up before, with pigeon holing yourself into a situation where you effectively have blinders on. When you come up with a definition, it becomes more difficult to look at anything outside of that definition, therefore soon as the definition is in place, that well might be all that you look at. I think that the idea of boundary testing is very important, but having a firm definition is not. Here is what I see boundary testing as:

The testing of conditions that YOU feel will cause problems. These feelings will more then likely be based on some sort of logic, but sometimes not. For example I was inputting values into this calendar and it was not clear what the boundary input was. So I just try some random year 1656; I get an error back stating dates for SQL must be between 1753 and 9999. Therefore I found the boundary pretty quickly by just putting in a random number, something that might not be done because there are guidelines to the boundary testing that you are doing.

To wrap things up a bit, boundary testing is an important concept, and should have some methodology to how it is done, but the more rigid the methodology and the more rigidly boundary testing is seen, the more likely it is that things are going to be missed, because once methodologies, and things in general are in place it is often difficult to look outside them.

–Steve

Post Script
-I know there are a lot of things that I am trying to get across here, any questions on anything are of course welcome
-I could quickly find what websites I grabbed the boundary test/equivalence case definitions from, I’ll update that when I find them

Advertisements

6 Responses to “What’s in a Name?”

  1. Stuart Murray said

    Wonderful thought, it is somewhat true that as test professionals we have locked ourselves into a set of definitions so as to ease understanding and save time bringing people up to speed on projects across the board but we should take a lesson from you, sometimes our definitions can work against us. The date example would not have occurred to most people as they would know that since 1999-2000 most systems only accept dates in the last 100 years or so, so a date well beyond this would not have being tested.

    Which reveals that the date range is 1753-9999, which to me raises the question of why 1753? is it something to do with historical data? or is it just a line in the sand?

  2. Steve said

    So I did a little bit of research and found that it was for historical reasons. Here is the explanation:

    “Good question. It is for historical reasons. In what we sometimes refer to as the western world, we have had two calendars in modern time: the Julian and the Gregorian calendars. These calendars were a number of days apart (depending on which century you look at), so when a culture that used the Julian calendar moved to the Gregorian calendar, they dropped from 10 to 13 days. Great Britain made this shift in 1752 (1752-09-02 were followed by 1752-09-14). An educated guess why Sybase selected 1753 as earliest date is that if you were to store an earlier date than 1753, you would also have to know which country and also handle this 10-13 day jump. So they decided to not allow dates earlier than 1753. Note, however that other countries did the shift later than 1752. Turkey, for instance, did it as late as 1927.
    Being Swedish, I find it a bit amusing that Sweden had the weirdest implementation. They decided to skip the leap day over a period of 40 years (from 1700 to 1740), and Sweden would be in sync with the Gregorian calendar after 1740 (but meanwhile not in sync with anyone). However, in 1704 and 1708 the leap day wasn’t skipped for some reason, so in 1712 which was a leap year, they inserted yet an extra day (imagine being born in Feb 30!) and then did the shift over a day like everyone else, in 1753.”

    This was taken from: http://www.karaszi.com/SQLServer/info_datetime.asp#Why1753, by Tibor Karaszi

  3. m_i_m said

    Hi there!

    Firstly, i’d like to commend you on your thoughts. Yes, i do agree that often definitions cannot be hard and fast, because they may not always be valid.

    However, if I look at the boundary value analysis concept, and this is but my view and opinion based on the little i have picked up so far, I would say that the concept per se seems to be a valid concept. I think the question we need to ask, is why are there issues at the boundaries?Why? From my understanding, and please correct me if i’m wrong, i believe it is a mechanism to check whether the developer has used the correct logics in his program. By logics i mean the conditional statements in a program. Those requirements that specify a range, sometimes inclusive or sometimes exclusive of the end points.

    For example:
    The student’s age should be greater than 18 and less than 22.
    For an ‘A’, the final mark should greater than and equal to 75.
    etc.

    Often what happens, when we program these statement/conditions, we may make a mistake. And the little tests we do as developers, may not catch the error(s). Therefore, the boundary value analysis would be quite instrumental in us finding the error(s), since it hits at the heart of these conditional statements.

    Please let me know what you think.
    Thanks.
    Regards,

  4. Joe Harter said

    Response to post by m_i_m:

    I’m not sure Steve’s article is trying to sway testers from doing the commonly used “boundary testing”, instead he is challenging testers to use critical thinking. All to often in my testing experience we get wrapped up in only testing at the boundaries, because that’s what we think we have to do. I think sometimes testers lose their autonomy and start to just repeat learned behaviors.

    Steve:
    I really liked this article. I can ask the same question of “What’s in a name?” to “System Testing”. I work in a System Testing group which to us is defined as “Black box testing after the unit testing phase in the SDLC”.

    What we actually are is the only IT Testing group in our company, so when I recommended that we start to consider white-box testing I was met with protests from the managers that “That isn’t what we are!”

    I think it is always important to evaluate what you are and not let nomenclature define you.

  5. Hi Steve, very out-of-the-box approach to boundary testing. This is more like the “glass half empty or half full” scenario to be honest. I mean, it has two sides. One one hand it is sometimes essential to have a certain framework, whereas in some situations it is better is we are not shackled by constraints and let our creativity looose. The answer, i feel lies somewhere in between- moderation. And what that middle ground is, varies for all of us.

  6. To be honest I think you have an excellent handle on labelling test techniques. Having read your post above I began thinking about my own thought processes when running through different techniques such as boundary or equivalence partioning etc.

    I found that some types of test tend to be limited to exactly what they are, whereas others (as you’ve identified) are open to your own interpretation, which is of course based on the general outline of that technique. Take orthogonal array tests as an example; an orthogonal array is by definition a specific technique, thus to apply it to a pice of software as a quick functionality test means simply that. On the other end of the scale though if someone says “boundary test” that’s open to the tester’s own view of what “boundaries” they are going to test.

    Definitely a great article, thanks for sharing!

    Dave

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: