Still Life

A Series of Mental Snapshots

Archive for the ‘Testing’ Category

Creating specific files in windows with fsutil

Posted by Steve on July 10, 2008

This post comes from a recent issue I had recently, that was actually solved very easily with a built in windows tool. While running a test scenario, I needed to fill up a 20 GB disk. I have had to do with before, and it is pretty simple to do, you just grab some large files a copy them over and over again. I will never be doing that again thanks to a fancy little tool, fsutil.

fsutil is a built in windows program that has a whole bevy of options. You can open up a command line window and type fsutil to get the following options:

—- Commands Supported —-
behavior Control file system behavior
dirty Manage volume dirty bit
file File specific commands
fsinfo File system information
hardlink Hardlink management
objectid Object ID management
quota Quota management
reparsepoint Reparse point management
sparse Sparse file control
usn USN management
volume Volume management

From here you can get more details about any of the above features by entering fsutil “Feature Name”. So to create the large file we want to see the file options but entering fsutil file. Typing this in the command window gives the following output:

—- FILE Commands Supported —-
findbysid Find a file by security identifier
queryallocranges Query the allocated ranges for a file
setshortname Set the short name for a file
setvaliddata Set the valid data length for a file
setzerodata Set the zero data for a file
createnew Creates a new file of a specified size

So to create the large file, you need to use the createnew. So you can type in fsutil file createnew to see the usage as well as an example!

Usage : fsutil file createnew <filename> <length>
Eg : fsutil file createnew C:\testfile.txt 1000

It is important to note that the length, actually is the length in bytes, so a value of 1000, will be 1KB. There we are, any file of any size can be created. I foresee using this many times in the future, I also see exploring the fsutil feature much more, as it seems like a useful tool.

–Steve

Was this post useful? Could I improve on the layout of how I present these quick little tutorials? If so leave me a line, and let me know, I would love some feedback!

Posted in Testing, Tools | Tagged: , , , , , , , , , | 4 Comments »

Bugs in the Wild, the Irony Edition

Posted by Steve on June 18, 2008

It has been a while since I wrote a post specifically about testing, and I ran into an opportunity that I couldn’t resist. I am attending a software test conference, CAST, that is in Toronto, in about a month. I am looking forward to this because there are some good speakers lined up, and I am attending a tutorial that is going to be done by Jerry Weinburg, who if you do not know, has written a whole host of books on various topics that relate in many different ways to testing.

Now where the testing part comes in is when I got an e-mail this morning stating that I have officially been signed up for CAST, and can go see what I am signed up for. I decide to click on the link to give the website a little look, and I was very glad I did, because it gave me, as you can seen in the screenshot below, a humourous dose of irony.

Yes, that is correct, there is a very large, noticeable and common mistake in the text, an apostrophe is accidentally displayed as: �

I did notify them of this error before putting up my post, just to give them a little heads up, but I did get a good laugh that there was a bug on a website for a software testing conference!

–Steve

Post Script:
If you have interest in the conference, you can check it out here: http://www.associationforsoftwaretesting.org/drupal/conference

Posted in Bugs in the wild, Testing | Tagged: , , , , , , , | 2 Comments »

Adventures With An Espresso Machine

Posted by Steve on May 23, 2008

The kitchen near my desk had a new coffee machine put in yesterday (May 22nd) and I decided to give it a whirl. I place my cup under the dispenser a see that there are four buttons to press, I was able to ascertain that one was for steam, one for an espresso, one for a full cup and one where it will fill until you tell it to stop. I decided that I wanted a full cup, so I press the full cup button… but a full cup does not come out, I get about a half cup of coffee; I was a little disappointed to say the least. Since the cup was still only half full I decided to try the fill until you stop button. I grab another cup just in case my first one overflows. So the machine starts piping out the coffee, and of course it does overflow, but not very much, maybe by an ounce or so. From this I ascertain that this button won’t give me a full cup either. Now I have used espresso machines before, in fact my parents own one and I used it consistently for 2 years before going off to University, and I KNOW there is a way to set the amount of liquid that comes out on any decent machine, but at this point I don’t know how, there does not seem to be an menu button.

Since I was stuck at the machine, I decide to check the internet, after a few pages, I find out that the amount of water that comes out can range from 2-12 ounces, but it doesn’t tell me how! So I start searching some more and low and behold I find an 2 page PDF of instructions, it tells me that I can access the menu by holding the large coffee button when the machine is turning on, Perfect! So I race back to the machine, after finishing the first coffee of course, eager to try to get a full cup of coffee out. I go into the menu, I get to the doses menu, and access the alrge coffee, the level is set at 425… I think to myself 425 whats? My first thought is that it is 4.25 ounces, so I try to set it to 800, but when I confirm that, it flips back to 425; hmmm seems I have hit a maximum. So I try 550, that works so I decide to see how much coffee comes out now. I take my cup hit the button… still only 3/5 of a cup.

I decided at this point I must have missed something in the instructions, so I go back to it a notice something peculiar under the instructions for setting the doses, it stated “Note: Dose values don’t refer to any measurement unit but the counts of the flowmeter”. I thought to myself, this machine has a computer in it, why could they not have converted this to Milliliters, or ounces!!! I really get riled up about this, especially because I have just finished that second, well third, (because the first one had 2 coffees in it) cup of coffee, knowing that the units are meaningless to me I decide to go back to the machine and do a few more tests. The first was to find the maximum, I did so by going up in increments of 100 and seeing if it would accept the value, I got to 725, and then started going up by 10s (knowing already that 800 was too much) I eventually find the max is 750 flowmeter counts of course. I place the cup, and press the button in anticipation, the coffee starts coming out, and keeps coming, and coming, and coming… It gets within a centimeter from the top and is STILL coming out, I realize I have definitely set it too high, I press the cancel button to avoid a mess. I have at last succeeded in getting a large cup, but one that is a little too large, therefore I go back to the menu and set the value to 700, this is my final adjustment!

I actually didnt test the 700 counts because I felt if I had another coffee I might be awake until the next day, so I left that till this morning, where when I got a coffee, the cup was filled right to the brim, which was a little bit too much. So I set it down another 50 counts, and there we have it the perfect cup of coffee, the cup fills to 7/8 the way, leaving just enough room if I want an extra shot of espresso.

So this may seem like just a mildly entertaining story, but in fact it reveals a lot about my thought process and how I approach problems.
My first attempt was to try all of the buttons to see if they would give me the desired volume
I then went on to trying to set the volume myself without instruction, since I couldn’t do that…
I then tried to find instructions, and once I did I looked for the key piece of information
I went to set the volume, but I was confronted with the unknown units
I then dug deeper into the instructions to find out the units were meaningless
I finally got the desired volume through a series of tests

This is in fact how I approach most problems, I first see if there is a solution already, I then try to find the solution myself. If that fails I go onto documentation, and if that fails I proceed to trial and error. Now if I could only find the person who set the units to flowmeter count and them just what they were thinking, my adventure would be complete.

–Steve

Posted in Testing, Work: General | Tagged: , , , , | 3 Comments »

It’s only a job title… right?

Posted by Steve on May 15, 2008

I have recently been putting quite a bit of thought into job titles and have come to somewhat of an impasse. Recently I said I would help revamp a job posting for the test coop position, and one of the road blocks that I have hit is what to put in the title. The posting is for the waterloo coop system and it would be an understatement to say that the system is less than ideal. There are two things about it that make a title so important:

1. The time window that a job posting is up is about 3 days and there are LOTS of jobs, enough that you cannot see them all in that short time period, thus the job title has to catch a potential coops attention if you want lots of applicants.
2. The only real way to do a search on the jobs is by job title; it is not possible to search what the actual posting says, thus the only way to try and pinpoint jobs you might want is by title

Well it’s just a title right? It shouldn’t be that hard to make one that is appealing; in most cases that might be true, but because this is a test position it is a little more difficult; unfortunately, especially in the coop program, Test (or QA) has a very bad rep, and rightfully so in some cases. I personally have worked a terrible Test coop job, we were glorified typing monkeys, I was given a very rigid test plan to go through that consisted of about 200 tests, and by the end I had done some of them up to 30-40 times! Furthermore creativity was also pretty much discouraged, going outside of our area was frowned upon and to top it off developers didn’t think much of testers, that basically saw us as a walking, typing monkey. So because of this experience I can sympathize with those who fear that testing job, because it is difficult to know if it will be good or not.

In getting back to the job title, I was considering if there was a way to put less emphasis on test in the title, so that people will actually look further into the job, and not disregard it simply based on the fact that the word test is in the title. I thought about it for a while, and decided that there would be no real way to represent this job properly without having test in the title, but I did come up with an idea that would actually represent the job better and might appeal to more people,

Software Testing and Development Specialist

Now the specialist might not be in the final rendition, but I think the development is actually an important thing to put in the title. The job definitely has room for development, and it something that seems to interest a lot of people and it demonstrates that the job is multi faceted.

I was contemplating putting this up or not, simply because it’s a very specific problem for me, but I figured heh why not!

An annoying problem can often be seen as an interesting challenge, as with many things in life, it’s all about perception.

Posted in Testing | Tagged: , , , | 2 Comments »

Bugs in the Wild: WordPress Total Views

Posted by Steve on February 21, 2008

There was a bug that I noticed today (as well as in the past few months) right here in wordpress. There seems to be an issue with the ‘Best Day’ and ‘Total view’ count on the blog stats page.

As can be seen in the attached picture, it says that my best day was ‘ Saturday September 23rd 2006’. This is impossible since I did not start this blog until September 2007. Furthermore the Total view count is terribly low, it is marked as 35, when I know for a fact I have many times received 35 views in a single day.

I find it odd that this bug has been present for about 2 months or so now. The problem is intermittent though. I have seen days were it actually has been fixed and does display the correct numbers. I also am very curious as to how they calculate those numbers, because all of the information is available from different sources. There is a graph that displays page views per day and there is also a section where an actual count is given per post that you have made. It seems odd to me that these functions work properly but the one mentioned above does not. It just shows that because one similar feature works does not necessarily mean that all the related others will.

–Steve

PS: I have done a little research and it appears tickets have been opened to let wordpress know of the issue.

wordpresserror.jpg

Posted in Bugs in the wild, Testing | Tagged: , , , | 1 Comment »

Trouble with compatability…

Posted by Steve on December 26, 2007

So recently I found a bug in windows. So I thought ‘cool, I’ll take some screen capture of it and then upload it to youtube and write a blog post about it.’ So I get two of these three things done properly; and based on the title a bet you can assume which one I couldn’t get to function!

Firstly I’ll describe the bug. The bug occurs when editing the details of a music file. If you select one of the text boxes and scroll the mouse wheel the text box floats as you scroll. THis clearly is not designed functionality.

So my first step was to get the screen capture and export it into a .avi file. I use software meant for screen capture (I am avoiding the name because I don’t want to bring negative attention to a product that in general I really enjoy) to get a nice 30 second video of the bug. When exporting to a .avi I fiddle with the settings slightly so the file is a reasonable 2 Mb size (instead of the original 170 Mbs!). I view the video file everything seems to be fine, the video looks ok and the framerate seems good as well.

From here I start writing a post on this blog to let the world know of this grievous error (heh heh)! I get my post all set and save it before posting it and surf over to youTube to upload my video (as this blog does not accept video uploads).

Here is where the trouble begins. I try to upload my screen captured .avi file, and everything seems to be going fine but when I watch the video it is only one second long! It was like watching the bug in fast motion, all of the 30 seconds where compressed into one.

I have a theory of why this happened; I am ‘fairly’ confident that it was because the frame rate I chose to take the video in was different then from what youTube desired. What I found odd about this whole thing was that the video plays fine in Winamp, windows media player and VLC player but once its uploaded to youTube it ends up being one second long!

My contemplation here lies with the following question: ‘Is this user error, or a bug in the youTube software?’.

Frankly I am not sure. I feel that I am pretty computer savvy but I have had no end of trouble with this youtube upload. I could, with reasonable certainty, identify what the issue was, but for some reason I could not seem to fix it (I tried a few times). Am I at fault and should be more aware of the .avi I exported or should youTube’s software be able to rectify the disparities between the video I am giving and what it expects.

Just something to think about while I get some new video capturing software and see if I can get that bug uploaded!
–Steve

Discussions and arguments are not about who is right or wrong; they are about having new ideas, thoughts and expanding the mind.

Posted in Testing | Tagged: | 1 Comment »

What’s in a Name?

Posted by Steve on December 19, 2007

Why does there need to be a name for all different kinds of testing. Why do you need to identify if one case is a boundary case and another is not. There are clearly a class of test areas that could be considered ‘boundary tests’ but why need the name? I it as a convention that just exists for the sake of cross communication between people. IE so when you say boundary testing, I know what you are talking about, just as in we can identify that a shoe is a shoe and communicate that to someone else.

Other then for that reason I do not see much use for it; what’s the point of having names for things? To me having a name limits what you see and limits creativity. If you feel that certain things are not to be considered boundary tests, then maybe you won’t do them. Maybe you are pigeon holing yourself into constraints that do not need to be there. Furthermore it seems that everyone has a different idea of what a ‘boundary’ test is. IF that is the case then why even have a name for it? I looked online to see if I could get some definitions and here is what I hit:
“Boundary testing has definition A testing strategy based on testing at the boundaries of equivalence classes – since this is where most errors occur.”

So from that you need to look up equivalence classes:
“Equivalence Class: A testing strategy based on determining the possible equivalence classes and creating a test case for each”

Again this begs the question of what they define as an equivalence class. Found another website that defines things as such:
• Case #1: domain of data is an interval
– Class(es) of valid values
– Class of invalid values (out of boundary, inf. and sup.)
– Class of non member (for numerical values, characters are non members)

So from this it is a little clearer as to what a formal definition of what a ‘boundary’ test can be considered. Its a way to test such that elements that ‘would’ have the same outcome are not retested, thereby identifying equivalence classes and tests the elements that would yield only different results Here is an example to illustrate a simple case of equivalence cases and boundary conditions.

Example: There is a field that is only suppose to be able to take in Unsigned short ints (programmed in c++ MIN: 0 Max: 65535).
In this case I see the equivalence class here to be {1 – 65534}. It would be expected that these values would yield ok results. This would leave the ‘boundary’ cases to test, in this case {0, 65535} I see those as the working boundaries… and then there are the non working boundaries {-1, 65536, ‘A’, ü}. That class covers the min, the max, a letter and a symbol. All of the rest of the possible entries could be lumped into another equivalence case of improper values {The Rest}

So from this example we can see two equivalence cases and 6 entries for boundary testing:

Equivalence cases:
————————-
Valid Values {1 to 65534}
Invalid Values {-Infity to -2, 65537 to infinity, all leters (but one), all symbols(but one)}

Boundary cases:
———————
Valid Boundary Values {0},{65535}
Invalid Boundary Values {-1}, {65536}, {A}, {ü}
So from this example there is now a better definition and hopefully leads to better understanding of boundaries and equivalence cases. So not that we have a sort of loose definition for boundary cases we can look at, well maybe a ‘simple definition’ is a better term for that.

Going with this example I can already see places where I could have made a mistake, maybe something happens at -65536, maybe the numbers switch back over to 1? That kind of behaviour is not unheard of, but to me does not come up as what I would consider a ‘boundary’. This is what I was bringing up before, with pigeon holing yourself into a situation where you effectively have blinders on. When you come up with a definition, it becomes more difficult to look at anything outside of that definition, therefore soon as the definition is in place, that well might be all that you look at. I think that the idea of boundary testing is very important, but having a firm definition is not. Here is what I see boundary testing as:

The testing of conditions that YOU feel will cause problems. These feelings will more then likely be based on some sort of logic, but sometimes not. For example I was inputting values into this calendar and it was not clear what the boundary input was. So I just try some random year 1656; I get an error back stating dates for SQL must be between 1753 and 9999. Therefore I found the boundary pretty quickly by just putting in a random number, something that might not be done because there are guidelines to the boundary testing that you are doing.

To wrap things up a bit, boundary testing is an important concept, and should have some methodology to how it is done, but the more rigid the methodology and the more rigidly boundary testing is seen, the more likely it is that things are going to be missed, because once methodologies, and things in general are in place it is often difficult to look outside them.

–Steve

Post Script
-I know there are a lot of things that I am trying to get across here, any questions on anything are of course welcome
-I could quickly find what websites I grabbed the boundary test/equivalence case definitions from, I’ll update that when I find them

Posted in Testing | Tagged: , , , , , , , | 6 Comments »

Testing is everywhere…

Posted by Steve on November 22, 2007

This will just be a quick one before I rush off to the gym, but today I was noticing how testing appears in one form or another in my daily life.

 An example today was when I was plugging in my ipod and hanging it over a towel rack. I was concerened that the cable would slip on the rack and the ipod would fall to the ground. So what did I do to make sure it wouldnt happen; I let it go a little bit with my hand other hand ready to catch it. This was to see how far it would fall when I let it go; or essentially a test (if anyone’s curious it didnt fall very far, and then I got on to thinking about static/kinetic friction but we won’t go there…)

 I just found it interesting how small things in life manifest themselves into situation where testing is required (that is if you want a good outcome = P ).

 –Steve

Posted in Testing | Leave a Comment »

Dealing with Bugs…

Posted by Steve on November 21, 2007

Filing Bugs / Bug Scrubs
—————————-

I have been giving thought to how bug scrubs go and it seems like there is some room for improvement. From what I have seen bug scrubs are done every day and they last about an hour, and have ~4-5 people in them. Say at an average of 40$ an hour, it wastes about 1000 dollars a week (which I guess isnt terrible) but there are bug scrubs for each product, so say 3-4 products we are up to 4000 dollars a week, which equates to about 200K a year, now we are talkin! Other then the money this wastes TIME, time that could be spent testing, developing, getting the product out the door etc… It just seems that there is a more efficient way to go through bugs. So I gave it some thought and came up with a possible algorithm.

The plan I came up with is as follows:
——————————

At the beginning of the test cycle introduce a few key things. I am going over what to introduce and what I think that would accomplish

The Testers and developers should know each other.
–> Face to face communication(encourage through a meeting with all developers and testers) can save a lot of misunderstanding and time.
–> When people know who they are working with they are more likely to work harder (because they know what they are working for).
–> Might result in more collaboration and idea sharing. Also a better team environment.

Testers should also know what area each developer works on.
–> This will allow a tester to assign a bug to a certain developer, and skip the bug scrub step for that bug
–> Less bugs in bug scrubs means less wasted time.

The Developers should know the testers
–> If a tester files a bug that they think is by design, they can face to face with the tester to find out if that is correct or not
–> If there is a disagreement, it can be placed in with other cases to be scrubbed.

How to assign Priority of Cases should be clear
–> If the priority of cases is clear, testers can assign priority on their own.
–> If the tester is not sure they can assign it to the developer and he can assign priority based on his thoughts
–> If the developer does not know, then it can go to the unscrubbed pile

Developers should be clear on what they desire in Bug reports
–>when are diags needed, when are stack traces needed, is a video always good?, should there always be screenshots?, what level of detail is needed?
–> This will allow for greater efficiency because no information will be missed (for example, I missed a stack trace that was needed once, simply because I didnt know I needed it)

I think that putting these processes in place could help do the following

–>Build a better sense of fellowship between developers and testers

–>More efficiency in bug resolution and bug scrubs

–>Less bugs to go through in bug scrubs

Any comments are always welcome

–Steve

Posted in Testing, Work: General | Tagged: , , , , , , | 2 Comments »