Still Life

A Series of Mental Snapshots

Archive for the ‘Testing’ Category

Taking action on JavaScript Popups with Ruby/Watir in Firefox and Internet Explorer

Posted by Steve on July 12, 2010

A long time ago I did a post on catching JavaScript popups, but have a much better way of catching them.

These JavaScript popups cause trouble as they interrupt the page from fully loading, causing Watir to wait (as the page is waiting), which means the next command in your script will never be reached. Previous work arounds to this were to use watirs built in click_no_wait, but I have that to be extremely temperamental and did not always work depending on which element the click was being performed on.

The new and improved method is to have a completely separate process that runs in the background and is continually checking for JavaScript pop ups. AutoIt commands are used to first locate the pop-up and then depending on what text or title is present in the pop up and different action can be performed on it. Unfortunately the same code cannot be used for both IE and FF due to the fact that the AutoIt controls cannot perform the same actions on IE pop-ups as it can on FF pop-ups.  I have included the code for both below:

clickPopupsIE.rb

require 'win32ole'
begin
    autoit WIN32OLE.new('AutoItX3.Control')
    loop do
       autoit.ControlClick("Windows Internet Explorer",'', 'OK')
       autoit.ControlClick("Security Information",'', '&Yes')
       autoit.ControlClick("Security Alert",'', '&Yes')
       autoit.ControlClick("Security Warning",'', 'Yes')
       autoit.ControlClick("Message from webpage",'', 'OK')
       sleep 1
    end
rescue Exception > e
    puts e
end

clickPopupsFF.rb

require 'win32ole'
websiteName = "w3schools.com"
begin
    autoit = WIN32OLE.new('AutoItX3.Control')
    loop do
       autoit.winActivate("The page at http://#{websiteName} says:")
       autoit.Send("{ENTER}") if(autoit.WinWait("The page at http://#{websiteName} says:",'',2) == 1)
    end
rescue Exception => e
    puts e
end

These two scripts can then be called from any of your other Watir scripts using the following two functions scripts:

require 'win32/process'
def callPopupKillerFF
    $pid = Process.create(:app_name => 'ruby clickPopupsFF.rb', :creation_flags => Process::DETACHED_PROCESS).process_id
end

def callPopupKillerIE
    $pid = Process.create(:app_name => 'ruby clickPopupsIE.rb', :creation_flags => Process::DETACHED_PROCESS).process_id
end

def killPopupKiller
    Process.kill(9,$pid)
end

As you can see above you do need to require one more ruby gem, ‘win32/process’, this is used to run the popup clicker as a separate process that runs in the background. Once you have those functions in place you can simply call:

callPopupKillerIE #Starts the IE popup killer
#Some watir code that results in a popup#
killPopupKiller #Kills the popup killer process, so that you do not end up with 5 of them running!

Well there you have it, a robust and effective popup killer for both IE and FF. If you have any questions let me know!

–Steve

Advertisements

Posted in IE Automation/Watir/Ruby, Testing | Tagged: , , , , , , , , , , , , , , | 12 Comments »

Review of Jawbone Prime vs. Plantronics Voyager Pro Bluetooth Headsets: A Real World Test

Posted by Steve on November 15, 2009

I recently was in need of a new bluetooth headset, the first one was far too loud on the receiving end up to the point that I was told to just go on mute on a conference call as others could not hear what I was saying at all. The first thing that I did, as often in testing, was to define what my desired end result was. For me what mattered most, above anything else, was the call quality on the receiving end. The other important factors were comfort, and a mute function (as my phone did not have one, and to mute with the conference call system, you had to press *1, something that is not easy while driving).

With the desired qualities in mind, I proceeded to do some research to try and figure out what device what meet my requirements. I found it was easy to narrow down the pool to a couple devices, the Jawbone Prime and the Plantronics Voyager Pro based mainly on editor and personal reviews. Both units were similarly priced, at the time the Jawbone prime was 120$ and the Plantronics Pro was 100$, so they were both upper end devices. This is where I ran into some difficulties; once I was down to the two devices, it was impossible to tell, from reading at least, which device would have the superior call quality. In words it is almost impossible to quantify call quality, often reviewers would say the quality was a four stars, or that it was better than its predecessor. I even found articles comparing the two devices, but I was surprised to discover none of them had any sound clips of what the receiving call sounded like. To me it seemed the only real way to get a feel for how good the call quality was.

This is when I decided move forward and do some testing of my own. I purchased both the Jawbone Prime and the Plantronics Voyager Pro headsets, and I then decided to leave myself a message on my phone, that way I could do a straight comparison of the receiving call quality. I actually ended up leaving messages twice for each device, as the first time I did not have a good idea of what I wanted to test, this was a good example of the importance of having various testing scenarios in mind. I wanted to hear the call quality in a variety of scenarios including, music playing, windows down, air conditioning pointed at face and different voice volume levels. (Unfortunately to hear the tests, you will need to actually download the .wav file, hopefully I can post them on another site that will stream them)

The Plantronics Voyager Pro:
http://www.mediafire.com/file/zuyznjijldm/Plantronics.WAV

The Jawbone Prime:
http://www.mediafire.com/?m5mgwtn3xny

After listening to both clips it was clear that the Plantronics Voyager pro had superior receiving call quality. The Jawbone Prime, was almost perfect when I was not talking, but when I was the background noise started to leak through. Overall comfort was about equivalent for both devices, and the Plantronics Pro had mute functionality where as the Jawbone Prime does not.

I would therefore state that the Plantronics Voyager Pro is the superior device based on my criteria, but both devices performed quite well (especially in comparison to some of the cheaper products). I have had the voyager pro for about 4 months now, and it has continued to perform well. To tie this back to testing, it was interesting because I got to test a product in a non traditional manner and in a realistic setting as a customer. It also reinforced the importance of knowing the needs of the customer when testing, because had my criteria been different a different headset could have come out on top.

–Steve

Posted in Testing | Tagged: , , , , , , , , , , , , , | 2 Comments »

Bugs In the Wild: WYSIWYG in WordPress Edition

Posted by Steve on April 22, 2009

I ran into this issue when creating my post for importing/exporting individual tables in MySQL. The issue resolves around the principal of WYSIWYG (What you see is what you get).

For one of the MySQL commands it was necessary to have two dashes (or minuses) next to each other, as can be seen in my the post mentioned above.  In the WordPress text editor displayed the two dashes properly in both the visual and HTML views, but after I had published the post, the two dashes where displayed (and when copied acted) as a single dash! For example the character in the quotation is seen as two dashes in the editor “–” but as can be clearly seen it is displayed as a single dash.

I tried for a while to resolve the issue, but I ended up just spacing out the dashes, like so “- -“, so that it could be seen that there were actually two dashes.

I feel this falls under a WYSIWYG type issue as in the text editor I was able to see the two dashes properly, but then when the post was published the same thing was not displayed therefore what I was seeing n the editor was not what I got in the post.

–Steve

Posted in Bugs in the wild, Testing, Uncategorized | Tagged: , , , , , , , | 4 Comments »

Anatomy of an Error Message – A Windows Vista Example

Posted by Steve on April 2, 2009

I have recently had a somewhat frequent recurrence of an error message on my laptop, which is running Windows Vista. The error message can be seen below:

Windows Vista Error Message

Windows Vista Error Message

The error message popped up (for seemingly no apparent reason), and to make matters worse, when I clicked cancel a new instance of the error message popped up, and I had to close it about 30-40 times before it stopped popping up!

You’ll notice that the first issue with this error message is that it is extremely uninformative and has indecipherable information, something the user should never see. This lack of information made so that I was unable to identify what was causing the error therefore I was not able to stop it from happening (as a quick note, I did not unplug any disks at the time, so it did not have to do with that). Subsequently, since the first incident, this has occurred a few more times with no real pattern as to what is causing it.

A bad error message is good illustrator to what a good error message should be. A good error message should have the following:

  • An informative title – A user should know what caused the issue
  • Actual error message should provide user with information on how to fix the problem
  • Finally it should provide the user with actions that will help fix the problem identify in the message

The only thing the above error message has going for it is that its title gives a clue as to the issue.

To quickly summarize, the goal of an error message is to inform the user that there has been an error, what caused the error, how to fix the error and finally some actions you can take to fix it.

Posted in Testing | Tagged: , , , , , , , , , , , , | Leave a Comment »

Book Review and Thoughts: “Don’t Make Me Think!” by Steve Krug

Posted by Steve on October 13, 2008

Don’t Make me Think is a book on web usability; it is mainly about how to improve the design of web sites and web applications so that they are more usable. This is one book that practices what it preeches in the respect that it is very readable and very nicely presented rendering it what I would classify a nice read. That being said, I do agree with some other reviews of the book, the content does tend to be a bit ‘fluffy’ (for lack of a better term). A lot of the information presented is more common sense than anything else. Now with that being said, there is no harm in hearing some good common sense, and it is always good to here from another source.

There were a few parts in particular that I thought were very important, the thing that I saw as being the most important was the section on testing (surprise, surprise). It happens to be one of the last chapters of the book, but I would say by far the most important.

It discusses the simplicity in how web usability testing can be done, for example, the importance of NOT figuring out your target user (at least in too much detail. Instead it is suggested to go out and offer a small stipend to someone (anyone!) to come in to do an hour or so of usability testing. I think this is excellent advice, it takes a lot of the overhead away from the testing and realistically being able to pin down and then find your target user is not the easiest of tasks.

Another point on usability testing that is extremely important is to do it EARLY. I know first hand how difficult it is to have people test things early, your projects, sometimes you think things just are not ready. But the reality is that if you have some usability testing done early, it can provide a lot value, as at the time the changes suggested will be much easier to fix than later on in the designs life. Therefore do smaller amounts of usability testing earlier, than large scale later!

Overall it was a good read and confirms some of those ‘common sense’ kind of feelings that you may have towards web design. If you have anything to do with any web page or application I would suggest giving it a read.

–Steve

Posted in Product Review, Testing | Tagged: , , , , , , , , , | 3 Comments »

The use of videos in bug reports

Posted by Steve on October 7, 2008

I recently came across a topic at the software testing club website that was discussing the use of videos in bug reports. After reading through the comments, it was clear there were a few main concerns:

1. file size
2. Serial viewing order – harder to skip and get to the point.
3. No search options.
4. Requires detailed attention in whole viewing process.

Firstly I want to address what the main ‘limitations’ are and then I will go into the benefits I see.
1. File Size.
If you get a decent tool, you don’t have to worry about this. You can export your videos in flash (if high res isn’t necessary) making them ~100KB. Now a days at a normal big box store you can get a 500GB hard drive for under 100$, so you are looking at about 5 million videos for 100$ (or 500 000 videos at 1 MB each). To me that isn’t really what I’d consider a limiting factor. Note, this does not really address any issues that may happen in a bug tracking database, as it really does depend on how the database is set up.

2. Serial Viewing order – harder to skip and get to the point.
The way that I use videos (described below) this is not an issue. A test video used properly should not be long enough for this to be a problem.

3.No search options
Again if you have a short video, and you use the video accompanying another bug report, you will not have any problems.

4. Requires detailed attention
Again if you are using videos correctly this should not be an issue.

So how should videos be used in reference to bug reports…?

Firstly in my opinion a bug report will never be replaced by a video (well unless it is mashup of video with text) because there are things that you can easily and quickly describe in text. Videos are only helpful for certain things, a few of them are as follows:

  • Remembering your course of actions when you hit a bug. I am sure it has happened to everyone once where they notice a bug but they have no idea how they got there.
  • Helping to illustrate time sensitive bugs; for example you find something where, in a web form, you press save back save quickly and it causes a problem.
  • When a developer may not believe you. Often I have filed bugs that developers do not believe happen (sometimes because it is intermittent behaviour), a video is proof, words are just that… words.
  • When someone else on the test team has to explain a bug you filed. Wording can be tricky and filing a bug report is somewhat of an art. At times it is difficult to understand what the person that filed the bug meant, but if you have a video you can often figure it out.

There are other uses for videos, those are just the ones that jumped to the fore front.

What has to be remembered is that, like anything else, it is all fluid! Sometimes videos are appropriate, sometimes not, it is always a judgement call; you have to think to yourself will a video add value. If you are unsure maybe ask the developers or your team lead what will be most helpful to them! I think video is extremely useful with certain cases, so try not to disregard it too quickly.

–Steve

Posted in Testing | Tagged: , , , , , | 4 Comments »

IE Automation with Ruby: Catching pop-up Windows

Posted by Steve on September 29, 2008

Firstly I need to define what I mean my pop-up Windows. The pop-up windows that cause trouble are not internet explorer based windows, they are actually ‘Windows’ windows, (sorry if thats confusing). The ones that I am referring to are the ones that come up when, for example, you click on a download link. These are inherently a pain because of the fact that they are not IE windows. Luckily there is a pretty simple work around that i have come up with.

Ruby has access to the WIN32OLE library , which is basically like an API for windows applications. What you can do is use this library  to catch these pop up windows. Below is the code that you’ll need to run in a Ruby script:

require ‘win32ole’                                #Loads the win32ole library
wsh = WIN32OLE.new(Wscript.Shell) #For more info click here
wsh.AppActivate(‘Connect’)                #Focuses on a given application based on its Title

At this point you can manipulate the window, for example, with a SendKeys command:

wsh.SendKeys(“%{F4}”) #This would close the program with Alt-F4

This clearly has it’s limitations because during this time you cannot be doing things on your computer, because the AppActivate would fail. I am still looking for a lower level at which I can address this problem.

Now everyone likes to see code at work, so I have written a quick script that goes to the Notepad++ downloads page, clicks the download link, and then closes the pop-up download window. As a quick side note, if you do not already use notepad++ I highly recommend it!

require ‘win32ole’
require ‘watir’

wsh = WIN32OLE.new(‘Wscript.Shell’)

ie= Watir::IE.new
ie.goto(“http://notepad-plus.sourceforge.net/uk/site.htm”)
ie.frame(:name, “index”).link(:text, “Download”).click #Good example of how to execute a link in a Frame
ie.frame(:name, “index”).link(:text, “Download Notepad++ executable files”).click

sleep 20 #need to wait for source forge to load it is slow
ie1 = Watir::IE.attach(:title, /Source/)
ie1.link(:id, “showfiles_download_file_pkg0_1rel0_2”).click

wsh.AppActivate(“File Download – Security Warning”) #Focuses on the pop up window
wsh.SendKeys(“%{F4}”) #Sends the alt-F4 command to the window to close it

Watir::IE.close_all #Closes all open IE windows

So if you are interested try the script out, I did try it out myself to make sure it worked, but let me know if you have any difficulties!

–Steve

Posted in IE Automation/Watir/Ruby, Testing | Tagged: , , , , , , , , , , , | 3 Comments »

Anatomy of a Bug Report — Presented via a MSOffice ‘Bug in the Wild’

Posted by Steve on September 13, 2008

The importance of a bug report is definitely understated; it is the difference between something getting fixed, and something getting filed away. I am going to present my version of a ‘good’ bug report through reporting a bug I found while I was MS Office 2007. First though I am going to outline the sections:

Title
The title is actually the most important part of the bug report, it is essential that it is possible to know what the whole problem is simply based on the title. If you cannot express what the bug is completely in the title, you have to think more about before reporting it.

Preconditions
Pretty straight forward, put what needs to be setup before the bug occurs, it can also include what machine configuration you used (if its meaningful)

Steps To Reproduce
Another straight forward area of the bug report. what you want to aim for here is that someone who knows nothing about the bug and possibly very little about the product can reproduce it. There preferrably should be no questions about how to hit the problem again (I do realize this is problematic with the bugs that appear to happen ‘randomly’)

Bug Details
The important things to include here are what the problem is (as you see it) and what you think should be happening. If you do not but a suggestion or desired outcome, the bug is not nearly as useful.

Frequency
This is a field that does not always apply, it generally is important with bugs that have to do with scalability and ones that seem to happen intermittently.

The Bug Report:

Title: Having the ‘Create Source’ window open results in any other open window becoming unusable

Preconditions:

  • Two instances of MS Office Open

Steps to Reproduce

  • In the first MS office window click the references tab
  • In the toolbar click ‘Insert Citation’
  • In the drop down list click ‘New Source’, the create source window will open
  • Click the mouse onto the second Office window

Bug Details

  • At this point nothing can be done in the second open window, you cannot click, or manipulate the window in any way.
  • Desired Outcome: The second MS Office window should not be affected by the Create Source window in any fashion
  • Note: A further issue is that a new Office window does cannot be opened (for example from the start window)

Frequency

  • Anytime the Create Source window is open

So there you have the bug report, if I had access to the proper software I would also have made and attached a video of the problem, video is a great bug reporting tool, one that I suggest is BB TestAssistant .

Let me know what you thought of my bug report! Does it match how you file bugs?

–Steve

Posted in Bugs in the wild, Testing | Tagged: , , , , , , , | Leave a Comment »

Web Usabilities Achilles Heel – The Back Button

Posted by Steve on September 12, 2008

Since the introduction of a lot of new web technologies including, (but of course not limited to) PHP and Javascript etc… the back button has never been the same. It used to be that when one pressed the back button it was expected, neh, certain that you would see the previous page that you were on; unfortunately this is no longer true! There are hoardes of issues that revovle around the back button, to follow is one example that I find terribly annoying.

Bestbuy.ca and Futureshop.ca Search
Like all web stores, best buy has a search option (and like most stores its pretty bad!). You arrive at the sight, and say you are looking to pick up an xbox 360, go ahead an enter that to the search bar. You come up with 368 hits, and you see a tree view:

Bestbuy Tree View

Bestbuy Tree View

You can then navigate down to the tree to finally find hardware:

Narrowed down selection

Narrowed down selection

Lets say you select the Pro Console, it takes you to the items page, and you see that it is sold out! So you press the back button to look at another console. This should take you back to the narrowed down selection… right? Not so much. It takes you back to the original search view, so you have to navigate through the tree again… GAH!

Have you had any interesting adventures with your back button, anything ever happen that you didnt expect? I may be adding to this myself the next time something strange happens!

–Steve

Posted in Testing | Tagged: , , , , , , , | 2 Comments »

Bugs in the Wild: In Flight Edition

Posted by Steve on August 29, 2008

I was recently on an air canada flight down to San Francisco to visit my parents and I had a touch screen in flight entertainment system (here is a link to a picture if you want to see).

The first thing that I start with is simply watching a TV show, as I wanted to relax a little, but as the flight progressed I decided to see what the entertainment system had to offer. I notice there is an option to listen to music, so I navigate over to there and start setting up a playlist. This is where I noticed the problem. The whole playlist (if it is over 5 songs long) does not fit on the screen, therefore there are arrows that you can click on to go up and down on the playlist; and if you get to the top or bottom of the playlist the respective up or down arrow gets disabled. Makes sense, why would you need to go up anymore if you are at the top of the playlist, right? Because of the title of the post, obviously this is wrong! What they forgot to take into account is that when you click on song that is at the bottom of those displayed on the screen, the screen re-centres around that song, thus if you click the fourth song on the list, you can no longer see the first song, and at this point in time the go up arrow is also still disabled!

There is of course a fix to this, you can simply click the down arrow and the up arrow will fix itself, but if you were really malicious you could have so that both arrows got diabled and you would have to exit and re-enter the audio screen.

I thought this was a fun little bug in a real world situation.

–Steve

Posted in Bugs in the wild, Testing | Tagged: , , , , , , | 2 Comments »