Still Life

A Series of Mental Snapshots

Posts Tagged ‘Testing’

Taking action on JavaScript Popups with Ruby/Watir in Firefox and Internet Explorer

Posted by Steve on July 12, 2010

A long time ago I did a post on catching JavaScript popups, but have a much better way of catching them.

These JavaScript popups cause trouble as they interrupt the page from fully loading, causing Watir to wait (as the page is waiting), which means the next command in your script will never be reached. Previous work arounds to this were to use watirs built in click_no_wait, but I have that to be extremely temperamental and did not always work depending on which element the click was being performed on.

The new and improved method is to have a completely separate process that runs in the background and is continually checking for JavaScript pop ups. AutoIt commands are used to first locate the pop-up and then depending on what text or title is present in the pop up and different action can be performed on it. Unfortunately the same code cannot be used for both IE and FF due to the fact that the AutoIt controls cannot perform the same actions on IE pop-ups as it can on FF pop-ups.  I have included the code for both below:

clickPopupsIE.rb

require 'win32ole'
begin
    autoit WIN32OLE.new('AutoItX3.Control')
    loop do
       autoit.ControlClick("Windows Internet Explorer",'', 'OK')
       autoit.ControlClick("Security Information",'', '&Yes')
       autoit.ControlClick("Security Alert",'', '&Yes')
       autoit.ControlClick("Security Warning",'', 'Yes')
       autoit.ControlClick("Message from webpage",'', 'OK')
       sleep 1
    end
rescue Exception > e
    puts e
end

clickPopupsFF.rb

require 'win32ole'
websiteName = "w3schools.com"
begin
    autoit = WIN32OLE.new('AutoItX3.Control')
    loop do
       autoit.winActivate("The page at http://#{websiteName} says:")
       autoit.Send("{ENTER}") if(autoit.WinWait("The page at http://#{websiteName} says:",'',2) == 1)
    end
rescue Exception => e
    puts e
end

These two scripts can then be called from any of your other Watir scripts using the following two functions scripts:

require 'win32/process'
def callPopupKillerFF
    $pid = Process.create(:app_name => 'ruby clickPopupsFF.rb', :creation_flags => Process::DETACHED_PROCESS).process_id
end

def callPopupKillerIE
    $pid = Process.create(:app_name => 'ruby clickPopupsIE.rb', :creation_flags => Process::DETACHED_PROCESS).process_id
end

def killPopupKiller
    Process.kill(9,$pid)
end

As you can see above you do need to require one more ruby gem, ‘win32/process’, this is used to run the popup clicker as a separate process that runs in the background. Once you have those functions in place you can simply call:

callPopupKillerIE #Starts the IE popup killer
#Some watir code that results in a popup#
killPopupKiller #Kills the popup killer process, so that you do not end up with 5 of them running!

Well there you have it, a robust and effective popup killer for both IE and FF. If you have any questions let me know!

–Steve

Posted in IE Automation/Watir/Ruby, Testing | Tagged: , , , , , , , , , , , , , , | 12 Comments »

Bugs In the Wild: WYSIWYG in WordPress Edition

Posted by Steve on April 22, 2009

I ran into this issue when creating my post for importing/exporting individual tables in MySQL. The issue resolves around the principal of WYSIWYG (What you see is what you get).

For one of the MySQL commands it was necessary to have two dashes (or minuses) next to each other, as can be seen in my the post mentioned above.  In the WordPress text editor displayed the two dashes properly in both the visual and HTML views, but after I had published the post, the two dashes where displayed (and when copied acted) as a single dash! For example the character in the quotation is seen as two dashes in the editor “–” but as can be clearly seen it is displayed as a single dash.

I tried for a while to resolve the issue, but I ended up just spacing out the dashes, like so “- -“, so that it could be seen that there were actually two dashes.

I feel this falls under a WYSIWYG type issue as in the text editor I was able to see the two dashes properly, but then when the post was published the same thing was not displayed therefore what I was seeing n the editor was not what I got in the post.

–Steve

Posted in Bugs in the wild, Testing, Uncategorized | Tagged: , , , , , , , | 4 Comments »

Anatomy of an Error Message – A Windows Vista Example

Posted by Steve on April 2, 2009

I have recently had a somewhat frequent recurrence of an error message on my laptop, which is running Windows Vista. The error message can be seen below:

Windows Vista Error Message

Windows Vista Error Message

The error message popped up (for seemingly no apparent reason), and to make matters worse, when I clicked cancel a new instance of the error message popped up, and I had to close it about 30-40 times before it stopped popping up!

You’ll notice that the first issue with this error message is that it is extremely uninformative and has indecipherable information, something the user should never see. This lack of information made so that I was unable to identify what was causing the error therefore I was not able to stop it from happening (as a quick note, I did not unplug any disks at the time, so it did not have to do with that). Subsequently, since the first incident, this has occurred a few more times with no real pattern as to what is causing it.

A bad error message is good illustrator to what a good error message should be. A good error message should have the following:

  • An informative title – A user should know what caused the issue
  • Actual error message should provide user with information on how to fix the problem
  • Finally it should provide the user with actions that will help fix the problem identify in the message

The only thing the above error message has going for it is that its title gives a clue as to the issue.

To quickly summarize, the goal of an error message is to inform the user that there has been an error, what caused the error, how to fix the error and finally some actions you can take to fix it.

Posted in Testing | Tagged: , , , , , , , , , , , , | Leave a Comment »

Book Review and Thoughts: “Don’t Make Me Think!” by Steve Krug

Posted by Steve on October 13, 2008

Don’t Make me Think is a book on web usability; it is mainly about how to improve the design of web sites and web applications so that they are more usable. This is one book that practices what it preeches in the respect that it is very readable and very nicely presented rendering it what I would classify a nice read. That being said, I do agree with some other reviews of the book, the content does tend to be a bit ‘fluffy’ (for lack of a better term). A lot of the information presented is more common sense than anything else. Now with that being said, there is no harm in hearing some good common sense, and it is always good to here from another source.

There were a few parts in particular that I thought were very important, the thing that I saw as being the most important was the section on testing (surprise, surprise). It happens to be one of the last chapters of the book, but I would say by far the most important.

It discusses the simplicity in how web usability testing can be done, for example, the importance of NOT figuring out your target user (at least in too much detail. Instead it is suggested to go out and offer a small stipend to someone (anyone!) to come in to do an hour or so of usability testing. I think this is excellent advice, it takes a lot of the overhead away from the testing and realistically being able to pin down and then find your target user is not the easiest of tasks.

Another point on usability testing that is extremely important is to do it EARLY. I know first hand how difficult it is to have people test things early, your projects, sometimes you think things just are not ready. But the reality is that if you have some usability testing done early, it can provide a lot value, as at the time the changes suggested will be much easier to fix than later on in the designs life. Therefore do smaller amounts of usability testing earlier, than large scale later!

Overall it was a good read and confirms some of those ‘common sense’ kind of feelings that you may have towards web design. If you have anything to do with any web page or application I would suggest giving it a read.

–Steve

Posted in Product Review, Testing | Tagged: , , , , , , , , , | 3 Comments »

The use of videos in bug reports

Posted by Steve on October 7, 2008

I recently came across a topic at the software testing club website that was discussing the use of videos in bug reports. After reading through the comments, it was clear there were a few main concerns:

1. file size
2. Serial viewing order – harder to skip and get to the point.
3. No search options.
4. Requires detailed attention in whole viewing process.

Firstly I want to address what the main ‘limitations’ are and then I will go into the benefits I see.
1. File Size.
If you get a decent tool, you don’t have to worry about this. You can export your videos in flash (if high res isn’t necessary) making them ~100KB. Now a days at a normal big box store you can get a 500GB hard drive for under 100$, so you are looking at about 5 million videos for 100$ (or 500 000 videos at 1 MB each). To me that isn’t really what I’d consider a limiting factor. Note, this does not really address any issues that may happen in a bug tracking database, as it really does depend on how the database is set up.

2. Serial Viewing order – harder to skip and get to the point.
The way that I use videos (described below) this is not an issue. A test video used properly should not be long enough for this to be a problem.

3.No search options
Again if you have a short video, and you use the video accompanying another bug report, you will not have any problems.

4. Requires detailed attention
Again if you are using videos correctly this should not be an issue.

So how should videos be used in reference to bug reports…?

Firstly in my opinion a bug report will never be replaced by a video (well unless it is mashup of video with text) because there are things that you can easily and quickly describe in text. Videos are only helpful for certain things, a few of them are as follows:

  • Remembering your course of actions when you hit a bug. I am sure it has happened to everyone once where they notice a bug but they have no idea how they got there.
  • Helping to illustrate time sensitive bugs; for example you find something where, in a web form, you press save back save quickly and it causes a problem.
  • When a developer may not believe you. Often I have filed bugs that developers do not believe happen (sometimes because it is intermittent behaviour), a video is proof, words are just that… words.
  • When someone else on the test team has to explain a bug you filed. Wording can be tricky and filing a bug report is somewhat of an art. At times it is difficult to understand what the person that filed the bug meant, but if you have a video you can often figure it out.

There are other uses for videos, those are just the ones that jumped to the fore front.

What has to be remembered is that, like anything else, it is all fluid! Sometimes videos are appropriate, sometimes not, it is always a judgement call; you have to think to yourself will a video add value. If you are unsure maybe ask the developers or your team lead what will be most helpful to them! I think video is extremely useful with certain cases, so try not to disregard it too quickly.

–Steve

Posted in Testing | Tagged: , , , , , | 4 Comments »

IE Automation with Ruby: Catching pop-up Windows

Posted by Steve on September 29, 2008

Firstly I need to define what I mean my pop-up Windows. The pop-up windows that cause trouble are not internet explorer based windows, they are actually ‘Windows’ windows, (sorry if thats confusing). The ones that I am referring to are the ones that come up when, for example, you click on a download link. These are inherently a pain because of the fact that they are not IE windows. Luckily there is a pretty simple work around that i have come up with.

Ruby has access to the WIN32OLE library , which is basically like an API for windows applications. What you can do is use this library  to catch these pop up windows. Below is the code that you’ll need to run in a Ruby script:

require ‘win32ole’                                #Loads the win32ole library
wsh = WIN32OLE.new(Wscript.Shell) #For more info click here
wsh.AppActivate(‘Connect’)                #Focuses on a given application based on its Title

At this point you can manipulate the window, for example, with a SendKeys command:

wsh.SendKeys(“%{F4}”) #This would close the program with Alt-F4

This clearly has it’s limitations because during this time you cannot be doing things on your computer, because the AppActivate would fail. I am still looking for a lower level at which I can address this problem.

Now everyone likes to see code at work, so I have written a quick script that goes to the Notepad++ downloads page, clicks the download link, and then closes the pop-up download window. As a quick side note, if you do not already use notepad++ I highly recommend it!

require ‘win32ole’
require ‘watir’

wsh = WIN32OLE.new(‘Wscript.Shell’)

ie= Watir::IE.new
ie.goto(“http://notepad-plus.sourceforge.net/uk/site.htm”)
ie.frame(:name, “index”).link(:text, “Download”).click #Good example of how to execute a link in a Frame
ie.frame(:name, “index”).link(:text, “Download Notepad++ executable files”).click

sleep 20 #need to wait for source forge to load it is slow
ie1 = Watir::IE.attach(:title, /Source/)
ie1.link(:id, “showfiles_download_file_pkg0_1rel0_2”).click

wsh.AppActivate(“File Download – Security Warning”) #Focuses on the pop up window
wsh.SendKeys(“%{F4}”) #Sends the alt-F4 command to the window to close it

Watir::IE.close_all #Closes all open IE windows

So if you are interested try the script out, I did try it out myself to make sure it worked, but let me know if you have any difficulties!

–Steve

Posted in IE Automation/Watir/Ruby, Testing | Tagged: , , , , , , , , , , , | 3 Comments »

Anatomy of a Bug Report — Presented via a MSOffice ‘Bug in the Wild’

Posted by Steve on September 13, 2008

The importance of a bug report is definitely understated; it is the difference between something getting fixed, and something getting filed away. I am going to present my version of a ‘good’ bug report through reporting a bug I found while I was MS Office 2007. First though I am going to outline the sections:

Title
The title is actually the most important part of the bug report, it is essential that it is possible to know what the whole problem is simply based on the title. If you cannot express what the bug is completely in the title, you have to think more about before reporting it.

Preconditions
Pretty straight forward, put what needs to be setup before the bug occurs, it can also include what machine configuration you used (if its meaningful)

Steps To Reproduce
Another straight forward area of the bug report. what you want to aim for here is that someone who knows nothing about the bug and possibly very little about the product can reproduce it. There preferrably should be no questions about how to hit the problem again (I do realize this is problematic with the bugs that appear to happen ‘randomly’)

Bug Details
The important things to include here are what the problem is (as you see it) and what you think should be happening. If you do not but a suggestion or desired outcome, the bug is not nearly as useful.

Frequency
This is a field that does not always apply, it generally is important with bugs that have to do with scalability and ones that seem to happen intermittently.

The Bug Report:

Title: Having the ‘Create Source’ window open results in any other open window becoming unusable

Preconditions:

  • Two instances of MS Office Open

Steps to Reproduce

  • In the first MS office window click the references tab
  • In the toolbar click ‘Insert Citation’
  • In the drop down list click ‘New Source’, the create source window will open
  • Click the mouse onto the second Office window

Bug Details

  • At this point nothing can be done in the second open window, you cannot click, or manipulate the window in any way.
  • Desired Outcome: The second MS Office window should not be affected by the Create Source window in any fashion
  • Note: A further issue is that a new Office window does cannot be opened (for example from the start window)

Frequency

  • Anytime the Create Source window is open

So there you have the bug report, if I had access to the proper software I would also have made and attached a video of the problem, video is a great bug reporting tool, one that I suggest is BB TestAssistant .

Let me know what you thought of my bug report! Does it match how you file bugs?

–Steve

Posted in Bugs in the wild, Testing | Tagged: , , , , , , , | Leave a Comment »

Creating specific files in windows with fsutil

Posted by Steve on July 10, 2008

This post comes from a recent issue I had recently, that was actually solved very easily with a built in windows tool. While running a test scenario, I needed to fill up a 20 GB disk. I have had to do with before, and it is pretty simple to do, you just grab some large files a copy them over and over again. I will never be doing that again thanks to a fancy little tool, fsutil.

fsutil is a built in windows program that has a whole bevy of options. You can open up a command line window and type fsutil to get the following options:

—- Commands Supported —-
behavior Control file system behavior
dirty Manage volume dirty bit
file File specific commands
fsinfo File system information
hardlink Hardlink management
objectid Object ID management
quota Quota management
reparsepoint Reparse point management
sparse Sparse file control
usn USN management
volume Volume management

From here you can get more details about any of the above features by entering fsutil “Feature Name”. So to create the large file we want to see the file options but entering fsutil file. Typing this in the command window gives the following output:

—- FILE Commands Supported —-
findbysid Find a file by security identifier
queryallocranges Query the allocated ranges for a file
setshortname Set the short name for a file
setvaliddata Set the valid data length for a file
setzerodata Set the zero data for a file
createnew Creates a new file of a specified size

So to create the large file, you need to use the createnew. So you can type in fsutil file createnew to see the usage as well as an example!

Usage : fsutil file createnew <filename> <length>
Eg : fsutil file createnew C:\testfile.txt 1000

It is important to note that the length, actually is the length in bytes, so a value of 1000, will be 1KB. There we are, any file of any size can be created. I foresee using this many times in the future, I also see exploring the fsutil feature much more, as it seems like a useful tool.

–Steve

Was this post useful? Could I improve on the layout of how I present these quick little tutorials? If so leave me a line, and let me know, I would love some feedback!

Posted in Testing, Tools | Tagged: , , , , , , , , , | 4 Comments »

Bugs in the Wild, the Irony Edition

Posted by Steve on June 18, 2008

It has been a while since I wrote a post specifically about testing, and I ran into an opportunity that I couldn’t resist. I am attending a software test conference, CAST, that is in Toronto, in about a month. I am looking forward to this because there are some good speakers lined up, and I am attending a tutorial that is going to be done by Jerry Weinburg, who if you do not know, has written a whole host of books on various topics that relate in many different ways to testing.

Now where the testing part comes in is when I got an e-mail this morning stating that I have officially been signed up for CAST, and can go see what I am signed up for. I decide to click on the link to give the website a little look, and I was very glad I did, because it gave me, as you can seen in the screenshot below, a humourous dose of irony.

Yes, that is correct, there is a very large, noticeable and common mistake in the text, an apostrophe is accidentally displayed as: �

I did notify them of this error before putting up my post, just to give them a little heads up, but I did get a good laugh that there was a bug on a website for a software testing conference!

–Steve

Post Script:
If you have interest in the conference, you can check it out here: http://www.associationforsoftwaretesting.org/drupal/conference

Posted in Bugs in the wild, Testing | Tagged: , , , , , , , | 2 Comments »

Basic Scripting of VM operations on ESX Server Using vmware-cmd

Posted by Steve on June 12, 2008

I’m writing a little more about ESX scripting because I had a lot of trouble tracking down some of this information, and if it had been all in one spot I would have saved a lot of time and trouble! I will be going over a few basic commands, as well as a few errors that I found. All of this was done on a ESX 3.5 machine.

This all started recently when I needed to reproduce a bug to show that it actually was an ESX specific issue. I learned quite a bit about ESX scripting from this experience, and hopefully some of the knowledge I am about to impart upon the vast interwebs helps someone out!

Now in starting the main command, that can be entered from anywhere in the command, that you will be using is vmware-cmd. Entering ‘vmware-cmd‘ will provide the following output:

Usage: /usr/bin/vmware-cmd <options> <vm-cfg-path> <vm-action> <arguments>
/usr/bin/vmware-cmd -s <options> <server-action> <arguments>

Options:
Connection Options:
-H <host> specifies an alternative host (if set, -U and -P must also be set)
-O <port> specifies an alternative port
-U <username> specifies a user
-P <password> specifies a password
General Options:
-h More detailed help.
-q Quiet. Minimal output
-v Verbose.

Server Operations:
/usr/bin/vmware-cmd -l
/usr/bin/vmware-cmd -s register <config_file_path>
/usr/bin/vmware-cmd -s unregister <config_file_path>
/usr/bin/vmware-cmd -s getresource <variable>
/usr/bin/vmware-cmd -s setresource <variable> <value>

VM Operations:
/usr/bin/vmware-cmd <cfg> getconnectedusers
/usr/bin/vmware-cmd <cfg> getstate
/usr/bin/vmware-cmd <cfg> start <powerop_mode>
/usr/bin/vmware-cmd <cfg> stop <powerop_mode>
/usr/bin/vmware-cmd <cfg> reset <powerop_mode>
/usr/bin/vmware-cmd <cfg> suspend <powerop_mode>
/usr/bin/vmware-cmd <cfg> setconfig <variable> <value>
/usr/bin/vmware-cmd <cfg> getconfig <variable>
/usr/bin/vmware-cmd <cfg> setguestinfo <variable> <value>
/usr/bin/vmware-cmd <cfg> getguestinfo <variable>
/usr/bin/vmware-cmd <cfg> getproductinfo <prodinfo>
/usr/bin/vmware-cmd <cfg> connectdevice <device_name>
/usr/bin/vmware-cmd <cfg> disconnectdevice <device_name>
/usr/bin/vmware-cmd <cfg> getconfigfile
/usr/bin/vmware-cmd <cfg> getheartbeat
/usr/bin/vmware-cmd <cfg> gettoolslastactive
/usr/bin/vmware-cmd <cfg> getresource <variable>
/usr/bin/vmware-cmd <cfg> setresource <variable> <value>
/usr/bin/vmware-cmd <cfg> hassnapshot
/usr/bin/vmware-cmd <cfg> createsnapshot <name> <description> <quiesce> <memory>
/usr/bin/vmware-cmd <cfg> revertsnapshot
/usr/bin/vmware-cmd <cfg> removesnapshots
/usr/bin/vmware-cmd <cfg> answer

These are all the different commands you can run with vmware-cmd. This in itself is only somewhat useful because for the more detailed commands, such as getconfig and setconfig, it does not tell you what variables to enter. I will go over a few of the ones that I have played with next.

The first useful command is:
vmware-cmd -l

This lists out the location of all of the VMs’ .vmx files on the ESX server. This is very useful as you will need this when you run any of the vmware-cmd commands. As stated above the syntax for the command is:
vmware-cmd <options> <vm-cfg-path> <vm-action> <arguments>

Now onto a few of the commands, there are the ones that are pretty self explanatory,
Remove All VM Snapshots: vmware-cmd <cfg> removesnapshots
Check if VM has any Snapshots: vmware-cmd <cfg> hassnapshot
Create A Snapshot[note everything but the name is optional]: vmware-cmd <cfg> createsnapshot <nameofsnapshot> <descriptionofsnapshot> <quiesce> <memory>
Revert to the previous snapshot: vmware-cmd <cfg> revertsnapshot
Start a VM: vmware-cmd <cfg> start <powerop_mode>
Get the state of the VM: vmware-cmd <cfg> getstate

Those are some of the simple useful ones, now here are a few that are a little finicky.
Stopping a VM: vmware-cmd <cfg> stop <powerop_mode>
Resetting a VM: vmware-cmd <cfg> reset <powerop_mode>

Here the <powerop_mode> is the tricky option. There are three options:
Soft: VMware tools need to be installed and running, therefore there are many cases when this will not work!
Hard: Performs the operation no matter what
Trysoft: Trys a soft and if that fails proceeds to a Hard. This is the best one in my opinion

get/setconfig:
vmware-cmd <cfg> setconfig <variable> <value>
vmware-cmd <cfg> getconfig <variable>

Setting or getting a configuration is a bit confusing at first because it asks for a variable, where is this variable coming from!?!? Luckily its actually pretty simple, you can set/getconfig on any of the settings in the .vmx file. There unfortunately is an issue with ESX 3.x ; some of the parameters that you set, only get set if you reset the hostD service by entering: service mgmt-vmware restart . This really is only a workaround as running this command will reset all of the connections that the ESX server has with the VMs, and it does take some time for it to come back up.

There you have it, this should be enough info to get you started on some ESX scripting. If you have any questions on how some of these commands work feel free to drop me a comment, and I’ll do my best to help you out. Below are also a few resources that I found helpful, but keep in mind if you are running ESX 3.x some things may work differently!

Useful References
Guide for driving the ESX server via Command line – http://www.rtfm-ed.eu/docs/vmwdocs/esx3.x-vc2.x-serviceconsole-guide.pdf
VMware Scripting API – http://wiki.eclab.byu.edu/plugin/attachments/ScriptingAPI/Scripting_API_215.pdf

Posted in ESX Scripting | Tagged: , , , , , , , , , | 10 Comments »