I found something that made me go "What The Frak" last week that I would love to share.
Running the tests locally I was getting the ever so annoying sometimes-it-fail-sometimes-it-just-works behavior on a few test cases. The same scenario every time. Test ran as expected up until a point where it's suppose to click a button, after the click is made the new page is not loaded.
I was looking in the logs and it look like everything was ok. It would locate the correct object to click. Calculate the correct x and y co-ordinates. Move the mouse to the correct position. Fire a click. Looking at the screen shoot that is taken it looks like TestComplete was clicking exactly where it was suppose to click. But still the new page wouldn't load.
I enter the system manually to click the button in different scenarios only half expecting it randomly not to work. But the button worked perfectly every time.
I then realized that it only happened on a few select buttons all located in the upper right corner. So I ran through the tests that sometimes would fail watching the upper right corner like a hawk. And that's when I noticed that the TestComplete Indicator has a pop up text box displaying log information, the pop up changes size depending how much text is in it. We had made some changes to logging and sometime it would get pretty large. Maybe 3 times the width of the Indicator itself.
When I watched it carefully I saw that sometimes the popup text would get big enough to cover some of the buttons in the system that we use in the text.
Not believing it could actually be true I did some tests. After starting a test I immediately paused it.
Then placed a windows with a button half way under the pop up. When I tried to click through the pop up nothing happened. Trying to clicking outside the pop up, of course, worked perfectly.
So what happened to the test where that the Indicator pop up blocked the button and my tests clicked the pop up instead! Which, naturally, didn't load the expected page.
 |
| *Sigh* |
Image 1: Hovering over the blocked part of the settings button gives no indication that I have the mouse over it. Clicking does nothing.
Image 2: Hovering over the unblocked part of settings button gives an indication that has a mouse over it. This is happening even if the window is not active. Clicking it opens the menu.
I still have a hard time believing how SmartBear could do this. It feels like such an obvious mistake.
Solution
The solution was to hide the indicator before we clicked and showing it after clicking has been done.
// Do the clicking
When we run the tests in our continuous integration environment we hide it permanently. Just to make sure it does not mess with any of our tests in some way.
Now all the tests run smoothly and all clicks go through as expected!
 |
| And there was much rejoicing. |
