Configuring Playback in Visual studio, Coded UI

The various settings for Playback and how you can modify them to improve the resilience of your Coded UI Tests. Playback is configured by modifying the fields in Playback.PlaybackSettings class.

1. Continue on Error

When you are recording a Coded UI test on a login page, the “Auto Complete passwords” popup may have appeared.

When you playback actions on the login page, this dialog will *not* come up. So the action that you performed will cause an error. To improve test resilience, the Record & Playback engine automatically marks this action as “Continue on Error”.  When the test is being played back and the error occurs, the engine will continue on to the next action. Specific actions (IE popups, implicit Hovers) when performed by the user are tagged by the engine as “Continue on Error”.

In Coded UI Test, you have the flexibility to turn on “Continue on Error” for any action. This is done by the following code snippet.

Playback.PlaybackSettings.ContinueOnError = true;

Remember to turn this flag to false after the section where there are potential failures.  Otherwise the engine will ignore all errors and your test will return incorrect results. By default this flag is set to false in Coded UI Test.

2. Match Exact hierarchy

The Record and Playback Engine uses a hierarchical structure to locate controls.  The Yes button in the dialog above is identified as

To locate YesButton during playback, the engine will search for AutoCompletePasswordWindow,then for YesWindow in it, and finally YesButton in YesWindow.

Sometimes application may change so that the hierarchy gets modified. Record and Playback engine attempts to locate the control, in such cases, by skipping the intermediate elements in the hierarchy. Thus in the example, if it is unable to locate YesWindow, it will search for YesButton directly under AutoCompletePasswordWindow and all its children.  This behavior can be controlled by the “Match Exact Hierarchy” setting. This is done by the following code snippet.

Playback.PlaybackSettings.MatchExactHierarchy = true;

If the flag is set to true, every control is the hierarchy will be searched in order. Any failure will result in failure to locate the control.

By default, this flag is set to false. Note that this may sometimes lead to false positives i.e an unintended control may be identified. In such cases, you will have to change this setting.

3. Search in minimized windows

The Record and Playback Engine searches for controls even inside minimized windows. The minimized window is restored and actions performed on the control inside it.

You have the ability to turn this feature off i.e. prevent the engine from looking at minimized windows.  This is done by the following code snippet.

 Playback.PlaybackSettings.SearchInMinimizedWindows = false;

4. Smart Match

The Record and Playback Engine identifies controls on the user interface by its search properties.  e.g:- The standard calculator window is identified as below.

SearchProperties[WinProperties.Window.ControlType] = ControlType.Window.Name;

SearchProperties[WinProperties.Window.Name] = "Calculator"; 
SearchProperties[WinProperties.Window.ClassName] = "CalcFrame";

Some of the search properties may change over time. Record and Playback engine uses a Smart Match algorithm to identify controls if they cannot be located using the exact properties. The smart match algorithm uses heuristics and attempts to locate the control using variations on the search properties.

You can control when Smart Match should be applied. By default Smart Match is applied for Top Level Windows and all controls. You can turn off Smart match using the following code snippet.

Playback.PlaybackSettings.SmartMatchOptions = SmartMatchOptions.None;

The default settings (Smart match top level windows and controls) is optimal for a resilient coded UI test. But sometimes it may lead to false positives and then you will have to modify this setting.

5. Wait For Ready Level

The Record and Playback engine ensures that each UI control is ready to be acted upon before performing any action.  The smart “Wait For Ready”  algorithm significantly improves the resilience of your Coded UI Test.  Now you don’t need to add the annoying Sleep statements whenever a UI Control is busy and not ready to receive input. By default, the engine checks the UI Thread (foreground thread) to determine if a control is ready.

You can turn off Wait For Ready completely by the following code snippet.

Playback.PlaybackSettings.WaitForReadyLevel = WaitForReadyLevel.Disabled;

Alternately you may want to wait for all threads including background threads. This may be required if the application is making asynchronous calls that results in UI changes.  You can do this by the following code snippet.

Playback.PlaybackSettings.WaitForReadyLevel = WaitForReadyLevel.AllThreads;

NOTE that this setting will slow down the engine and should be carefully used.

6.  Set Property Verification

After setting the property of any UI control, the record and playback engine performs a verification pass to ensure that the set succeeded and the UI control now has the value assigned to it.

e.g:-  You type the text “MSFT” in the search edit box in Bing. After playing back this action, the engine reads the text property of the search edit box to ensure that this set has succeeded.

In certain applications, the set property may trigger off some business logic which then changes that property itself. In such cases, the verification fails. If your application exhibits such a behavior you can turn of the set property verification using the following code snippet.

Playback.PlaybackSettings.SkipSetPropertyVerification = true;

7.  Fail Fast

The Record and playback engine’s search algorithm has a fail fast logic. i.e. If the engine is reasonably confident that the search will not succeed, it will return with a failure. This ensures that you are not waiting for a non-existent control for 2 mins (the default search timeout).

Sometimes you may want to disable fail fast. Typically, you do this along with increasing your search timeout. You are confident that the UI control will become available within a specified duration  (which is longer than the search timeout) and you want the engine to wait for that duration.  You can do this by using the following code snippet.

Playback.PlaybackSettings.ShouldSearchFailFast = false;

8. Misc

Other playback settings which can be modified include

a.         Delay Between Actions: How long should the engine wait before performing two successive actions? By default, 100 ms.

b.         Search timeout – How long should the engine attempt to locate a control? By default, 2 mins.

c.         Wait for Ready Timeout – How long should the engine continue waiting for a control to be ready?  By Default, 60s

d.         Think Time multiplier – For slow machines, assign a high multiple so that the recorded think time is scaled up to handle slower application responses.

e.         Send Keys As ScanCode - By default, the engine sends keys to the application under test as Unicode. But in certain cases, it may need to send keys as scancode. Sending Unicode to such applications will result in failure

NOTE: The Test runner tool in VSTT 2010 also uses the Record and Playback Engine. You can configure the playback settings described above in Test runner tool from

a.         mtlm.exe.config – located in %VSINSTALLDIR%\Common7\IDE

b.         Microsoft Test & Lab Manager, navigate to Lab Center –> Test Settings , open the applicable Test Settings, select Data and Diagnostics section and click on the Configure button next to Action Log and Action Recording. In the Advanced section of the dialog, some of the Playback settings described above may be configured.

Adding a UI Control to a UIMap Without adding a UI action

Whether you are working with one or many UIMaps the best way to add a UI Control to a UI Control Map , without adding a UI Action, is to right click on the UIMap to which you want to add the control and select

“Edit With Coded UI Test Builder”.



This will open the Coded UI Test Builder




Be sure that the application under test is running.



Once the application under test is running, click and drag the bull’s-eye to the desired control. The selected control will be identified by a blue border.




Releasing the left mouse button over the desired control will bring up a new window containing information about the control.
Clicking on the “UI Control Map” tab to the left of this new window will give you more options.
Amongst the options presented to you, is a button that allows you to add the control to your
UI Control Map without adding a new UI Action to the UIMap.
Click on the first icon:
“Add control to UI Control Map”





Once the control is added the checkmark located next to the control will become black. To save this control to the UI Control Map click on the 4th icon located in the Coded UI Test Builder “Generate Code”
A window will popup with a note mentioning that no Method Name is required because no UI Action will be generated.
The last step, clicking “Generate” will generate the elements required to include the newly mapped control in the UIMap.
The UI Control will available from code once the project gets built.

SpecFlow with CodedUI - BDD framework

Here mainly we are not discussing in deep, instead just a brief about how exactly the BDD framework can be developed by using CodedUI and its integration with the SpecFlow

Final look of the Solution Explorer

This is how the Solution explorer looks for the sample test written for testing a Addition test scenario for the Calculator.

Pre Setup to be done:

·         Add the following references

·         TechTalk.Specflow.dll is to be downloaded from SpecFlow_v1.9.0_bin.zip and added in the reference.

·         Download and install SpecFlow 1.9 IDE for integrating it with the Visual studio 2012 premium.

http://visualstudiogallery.msdn.microsoft.com/9915524d-7fb0-43c3-bb3c-a8a14fbd40ee

                once it is installed, right click on the project and select “Add an new item” and add the SpecflowFeature1.feature and then a StepDefinition.cs file

·         Add the App.Config file with the following data

<?xml version="1.0" encoding="utf-8"?>

<configuration>

  <configSections>

    <section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow" />

  </configSections>

  <specFlow>

    <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config -->

  <unitTestProvider name="MsTest" generatorProvider="Specflow.CodedUI.MsTestCodedUiGeneratorProvider, Specflow.CodedUI" runtimeProvider="TechTalk.SpecFlow.UnitTestProvider.MsTest2010RuntimeProvider, TechTalk.SpecFlow" /></specFlow>

</configuration>

This makes the initial setup done for the project.

Coding Part

Now comes the coding part.

To make things understandable, or make things easy, we defined 3sets of UIMaps

GeneralUIMap – Consists of general functions likes launching Calculator, closing Calculator

StandardUIMap – Consists of all the functions related to the Standard Calculator

ScientificUIMap – Consists of all the functions related to the Scientific Calculator

These UIMaps file can be generated separately in a CodedUI Project and then added into the UIMap folder.

UIMapLoader folder consists of the the cs file where we define a class and the instances of all the three UIMap class.

using Microsoft.VisualStudio.TestTools.UITest.Common.UIMap;

using SpecflowAndCodedUI.UI.GeneralUIMapClasses;

using SpecflowAndCodedUI.UI.ScientificUIMapClasses;

using SpecflowAndCodedUI.UI.StandardUIMapClasses;

namespace SpecflowAndCodedUI.UI.UIMapLoader

{

    public static class Calculator

    {

        public static ScientificUIMap Scientific

        {

            get { return _scientific ?? (_scientific = new ScientificUIMap()); }

        }

        private static ScientificUIMap _scientific;

        public static StandardUIMap Standard

        {

            get { return _standard ?? (_standard = new StandardUIMap()); }

        }

        private static StandardUIMap _standard;

      

 public static GeneralUIMap General

        {

            get { return _general ?? (_general = new GeneralUIMap()); }

        }

        private static GeneralUIMap _general;

    }

}

The sample scenario which can be specified in the SpecflowFeature1.feature file

Feature: SpecFlowFeature1

                In order to avoid silly mistakes

                As a math idiot

                I want to be told the sum of two numbers

@mytag

Scenario: Add two numbers

                Given I have entered 50 into the calculator

                Given I press add

                And I have entered 70 into the calculator

                When I press equals

                Then the result should be 120 on the screen

// Below scenario is for the Data Driven Approach

@mytag

Scenario Outline: Add two numbers for different scenarios

                Given I have entered "<Input1>" into the calculator

                Given I press add

                And I have entered "<Input2>" into the calculator

                When I press equals

                Then the result should be <Result> on the screen

Examples:

| Input1 | Input2 | Result |

| 20     | 70     | 90     |

| 50     | 100    | 150    |

The StepDefinitions.cs file looks like this

namespace SpecflowAndCodedUI.StepDefinitions

{

    using SpecflowAndCodedUI.UI.UIMapLoader;

    using System;

    using TechTalk.SpecFlow;

    [Binding]

    public class StepDefinitions

    {

        [BeforeScenario()]

        public static void PrepareForTest()

        {

            Calculator.General.LaunchCalculator();

        }

        [AfterScenario]

        public void CleanUpTest()

        {

            Calculator.General.CloseCalculator();

        }

        [Given(@"I have entered (.*) into the calculator")]

        public void GivenIHaveEnteredIntoTheCalculator(string input)

        {

            Calculator.Standard.TypeAValue(input);

        }

        [Given(@"I press add")]

        public void GivenIPressAdd()

        {

            Calculator.Standard.ClickAddButton();

        }

        [When(@"I press equals")]

        public void WhenIPressEquals()

        {

            Calculator.Standard.ClickEqualsButton();

        }

        [Then(@"the result should be (.*) on the screen")]

        public void ThenTheResultShouldBe120OnTheScreen(string expectedResult)

        {

            //Is is the alternative to modifying the Coded UI Test Method to accept a parameter as we did for the TypeAValue method

            //Instead you can use the Coded UI Tests Params properties to pass the value.

            Calculator.Standard.AssertTheResultExpectedValues.InputLableValueDisplayText = expectedResult;

            Calculator.Standard.AssertTheResult();

        }

    }

}

The coding part is done

Now to execute the scenarios, right click on the SpecFlowFeature.feature and select “Run SpecFlow Scenarios”

Click to downloaded Sample project 

Happy coding.

Let me know if anyone needs any more details information or sample project to be shared

Store all rows and columns of Web table in List using Coded UI Test (For performance improvement)

There are many situations where we might need to verify all the columns and rows of one or more table in a page against Database table to check its data.

This can be done if we store all the UI table rows and columns in List<> and then compare it with any data source.

This is a very time consuming process for Coded UI Test if we write the code with the built-in function provided by Microsoft.

All we do while writing code is this.

§  First get the number of rows

§  Next get the number of columns

§  Iterate through the rows and columns

§  Using GetCell method, get the value of the cell

public static void UITableToArray(HtmlTable table)  {  //Declare List<> or Dictionary  try  {  int rowCount = table.RowCount;  int colCount = table.ColumnCount;  for (int rowIndex = 1; rowIndex &lt; rowCount; rowIndex ++)  {  for (int colIndex = 1; colIndex &lt; colCount; colIndex ++)  {  if (table.GetCell(row, col).InnerText != null)  {  //Store in list or dictionary  }  }  }  }  catch (Exception e)  {  }  }

But the problem is, the code is very slow.

If you would like to read a table in page with 100 rows and 25 columns, then it will take you more than 20 minutes to read all the data!!!

Believe me, it’s true fact.

The reason is this

The GetCell method, the method is slow because every time it does the following

• §  Identifies the page

• §  Identifies the table

• §  Identifies the Row and its column with index supplied, then reads the data

The process is fast if we read only one data, but what happen if we want to read hundreds of data, the above will surely very slow.

So, whats the solution?

As you could see above, the table is identified ONLY ONCE and so is the row collection, which means all the rows and its related values are found in just one single shot, which is awesome.

This will eradicate the above problem faced by GetCell() method which we were discussing about earlier.

As you could see, the code will now look like one shown below.

public static void UITableToArrayList(HtmlTable table) { //Declare List<> or Dictionary     try     {         UITestControlCollection rowcontrol = table.Rows;         string cellval = null;         int rowcount = 0;         //col Names         string[] colNames = table.GetColumnNames();         foreach (UITestControl item in rowcontrol)         {             int colCount = 0;             if (item is HtmlRow)             {                 rowcount++;                 foreach (HtmlControl cell in item.GetChildren())                 {                     cellval = cell.InnerText;                     if (colNames[colCount] != null)                     {                     //Store it in List or Dictionary                     }                     colCount++;                 }             }         }     }     catch (Exception e)     {     } }

Getting Detailed Test Results From Coded UI Tests (CUIT)

Coded UI Tests can fail for a number of reasons, producing detailed logs that can be used to rapidly identify what caused the failure. The interesting characteristic about the logs produced by the tests, is that each action results in a detailed entry consisting of information about the how to find the control, the technology used to find the control and a screenshot of the action including a red box above the control upon which the action is performed. These details can help identify if the CUIT has been able to find the right control. It can also help locate instances where the target control is missing.



To enable this level of detail in the test results, open explorer and go to the following folder

C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\

In this location you will find QTAgent32.exe.config. This is that first file that I usually modify to augment the verbosity of the logs. Before opening files in this directory, be sure that you have the permissions to edit and save files.

Locate the following section in the configuration file and set the “EqTraceLevel” to 4.

<system.diagnostics>
  <switches>
    <!-- You must use integral values for "value".
          Use 0 for off, 1 for error, 2 for warn, 3 for info, and 4 for verbose. -->
    <add name="EqtTraceLevel" value="4" />
  </switches>
</system.diagnostics>

Then scroll down to the appSettings section and a key “EnableHttpLogger” with a value of “true”. Then add another key “EnableSnapshotInfo” with a value of “true”. I read that “EnableSnapshotInfo” was not required for the RTM version of Visual Studio 2012, but I kept it around for my last test.

  <appSettings>

    <add key="EnableSnapshotInfo" value="true"/>
    <add key="EnableHttpLogger" value="true"/>

    <add key="StopTestRunCallTimeoutInSeconds" value="5"/>
    <add key="LogSizeLimitInMegs" value="20"/>
    <add key="CreateTraceListener" value="no"/>
    <add key="GetCollectorDataTimeout" value="300"/>
  </appSettings>

The following are the names of the other QTAgent configuration files found in the same location.

QTAgent.exe.config
QTAgent_35.exe.config
QTAgent32_40.exe.config
QTAgent_40.exe.config
QTAgent32_35.exe.config
QTAgent32_40.exe.config
QTDCAgent.exe.config
QTDCAgent32.exe.config

How to identify all the child elements within a page in Coded UI Test (CUIT)

There are many situations where we might need to get all the elements available from within a Page, Div, Table etc. The elements can be either list of text box, checkbox, hyperlinks etc. In QTP we can do this very easily by means of ChildCount method to return the list of child within a particular object, similarly in Visual Studio 2010 Coded UI Testing we can do the same via UITestControlcollection collection and FindMatchingControls method.

Consider a scenario where you are asked to list all the Hyperlinks within a Div element of a page. Then all you have to do is follow the three simple steps

1.       Identify the Div name (It can be for e.g. HeaderContent)

2.       Pass the Div name instance object to HTML control class (Now this will have all your controls within the particular Div, note this can be anything not just hyperlink alone which we are searching)

3.       Using FindMatchingControl method, search for the control you are interested in searching for, here its hyperlink (HtmlHyperLink class in VSTS) and pass it to UITestControlCollection.

4.       Now you have the handle for getting the controls, you can now iterate through the collection and get all the matching controls value out from it. That’s it !!!

Let’s see the above operation in code

public void SearchhyperLinks()         {             //Search for the Div which has the list of hyperlinks we are searching             HtmlDiv div = new HtmlDiv();             div.SearchProperties[HtmlDiv.PropertyNames.Name] = "HeaderContent";             //Pass the instance of Div to HtmlControl class             HtmlControl controls = new HtmlControl(div);             controls.SearchProperties.Add(HtmlControl.PropertyNames.ClassName, "HtmlHyperlink");             UITestControlCollection collection = controls.FindMatchingControls();             foreach (UITestControl links in collection)             {                 //cast the item to HtmlHyperlink type                 HtmlHyperlink mylink = (HtmlHyperlink)links;                 //get the innertext from the link, which inturn returns the link value itself                 Console.WriteLine(mylink.InnerText);             }       }

Source :- executeautomation blog