Optics Framework

Overview

Application and Setup Control

Initialise Setup

This keyword reads the robot_record_port from environmental variables and establishes a TCP connection to the record_to_file server set by the controller. This keyword needs to be called at the start of the tests for setup where recording is performed by the controller. A TCP connection is required for the test to fetch screenshots from the recording pipeline.

Webcam Setup

This keyword performs the external webcam recording from the script side and also creates a TCP connection for screenshot capture. There’s a time buffer of 10 seconds added for the connection to be established and stable. This recording is done by reading config data such as deskew, rotation, and out width/height from CSV. However, automatic deskew will be implemented and this will be a standalone keyword with only the camera index as input.

Start Appium Session

This keyword creates an Appium session and launches the app that is mapped to the device in device_app_mapping.csv.

Start Other App

Arguments: App package

This keyword launches the app based on the input arguments. This keyword allows the user to launch applications not mapped in the device_app_mapping.csv.

Launch App

This keyword launches the application mapped in the device_app_mapping.csv without

creating an Appium session. This keyword can be used for both Android and iOS platforms since the framework uses the respective working commands based on the PLATFORM variable set. This keyword

can be used to launch applications where direct appium session creation is not available ( for example, launching app after launching settings app )

Send All Events

This keyword sends the payload ( combined events captured by keyword {Capture Event} during the test ) to the event API. This keyword needs to be called at the end of the test. Please verify that this keyword is added to Test Teardown steps for projects that require events and KPIs.

User Interaction

Press Element

The input to Press Element could be full text, partial text, XPath, image path, etc. The framework uses matching strategies to find the element and then press on it.

Arguments

  • Element: Element is described by text, image path, XPath, etc.

  • Offset x: offset in x axis , unit in percentage of device’s screen pixel

  • Offset y: Offset in y axis, unit in percentage of device’s screen pixel

User Guide:

Use Validate Screen keyword before Press Element for scenarios where waiting time is required for the element to appear.

Offset clicking is available for images only for now. Output:

Annotated Screenshot of the element on the screen.

Long Press Element

The input to Long Press Element could be full text, partial text, XPath, image path, etc. The framework uses matching strategies to find the element and then press on it.

Arguments

  • Element: Element is described by text, image path, XPath, etc.

  • Class: UI Class name EditText, TextView, LinearLayout, etc.

  • Time: amount of time to hold press - default 3 sec

User Guide:

Use Validate Screen keyword before Press Element for scenarios where waiting time is required for the element to appear.

Output

  • Annotated Image when the action is taken

Press Percentage

Presses based on the input percentage coordinates.

Arguments

  • x_percentage

  • y_percentage

Press Text With Index

Arguments

  • element : text element ( a string )

  • index : 0 indexing applied ( meaning 1st element corresponds to 0, 2nd corresponds to 1 and so on )

This keyword searches the UI Tree and finds all the matches of the input text using all available matching algorithms. From the found matches , it will select the match based on the input index and performs press.

This keyword can be used to click on texts that appear more than once on a single screen ( UI Tree ).

Detect And Press

Look for an element/elements and perform an action ( Press Element ) based on the result.

Arguments

  • Element to detect : Element is described by text, image path, Xpath, etc.

  • Element to Press : Element is described by text, image path, Xpath, etc.

  • Timeout : Time duration in seconds to wait for the element to be detected to appear.

User Guide:

Possible use cases-

  • Detecting a popup ( Element to detect ) and clearing it ( element to press )

  • Toggling between 2 options such as Increasing and Decreasing Limits by using this keyword twice

  • Appropriate timeout can be passed as argument to wait for the popup to appear.

Enter Text

Arguments

  • Element: Element is described by text, image path, XPath, etc.

  • Input Text: The value of text to be set to the Element.

Output

  • Annotated Image when the text is set into the element specified ( pending )

Enter Text Using Keyboard

Alternative option when Enter Text isn’t working, this keyword allows entering text using the keyboard.

Arguments

  • input text

Enter Number

Arguments

  • Element: Element is described by text, image path, XPath, etc.

  • Number: The value of the number is to be set to the Element.

Output

  • Annotated Image when the number is set into the element specified

Press Checkbox

Arguments

  • Element: Element is described by text, image path, XPath, etc.

Press Radio Button

Arguments

  • Element: Element is described by text, image path, XPath, etc.

Select From Drop Down

Arguments

  • Element: Element is described by text, image path, XPath, etc.

  • option: Option to be selected from the drop down options

Swipe Right

Arguments

  • Start X: % distance from the left of the screen

  • Start Y: % distance from the bottom of the screen

  • Swipe Percentage: % distance from start position

Swipe Left

  • Start X: % distance from the left of the screen

  • Start Y: % distance from the bottom of the screen

  • Swipe Percentage: % distance from start position

Swipe Right From Element

  • Element: Element is described by text, image path, XPath, etc.

  • Swipe Length: Default is 1000 pixel, can be overwritten by passing the argument.

Swipe Left From Element

  • Element: Element is described by text, image path, XPath, etc.

  • Swipe Length: Default is 1000 pixel, can be overwritten by passing the argument.

Scroll Page Up

Arguments

  • Scroll Percentage: % distance from start position

User Guide:

Scroll Page Up moves the page upwards by percentage input ( percentage of screen ). This keyword can be used to scroll up the page without performing the scroll on a specific locator/element.

Scroll Page Down

Arguments

  • Scroll Percentage: % distance from start position

User Guide:

Scroll Page Down moves the page downwards by percentage input ( percentage of screen ). This keyword can be used to scroll down the page without performing the scroll on a specific locator/element.

Scroll Page Up Until Element Appears

Arguments

  • Elements : Elements described by text, image path, Xpath

  • timeout : maximum time limit to scroll and look for the element

User Guide:

This keyword can be used when it is difficult to determine how much to scroll up. This keyword can be used when an element you are looking for has varying positions on the page at different runs. This keyword will stop scrolling once the element is detected on the page.

Scroll Page Down Until Element Appears

Arguments

  • Elements : Elements described by text, image path, Xpath

  • timeout : maximum time limit to scroll and look for the element

User Guide:

This keyword can be used when it is difficult to determine how much to scroll down. This keyword can be used when an element you are looking for has varying positions on the page at different runs. This keyword will stop scrolling once the element is detected on the page.

Scroll Up From Element

  • Element: Element is described by text, image path, XPath, etc.

  • Scroll Percentage: % distance from start position

Scroll Down From Element

  • Element: Element is described by text, image path, XPath, etc.

  • Scroll Percentage: % distance from start position

Close and Terminate Application

  • Closes the app which was launched while creating the Appium session

Clear Element Text

Clears text from a text field, EditText element, ideally identified by xpath.

Arguments

  • Element: Element is described by text and XPath

Get Text From Element

Arguments

  • Element: Element is described by text and XPath

Data Reader

Read Data Using Text Search

Arguments

  • text_to_search

  • variable

  • index

This keyword searches all possible matches that contains text given by input ${text_to_search} and returns the numerical values in the text field. Index can be given as input if there are more than one elements that contain the same text to search. This keyword also sets the value as a global variable using the input ${variable} name, which allows user to reuse the data in following test steps by calling ${variable}.

Usage

This keyword can be used to read account balances or amounts which are displayed on screen as numbers followed by currency , for example , 100.00 SGD. This keyword will fetch the value 100 and set it as global

using the input ${variable}.

Read Data Using Pattern Search

Arguments

  • pattern to search : in regex expression format

  • variable

  • index

This keyword searches all possible matches that contains values in regex pattern given by input pattern to search and returns the numerical values in the text field. Index can be given as input if there are more than one elements that contain the same text to search. This keyword also sets the value as a global variable using the input ${variable} name, which allows user to reuse the data in following test steps by calling ${variable}.

Data Writer

Arguments

  • variable

  • value

This keyword stores the data in a csv file which is uploaded at the end of the test which can be downloaded from the portal. This function allows extraction of data from journeys for future uses outside of test cases. This keyword can be called for writing the data fetched by Read Data keywords.

Usage 1

If input ${variable} is already a global variable , this function will read its value and store its value in csv.

${amount} 100

Data Writer ${amount}

${amount},100 will be written in csv

Usage 2

If input ${variable} is not a globally defined variable yet, input value needs to be passed.

Data Writer ${new_amount} 50

${new_amount},50 will be written in csv.

Assertions

Assert Equal

Arguments

  • expected output : this is the expected or correct output of the expression to be evaluated

  • expression: mathematical expression, can contain robotframework variable , numbers

This keyword evaluates the expression input and checks if it is equal to the expected outcome. It fails the test if the expression output is not equal to the expected outcome.

Assert Format

Arguments

  • element(s) : one or more elements to be checked

  • format : format that the element should follow in regular expression

  • value ( optional ) : the value that the element should have, on top of following the format

This keyword checks if the elements follow a particular format given by format input described in regular expression. If value is also given as an input, it will check if the element is equivalent to the value input, similar to keyword Assert Equal.

Validate Screen

Arguments

  • Elements: One or many(up to 5) elements described by text, image path, XPath, etc.

  • Timeout: Duration of wait time

  • event_name ( not compulsory ): event name defined on KPI configuration on portal

  • rule: any or all, default rule is any

User Guide:

Validate Screen will look for elements at a particular screen for the given timeout. By default, rule is any but can be overwritten to set the rule as all by passing argument rule as ‘all’.

Validate Screen will not fail the test when the element is not found. This keyword can be used to wait for elements until they appear on the screen right before attempting to perform actions on them.

When event_name is passed as a parameter, it will trigger the event using the timestamp when page source was fetched , making sure the latency KPI calculations are accurate.

Start Appium Session Capture Event AppLaunched Validate Screen ${homepage_element} 10 event_name=homepageAppeared

Expected duration:

This keyword should take approximately timeout given + 1 sec when the elements are not found.

Output:

  • Annotated Image showing the elements matched

Assert Presence

Arguments

  • Elements: One or many(up to 5) elements described by text, image path, XPath, etc.

  • Timeout: Duration of wait time

  • event_name ( not compulsory ): event name defined on KPI configuration on portal

  • rule: any or all, default rule is any

User Guide:

Assert Screen will look for elements at a particular screen for the given timeout. By default, rule is any but can be overwritten to set the rule as all by passing argument rule as ‘all’.

Assert Screen will fail the test when the element is not found. This keyword can be used to check if elements needed for the next steps are on the page or not.

When event_name is passed as a parameter, it will trigger the event using the timestamp when page source was fetched , making sure the latency KPI calculations are accurate.

Start Appium Session Capture Event AppLaunched Assert Presence ${homepage_element} 10 event_name=homepageAppeared

Expected duration:

This keyword should take approximately timeout given + 1 sec when the elements are not found.

Output:

  • Annotated Image showing the elements matched

Is Element Enabled

Arguments

  • Elements: an element described by text, image path, XPath, etc.

  • timeout : duration to check

Is Element Disabled

Arguments

  • Elements: an element described by text, image path, XPath, etc.

  • timeout: duration to check

Element Should Not be Visible

Arguments

  • Elements: an element described by text, image path, XPath, etc.

Wait Until Screen Match ( will be deprecated )

Arguments

  • elements as a list

  • rule

  • timeout

This keywords take in a list of elements. Please use Validate Screen and Assert Presence instead.

Wait Until All Elements Present ( will be deprecated )

Arguments

  • Elements: an element described by text, image path, XPath, etc.

Wait Until Any Element Present ( will be deprecated )

Arguments

  • Elements: an element described by text, image path, XPath, etc.

Capturing Events

Capture Event

Arguments

  • event name : string, this should be identical to the event created on the portal

  • event_attributes ( optional )

This keyword sends the events to the API which are processed to calculate KPIs after the test. This keyword Capture Event can be used to trigger events during the journeys. It adds dateTime, testCaseName and fetches mozark_event_attributes to prepare the payload. It collects all the events during a single test ID and sends one single combined payload to the API at the end of the test.

Debugging & Helper Keywords

Template QA

This keyword checks if the images defined in the data csv are available on the controller after test is started. It will check for each images and reports to console if there are any image files that cannot be loaded using opencv. Add this keyword after Initialize Csv Keyword to use.

Time Keyword

This keyword logs the start time and time taken to run/process any other keywords available. This keyword can be used to check the duration of each keyword processing time to debug if the keywords are behaving as they should.

Usage example : Time Keyword Assert Presence ${argument for Assert Presence}

This will tells you how long it takes to run Assert Presence keyword using its arguments , results can be observed on console.

Element Locator Strategy

Specifying Element

Elements can be specified in one of the following ways

  • Text - Complete or partial

  • Image path - path to the template image

  • XPath - Xpath captured using tools like Appium Inspector or Selenium IDE

Locator Strategy

Based on the input specified the framework determines whether the passed input is Text, Image Path, or XPath with simple rules such as:

  • if the input contains .png or .jpg then the input is considered to be an Image

  • If the input starts with / or // then the input is considered to be XPath

  • Else the input is considered Text

For Text and XPath, the following are the common steps when the framework can access the page source of the app:

  • Text or XPath is matched fully

  • Text or XPath is matched partially

  • Text or XPath is matched using Fuzzy Matcher

XPath is optimized to handle the variations possible during the collecting of the XPath or structural changes done to page source by the developer - this includes:

  • Moving the elements within the hierarchy

  • Small updates to the text or the element attribute values

The following steps are taken before matching the XPath within the page source

Making it relative

Given:

//android.widget.LinearLayout[@content-desc="TEST"]/android.widget.LinearLayout/android.widget.LinearLayout/androidx.appcompat.widget.LinearLayoutCompat[1]/android.widget.TextView

Updated:

//android.widget.LinearLayout[@content-desc="TEST"]//android.widget.TextView

For using a partial match

Given

//android.widget.LinearLayout[@content-desc="User name"]//android.widget.TextView

Updated:

//android.widget.LinearLayout[contains(@content-desc, "User") or contains(@content-desc, "name")]//android.widget.TextView

For using fuzzy match

Given

//android.widget.TextView[@resource-id="com.example.uat:id/tv_card_name"] //android.widget.TextView[@resource-id="com.example.prod:id/tv_card_name"]

Updated(for partial match)

//android.widget.TextView[contains(@resource-id, 'tv_card_name')]

Updated(for fuzzy match)

  • gets all the android.widget.TextView elements within a page source - XML tree

  • determine fuzzywuzzy.fuzz.ratio() for given attributes (resource-id, content-desc, name, value, etc. ) values.

    • given: com.example.uat:id/tv_card_name

    • to_be_matched: com.example.prod:id/tv_card_name

    • ratio = fuzzywuzzy.fuzz.ratio(given, to_be_matched)

    • given: User Name

    • to_be_matched: Username

    • ratio = fuzzywuzzy.fuzz.ratio(given, to_be_matched)

  • use difflib.SequenceMatcher

    • expected_text = send money

    • element_text = Send money

    • match_score = difflib.SequenceMatcher(None, expected_text.lower(), element_text.lower()).ratio()

Element detection using computer vision

In cases where Appium-based inspection and matching aren’t possible, the framework automatically uses computer vision techniques to detect the elements(typically text and image). In addition to the scenario where Appium is not supported following use cases are seamlessly covered using computer vision techniques within the framework without the need for making changes in the test automation scripts(no-code scripts developed using CSVs). These scenarios are:

  • USB debugging is not allowed by the Application

  • Some pages are secured within the Application

  • Content(both text and visualizations) on some pages rendered as image

  • etc.

A few of the techniques used are:

  • Text detection using OCR

  • Image template matching within a page

  • Structural screen match

  • Dominant color detection

  • etc.

Creating test cases for functional testing

A typical functional testing test case scenario includes a sequence of test steps. A test step can combine user input and visual checks.

A sample test scenario could be:

  • Launch Application

  • Login using a valid username and password

  • Check whether the user lands on the home screen

  • Navigate to the user profile page

  • Check whether the premium subscription is active

Another scenario could be:

  • Launch Application

  • Login using an OTP

  • Check whether the user lands on the home screen

  • Navigate to the user profile page

  • Check whether the premium subscription is active

The number of user inputs and visual checks varies depending on the test step. To allow the reuse of the test steps in other test scenarios and speed up the test case creation, it is important to make those modular. Similar principles like SOLID in software development can be followed.

The test steps can be alternatively referred to as a Module.

In the above test scenarios, the Login module differs.

Creating a module

A module is simply a sequence of user interaction and visual check steps. These steps/sub-steps can be defined using the keywords listed in the Keywords section.

The definition of the module is written in the no-code editor like Excel/CSV editor.

A sample module can be defined as follows:

module_name

module_step

params

Login using a valid username and password

Press Element

${username_input_field_text}

Login using a valid username and password

Enter Text

${username}

Login using a valid username and password

Press Element

${password_input_field_text}

Login using a valid username and password

Enter Text

${password}

Login using a valid username and password

Press Element

${submit_button_field_text}

Creating a test scenario

A test case scenario is a sequence of modules arranged to test a certain functionality.

test_case

test_step

TC 001 Validate Premium Subscription

Launch Application

TC 001 Validate Premium Subscription

Login using a valid username and password

TC 001 Validate Premium Subscription

Check whether the user lands on the home screen

TC 001 Validate Premium Subscription

Navigate to the user profile page

TC 001 Validate Premium Subscription

Check whether the premium subscription is active

Using test data parameters in a test scenario

Another important component to designing the best test case scenario is to use the test data instead of using constant values in the scenario.

There are two types of test data parameters:

  • Application user interface specific - let's call it elements.csv

  • Use case specific - there can be multiple of these like - credentials.csv, addresses.csv

Defining use case data parameters

Notice ${username} and ${password} in the above sample test case scenario, instead of setting it to constant values like ashokkumar and ashokpasswd respectively, these values should be set dynamically to test various scenarios. E.g. ${username} and ${password} set dynamically during test case execution. These values can be read from a data CSV. A sample test data CSV for authentication can be:

username

password

ashokkumar

password1

alok.kumar

alonk124

kumar.vijay

vijay333

There can be multiple test data CSV files, for each test case scenario. Instead of combining all the data for a test scenario in a single file, it is a good practice to separate those based on the use case. Separating these into multiple CSV enables reuse and ease of maintenance. Sample CSVs for use case could be:

  • credentials.csv, accounts.csv, invalid_credentials.csv, and so on.

Initializing test case data parameters

For setting values to the variables used within the test case scenario, the rules for setting the values can be set in test_data_intializer.csv

A test case contains several variables those are used in the modules to perform a user journey in automation.

The initialization rule follows the syntax similar to the SQL or Javascript Query Selector.

The syntax will be abstracted into a simple UI to connect test data to a test case scenario.

variable_name

initialization_rule

${username}

${credentials}.${username}[username='ashokkumar']

${pin_code}

${addresses}.${pin}[5]

Elements CSV

For user interaction and visual checks, the elements need to be listed in the elements.csv, this also enables de-coupling element data with test case scenarios. Keeping the element metadata in a separate CSV instead of hardcoding allows re-use and ease of maintenance.

Following is a typical structure of elements CSV:

element_name

element_id

element_region

other_metadata

PreviousAutomationNextAppium

Last updated