Makima - Automated Testing on desktop

• English
• 中文版

Background

This is a desktop automated testing framework based on accessibility api. At the same time, with the help of the open source framework of Ctype, the purpose of calling the Mac and Windows system-level API is achieved.

Install

pip install Makima

Getting started

For Mac

pass

For Windows

Requires system version >= Windows 7

Application element locate tool

For Mac

Accessibility Inspector：Xcode -> Open Developer Tools

Using Accessibility Inspector can provide a quick way to find these attributes.

For Windows

download inspect.exe

Using inspect.exe can provide a quick way to find these attributes.

Init

initialize an UIElementRef by app name/window name

For Mac：

from makima.helper.operation_mac import Init_App_Ref_For_Mac

makima = Init_App_Ref_For_Mac()    
teams_ins = makima(name="Microsoft Teams")

For Windows：

from makima.helper.operation_win import Init_App_Ref_For_Win

makima = Init_App_Ref_For_Win()   
teams_ins = makima(name = "Microsoft Teams")

support query：

Both mac and windows support, name = is used as the search criterion by default when the query type is not specified,

name = full matching

name = "Calendar | Microsoft Teams"

name_contains= partial matching

name_contains = "Calendar | Microsoft Teams"

name_matchesregular expression matching

name_matches = "^Automation.*Teams$"

for Windows Additional support for class_name as an additional lookup condition e.g. teams_ins = makima(name_contains="| Microsoft Teams",class_name="")

or pass a handle object directly e.g.

teams_hwnd = makima_common.find_windos(name="Microsoft Teams")[0]
teams_ins = makima(hwnd=teams_hwnd)

for Mac Additional support for pid as an lookup condition e.g. self.makima(pid="1234")

UIElementRef

UIElement is return by Init_App_Ref_For_Win/Init_App_Ref_For_Mac

Attribute Windows only:

Attribute@property:

• get_current_hwnd -> str:
  Return a handle of current UIElement related Window

• get_toggle_state -> str:
  Return the toggle status of the current element (provided the element is a toggle element).

• get_acc_value -> str:
  Return UIElement's value, corresponding to LegacyIAccessible.Value

• get_acc_keyboardshortcut -> str:
  Return UIElement's keyboardshortcut, corresponding to LegacyIAccessible.KeyboardShortcutProperty

• get_automation_id -> str: Return UIElement's automationid,

• get_class_name -> str: Return UIElement's class name

• get_control_type_name -> str: Return UIElement's control type name

• get_is_enabled -> bool: Return UIElement's enabled status

• get_acc_name -> str: Return UIElement's accessible name

• get_default_action -> str:
  Return UIElement's default_action

• get_description -> str:
  Return UIElement's accessible description, corresponding to LegacyIAccessibleDescriptionProperty

• get_acc_role -> str:
  Return UIElement's accessible role, corresponding to LegacyIAccessibleRoleProperty

• get_help -> str:
  Return UIElement's help, corresponding to LegacyIAccessibleHelpProperty

• get_state -> str:
  Return UIElement's state text, corresponding to LegacyIAccessibleStateProperty

• get_window_state -> str:
  Return Current window state, e.g. 0: "standard", 1: "maximum", 2: "minimize"

• get_last_ele -> WinUIElement: Return last element

• get_next_ele -> WinUIElement: Return next element

Attribute@non-property:

• get_clickable_point() -> tuple(x,y):
  Return tuple (x, y) if a clickable point was retrieved, or None otherwise

• get_acc_children_elements() -> list[WinUIElement]: Return children elements

• get_acc_location() -> tuple (left, top, right, bottom):
  Return the coordinates of the rectangle that completely encloses the UIElement.Return tuple (left, top, right, bottom)

• get_parent() -> WinUIElement:
  Return parent element

Attribute Mac only:

Attribute@property:

• get_role -> str: Return UIElement's role value

• get_identifier: -> str: Return UIElement's identifier

• get_title: -> str: Return UIElement's title

• get_value -> str: Return UIElement's value

• get_label -> str: Return UIElement's label

• get_role_description -> str: Return UIElement's role description

• get_help -> str: Return UIElement's help

• get_sub_role -> str : Return UIElement's sub role

• get_selected -> bool : Return UIElement if selected

Attribute@non-property:

• get_attributes() -> str: Return all attribute ofr UIElement

• get_actions() -> list: Return a list of the actions available on the UIElement

• get_pid(): Return the PID of the AXUIElement

• perform_action(action): Perform the specified action on the UIElement object

• get_element_at_position(x,y) ->MacUIElement: Gets the UIElement object at the specified coordinates

• get_acc_children_elements(): Return children elements

• get_position(): Gets the current element coordinates

• get_size(): Gets the current element width, height

• get_parent() -> MacUIElement:
  Return parent element

• get_center_coordinates() Return center coordinates of UIElement object

Action For Windows and Mac

Method:

Find method

For all find method, **query can use all attribute from attribute@property, and support contains and matches e.g

ele(acc_name = "Microsoft Teams")
ele(acc_name = "Microsoft Teams",acc_role = "input")
ele(acc_name_contains = "teams",acc_role = "input")
ele(acc_name_matches = "*.teams*.")

• ele(timeout =5, **query) -> WinUIElement: Find element, The default timeout period for searching elements is 5s，Timeout will throw an error. Return a single-element object

• eles(timeout =5, **query) -> List[WinUIElement]:
  Find elements, The default timeout period for searching elements is 5s，Timeout will throw an error. Return multiple element objects

• check_element_exist(timeout =5, **query) -> bool: Check whether the element exists, The default timeout period for searching elements is 5s，Timeout will throw an error. Return True or False

• scroll_to_find_element(scroll_time=15, timeout =5, **query) -> WinUIElement:
  Find element by scroll, The default timeout period for searching elements is 5s，Timeout will throw an error.

Action Method

For all action method
x_coordinate, y_coordinate are optional and defaults to None, which means use the coordinate of the current element.
x_offset, y_offset are optional. The offset xy is offset based on the current xy axis

• click(x_coordinate=None, y_coordinate=None, x_offset: float = None, y_offset: float = None): Simulates mouse click events

• hover(x_coordinate=None, y_coordinate=None, x_offset: float = None, y_offset: float = None): Simulates mouse hover events

• double_click(x_coordinate=None, y_coordinate=None, x_offset: float = None, y_offset: float = None): Simulate a double mouse click

• right_click(x_coordinate=None, y_coordinate=None, x_offset: float = None, y_offset: float = None): Simulates mouse right click events

• drag_to(self,to_x, to_y, duration, x_coordinate, y_coordinate, x_offset, y_offset): Simulates dragging an element to the specified coordinates

• input_text(text): Simulates input event

• clear(text): Clear input field

Keyboard event

Windows only

Supports single or multiple key combinations

from makima.windows.utils.keyboard import WinKeyboard

makima_kb = WinKeyboard()

# Simulate the Enter key
makima_kb.send_keys(makima_kb.codes.RETURN)

# Simulating copy and paste
makima_kb.send_keys(makima_kb.codes.KEY_C,makima_kb.codes.KEY_V)

# Simulating ctrl + alt + A
makima_kb.send_keys(self.makima_kb.codes.ALT,self.makima_kb.codes.CONTROL,self.makima_kb.codes.KEY_A)

Mac only

Supports single or multiple key combinations

from makima.mac.utils.keyboard import MacKeyboard

makima_kb = MacKeyboard()

# Simulate the Enter key
makima_kb.send_keys(makima_kb.codes.RETURN)

# Simulating copy and paste
makima_kb.send_keys(makima_kb.codes.KEY_V, makima_kb.mask_codes.COMMAND)


# Simulating ctrl + command + A
makima_kb.send_keys(makima_kb.codes.KEY_A, makima_kb.codes.Ctrl, makima_kb.mask_codes.COMMAND)

Notice! When there are two or more keys, the Mac and Windows parameters are not the same, and when the operating system is Mac, the Mac key combination needs to be matched with the keycode of the mask type. The order in which arguments are passed also affects the actual performance of keypresses.

Mouse event

Most mouse events are contained within the element object operations, so we won't cover them here

Windows only

from makima.windows.utils.keyboard import WinMouse

makima_mouse = WinMouse()

# The wheel moves up 100, x,y means start position 
makima_mouse.scroll_wheel(x,y,-100)

# The wheel moves down 100, x,y means start position 
makima_mouse.scroll_wheel(x,y,100)

Mac only

from makima.mac.utils.keyboard import MacMouse

makima_mouse = MacMouse()

# The wheel moves up 100, x,y means start position 
makima_mouse.scroll_wheel(x,y,-100)

# The wheel moves down 100, x,y means start position 
makima_mouse.scroll_wheel(x,y,100)

Common

Windows only

from makima.windows.utils.keyboard import WinCommon

makima_common = WinCommon()

# Waiting for the window to appear,The default timeout is 5 seconds, The rules for passing parameters are the same as
`Init_App_Ref_For_Mac`
makima_common.find_window_by_wait(name=None, timeout=5, **kwargs)

# Finds the window and returns a list of `HWND_OBJ`
makima_common.find_windows(name=None, **query)

# Open the application from the start menu
makima_common.open_app_by_name(name)

Additional Windows Support HWND_OBJ

teams_obj = makima_common.find_windows(name="Teams", **query)

#get window title
teams_obj.get_window_title

#get window class name
teams_obj.get_window_class_name

#Force the window to appear at the front
teams_obj.focus_window()

""" 
Maxinum or Mininum windows
SW_HIDE = 0
SW_SHOWNORMAL = 1
SW_SHOWMINIMIZED = 2
SW_SHOWMAXIMIZED = 3
SW_SHOWNOACTIVATE = 4
SW_SHOW = 5
SW_MINIMIZE = 6
SW_SHOWMINNOACTIVE = 7
SW_SHOWNA = 8
SW_RESTORE = 9
SW_SHOWDEFAULT = 10
""" 
teams_obj.show_window(number)

Mac only

from makima.mac.utils.common import MacCommon

makima_common = MacCommon()

# Activate the window so that it appears at the front
makima_common.active_window(name)

# Hide the window
makima_common.hide_window(name)

# Unhide the window
makima_common.unhide_window(name)

# Determine if the application has started
makima_common.is_finished_launching(name)

背景

这是一个基于无障碍api的桌面自动化测试框架。同时，借助开源的Ctype框架，实现了对Mac和Windows系统级API的调用。

安装

pip install Makima

版本要求

对于 Mac

无

对于 Windows

需要系统版本>= Windows 7

应用程序元素定位工具

对于 Mac

Accessibility Inspector：Xcode -> 打开 Developer Tools

使用Accessibility Inspector可以提供一种快速查找这些属性的方法。

对于 Windows

下载 inspect.exe

使用inspect.exe可以提供一种快速查找这些属性的方法。

初始化

通过应用名称/窗口名称初始化UIElementRef

对于 Mac：

from makima.helper.operation_mac import Init_App_Ref_For_Mac

makima = Init_App_Ref_For_Mac()    
teams_ins = makima(name="Microsoft Teams")

对于 Windows：

from makima.helper.operation_win import Init_App_Ref_For_Win

makima = Init_App_Ref_For_Win()   
teams_ins = makima(name = "Microsoft Teams")

支持的查找方式：

Windows 和Mac 通用，当不指定查询类型时，默认使用' name = '作为搜索条件。

name = 完全匹配

name = "Calendar | Microsoft Teams"

name_contains= 模糊匹配

name_contains = "Calendar | Microsoft Teams"

name_matches 正则表达式匹配

name_matches = "^Automation.*Teams$"

对于 Windows

额外支持class_name作为额外的查询条件

e.g. teams_ins = makima(name_contains="| Microsoft Teams",class_name="")

or pass a handle object directly e.g.

teams_hwnd = makima_common.find_windos(name="Microsoft Teams")[0]
teams_ins = makima(hwnd=teams_hwnd)

对于 Mac

额外支持pid作为查询条件

e.g. self.makima(pid="1234")

UIElementRef

UIElement通过Init_App_Ref_For_Win/Init_App_Ref_For_Mac返回

Windows支持的属性:

Attribute@property:

• get_current_hwnd -> str:
  返回当前元素的相关窗口的句柄

• get_toggle_state -> str:
  返回当前元素的开关状态(前提是元素是开关类型的元素)。.

• get_acc_value -> str:
  返回当前元素的 Value, 对应 LegacyIAccessible.Value

• get_acc_keyboardshortcut -> str:
  返回当前元素的 keyboardshortcut, 对应 LegacyIAccessible.KeyboardShortcutProperty

• get_help -> str:
  返回当前元素的 help, 对应 LegacyIAccessible.LegacyIAccessibleHelpProperty

• get_automation_id -> str: 返回当前元素的 automationid,

• get_class_name -> str: 返回当前元素的 class name

• get_control_type_name -> str: 返回当前元素的 control type name

• get_is_enabled -> bool: 返回当前元素的 enabled status

• get_acc_name -> str: 返回当前元素的 accessible name

• get_default_action -> str:
  返回当前元素的 default_action

• get_description -> str:
  返回当前元素的 accessible description, 对应 LegacyIAccessibleDescriptionProperty

• get_acc_role -> str:
  返回当前元素的 accessible role, 对应 LegacyIAccessibleRoleProperty

• get_state -> str:
  返回当前元素的 state text, 对应 LegacyIAccessibleStateProperty

• get_window_state -> str:
  返回当前窗口状态，例如 0:"standard"， 1: " maximuim "， 2: "minimize"

• get_last_ele -> WinUIElement: 返回上一个元素

• get_next_ele -> WinUIElement: 返回下一个元素

Attribute@non-property:

• get_clickable_point() -> tuple(x,y):
  如果检索到了可单击的坐标，则返回元组(x, y)，否则不返回

• get_acc_children_elements() -> list[WinUIElement]: 返回子级元素

• get_acc_location() -> tuple (left, top, right, bottom): 返回完全包围UIElement的矩形的坐标。返回元组(left, top, right, bottom)

• get_parent() -> WinUIElement:返回父级元素

Attribute Mac only:

Attribute@property:

• get_role -> str: 返回当前元素的 role value

• get_identifier: -> str: 返回当前元素的 identifier

• get_title: -> str: 返回当前元素的 title

• get_value -> str: 返回当前元素的 value

• get_label -> str: 返回当前元素的 label

• get_role_description -> str: 返回当前元素的 role description

• get_help -> str: 返回当前元素的 help

• get_sub_role -> str : 返回当前元素的 sub role

• get_selected -> bool: 返回当前元素是否被选中

Attribute@non-property:

• get_attributes() -> str: 返回当前元素的所有属性

• get_actions() -> list: 返回当前元素上可用的操作列表

• get_pid(): 返回当前元素的pid

• perform_action(action): 在UIElement对象上执行指定的操作

• get_element_at_position(x,y) ->MacUIElement: 获取指定坐标处的UIElement对象

• get_acc_children_elements(): 返回子元素

• get_position(): 获取当前元素坐标

• get_size(): 获取当前元素的宽度、高度

• get_parent() -> MacUIElement:
  返回父元素

• get_center_coordinates() 返回UIElement对象的中心坐标

适用于Windows和Mac操作系统

方法:

查找方法

对于所有find方法， query可以使用attribute@property中的所有属性，并支持contains和matches 如

ele(acc_name = "Microsoft Teams")
ele(acc_name = "Microsoft Teams",acc_role = "input")
ele(acc_name_contains = "teams",acc_role = "input")
ele(acc_name_matches = "*.teams*.")

• ele(timeout =5, **query) -> WinUIElement: 查找元素，搜索元素的默认超时时间是5秒, timeout将抛出错误。返回一个单元素对象

• eles(timeout =5, **query) -> List[WinUIElement]:
  查找元素，搜索元素的默认超时时间是5秒, timeout将抛出错误。返回多个元素对象

• check_element_exist(timeout =5, **query) -> bool: 检查元素是否存在，搜索元素的默认超时时间是5秒, timeout将抛出错误。返回True或False

• scroll_to_find_element(scroll_time=15, timeout =5, **query) -> WinUIElement:
  通过滚动查找元素，搜索元素的默认超时时间是5秒, timeout将抛出错误。

操作方法

对于所有操作方法 x_coordinate、y_coordinate 是可选的，默认值为None，这表示使用当前元素的坐标。 x_offset、y_offset是可选的。偏移量xy是基于当前xy轴的偏移量

• click(x_coordinate=None, y_coordinate=None, x_offset: float = None, y_offset: float = None): 模拟鼠标点击事件

• hover(x_coordinate=None, y_coordinate=None, x_offset: float = None, y_offset: float = None): 模拟鼠标悬停事件

• double_click(x_coordinate=None, y_coordinate=None, x_offset: float = None, y_offset: float = None): 模拟鼠标双击

• right_click(x_coordinate=None, y_coordinate=None, x_offset: float = None, y_offset: float = None): 模拟鼠标右键事件

• drag_to(self,to_x, to_y, duration, x_coordinate, y_coordinate, x_offset, y_offset): 模拟拖动一个元素到指定的坐标

• input_text(text): 模拟输入事件

• clear(text): 清空输入字段

键盘事件

仅限Windows

Supports single or multiple key combinations

from makima.windows.utils.keyboard import WinKeyboard

makima_kb = WinKeyboard()

# Simulate the Enter key
makima_kb.send_keys(makima_kb.codes.RETURN)

# Simulating copy and paste
makima_kb.send_keys(makima_kb.codes.KEY_C,makima_kb.codes.KEY_V)

# Simulating ctrl + alt + A
makima_kb.send_keys(self.makima_kb.codes.ALT,self.makima_kb.codes.CONTROL,self.makima_kb.codes.KEY_A)

仅限Mac

支持单键或多键组合

from makima.mac.utils.keyboard import MacKeyboard

makima_kb = MacKeyboard()

# Simulate the Enter key
makima_kb.send_keys(makima_kb.codes.RETURN)

# Simulating copy and paste
makima_kb.send_keys(makima_kb.codes.KEY_V, makima_kb.mask_codes.COMMAND)


# Simulating ctrl + command + A
makima_kb.send_keys(makima_kb.codes.KEY_A, makima_kb.codes.Ctrl, makima_kb.mask_codes.COMMAND)

注意! 当有两个或两个以上的键时，Mac和Windows参数不相同，当操作系统为Mac时，需要将Mac键组合与mask类型的keycode进行匹配。传递参数的顺序也会影响按键的实际表现。

鼠标事件

大多数鼠标事件都包含在元素对象操作中，因此这里不做介绍

仅限Windows

from makima.windows.utils.keyboard import WinMouse

makima_mouse = WinMouse()

# The wheel moves up 100, x,y means start position 
makima_mouse.scroll_wheel(x,y,-100)

# The wheel moves down 100, x,y means start position 
makima_mouse.scroll_wheel(x,y,100)

仅限Mac

from makima.mac.utils.keyboard import MacMouse

makima_mouse = MacMouse()

# The wheel moves up 100, x,y means start position 
makima_mouse.scroll_wheel(x,y,-100)

# The wheel moves down 100, x,y means start position 
makima_mouse.scroll_wheel(x,y,100)

通常接口

仅限 Windows

from makima.windows.utils.keyboard import WinCommon

makima_common = WinCommon()

# Waiting for the window to appear,The default timeout is 5 seconds, The rules for passing parameters are the same as
`Init_App_Ref_For_Mac`
makima_common.find_window_by_wait(name=None, timeout=5, **kwargs)

# Finds the window and returns a list of `HWND_OBJ`
makima_common.find_windows(name=None, **query)

# Open the application from the start menu
makima_common.open_app_by_name(name)

Windows额外支持HWND OBJ

teams_obj = makima_common.find_windows(name="Teams", **query)

# 获取窗口标题
teams_obj.get_window_title

# 获取窗口类名
teams_obj.get_window_class_name

# 强制窗口显示在前面
teams_obj.focus_window()

""" 
Maxinum or Mininum windows
SW_HIDE = 0
SW_SHOWNORMAL = 1
SW_SHOWMINIMIZED = 2
SW_SHOWMAXIMIZED = 3
SW_SHOWNOACTIVATE = 4
SW_SHOW = 5
SW_MINIMIZE = 6
SW_SHOWMINNOACTIVE = 7
SW_SHOWNA = 8
SW_RESTORE = 9
SW_SHOWDEFAULT = 10
""" 
teams_obj.show_window(number)

仅限Mac

from makima.mac.utils.common import MacCommon

makima_common = MacCommon()

# 激活窗口，让它出现在前面
makima_common.active_window(name)

# 隐藏窗口
makima_common.hide_window(name)

# 打开窗口
makima_common.unhide_window(name)

# 确定应用程序是否已经启动
makima_common.is_finished_launching(name)