QC automation : Development Results and Guide
What is QC automation?
The built game will be distributed to each store on the platform after verifying that all UIs and contents are working properly through a step called QC Test before release. Until now, testers have checked a series of processes from normal download of builds to termination of apps, and have consumed tens of minutes to hours of test labor depending on the platform and device types supported by the game’s scale.
QC automation was developed in script form to provide the tester with the basis for determining whether a typical menu button works, processing repetitive test items that were executed per build in a short time, allowing the tester to focus on Major Issues.
Areas of QC automation
Not all test items can be dependent on Automation QC. Typical test items in the game, for example, the areas of manual and automatic can be distinguished as follows:
Automation QC tests whether the menu’s sequential input and next steps are successful in accordance with the user’s Play Flow or UI Flow, and manual QC can focus on major issues such as reading, payment, and reward of visual resources such as images, production or sound.
QC Automation Production
The process of automating QC can be largely divided into four stages. Document the UI Flow, which defines all processes that require Input as Step, as shown in the reference figure below. The Step 2 <Resource Acquisition> process cuts out the original size button image for each step and stores it step by step. Next, register the resources in order in Step 3 <Script>. Finally, when you <Run> the Script, you will run an automated QC test from Download to Install > Run > Input > • • • > Exit > Results Report.
How scripts run
The test is conducted in the following structure:
Open the Appium server on the device (PC) to which the mobile device is connected and interact with the device through its port. The python script runs on the PC, and interactions such as pressing a button based on the QC checklist json file of the project to conduct QC are initiated.In the process, a log is left for each UI you press, and screenshots are automatically taken for each step and saved separately.
Screen shot the entire screen, find the UI you want through openv, and touch input with web driver. If the accuracy (similarity) of the result found through openv is less than 0.5, it is judged that it is not the image you want to find, and the location of the screenshot you took with the error log. Of course, there is also an option to skip the error log even if you can’t find it.
Tips for preparing for QC automation!
– Interval between UI Flow Steps: Define the time between 0.x and several seconds in case of steps that require time considering the test environment and the loading speed difference of UI for each device, and apply this element to the script.
– Unique Resource: Save within the screen of the game • It is an automated script that matches the registered button image resource and touches the center of the area, so only parts that have never changed in the <Resource> process must be cut and stored.
– It is convenient to know the UDID of the device in advance when connecting a physical mobile device rather than a remote QC, such as BrowserStack. This allows you to choose which device to send commands to when you connect multiple devices.
Automation QC Challenges
Automation QC conducts a one-way test according to Flow, making it difficult to test branch menus, swipes, and drag-type input, so in-game tests are not possible except for picking-input play. However, in the branch menu test mentioned earlier, multiple mobile devices and platforms can be tested in one test machine, so the problem can be solved by adding Flow according to the branch, but it is a fundamental part of the study. The following in-game tests are expected to be resolved either by auto-play support or by inserting other auto-test functions into UI Flow.