TextExerciser WorkFlow
TextExerciser provides a novel approach to automatically generate valid text inputs in testing Android apps. The key insight here is that if a text input does not satisfy the enforced constraints, the Android app will provide clues or hints for the malformed input, which can be used as a feedback for improvement. The workflow of our methodology is as follow :
- Hint Extractor(HE) : The main function of HE is to extract hints shown on UI. It first extracts all the texts in the app’s UI and then identifies static hints via a learning based method and dynamic hints via a structure-based differential analysis.
- Hint Parser(HP) : HP is mainly used for parsing and extracting key elements(i.e. constraints) from hints. It parses all the extracted hints via three steps: classifying hints into different categories, generating syntax trees for each hint, and interpreting the generated tree into a constraint representation form.
- Input Generation Engine(IG) : In this module, IG generates text input based on constraints. Specifically, it first maps constraints into the solver, and then generates a concrete input by solving the problem based on hints.
In the case of another hint, TextExerciser will iterate the above procedure until TextExerciser finds a valid input.
Tool Implementation
We implement it in python code based on Uiautomator2 to interacte with tested app in phone/emulator, such as fetching dynamic layout information on current UI, exploring the widgets on screen, etc. And a CNN-RNN Neural Network Model is applied to identify and classify hints. We use Standford NLP tools to parse hint texts in HP module and Z3StrSolver to solve the input constraints in IG module.
This is the working flow of TextExerciser. The Physical Phone on the left is used for running the smsOberserver app which transmits the varify code SMS to PC. In the middle, the PC executes the main function of TextExerciser in Python.
