Benoit Marty
2 years ago
committed by
GitHub
18 changed files with 231 additions and 4 deletions
@ -0,0 +1,36 @@
@@ -0,0 +1,36 @@
|
||||
name: Meastro |
||||
|
||||
on: |
||||
pull_request: { } |
||||
push: |
||||
branches: [ main, develop ] |
||||
|
||||
# Enrich gradle.properties for CI/CD |
||||
env: |
||||
GRADLE_OPTS: -Dorg.gradle.jvmargs="-Xmx3072m -Dfile.encoding=UTF-8 -XX:+HeapDumpOnOutOfMemoryError" -Dkotlin.daemon.jvm.options="-Xmx2560m" -Dkotlin.incremental=false |
||||
CI_GRADLE_ARG_PROPERTIES: --stacktrace -PpreDexEnable=false --max-workers 2 --no-daemon |
||||
|
||||
jobs: |
||||
maestro-cloud: |
||||
name: Meastro test suite |
||||
runs-on: ubuntu-latest |
||||
if: github.ref != 'refs/heads/main' |
||||
strategy: |
||||
fail-fast: false |
||||
# Allow all jobs on develop. Just one per PR. |
||||
concurrency: |
||||
group: ${{ github.ref == 'refs/heads/develop' && format('maestro-develop-{0}', github.sha) || format('maestro-debug-{0}', github.ref) }} |
||||
cancel-in-progress: true |
||||
steps: |
||||
- uses: actions/checkout@v3 |
||||
- name: Assemble debug APK |
||||
run: ./gradlew assembleDebug $CI_GRADLE_ARG_PROPERTIES |
||||
- uses: mobile-dev-inc/action-maestro-cloud@v1.1.1 |
||||
with: |
||||
api-key: ${{ secrets.MAESTRO_CLOUD_API_KEY }} |
||||
app-file: app/build/outputs/apk/debug/app-debug.apk |
||||
env: | |
||||
USERNAME=maestroelement |
||||
PASSWORD=${{ secrets.MATRIX_MAESTRO_ACCOUNT_PASSWORD }} |
||||
ROOM_NAME=MyRoom |
||||
APP_ID=io.element.android.x.debug |
||||
@ -0,0 +1,71 @@
@@ -0,0 +1,71 @@
|
||||
# Maestro |
||||
|
||||
Maestro is a framework that we are using to test navigation across the application. |
||||
To setup, please refer at [https://maestro.mobile.dev](https://maestro.mobile.dev) |
||||
|
||||
<!--- TOC --> |
||||
|
||||
* [Run test](#run-test) |
||||
* [Output](#output) |
||||
* [Write test](#write-test) |
||||
* [CI](#ci) |
||||
* [iOS](#ios) |
||||
* [Future](#future) |
||||
|
||||
<!--- END --> |
||||
|
||||
## Run test |
||||
|
||||
From root dir of the project |
||||
|
||||
*Note: Since ElementX does not allow account creation nor room creation, we have to use an existing account with an existing room to run maestro test suite. So to run locally, please replace `user` and `123` with your test matrix.org account credentials, and `my room` with one of a room this account has join. Note that the test will send messages to this room.* |
||||
|
||||
```shell |
||||
maestro test \ |
||||
-e APP_ID=io.element.android.x.debug \ |
||||
-e USERNAME=user \ |
||||
-e PASSWORD=123 \ |
||||
-e ROOM_NAME="my room" \ |
||||
.maestro/allTest.yaml |
||||
``` |
||||
|
||||
### Output |
||||
|
||||
Test result will be printed on the console, and screenshots will be generated at `./build/maestro` |
||||
|
||||
## Write test |
||||
|
||||
Tests are yaml files. Generally each yaml file should leave the app in the same screen than at the beginning. |
||||
|
||||
Start the ElementX app and run this command to help writing test. |
||||
|
||||
```shell |
||||
maestro studio |
||||
``` |
||||
|
||||
Note that sometimes, this prevent running the test. So kill the `meastro studio` process to be able to run the test again. |
||||
|
||||
Also, if updating the application code, do not forget to deploy again the application before running the maestro tests. |
||||
|
||||
## CI |
||||
|
||||
The CI is running maestro using the workflow `.github/worflow/maestro.yaml` and [maestro cloud](https://cloud.mobile.dev/). For now we are limited to 100 runs a month. |
||||
Some GitHub secrets are used to be able to do that: `MAESTRO_CLOUD_API_KEY`, for now api key from `benoitm@element.io` maestro cloud account, and `MATRIX_MAESTRO_ACCOUNT_PASSWORD` which is the password of the account `@maestroelement:matrix.org`. This account contains a room `MyRoom` to be able to run the maestro test suite. |
||||
|
||||
## iOS |
||||
|
||||
Need to install `idb-companion` first |
||||
|
||||
```shell |
||||
brew install idb-companion |
||||
``` |
||||
|
||||
Also: |
||||
https://github.com/mobile-dev-inc/maestro/issues/146 |
||||
https://github.com/mobile-dev-inc/maestro/issues/107 |
||||
So you have to change your input keyboard to QWERTY for it to work properly. |
||||
|
||||
## Future |
||||
|
||||
- run on Element X iOS. This is already working but it need some change on the test to make it works. Could pass a PLATFORM parameter to have unique test and use conditional test. |
||||
- run specific test on both iOS and Android devices to make them communicate together. Could be possible to test room invite and join, verification, call, etc. To be done when Element X will be able to create account and create room. A main script would be able to detect the Android device and the iOS device, and run several maestro tests sequentially, using `--device` parameter to perform a global test. |
||||
@ -0,0 +1,7 @@
@@ -0,0 +1,7 @@
|
||||
appId: ${APP_ID} |
||||
--- |
||||
- runFlow: tests/init.yaml |
||||
- runFlow: tests/account/login.yaml |
||||
- runFlow: tests/settings/settings.yaml |
||||
- runFlow: tests/roomList/roomList.yaml |
||||
- runFlow: tests/account/logout.yaml |
||||
@ -0,0 +1,5 @@
@@ -0,0 +1,5 @@
|
||||
appId: ${APP_ID} |
||||
--- |
||||
- tapOn: "Change" |
||||
- takeScreenshot: build/maestro/200-ChangeServer |
||||
- tapOn: "Continue" |
||||
@ -0,0 +1,21 @@
@@ -0,0 +1,21 @@
|
||||
appId: ${APP_ID} |
||||
--- |
||||
- tapOn: "Get started" |
||||
- runFlow: ../assertions/assertLoginDisplayed.yaml |
||||
- takeScreenshot: build/maestro/100-SignIn |
||||
- runFlow: changeServer.yaml |
||||
- runFlow: ../assertions/assertLoginDisplayed.yaml |
||||
- tapOn: "Username or email" |
||||
# ios |
||||
# - tapOn: |
||||
# id: "usernameTextField" |
||||
# index: 0 |
||||
- inputText: ${USERNAME} |
||||
- tapOn: "Password" |
||||
# iOS |
||||
#- tapOn: |
||||
# id: "passwordTextField" |
||||
# index: 0 |
||||
- inputText: ${PASSWORD} |
||||
- tapOn: "Continue" |
||||
- runFlow: ../assertions/assertHomeDisplayed.yaml |
||||
@ -0,0 +1,12 @@
@@ -0,0 +1,12 @@
|
||||
appId: ${APP_ID} |
||||
--- |
||||
- tapOn: "Settings" |
||||
- tapOn: "Sign out" |
||||
- takeScreenshot: build/maestro/900-SignOutDialg |
||||
# Ensure cancel cancels |
||||
- tapOn: "Cancel" |
||||
- tapOn: "Sign out" |
||||
- tapOn: |
||||
text: "Sign out" |
||||
index: 1 |
||||
- runFlow: ../assertions/assertInitDisplayed.yaml |
||||
@ -0,0 +1,5 @@
@@ -0,0 +1,5 @@
|
||||
appId: ${APP_ID} |
||||
--- |
||||
- extendedWaitUntil: |
||||
visible: "All Chats" |
||||
timeout: 10_000 |
||||
@ -0,0 +1,5 @@
@@ -0,0 +1,5 @@
|
||||
appId: ${APP_ID} |
||||
--- |
||||
- extendedWaitUntil: |
||||
visible: "Own your conversations." |
||||
timeout: 10_000 |
||||
@ -0,0 +1,5 @@
@@ -0,0 +1,5 @@
|
||||
appId: ${APP_ID} |
||||
--- |
||||
- extendedWaitUntil: |
||||
visible: "Welcome back!" |
||||
timeout: 10_000 |
||||
@ -0,0 +1,8 @@
@@ -0,0 +1,8 @@
|
||||
appId: ${APP_ID} |
||||
--- |
||||
- clearState |
||||
- launchApp: |
||||
clearKeychain: true |
||||
- tapOn: "Close showkase button" |
||||
- runFlow: ./assertions/assertInitDisplayed.yaml |
||||
- takeScreenshot: build/maestro/000-FirstScreen |
||||
@ -0,0 +1,6 @@
@@ -0,0 +1,6 @@
|
||||
appId: ${APP_ID} |
||||
--- |
||||
- takeScreenshot: build/maestro/300-RoomList |
||||
- runFlow: searchRoomList.yaml |
||||
- runFlow: timeline/timeline.yaml |
||||
|
||||
@ -0,0 +1,15 @@
@@ -0,0 +1,15 @@
|
||||
appId: ${APP_ID} |
||||
--- |
||||
- tapOn: "search" |
||||
- inputText: ${ROOM_NAME.substring(0, 3)} |
||||
- takeScreenshot: build/maestro/400-SearchRoom |
||||
- tapOn: ${ROOM_NAME} |
||||
# Close keyboard |
||||
- hideKeyboard |
||||
# Back from timeline |
||||
- back |
||||
# Close keyboard |
||||
- hideKeyboard |
||||
# Back from search |
||||
- back |
||||
- runFlow: ../assertions/assertHomeDisplayed.yaml |
||||
@ -0,0 +1,14 @@
@@ -0,0 +1,14 @@
|
||||
appId: ${APP_ID} |
||||
--- |
||||
# This is the name of one room |
||||
# TODO Create a room on a new account |
||||
- tapOn: ${ROOM_NAME} |
||||
- takeScreenshot: build/maestro/500-Timeline |
||||
- tapOn: "Message…" |
||||
- inputText: "Hello world!" |
||||
- tapOn: "Toggle full screen mode" |
||||
- tapOn: "Toggle full screen mode" |
||||
- tapOn: "Send" |
||||
- hideKeyboard |
||||
- back |
||||
- runFlow: ../../assertions/assertHomeDisplayed.yaml |
||||
@ -0,0 +1,12 @@
@@ -0,0 +1,12 @@
|
||||
appId: ${APP_ID} |
||||
--- |
||||
- tapOn: "Settings" |
||||
- assertVisible: "Rage shake to report bug" |
||||
- takeScreenshot: build/maestro/600-Settings |
||||
- tapOn: |
||||
text: "Report bug" |
||||
index: 1 |
||||
- assertVisible: "Describe your problem here" |
||||
- back |
||||
- back |
||||
- runFlow: ../assertions/assertHomeDisplayed.yaml |
||||
Loading…
Reference in new issue