Introduction:
Given the wide variety of convenience and functionality that mobile applications bring, they are now a critical part of the mobile ecosystem. Unfortunately, most of the apps are buggy. The main reason being that they need to be able to handle a wide variety of system settings and user interactions. An app needs be switched to the background at any time while restoring its state and later proceed as if no interruption ever occurred. Mobile OS are enabled to kill apps in the background to save battery, while the apps need to auto save their current state.
Testing of such actions takes time and testing them over many device configurations take even more time. Most of the bugs and vulnerabilities are blamed on mobile devices. But this may not be the case. Mobile apps can equally house bugs and vulnerabilities, and serve as major vectors of attack. Some of the factors of mobile application vulnerabilities are bad practices for data storage, malware, lack of encryption and side-loading.
While testing, bugs that are found can appear to be small and insignificant, but on further digging, larger inconsistencies may be disclosed. For finding vulnerabilities and bugs in mobile apps, answering the following questions can help.
- Does the app function as specified in the requirement documents.
- Does the app function beyond specifications.
- How does the app perform when used for a long time under constant load. How is it supposed to perform under such conditions? If its performance is not optimal, then what are the reasons?
- Is the data secure?
- What are the third party integrations of the app.
- How does the app redirection occur? What are the data transmitted in this redirection.
Common test activities:
- Review the crash reports.
- To test an app, all states of the app need to be monitored. Collect all the available actions in the app and perform the actions in the app.
- Generate a repository of cleanly booted emulator snapshots per configuration. While checking an app start for the specific snapshot instead of booting an emulator from scratch. Check multiple executions of the app by reusing the same emulator instance. This approach reduces the preparation overhead and ensures more testing time.
- While implementing possible executions of an app, it is more productive to decide the next action yourself. This gives you more control over exploratory testing process and has the maximum opportunities of detecting bugs.
- Often, input is required to move from one point to the other. This can be accomplished by leveraging an input generation component that generates inputs as per the specified rules.
- The record and play technique is a good way of tracing back to a corresponding failure.
- One bug can manifest many many redundant bug reports. After collecting test reports from all servers, filter the redundant report based on the type of failure.
- Unpack the application and analyze the.xml file to find necessary information, identifiers, library dependencies and startup activities. Use this information to start the app.
- Check what happens when the mobile device is full of data as well as devoid of all data.
- Uninstall the app and review the data. Reinstall and review again.
- Work under different time zones and in different times.
- Use invalid data.
- Use duplicate data.
- Effect of data update on other services
- Check if the errors adhere to good practices. Check the state of the user after an error is generated.
- Test on a new phone as well an old phone.
- Consider crowd sourcing the testing.
- Always check what the users see is correct.
- Check the app with variable connectivity, OS updates.
It is recommended to test the mobile app after every release. A set of priority tests can be defined and they should be covered under a variety of conditions.
Testing is mostly thought of as an inherently logical, process filled test scripts and test plans, pass and fail, red and green indicators. Good testing is all about asking challenging and to the point questions. Ideally, testers are expected to have all the information about what and how an application is being developed. But practically this is not so and testers need to do with whatever they have at hand. Information required for testing can be collected from many internal and external resources. Based on the information collected, a testing approach can be planned.