Examining and improving existing tests and procedures
Testing without a specification
When the software has evolved, original requirements are no longer relevant, test coverage is incomplete and yet the team are able to maintain and improve a high quality, defect free product how can we limit the possible risk that a significant change might result in an undetected defect being released into a live environment?
Experience based testing
If the explicit requirements are incomplete, out of date, or just absent you can initially turn to implicit requirements – the behaviours we come to expect in online products and services like well formatted presentation, a consistent user experience and clear communication of intent, etc.
Consult the oracles
Going deeper, you should be able to learn a great deal about a product or service from speaking to the stakeholders and product owners about what value it brings to the business – capture their knowledge of how the products were designed to be used.
Similarly, speak to the people who support the users – what are the common problems encountered? and if possible also speak to the end users (obviously easier if they’re internal to the business) to see how they’re using it.
Mnemonics – it doesn’t need a cool acronym
There are many mnemonics but they may not fit the context of your product, so develop one that fits by listing the quality aspects of the product as prompts to what needs to be investigated. You could start with some of the keywords in this rather handy list of testing mnemonics:
In addition, consider adding any keywords that spring to mind – technologies in use, modes of operation, business domain specific terminology, etc.
For a backend API service this might include:
- Latency
- Capacity
- Schema
- Parameters
- Paging
- Kafka
For a back office UI:
- Authentication
- Inputs
- Validation
- Workflow
- Accessibility
- Audit Trail
Whatever you come up with, bear in mind it’s used as an aid in remembering. Test mnemonics help guide thinking around the aspects of test areas to focus on and are a form of heuristic, and while many of them have a (sometimes a very tenuous) acronym, it’s not necessary (although quite fun if you do manage to come up with one ? )
Running a session
Once you have some initial keywords (and bear in mind you can and should change them during the session if necessary), follow the guidance in Running an exploratory session – set a charter, adopt a persona and timebox a session. Take notes or use a screen recorder to record the session.
Once complete, analyse your findings – where there any new defects discovered?
Analyse the existing test material
With the mnemonic and session in mind, assess the current test coverage – is there any obvious gaps in that should be addressed?
For tests that passed, Is there anything you feel should be performed frequently – could you enhance your existing tests?
Close the loop
Document all your findings, create tickets for any process improvements that needs to be made and get them on the backlog. Schedule some time in the future to repeat your analysis.