I’ve been experimenting with MCP and tool calling to learn a bit more about how it works, and what its capabilities are, and ultimately how that can be applied to Quality Engineering through test automation. My initial aim was to investigate the MCP server for Playwright but I wanted to…
Playwright is a UI automation test tool that supports Chrome, Edge, Firefox and Safari browsers. It was released in January 2020, has 74.7k stars on GitHub, and is fully open sourced and maintained by Microsoft. Playwright uses the Chrome DevTools Protocol, Firefox Remote Debug Protocol & WebKit Debug Protocol to…
Nightwatch is a UI automation test tool that supports Chrome, Edge, Firefox and Safari browsers. It was released in January 2014, has 11.3k stars on GitHub, and is fully open sourced and maintained by BrowserStack. Nightwatch uses the WebDriver protocol to perform the browser automation tasks, and can be used…
Cypress is a UI automation test tool that supports Chrome, Edge, and Firefox browsers, has a GUI test runner to aid in debugging. It was released in September 2017, has 48.7k stars on GitHub, and is maintained by Cypress. Cypress tests run inside the JavaScript engine of the browser, however…
As a follow up to the previous post, I repeated the tuning process using LoRA & QLoRA to see if they have the same desired output and how the tuning effects the limited evaluations I have been running on the models. The additional scripts are located in the ‘simple-tuning‘ directory…
Open-weight models can be tuned to customise them for specific tasks that they were not originally trained to do by fine-tuning them, and there are a number of techniques and tools to perform this. For this experiment I wanted to try full fine-tuning with https://pytorch.org/torchtune/ to teach Llama 3.1 8b…
Commercial and openly available large language models can help software testers be faster and more creative in their tasks, and there’s a rich body of best practices to help get that best out of these tools. Propmpt engineering is an ever-evolving subject though, and what works today may not be…
Large language models can help software testers be faster and more creative in their tasks, and there’s a rich body of best practices to help get the most out of these tools. Prompt Engineering tips Prompt engineering is an ever-evolving subject, and what works today may not be the best…
Commercially available large language models such as OpenAI’s ChatGPT4, Google’s Gemini and Anthropic’s Claude continue to dominate the headlines and have been heavily discussed within the test community, however there has been a great deal of research and development in openly available models that can be run on locally on…
In order to train the GPT-2 model we need some data, and to investigate how bias can affect the output of text generation we will experiment with political manifestos to see how text generation can be skewed to the political left and right. Getting the raw data The manifestos for…
