Analytical Quality Assurance
Our great experience in analysing data and building models means that we are well placed to review in-house or third-party analytical projects for our clients. These reviews can take many forms:
- Reviews of the Quality Assurance (QA) procedures and implementation for a project – effectively how well the project has applied QA and whether the way this has been done covers all the key aspects of quality. A team of three consultants worked with Defra and BEIS to perform a detailed review of the QA applied to a large suite of models used to report on current and forecast air quality across the UK – ensuring that the models were consistent with the departmental QA standards, the UK Government AQuA standards and the recommendations from the McPherson Report.
- Advising clients to help develop Analytical QA standards and procedures, either for specific projects or for the organisation as a whole. For example, we worked closely with WRAP to develop testing strategies for a new forecasting model they were developing with Defra – this then fed into the general testing procedures for model development at WRAP.
- Undertaking third-party assurance of analytical models. We are often commissioned to test models built by client teams or by third-parties for our clients. Hartley McMaster has even developed its own set of tools to aid in the testing of models built in Excel (including VBA modules). Testing can range in scope – based on client needs – but usually includes review of the model specification (is the approach adopted sensible, are the assumptions documented and tenable, etc), review of the model implementation (testing and review using the company tools), sensitivity analysis and a review of documentation. Additionally, this assurance process might examine data quality, the real-life business processes using a model, or a model’s susceptability to bias. We have undertaken many such reviews for a range of clients including MoJ and MCA in government and Ricardo EE in the private sector.