Methodology Board

At ecda, we deliver data projects to demonstrate the power of analytics in supporting decision making in policy and practice in public services. To support good decisions, our projects must apply high methodological standards - the right analytical methods need to be used and in the right way. To ensure the robustness of our work, ecda has established a Methods Board as a key part of its governance processes.

The who

Together Data Scientists, analytical experts, academics, and data practitioners from ecda’s three partners, Essex Police, Essex County Council and the University of Essex, make up the Methods Board and provide a wide range of skills and experience.

The Board is open to all analysts working within the Essex public sector who may benefit from some support and review of analytical work they are doing. It provides an opportunity for all those involved to develop a greater understanding of a range of methods within the analysts’ tool kit – this applies to both members and those involved in projects.

The how

The Methods Board has been set up to work with project teams throughout the lifecycle of an ecda project. At an early stage, the project team will outline the project to the Board, explaining the research questions, describing the data and setting out in broad terms the type of analysis that is expected. At this step, before a detailed assessment of the data, it can be hard to specify precisely the methods to be used but this early view allows the Board to comment on the proposed approach and to suggest additional avenues to explore.

During the analysis phase, the Board is available for consultation with the project team giving them access to expert advice. The Board can also input ideas during this time. At the end of the project, the Board carries out a detailed review of the analysis, based on documentation supplied by the project team as well as the code created and data. This extensive information allows a full review of the analysis and, potentially, a check to ensure replicability of the results.

Although loosely based on the model for peer review within academia, we were keen to establish the Board as a collaborative and supportive partnership that provides a space for respectful interaction and personal development.

The what

The Board brings methods to ecda from two different angles. First and foremost, we approach projects from a scientific perspective. This entails research design, literature review and theoretical work, data collection and pre-processing, and the selection and sound application of statistical, econometric, or data science methods. While these terms are often used interchangeably, at ecda we do consider their nuances to decide how to tackle a particular research question using data. We then think about improvements, gaps, and future work, and specially how to articulate the results of our analyses for different audiences while keeping an eye on replicability.

In other words, ecda relies on, and promotes, science and scientific methods to data projects in public policy. This is one of the foundations of our work.

Second, we bring specific methods and techniques to particular data projects, and promote innovation in data science for public policy. The insight that can be extracted from data depends on the type of variables in a database, as well as their organisation and quality. In some cases, we have enough information to use supervised learning and explore relationships between outputs (e.g. employed/unemployed) and inputs (e.g. age). In these cases, we can use a number of methods for inference or for prediction, and combine them with machine learning methods and robustness tests. In other cases, we do not have inputs and outputs and only a set of features about our units of interest, such as districts or parishes. In this case, we can bring unsupervised methods to discover structure.

While ecda relies on a group of analysts across the partnership, we often work with academics who are experts on the techniques necessary to extract insight from data. Here ecda works closely with colleagues based at the institutes and departments at the University of Essex, such as the Business and Local Government Data Research Centre (BLG DRC), the Institute for Analytics and Data Science (IADS), or the School of Computer Science and Electronic Engineering. In a recent collaboration for the Alleged Perpetrators of Domestic Abuse project, we had excellent discussions about clustering methods for discrete variables. Clustering is widely applied to continuous variables, but discrete ones present interesting challenges. Conversations with academic experts and analysts led to a series of alternative algorithms that we continue to explore.

In sum, science, methodology, and methods are at the centre of ecda. They guide the technical strategy of the Centre and inform specific projects. At the same time, this generates knowledge exchange and collaboration, and improves capacity and supports skills development in the partnership.