Ignacio Moral-Arce is the Head of the Evaluation Unit of The Spanish Independent Fiscal Institution AIReF. Senior statistician with a Ph.D. in Econometrics, he has worked in evaluating public interventions both at a national and international level, evaluating European Founds impact in Spain and working for many underdeveloped countries such as Cape Verde, Tunisia, Costa Rica, Perú, Paraguay, Ecuador and Uruguay. He has been recently included by The European Commission as one of the best evaluators of 2016 during the 7th European Evaluation Conference “The result orientation: Cohesion Policy at work”.

Dr. Moral Arce, you have focused your interests on public policy evaluation working as an evaluator for the “Independent Authority for Fiscal Responsibility” of Spain. Starting with a more general question, what does being an evaluator mean? And what are his main duties?

The evaluator is a specialist who tries to “professionalize” the public administration. It is necessary to apply methods of private companies to the environment of the public administration: the public program has to be effective and efficient, and in order to obtain that, it is necessary to use useful information. The evaluator systematically analyzes information to determine the success of a public intervention and to improve the program in different areas: decision-making and planning (design), operations and management (implementation) and conclusive results and resources invested (Efficiency and impact).

The main duty consists in offering information with a double objective:
-An Instrument for taking decisions
-Generation of information that helps the decision making by the politicians

On the other hand, accountability is important too: it is appropriate that the society knows what has been done with the money (your taxes, for example). Everybody needs to know If the public intervention works or not.

Talking about policy evaluation, there are two various kinds of it: Quantitative evaluation and qualitative evaluation. Quantitative evaluation uses data, mathematical models and mainly statistics to find relations between different variables and obtain a measurable result. Instead, qualitative evaluation has a more case study approach. Closed to sociology, it tries to give a broad understanding of the subject in analysis, thanks to a more open and not measurable way of evaluating. Which are the point of weakness and strength of these two different methods? Do you think one is more significant than the other, or at least cannot work without the use of the other?  

Being synthetics:

Strengths of quantitative methods: Studies can be generalized to the population, we can estimate the amount of the impact and distribution.

Weaknesses of qualitative methods: They are related to some types of information which cannot be obtained through classical data collection instruments. Self-reported information can be inaccurate or incomplete and there is no information on contextual factors.

Strengths of qualitative methods: The flexibility to evolve, high-value subjects, looking at the big picture.

Weaknesses of qualitative methods: A lack of clear evaluation design and generalization of the results. Individual factors are not isolated and we usually have doubts about the subjectivity of the method.

In conclusion, I think that choosing between quantitative and qualitative approach depends on the question that you want to answer. Anyway, I believe that the use of mixed methods greatly improves any type of evaluation. It is certainly the best choice. However, we have to realize that nowadays, the dictatorship of the numbers has acquired a remarkable importance.

We all listen every day a lot of talking about the new era of Big Data. What makes it so important? In which way will it change our lives in the near future?

I think Big Data is changing the way things are done. Previously, there was an almost total lack of information to make decisions. Nowadays, the problem is that there is too much data. However, the fact that there are data does not imply that we have information to make decisions (data that have been “treated and cooked” in order to be useful for the analysis of a problem). The problem with such a large volume of data is that it is necessary to manipulate them in the right way to provide information. No doubt this task is not simple. The big data is important because it offers millions (literally) of data on a particular topic. Imagine a flight from Rome to London, there are, in each seat, about 15 sensors that every minute say how you are sitting, if you select music, what movies you are watching etc. With 200 passengers and 2 or 3 hours of flight, how much information does it generate? The same can be applied to other topics: the use of credit cards, public transports, labour market. Big data is the future in the management of projects, both public and private.

Big data, and the ways to treat this information, will change the ways of doing things. Many of the surveys now being carried out by statistical institutes will no longer make sense. These data already exist. From the point of view of job opportunities, without doubts, Big data manager is one of the professions of the future.

Do you think that evaluate the effectiveness of the policies is important to build a sense of political responsibility?

One of the great problems of society in general, and of politicians, is the lack of responsibility in the sense of assuming, for good or bad, the consequences of what has been decided (we only want to be responsible for good news). The evaluation of public policies clearly allows to increase the degree of allocation of responsibilities. Especially because it provides information on whether something has worked or not.

To conclude, recently, scientific and technical analysis are losing credit by the population. It is a spreading belief that social and economic experts do not have any clue of where we are and what should be done. Do you think there is some truth in these statements? And if yes, is refusing the scientific problem-solving approach a profitable solution?

There may be some truth. In the social sciences, because of the complexity of reality is very difficult to explain and predict. Firstly, it is not easy to perform good technical experiments as in medicine or laboratories. In addition, in some research works, there are biases that must be avoided (funders and political beliefs). Technical capacity and transparency are fundamental elements for credibility.