3.4.1 Robustness of results

Loading...

En provenance du cours de University of Geneva

Global Statistics - Composite Indices for International Comparisons

3 notes

University of Geneva

Global Statistics - Composite Indices for International Comparisons

3 notes

The number of composite indices that are constructed and used internationally is growing very fast; but whilst the complexity of quantitative techniques has increased dramatically, the education and training in this area has been dragging and lagging behind. As a consequence, these simple numbers, expected to synthesize quite complex issues, are often presented to the public and used in the political debate without proper emphasis on their intrinsic limitations and correct interpretations. In this course on global statistics, offered by the University of Geneva jointly with the ETH Zürich KOF, you will learn the general approach of constructing composite indices and some of resulting problems. We will discuss the technical properties, the internal structure (like aggregation, weighting, stability of time series), the primary data used and the variable selection methods. These concepts will be illustrated using a sample of the most popular composite indices. We will try to address not only statistical questions but also focus on the distinction between policy-, media- and paradigm-driven indicators.

À partir de la leçon

The steps of constructing a composite index

This module is organized along four lessons. The objective of this module is to familiarize you with the key steps to undertake when constructing a CI. The first lesson will develop a theoretical framework to support CIs’ construction. Notably, it will cover topics such as variables selection and data issues. The second lesson will introduce a unifying approach to construct CI by discussing aspects related to transformation functions and the elasticity of substitution. The entire third lesson will be devoted to an essential aspect in the construction of a CI: the choice of weights. Finally, the module will conclude by addressing questions arising after the construction of a CI. For instance, lesson four will discuss how to assess the robustness of the resulting CI. By the end of this module you will be familiar with all the most important technical (or say statistical) steps involved in constructing CIs.

3.4.1 Robustness of results7:24

Rencontrer les enseignants

Stefan Andreas Sperlich
Professor
GSEM, Research Center for Statistics
Juan Manuel Rodriguez Poo (in Partnership with UNIGE)
Professor
Departamento de Economía, Universidad de Cantabria

This section of the course is about to study the robustness and sensitivity

of our results to changes in the different components in the composite index.

As you already know, composite indices depend basically in three different issues.

The type of transformation we have proposed for the partial indicators,

the value of the elasticity of substitution of the beta,

the so-called beta parameter we previously defined,

and finally, the value of their weights.

As you probably remember also in another section of this course,

we established some simplifying assumptions that

basically would mean that we could express

the exchangeability relation between the different partial indicators as

a function of the relative ratio of the relative weights of the partial indicators.

If this is the case,

then please note that in most part of cases,

it is of great interest to change,

to vary the weights in

the composite indicator and to analyze the results we are obtaining.

This is important in terms of the conclusions we can obtain

or we can withdraw from a composite indicator.

Because otherwise if we don't make

a sensitivity analysis and we don't study its robustness,

we may be ranking or we may be out-performing or down-performing,

I guess some characteristic of a country or an individual because there has

been some problem or some outlying observations or

some wrong choice of some of the weights.

So please note that this is a very important part

in the making process of a composite indicator.

In the next slide, you are going to find

some examples that have been taken from Section Seven from

the already mentioned interesting work of Sharpe and Andrews, 2012.

So basically, if you note in this table and you realize in this table,

the ordinal rankings for all weighting methods

of the different countries do vary significantly.

This is the case, for example,

when we take equal weights with respect to them,

the first alternative in selecting their weights is

already designing in the previous slide.

We can see that Norway is still at the top and Spain is still or remains at the bottom.

But however, we can see that in

between there are some countries that change their position.

So somehow, of course,

these tables provide us with a very interesting information about rankings and

about the impact of changes in their weight in terms of the ranking of countries.

But it is also true that it give us a too wide picture,

it is hard to find really significant information from all these tables.

What do we usually do them?

Basically if you go to the next slide,

you will find what we call the dominance criteria.

This dominance criteria, which is rather simplified here,

basically determines under which conditions a country X

strongly dominates a country Y or a country X weakly dominates a country Y.

You can find in this slide the definitions given by

these two relationships and they are based always,

of course, in the values of the composite indicator for each country and

for every value of the partial indicators in this country.

So we will say, country X strongly dominates

country Y whether the composite index evaluated

for this country takes a larger value than the composite indicator for the other country.

And we say it weakly X weakly dominates Y,

if we are allow for the possibility that both composite indices are equal.

These are the two difference of dominance criteria we are going to use.

Once we have established these dominance criteria,

we go now to the next slide where it has been computed

that different relations in terms of dominance between or among the different countries.

Then we can see if we have a look at through rows,

we will see for example that Finland gets a zero with

respect to all other countries but Spain where he gets a one.

This means that Finland dominates Spain with respect

to the criteria we have studied in the choice of the weights.

For example, take another like Norway,

you can see in the rows that Norway takes a lot of ones,

this mean that Norway dominates

these countries where we get the ones with respect to these different criteria.

These would be, the first table in this slide refers

about the strong dominance and the second table in

the next slide refers about weak dominance.

With these tables we are already able to decide which type or

weights are more robust to the outcomes in the composite indicator.

Let me, just to finish,

let me point out here that robustness is something

necessary to get nice properties for composite indicators.

It's very interesting to show the audience and to show

public opinion that our results are robust to the choice of weights

and somehow this robustness is going to

reinforce the conclusions we are getting from the composite indicators.

Because otherwise, as you probably understand,

anyone could tell us that okay,

we have chosen or we have obtained these results.

But if the weights would have been another,

then the results probably would have changed also.

And this is something we don't want to have.

Explorer notre catalogue

Rejoignez-nous gratuitement et obtenez des recommendations, des mises à jour et des offres personnalisées.

Coursera propose un accès universel à la meilleure formation au monde, en partenariat avec des universités et des organisations du plus haut niveau, pour proposer des cours en ligne.

© 2018 Coursera Inc. Tous droits réservés.

Télécharger dans l'App Store

Disponible sur Google Play