Measuring advantage and disadvantage
Download the WEA commentaries issue ›
By Stuart Birks
A common feature of policy debate is the discussion of whether chosen groups are advantaged or disadvantaged. Such claims can be analysed using a basic three step structure:
1. Define groups for comparison;
2. Choose a measure for which values vary over groups;
3. Interpret differences as advantage or disadvantage for the chosen group.
These three steps can be used to look critically at any claims of advantage or disadvantage. Needless to say, given the reasons for wanting to show advantage or disadvantage, any real world examples that I use will be politically sensitive. Common ones today might relate to gender or ethnicity. Fifty years ago class distinctions or a division between capital and labour might have been more common. With the publicity surrounding Thomas Piketty’s book, Capital in the Twenty-First Century, we may now be moving back in that direction. In an attempt to retain attention on the broader issues, it may be helpful to use a politically neutral example (i.e. one currently without ‘traction’), such a height or age. Let’s consider the three steps applied to age.
First, we choose our groupings. This can be a highly political act. The chosen framing sets up an “us and them” scenario. This is despite wide variations in circumstances often occurring within groups along with large overlaps across groups even when group averages differ. We could compare the elderly with the rest of the population, or with the rest of the adult population. We could take the elderly as those over 65, or over 75. The choice could make a big difference to the results. This is an aggregation issue, as groups will be represented by some average measure. Hence, if we want a policy shift favouring those in their late 60s, we could group them together with those 70 and above, for example.
Then we choose our measure(s). Note that comparisons would differ according to whether we used income, wealth, home ownership, income from paid work, hours worked, health expenditure, benefits received, savings out of income, number of dependents, education levels, overseas trips, sporting activity, computer-literacy, food consumption, sick days per year, and so on. Should we look at individual incomes, or consider intra-family transfers, household or family income and the number of dependents? Should wealth be considered along with income? Should current and future entitlements such as state benefits and inheritances be included? Are material measures the right ones to use, or are there other dimensions of wellbeing (social inclusion, family-connectedness, freedom from crime and abuse)? If we are interested in elder abuse, should we use police data on incidents or offences, or justice data on charges or convictions, or survey data? Each will give a different picture.
Given the chosen measure(s), there is then the interpretation of differences. Does lower average income of older people indicate disadvantage, or should the focus be on transfers from younger to older people? Is the health difference an indication of disadvantage for the elderly, or are they actually advantaged and overly supported by the health services after ‘controlling for’ age-related health differences? Are working age people disadvantaged by putting in many more working hours per week, or are they advantaged by having higher income from work (should we focus on effort or reward)?
These points do not mean that assessments of advantage and disadvantage are of no use. They are important for any decision making in that this involves comparison of the desirability of available options. The points do highlight the nature of the framing underpinning the comparisons, however. Consequently they can be used to critically assess the decisions and perhaps to identify what would otherwise be ‘unanticipated consequences’.
It is inevitable that we undertake analyses based on groupings, if only as a result of the constraints of classifications through the use of language and the nature of quantitative data. There is a danger that this leads us to focus too heavily on these measures. They may be used to construct performance indicators whereby the objective is to improve those indicators even when this may conflict with broader, but more nebulous, aims of policy.
International comparisons of school performance use influential measures such as the OECD’s Programme for International Student Assessment (PISA). This is described as “Measuring student success around the world”, although it focuses on performance in three core subjects and there is a danger that countries have a strong incentive to “teach to the test”. It is also not clear to what extent student performance is a result of the input from schools as compared to home background, additional tutoring, social pressure to do well in tests, etc..
We could similarly consider measuring country differences in performance in university economics courses. How are we measuring this performance? What skills are we assessing? How closely do the perspectives in the theory match the culture and values of the students in different countries? How broad are the courses (as in the three types of pluralism described on pages 2-3 of this Newsletter) and is this breadth being assessed?
In summary, comparisons over groups are an important component of the rhetoric of policy debate. They have implications for the choice and implementation of policies. They are central components of the framing that shapes our perceptions of society and the world around us. They shape what issues we see and do not see, and the values that we place on those issues. Quantitative analyses commonly take the measures as given, as the data that are available and suitable for the techniques of analysis. A more pluralist perspective would involve critical assessment of the concepts and construction of the measures, recognising additional reserves, qualifications and adjustments that must be made when attempting to use findings to make real world decisions.
From: p.9 of World Economics Association Newsletter 4(2), April 2014 https://www.worldeconomicsassociation.org/files/newsletter/Issue4-2.pdf