Skip to main content

RPI versus CPI – The Definitive Account

By August 15, 2011No Comments
The UK government has recently changed the basis on which it calculates increases to benefits and pensions. It used to base them on the RPI, the Retail Price Index. Now it bases them on the CPI, the Consumer Price Index. What is the difference, and what does the change mean? 
It is a complex subject, and even experienced economists have asked the Royal Statistical Society for a definitive account of the changes and what they imply, and whether or not they are justified. Here we provide that definitive account.
It is not for beginners. For those who want a basic introduction, see the accounts by Andrew McCulloch that we published here and here back in May. Nor is it short. It is several times the length of our usual website pieces. Nevertheless we post it here; we believe that financial professionals, economists, analysts and others may find it useful, now and to refer to in the future. It is written by Jill Leyland, who is the Chair of the RSS National Statistics Working Party.
Julian Champkin, Editor, Significance website.
The RPI and CPI Issue
In 2010 the Royal Statistical Society raised a number of concerns about the Consumer and Retail Price indices with the UK Statistics Authority1. Against the background of the government’s decision to switch the uplifting of a number of benefits, pensions and, in future, tax bands from the Retail to the Consumer Price Index, the ensuing debate has evoked considerable interest. This note is produced at the request of a number of people who want a guide to a complex subject and who are keen to know if the issues raised are being addressed. It does, of course, reflect the RSS view of the issue.
1. Background
Two main consumer price indices are currently compiled in the UK: the Retail Price Index (RPI), and the Consumer Price Index (CPI) which elsewhere in the EU is normally called the Harmonised Index of Consumer Prices (HICP). Each has a limited number of variants (for example RPIX excludes mortgage interest payments; CPIY excludes indirect taxes).
The RPI was first compiled in the 1950s (with data going back to 1948), replacing some earlier indices. The HICP/CPI was introduced in the 1990s following the need within the EU for a consumer price index on harmonised definitions (it is currently used as the measure of eurozone inflation). Its coverage and methodology are on agreed EU principles to ensure comparability between countries. The UK is obliged under European law to compile and publish the HICP/CPI; it is not obliged (as far as I am aware) to use it for any particular purpose.
In 2003 the government decided that the Bank of England’s Monetary Policy Committee should use the HICP as its inflation target instead of RPIX. The then National Statistician decided that the index should be renamed the Consumer Price Index. The change of name is not a practice followed by all EU countries: France, Germany, Italy, the Netherlands, Spain and Sweden, at least, all use the name “consumer price index” for their national price index and use HICP for the harmonised index.
It was announced in the June 2010 budget that tax credits, certain benefits and public sector pensions were to be uprated from April 2011 in line with CPI rather than RPI. The 2011 budget extended this to tax bands from April 2012 (although some thresholds will remain linked to RPI).
2. The problem – and its causes
RPI usually shows a higher rate of inflation than CPI. This is a problem in itself but is particularly concerning when the indices are used for uprating pensions, benefits, wages, commercial contracts etc due to the cumulative effect over the years. If we look back over the 15 years since CPI was introduced, the cumulative inflation rate shown by RPI since 1996 is 53.6% while that for CPI is 35.6% 2. A notional private pensioner who retired in 1996 and whose pension had been uplifted by RPI would today be 13% better off than a notional person starting on a similar pension uplifted by CPI. Thus the coalition government’s decision to replace the RPI by the CPI for certain uses has highlighted the problem of the differences between the two indices.
There are two main reasons for the difference in inflation rates: the coverage of the two indices; and the way they are calculated.
Coverage. Both indices notionally cover a basket of goods and services bought by households with the components weighted according to their average share of household expenditure. They are compiled, essentially, from the same basic information on individual prices with weights and the basket’s composition updated annually. However, there are some important differences in the basket of goods and services covered by the two indices, while the RPI excludes some households at the top and bottom of the income scale and also purchases by people not living in households.
The best known coverage difference is that the CPI excludes most owner occupier housing costs while the RPI includes mortgage interest payments and house depreciation. But this is not the only factor. Council tax, vehicle excise duty, TV licences are among elements excluded from the CPI which also includes spending by overseas residents while visiting the UK. The RPI also has one or two exclusions such as university student accommodation costs and stockbroker and unit trust charges; these result essentially from the limitations on its population coverage.
The reason for many of these differences in coverage reflects the original purpose and aim of the two indices. While there was always an element of recognition in the RPI that it was used as a guideline for uprating wages, pensions etc, the CPI was firmly based on macroeconomic principles. Thus it is consistent with national accounts and excludes elements considered as taxes (with the notable exception of VAT). It also includes spending by foreign residents visiting the UK for similar reasons. Insurance premiums have a lower weight than their share of household spending. Conversely, council tax, vehicle excise duties and TV licences are included in the RPI as being essentially payments for services rendered, it excludes spending by foreign residents and insurance is weighted according to the relative importance of premiums.
While the macroeconomic principles underlying the CPI make it suitable for certain purposes (eg a monetary policy target) its coverage makes it less suitable for uses more closely related to household budget pressures such as wage and salary reviews, pensions uplift etc.
Owner occupier costs are primarily excluded from the CPI due to a lack of agreement among EU countries as to how they should be treated (an issue exacerbated by differences in national housing markets). The UK has a Consumer Prices Advisory Committee which has been considering different approaches to measuring owner occupier housing costs and has recommended two possible alternatives. However, the Committee’s deliberations (see its 2010 annual report) have clearly been along macroeconomic lines and if implemented will exacerbate the problem outlined in the previous paragraph.
Appendix 1 gives more details of coverage differences.
Calculation differences. The coverage differences by themselves would have resulted in the RPI showing a higher inflation rate most of the time during the past fifteen years – but not all of it. However, the calculation difference, known as the formula effect, inevitably – mathematically – means that the RPI will show a higher inflation rate. Changes introduced in 2010 have increased the difference; for the latest five months published (February to June2011) the formula effect on its own would have accounted for around one percentage point in the annual inflation rate (current month compared to corresponding month of the previous year).
There are two main stages in calculating a price index once individual price quotes for many items have been collected. First, the different quotes for individual items have to be averaged to produce an average price (or growth in prices) for that item. Second, the average prices for each item have to be weighted together to provide the overall index or indices. It is the first stage where the differences lie. The UK’s treatment at the second stage is similar for the two indices; it is not controversial (at least in the view of the RSS) and reflects best international practice.
However, at the first stage of calculation the RPI uses arithmetic means while the CPI uses mainly geometric means (arithmetic means are used for a minority of items). (To take the geometric mean of n items multiply them together and then take the nth root.) The decision as to which type of mean to use is highly complex and depends, inter alia, on assumptions made about consumers’ behaviour in relation to price changes. Internationally, the recent trend has been for countries to change to using the geometric mean as this allows for the fact that consumers switch their purchases, to some extent, to brands or varieties which become relatively cheaper whereas the arithmetic mean does not allow for this. On this basis it can certainly be argued that the RPI overestimates inflation as it does not allow for this reaction by consumers. The government and the Bank of England believe that this is the case. However, this argument has been challenged in some quarters, partly as it assumes that consumers make no change in their overall spending on an item when prices change, partly as it assumes that consumers will not only switch to cheaper brands or varieties of the same item but also that they will switch from one retail outlet to another, irrespective of the distance between them, and partly as it has been argued that the geometric mean does not reflect behaviour in cases where price changes are demand driven (eg a reaction to changes in consumer taste) rather than supply driven (eg one manufacturer being able to produce more cheaply than another)3. Also, some items, such as ladies' dresses, cover a range of goods so wide that it cannot be assumed that purchasers would be willing to switch to some of the other goods included in the item.
A peculiarity of the UK is the size of the formula effect. When other countries (eg USA, some other EU countries) have changed calculation systems from arithmetic to geometric means, reported differences in annual inflation rates have been around 0.1 to 0.2 percentage points, substantially lower than the size of the UK effect. The reasons for this are complex4 but one major factor is the use by the UK of the “average of relatives” version of the arithmetic mean (which averages the changes in the prices of individual quotes between the base and current period as distinct from calculating the change in the average of quotes in the base and current periods) coupled with the greater heterogeneity allowed within individual items of certain categories of expenditure. The changes introduced in 2010, which increased the size of the formula effect, were driven primarily by changes to the practices in the collection of prices for clothing and footwear. This is explained in the information note published by the ONS in early 2011 ( Clothing and footwear is the largest contributor overall to the formula effect. In December 2010, for example, clothing and footwear contributed 0.51 percentage points of the total formula effect of 0.86 percentage points.
The RSS does not have a view on whether the arithmetic or geometric mean is the better approach but it does consider the issue a major concern.
3. A measure of inflation vs a “cost of living” index
One of the key issues in devising a consumer price index is the extent to which it is supposed to measure general inflation pressures as distinct from being a “cost of living” index. The term “cost of living” is confusing since it can be taken as referring to basic necessities (in itself an ill-defined concept) or, more broadly, to all items included in the household budget. In general, official price indices are supposed to represent general inflation pressures (for example in the UK the former Retail Prices Advisory Committee specifically rejected the notion that the RPI should be a cost of living index) but the USA is a notable exception. The CPI/HICP is firmly based on the notion of being a measure of general inflation pressures but, as has been seen, this has resulted in its coverage being less suitable for certain purposes.
4. A complication
The retail price index has been used to uplift the value of inflation linked government bonds. Until July 2002, such bonds had a clause in their prospectus which, if the calculation of the RPI was altered in a way deemed by the Bank of England to be “materially detrimental” to the holders of index linked bonds, would enable bond holders to demand immediate repayment at uplifted par value. The potential impact on government finances, should this occur, has been and is a severe constraint on making changes to the RPI. (But, depending on one’s assumptions about investor behaviour, it could also be noted that if the RPI, as some would argue, overstates inflation, then holders of index-linked bonds may be over-compensated for inflation at tax-payers’ expense.)
5. What should be done?
Towards the end of 2010 the UK Statistics Authority (UKSA) published the results of its assessment5 of the CPI and RPI along with a report on communicating inflation. The Assessment report, like most of its assessments, imposed both a number of requirements on the ONS regarding the indices as well as making some recommendations while the report on communicating inflation made some further recommendations. See for a summary. The UKSA and RSS views have some synergy but also have some differences.
In the view of the Royal Statistical Society, the following steps need to be taken:
  1. The ONS needs to have a better understanding of the needs of all users of the CPI and RPI, not just those who use it for macroeconomic purposes. To facilitate this, a broadly-based user group should be established. The ONS has agreed to this and is keen to have it set up. We are aiming to have an initial meeting in late Autumn.
  2. Based on this better understanding of user needs, an index, or family of indices, should be established which reflects the household budget (a cost of living index in the broad sense); a family of indices would enable the experience of different population groups to be reflected (as seems needed within the limits of practicality). Partly in response to UKSA requirements, the ONS is preparing an article on the CPI and RPI. Among other topics it will look at the issues which need to be addressed if a “cost of living index” were to be compiled.
  3. The calculation procedure at the first stage of aggregation needs to be fully reviewed by the ONS, drawing on the body of international research and the insights that can be provided by relevant outside experts. This could include tapping into major retailers’ knowledge of consumer behaviour. The aim would be to develop calculation procedures for different items that are most statistically appropriate, with particular attention paid to reflecting consumer behaviour as closely as possible, and which are coupled with appropriate price collection methods. The results need to be incorporated into the new index. The ONS is now actively examining this issue, initially for clothing. The RSS welcomes this and the fact that some RSS members will be involved.
  4. The RPI would need to be improved in line with the two preceding points subject to the constraints of the index-linked bond issue.
  5. The CPI will need to be calculated and published in line with EU requirements but consideration should be given to calling it again the Harmonised Index of Consumer Prices, in line with the practice of other major EU members, to avoid the current confusion.
It is recognised that it is for the government to decide which price index to use for official purposes. Even if the ONS produces a more appropriate index, the government will not necessarily adopt it. But this is not a reason not to produce more appropriate indices which in any event can be used by the private sector.
Jill Leyland
Vice President, Royal Statistical Society and Chair, RSS National Statistics Working Party
August 2011
Appendix 1: Coverage differences
The CPI covers all private and institutional households; it also includes spending in the UK by foreign residents. The RPI excludes the top 4% of households by income and pensioner households where ¾ or more of income comes from the state. It excludes institutional households and foreign residents. These restrictions on the RPI perhaps reflect the notion that it should look at inflation as experienced by the majority of the population, the fact that it was (and still is) often used as a benchmark in wage negotiations, pensions uplift, business contracts etc6 . 
The CPI does not cover most owner occupier housing costs; it excludes mortgage interest payments and house depreciation. It also excludes council tax, vehicle excise duty, TV licences and trade union dues. It includes spending by foreign residents while in the UK, foreign students’ university tuition fees, university accommodation fees, unit trust and stockbroker charges all of which are excluded from the RPI. The RPI includes mortgage interest payments and house depreciation, council tax, vehicle excise duty, TV licences and trade union dues. It excludes university accommodation fees, spending by foreign residents, foreign students’ university tuition fees, unit trust and stockbroker charges.
The best known difference between the two indices is the exclusion of most owner occupier housing costs from the CPI. Their exclusion is primarily due to lack of agreement between EU countries on how they should be dealt with. (There are both theoretical and practical issues underlying this. Since house ownership provides not just “current” benefits – somewhere to live – but is also a capital investment some could argue that it does not automatically have a place in an index which notionally looks at “current” costs. There are also practical difficulties in deciding what is both appropriate and statistically measurable exacerbated by differences in national housing markets.) When the HICP was introduced a number of items were excluded due to absence of agreement between EU countries on how they should be measured. Most of these differences have long been settled but how owner occupier housing should be covered remains outstanding.
As has been seen the exclusion of owner occupier housing costs is not the only coverage difference. Some of the other differences are due to theoretical considerations. The CPI/HICP is based on economic principles – it is consistent with national accounts principles and uses the Standard International Classification System for its components (whereas the RPI has its own classification system). Council tax, vehicle excise duties and TV licences are excluded from CPI/HICP as they are taxes (although the VAT element of prices is included) whereas they are included in the RPI are they are considered to be implicitly payments for services. Spending by foreigners in the UK is included in the CPI as it is intended to measure general inflation pressures in the economy. I am not aware of why stockbroker and unit trust charges, or university accommodation fees, are excluded from the RPI although it could perhaps be argued that in the past they were mainly incurred by the top 4% by income of the population.
While not strictly a coverage issue, the difference in treatment of insurance is worth mentioning here. In the RPI insurance premiums are included and weighted according to their share of the typical household budget. However, the CPI/HICP notionally covers the net cost of insurance – premiums less claims received (ie the net income to insurers or the cost of the insurance service). Due to the difficulty in measuring claims paid out, premiums are used as an indicator of this, as they are in the RPI, but the weight is based on the net cost so is a much smaller share of the overall basket.
  1. The full correspondence can be seen at—correspondence/correspondence/index.html. See letters between Sir Michael Scholar and Professor David Hand of August, October and December 2010 and letters between Sir Michael Scholar and Jill Leyland of February and April 2011.
  2. April 1996 to April 2011.
  3. See in particular Ward et al, 2011, and Courtney, 2011
  4. See Fenwick, 1999
  5. The 2007 Statistics and Registration Service Act obliges the UKSA to carry out periodic assessments of all national statistics against the Code of Practice.
  6. The original index was compiled during the first world war and was designed as a compensation index to help protect ordinary workers from the rapid price increases experienced at that time.
Further Reading
  • The New Inflation Target: the Statistical Perspective, Office for National Statistics, December 2003
  • The impact of choice of base month and other factors on the relative performance of
  • different formulae used for aggregation of Consumer Price Index data at an elementary
  • aggregate level, David Fenwick, 1999
  • Consumer Prices Advisory Committee, 2010 Annual Report to the UK Statistics Authority, 2010
  • How Relevant are the United Kingdom’s Official Measures of Price Change? Michael Ward, Derek Blades and Carol Carson, in Statistical Journal of the IAOS, Vol 27 (2011) p.31-37
  • CPI and RPI Differences: The Formula Effect and the Identification Problem, Mark M Courtney (Forthcoming, available on request)
  • CPI and RPI: increased impact of the formula effect in 2010, Office for National Statistics, Information Note, 2011
  • Consumer Price Indices, Technical Manual, 2010 Edition, Office for National Statistics
  • Consumer Price Index Manual: Theory and Practice. Published in 2004 by: International Labour Organization/International Monetary Fund/Organisation for Economic Co-operation and Development/ Statistical Office of the European Communities/United Nations/The International Bank for Reconstruction and Development/ The World Bank
  • The measurement of consumer price inflation is highly complex. The Ottawa Group is an international forum, meeting every two years, for specialists in the subject to share views and exchange experiences. Papers presented at the forum’s meetings can be accessed via its website

Leave a Reply