Monitoring equity and research policy

Measuring under-five mortality: Validation of new low-cost methods
Rajaratnam JK, Tran LN, Lopez AD and Murray CJL: Public Library of Science Medicine 7(4), 13 April 2010

Although global under-five mortality is declining, this paper argues that it is unlikely that Millennium Development Goal 4 will be reached by 2015. The researchers used data about all children born and dead children extracted from 169 Demographic and Health Surveys covering 70 countries to develop four new methods to estimate under-five mortality. Their findings suggest that application of the new methods developed by the researchers could significantly improve the accuracy of estimates of under-five mortality based on summary birth history data. The researchers warn that although their methods can provide accurate estimates of recent under-five mortality, they might not capture rapid fluctuations in mortality such as those that occur during wars. However, they suggest, the two questions needed to generate the data required to apply these new methods could easily be included in existing survey programmes and in routine censuses. Consequently, systematic application of the methods proposed in this study should provide policy makers with the information about levels, recent trends, and inequalities in child mortality that they need to accelerate efforts to reduce the global toll of childhood deaths.

Monitoring the quality of public spending in the social sectors in developing countries: Lessons from public expenditure tracking surveys and other sources
Glassman A, Becker L and Bun C: Transparency and Accountability Project (TAP), Brookings Institution, January 2008

This paper reviewed 29 public expenditure tracking surveys (PETS) and related literature produced since the mid-1990s to identify common problems and lessons learned to improve the quality of public spending in the social sectors via civil society oversight and involvement. It examines ten of the most commonly reported problems in public expenditure management in the social sectors, like the limitations imposed by highly earmarked budgets, which do not allow for reallocation towards priorities, and public monies that do not reach the poor or are 'leaked' to unintended uses. Other problems include corruption, incomplete information and inefficient transfer and expenditure operations, health workers who face perverse incentives, and inefficient, low-quality and unresponsive health services that result in wastage. User fees and informal payments are also to blame for a lack demand from the population for health services. The paper argues that civil society organisations may have a critical role to play in improving the quality of social spending in developing countries. It recommends generating and disseminating information on flows of public spending and materials, advocating for and participating in organisational structures that incorporate citizens in oversight, preparing and disseminating citizen report cards, and carrying out PETS.

New model for HIV data takes into account new science
Odendal L: Health-e News, 23 March 2010

Researchers are designing a new model for determining the demographic impact of HIV and AIDS in South Africa. Modelers Leigh Johnson of the Centre for Infectious Disease Epidemiology and Research and Rob Dorrington of the Centre for Actuarial Research at the University of Cape Town have launched the ASSA 2008 model, which has replaced the ASSA 2003 model for estimating HIV prevalence, HIV-related deaths, the numbers of those in need ofanti-retrovirals (ARVs) and the impact of HIV interventions. According to Johnson, the reason the model needed to be updated was because the prevalence data projected was no longer correct because of the new data that emerged from South Africa’s antenatal HIV-prevalence survey. The survey increased the number of women who were tested for HIV and was thus more representative, although there was very little difference in the HIV prevalence results across the board from the ASSA 2003 and 2008 models. ASSA 2008 takes into account new epidemiological data to allow for more accurate projections of HIV prevalence and impact of interventions. It includes the ARV rollout data for up to the end of 2008. Because data shows that two-thirds of people starting ARVs are females, the ASSA 2008 model allows for different rates of ARV initiation in males and females, as well as for children and adults.

Overview of a formal scoping review on health system report cards
Brien SE, Lorenzetti DL, Lewis S, James K and William AG: Implementation Science 5(2), January 2010

Scoping is a novel methodology for systematically assessing the breadth of a body of literature in a particular research area. The objectives of this review were to showcase the scoping review methodology in the review of health system quality reporting, and to report on the extent of the literature in this area. A scoping review was performed based on the York methodology. The researchers searched fourteen peer-reviewed and grey literature databases, specific websites, reference lists and key journals for relevant material and also solicited input from key stakeholders. A total of 1,222 articles were included. These were categorised and catalogued according to the inclusion criteria, and further subcategories were identified through the charting process. Topic areas represented by this review included the effectiveness of health system report cards (194 articles), methodological issues in their development (815 articles), stakeholder views on report cards (144 articles), and ethical considerations around their development (69 articles). The review concluded that the scoping review methodology allowed for the convenient characterisation and cataloguing of the extensive body of literature pertaining to health system report cards. The resulting literature repository from this review could prove useful to researchers and health system stakeholders interested in the topic of health system quality measurement and reporting.

Peer review: A guide for researchers
Research Information Network: March 2010

This guide provides an overview of peer review and an assessment of the practice to date. It begins by discussing the main concepts related to peer reviewing of research papers before it outlines the peer review process generally, and specifically with relation to grant applications. It assesses the peer review process by addressing some of its shortcomings. For example, instances of malpractice and misconduct continue and, as reviewers themselves are fallible, peer review cannot provide a guarantee against the publication of bad research. So a number of published papers are retracted each year for a variety of reasons, and there is evidence that the number is rising. The core issues of transparency and subjectivity are discussed in the guide under the idea that peer review should foster fairness. However, financing the high costs of peer review mechanisms still proves problematic. On the positive side, major new opportunities in digital technology, such as the internet, have improved connectivity between stakeholders of the process.

What can we conclude from death registration? Improved methods for evaluating completeness
Murray CJL, Rajaratnam JK, Marcus J, Laakso T, Lopez AD: Public Library of Science Medicine 7(4), April 13, 2010

This paper systematically evaluates the performance of 234 variants of DDM methods in three different validation environments where the researchers knew or had strong beliefs about the true level of completeness of death registration. Using these datasets, it identified three variants of the DDMs that generally perform the best. It also found that even these improved methods yield uncertainty intervals of roughly one-quarter of the estimate. Finally, it demonstrates the application of the optimal variants in eight countries. In its conclusion, it notes that partial vital registration data in measuring adult mortality levels and trends still has a role, but such results should only be interpreted alongside all other data sources on adult mortality and the uncertainty of the resulting levels, trends, and age-patterns of adult death considered.

Are the investments in national HIV monitoring and evaluation systems paying off?
Peersman G, Rugg D, Erkkola T, Kiwango E and Yang JM: Journal of Acquired Immune Deficiency Syndromes 52(S2): S87-S96, December 2009

Globally, concerted efforts and substantial financial resources have gone toward strengthening national monitoring and evaluation (M&E) systems for HIV programmes. This paper explores whether those investments have made a difference in terms of data availability, quality and use for assessing whether national programmes are on track to achieve the 2015 Millennium Development Goal of halting and reversing the global HIV epidemic. It found a marked increase in the number of countries where the prerequisites for a national HIV M&E system are in place, as well as in human resources devoted to M&E at the national level. However, crucial gaps remain, such as available M&E data and data quality assurance. The extent to which data are used for programme improvement is difficult to ascertain. There is a potential threat to sustaining the current momentum in M&E, as governments have not committed long-term funding and current M&E-related expenditures are below the minimum needed to make M&E systems fully functional. Nonetheless, essential data gaps will need to be filled urgently to provide quality data to guide future decision making.

Implicit value judgments in the measurement of health inequalities
Harper S, King NB, Meersman SC, Reichman ME, Breen N and Lynch J: Milbank Quarterly Volume 88, Number 1, 2010

This study questions the assumption that the measurement of health inequalities is a value-neutral process that provides objective data that is then interpreted using normative judgments about whether a particular distribution of health is just, fair or socially acceptable. The study discusses five examples in which normative judgments play a role in the measurement process itself, through either the selection of one measurement strategy to the exclusion of others or the selection of the type, significance, or weight assigned to the variables being measured. Overall, it found that many commonly used measures of inequality are value laden and that the normative judgments implicit in these measures have important consequences for interpreting and responding to health inequalities. Because values implicit in the generation of health inequality measures may lead to radically different interpretations of the same underlying data, the study urges researchers to explicitly consider and transparently discuss the normative judgments underlying their measures. Policymakers and other consumers of health inequalities data should pay close attention to the measures on which they base their assessments of current and future health policies.

Systematic review of methods for evaluating healthcare research economic impact
Yazdizadeh B, Majdzadeh R and Salmasian H: Health Research Policy and Systems 8(6), 2 March 2010

The economic benefits of healthcare research require study so that appropriate resources can be allocated to this research, particularly in developing countries. This study took the form of a systematic review to identify the methods used to assess the economic impact of healthcare research, and the outcomes. The initial search yielded 8,416 articles, of which 18 articles were included in the analysis, as well as eleven other reports. It found that the outcomes assessed as healthcare research payback included direct cost-savings, cost reductions in healthcare delivery systems, benefits from commercial advancement, and outcomes associated with improved health status. The study found that different methods and outcomes can be used to assess the economic impacts of healthcare research. However, none of the research from low- and middle-income countries had evaluated the economic return of research. The authors recommend a consensus on practical guidelines at international level in order to build capacity, arrange for necessary informative infrastructures and promote necessary skills for economic evaluation studies in developing countries.

Tracking progress in advocacy: Why and how to monitor and evaluate advocacy projects and programmes
O’Flynn M: International NGO Training and Research Centre (INTRAC) M&E paper 4, October 2009

This paper begins by introducing the scope of and rationale for engaging in advocacy work as part of development interventions. It notes that effective advocacy can prove to be a very powerful and empowering strategy, and contrasts this with ineffective advocacy strategies, which can be hugely wasteful of time and resources and leave stakeholders feeling bewildered, disempowered or uninterested. It then focuses on the issue of monitoring and evaluating (M&E) these efforts – offering reasons why and when these processes should be planned and implemented, what’s involved, and who should be engaged in the process. The paper discusses organisational approaches to M&E of a number of agencies in the development sector, like Oxfam and the Institute of Development Research (IDR), and offers tools and methods for M&E of advocacy projects in international and national non-governmental organisations.

Pages