Monitoring and Evaluation

Monitoring and Evaluation

 

Monitoring is the routine tracking and reporting of high-priority information about a programme or project, its inputs and intended outputs, outcomes and impact.  In contrast, Evaluation is the rigorous collection of information about a programme and its outcomes to determine the extent to which they are achieving the stated objectives. The key distinction between the two is that evaluations are done independently to provide an objective assessment, and are typically more rigorous and involve more extensive analysis. The aims of both, however, are similar – to provide information that can help inform decisions, improve performance and achieve planned results.

 

M&E is about collecting, storing, analysing and finally transforming data into strategic information to be used by management to make decisions. It provides the strategic information needed to make good decisions for managing and improving programme performance, formulating policy and advocacy messages and planning programmes better.

 

The Global Fund requires a Monitoring and Evaluation system to ensure that:

  • Relevant, timely and accurate data is made available to national programme leaders and managers at each level of the programme and the health care system.
  • Selected high-quality data can be reported to national leaders.
  • The national programme can meet donor and international reporting requirements under a unified effort.

 

Essential Capacities required for Principal Recipients

 

Capacity Area and Scope

Capacities

Indicators

M&E Systems – to ensure that programme M&E is integrated into national M&E systems; there are comprehensive M&E guidelines and standard operating procedures; data collection and reporting forms flowing from service delivery point to central level; tools and guidelines for effective data quality assurance.

· To have in place M&E Systems which are linked or part of national M&E systems.      

· To have M&E tools to report accurate and quality assessed data from all levels of implementation.                      

· To have effective Management Information Systems.

· To upgrade and maintain hardware and software needs as required and identify any training needed.              

· To have in place an M&E Manual, and any Standard Operating Procedures for all implementers.

· To ensure an effective data quality assurance strategy, with clear procedures, reviews and verification processes.

· Timeliness of M&E reports.

· Availability and transparency of M&E data for management and partner review.

· Availability and use of M&E policies and procedures.

· Data quality assessments carried out and published within the past three years, using internationally agreed quality criteria, such as the Data Quality Assessment Framework.

 

M&E Plan – to have an effective M&E plan with clearly defined activities, timelines, financial resource estimates and responsibilities; clear and measurable indicators with baselines, targets, measurement methods, and data sources; comprehensive, adequate and feasible budget; and linked to the national strategy; with schedules for evaluations, surveys, etc.

 

· To have a national M&E Plan that is sufficiently detailed on how the National Programme and the Global Fund Grant(s) will be monitored and evaluated and results reported to the Global Fund and other donors.                      

· To ensure realistic and relevant indicator frameworks to allow routine monitoring of activities that are aligned to the goals and objectives of programmes.            

· To ensure a comprehensive and adequate M&E budget.                  

· To implement a M&E work plan with clearly identified activities; time lines for implementation; and who is responsible.                        

· To coordinate the M&E Plan under the guidance of a Technical Working Group.

· Availability of M&E plan.


· Frequency of M&E plan reviews and update 


· Level of integration of M&E plans into budgets and management plans.

· Level of integration of Global Fund M&E planning with national M&E planning processes.

· A national set of indicators with targets and annual reporting to inform annual health sector reviews and other planning cycles.

· Quality and comprehensiveness of indicators.

· Completeness in defining baselines and targets.

· Measurement methods identified.

M&E Analysis and Decision Making – to ensure effective data collection and analysis processes, including data repository, backup system, data compilation, processing and analysis; the ability to produce timely and accurate reports; M&E results that inform management decisions; and systems for integrating evaluation results into planning.

 

· To have systems for aggregating and analysing data, (including by age and sex for key populations).            

· To utilise Health Management Information Systems (HMIS) databases to improve programmatic reports.

· To put in place a HMIS data-assurance mechanism that can annually verify data.              

· To facilitate regular programme evaluations based on accurate data.

· To have systems to utilise evaluation outcomes for decision-making.

· To ensure strong epidemiological and health data at national and sub-national levels (including by age and sex for key populations).

· To prepare and share latest reports and evaluations including the disease status at the national and sub-national levels; and performance of each relevant disease programme and the heath system.                

· Quality and timeliness of health data.

· Comprehensiveness of health data capture: prevalence, incidence, qualitative social and behavioural data, disaggregated by age and gender.

· Availability of national Health Information System.

· Level of use of data collection systems for studies and evaluations.

· Level of integration of health data into management and forecasting reports and processes 


· Percentage of districts that submit timely, complete and accurate reports to national level.

· Percentage of evaluations completed per plan.

M&E Human Resources – to have in place adequate M&E human resource staffing and skill levels; adequate structures, roles and responsibilities for data collection, monitoring and reporting; effective oversight and review of sub-reporting entities; plans for addressing human resource gaps; and effective M&E stakeholder and technical working groups.

· To conduct a functional analysis to identify the Monitoring and Evaluation structures required.

· To utilise Human Resource policies and procedures with acceptable levels of compliance.

· To have a clear organogram and TORs for Monitoring and Evaluation specialists with clear accountability and reporting lines, roles and responsibilities, clear segregation of duties and, if necessary, review the mix of consultant vs. staff.                  

· To have in place a recruitment plan of key Monitoring and Evaluation positions to provide sufficient absorptive capacity, together with an oversight mechanism for the recruitment process, as well as the possibilities to enhance staff retention.

· To develop a training needs analysis and deliver training programmes for M&E specialists with opportunities to obtain recognised qualifications.

· Percentage turnover per year.

· Percentage or number of vacant posts.

· Timing from recruitment posting to staffed positions.

· Terms of reference for M&E team defined and clearly understood.

· Clear M&E management accountability structures from national and regional to district/local levels.

· Number of partners and stakeholders involved in M&E oversight.

 

 

Key Considerations when assessing Monitoring and Evaluation

 

Focus on M&E

Investing in strengthening a national monitoring and evaluation system is important as it will eventually save resources that may otherwise be spent in inefficient programmes or overlapping activities supported by different partners. As such, M&E should be carefully reviewed during capacity assessment exercises and supported in capacity development planning.

National Systems

Coordination of the overall M&E system across country and across donor requirements is an important first step in building a common M&E system that can meet a variety of needs. A strong unified M&E system may help to ensure that: relevant, timely and accurate data are made available to managers at each level of the health care system; selected high-quality data can be reported to national leaders; and the national programme can meet donor and international reporting requirements under a unified global effort.

Management Culture for M&E

Look for knowledge of and commitment to M&E and the M&E system among policy-makers, programme managers, programme staff and other stakeholders, as an indicator of information-based and results-based management.

Partnering for M&E

Partnerships among in-country and international stakeholders involved in planning and managing the national M&E system should be actively established and maintained. Stakeholder analysis and extent of their participation should be part of the M&E planning process. In addition, M&E should form an integral part of the Country Dialogue under the Global Fund Funding Model to enable evidence-based decision-making.

Disaggregated data collection

Data on age and gender and non-discriminatory accessibility and quality of services can be particularly critical in HIV prevention and care since population groups most vulnerable to infection continue to face technical, legal, and socio-cultural barriers in accessing health care. In many countries it can be challenging collecting data on most-at-risk populations due to the profound stigma and discrimination directed towards these key populations. Capacity to carry out qualitative behavioural studies and operations research should be assessed, to complement the data collection, data analysis and synthesis.

Data Quality

With the Global Fund's Performance-Based System, reporting based on a transparent assessment of programmatic data against time-bound targets secures further disbursements, grant renewals, and defines grant ratings.  Therefore the quality of programmatic data reported by PRs is a critical component of Performance-Based Funding.

Budgeting for M&E systems strengthening

As a general guideline the Global Fund recommends that 5-10 per cent of the national programme budget, including the Global Fund grant, be allocated for M&E systems strengthening.

Data Verification

The GF M&E framework includes various initiatives to strengthen M&E and quality of data including Data and Service Quality Assessment. Reports and recommendations from verification exercises and audits should be used to inform capacity development planning and implementation and bolster the grant outreach and performance.