Bangladesh: External evaluation of Plan International UK’s DEC funded response to the Rohingya refugee crisis in Bangladesh

An external evaluation of Plan International UK’s DEC funded response to the Rohingya refugee crisis in Bangladesh

1. Background and Introduction

Since extreme violence erupted in the Rakhine State of Myanmar in 25 August 2017, an estimated 702,160 Rohingya men, women and children have crossed the border into Cox’s Bazar, Bangladesh to find safety. Combined with pre-existing numbers of Rohingya population that fled earlier in previous years, the total Rohingya population in Cox’s Bazar amounts to approximately 914,678 as of 24 May 2018.^[1] The situation also negatively affected the host community. Cox’s Bazar is one of Bangladesh’s most vulnerable districts ranking 62^nd from the bottom on the Composite Deprivation Index (CDI). The rapid and massive increase in the Rohingya population, concentrated in Ukhia and Teknaf Upazilas in the south of Cox’s Bazar district, has impacted the host community’s market access, labour competition, deforestation, and inflation, resulting in increasing discontent between host and Rohingya communities.

55% of the Rohingya refugee population is under 18. 3.6% are infants below one year, 15% are children under five and 36% are children between five to seventeen years old. Due to children’s physical development, age, and relative vulnerability, they are considered one of the most at-risk groups. Compounding their vulnerability, the Rohingya population fleeing Myanmar have comparatively low social indicators, especially in health, education and nutrition, having suffered decades of oppression and marginalization in their native country. Women and girls experience social restrictions significantly more than men and boys: social and cultural norms dictate traditional gender roles, where women and girls, especially adolescents, are restricted to their homes, are often not able to realise their rights; and are underrepresented in decision making.

Plan International’s initial response focused on providing life-saving and immediate WASH and Protection needs to the most vulnerable arriving in Cox’s Bazaar, focusing in the Balukhahli Mega camp. This included the installation of gender, age and ability friendly latrines, bathing spaces and distribution of WASH and other NFI kits responding to the specific needs of boys, girls, adolescent girls, women and men. The Protection response has focused on ensuring safe spaces for children, particularly adolescent and young girls, and provision of gender and age appropriate protection services including case management, family tracing and reunification, and enhancing community-based child protection mechanisms. The response has now phased out WASH and is focusing on implementing child protection and education in emergencies programming.

The DEC Phase 2 response began in May 2018 is contributing to the Child Protection programming. It aims to address the protection needs of vulnerable boys and girls; and build their and the community’s capacity to manage existing risks. The key components are the establishment and running of both static and mobile safe spaces, provision of Psychosocial and life skills support, and the provision of quality case management services as identified to protection cases. Plan International has placed emphasis on ensuring its response is responding to the needs of adolescents, particularly girls, due to the huge risks they face, and the recognition of the need to provide adolescent specific services. Activities will be completed in June 2019.

1. Rationale of the evaluation

This evaluation is planned to allow Plan International to record the learning around the implementation of its DEC funded emergency programming in response to the Rohingya Refugee Crisis in Cox’s Bazaar, Bangladesh and analyse successes, areas for improvement, accountability strategies, and recommendations for future emergencies, particularly around responding to the needs and priorities of adolescents and youth. The objectives and key questions have also included references to the Core Humanitarian Standards to ensure the evaluation considers the extent to which the response is aligning to the 9 commitments. This evaluation is also part of the DEC’s Accountability system. The evaluation will therefore also be shared with the DEC, published on Plan UK’s website, ALNAP and used to demonstrate the impact of Plan International’s and the DEC’s work for this appeal.

The findings from the evaluation will primarily be used:

By the Plan Bangladesh project team and stakeholders

· to inform improvements in delivery and support sustainability and transition strategies;

· to learn and document lessons from the project for replicating good practices and/or taking up approaches and activities with evidence of success

· to share findings with the affected communities we worked with for the Rohingya Refugee Crisis emergency response.

By the Plan UK project team:

· to assess and demonstrate accountability for the funding received to communities we work with and the Disasters Emergency Committee;

· to learn and document lessons for replicating good practices and use findings to inform future responses

· to generate a body of evidence that supports scale up and sustainability of the activities /benefits delivered by the project;

By Plan Bangladesh and UK

· to continuously evaluate alignment to the Core Humanitarian Standards and progress towards Improvement Commitments;

· to inform continuous improvement on emergency responses and share lessons

By the DEC team

· to share findings, information and impacts of the Royingya Refugee Crisis Appeal to external stakeholders and demonstrate accountability to its supporters

1. Objectives of the evaluation

The broad objective of the evaluation is to understand the achievements of the project made so far in relevance with the expected outcomes of the project. Therefore the specific objectives are:

1. To assess the quality of achieved outputs, outcomes and results of the project (annex 2) throughout the period of implementation and its continuous improvement against the key evaluation criteria (relevance, timeliness, efficiency, effectiveness, sustainability, accountability, impact etc. (annex-1) considering how the response is aligning to the Core Humanitarian Standards(CHS) including analysing the reasons behind their successful or unsuccessful achievement.

2. To assess the effect of the project, including intended and unintended impacts and adaptation to the changing context, on the target communities and their environment; particularly considering the effects on adolescent and young girls and the impact of project strategies in social, economic and environmental terms (CHS commitments 2 and 3)

3. To evaluate the extent to which the programme encouraged meaningful participation of the different groups and communities it worked with- making particular reference to the opinions of young girls and young boys - throughout the programme cycle and identifying areas for improvement including in: effectively communicating with communities, encouraging and using feedback, supporting community decision making (CHS commitment 4), and responding to the priorities, needs and culture of the communities and groups the response is working with (CHS commitment 1).

4. To identify key good practices and key lessons learnt, including how these have been used throughout the programme to improve its delivery (CHS commitment 6), and make recommendations for future improvement of similar programs based on evaluation findings.

5. Methodological guideline

The consultant/consulting firm is expected to develop an appropriate methodology to meet the objectives of the evaluation. However, appropriate triangulation in data collection methods is anticipated in the proposed methodology as per need. The methodology and relevant instruments should be adjusted in consultation of Plan International and finalised before implementation. The evaluation is expected to follow the OECD-DAC criteria as a framework integrating the Core Humanitarian Standards within it. The evaluator(s) will be also responsible for assessing the programme, financial and management issues. Therefore, the consultant/consulting firm is expected to propose a methodology for how they will cover the key questions found in Annex 1 “Framework for Questions of Evaluation”.

1. Scope of Work

The assignment will preferably include, but not limited to:

· Specifically assess the DEC Funded Phase 2 project. While this will have to be assessed in the wider context of Plan International’s overall response, it is not expected for the evaluation to cover other projects.

· The evaluators will also not be expected to draw conclusions on the Plan International response strategy or approaches. The evaluation team however will be given access to information on the overall response for context and understanding the project as part of the wider programming. It is expected that assessing the key questions will involve how the project worked within the response such as coordinating with other actors and projects, contributing to overarching objectives, using response-wide beneficiary feedback mechanisms and general Plan systems**.**

· Develop appropriate methodology, including determining the appropriate sample size, and data collection instruments in consultation with Plan International Bangladesh and finalise the instruments by pre-testing.

· Administer data collection in the study area that includes supervision, cross validation, quality control and data transcription. The sampling criteria and sampling methodology will be clearly described in the final report of the consultant.

· Engage qualified enumerators as necessary, train them on quantitative and qualitative data collection and quality control.

· It is important in gathering data to ensure safe and meaningful participation of different groups, including women and young and adolescent girls through gender, age and inclusion friendly approaches.

· Data management and analysis in terms of coding, computer entry, cleaning, transcription and analysis as per the study themes in the objectives.

· Prepare a study brief in both Bangla and English in consultation with Plan International Bangladesh.

1. Expected competencies

Expected competency of the consultant/consulting firm includes:

· Expertise in conducting quantitative and qualitative study in the area of evaluating sudden-onset emergency response and recovery projects, humanitarian response programming in Bangladesh, child rights and child protection, children’s participation and inclusion, Gender Sensitive programming

· Strong analytical, facilitation and communication skills, especially with regard to working with young and adolescent girls and boys.

· A minimum of 4 years’ experience in carrying out impact evaluations, demonstrable relevant academic and practical experience in qualitative and quantitative research methodology, evaluation design and implementation; experience undertaking similar evaluations for reputed national and international organizations.

· Knowledge of child protection procedures when working with children, youths, and adolescent girls in particular, as well as experience with implementing child and/or youth friendly evaluation methodologies.

· Good spoken and written communication skills in Bangla and English.

· The Knowledge in local dialect of Chittagong would be an added advantage.

· Should have clear understanding on the challenges of administering data collection in the local context of Cox’s Bazar specifically in Rohingya Refugee Crisis areas.

· No history of violation of child rights.

· Proven experience of using participatory tools, appropriate for different vulnerable groups including boys and girls especially adolescents, as a means of data collection for project evaluation Experience of effective interaction with local and national organizations, government departments, and marginalized communities in rural and urban areas.

· Capacity to provide necessary training to human resource for carrying out data collection, quality control (reliability) and data entry management.

· Bangladeshi and female consultants are welcome and encouraged to apply

1. Deliverables and timeframe

The whole program evaluation process is expected to take around 30 working days including: preparation, field work with partners and stakeholders, and report drafting, feedback and finalisation. The expected date for commencing work is around the 1^st February 2019 and will follow the agreed work plan and deadlines based on that submitted with the application and discussed between the selected consultant/consulting firm and the Plan Evaluation Management Team. It may be possible for the evaluation to take place across more than one calendar month depending on agreed dates for travel and allowing input from the Plan International teams on the draft inception and final reports. The final report must be submitted to Plan International no later than 10 April 2019. Follow up meetings will be held time-to-time between the contracted consultant/consulting firm and Plan Evaluation Management Team.

The final report should have the following structure:

· Title page

· Acknowledgments

· Executive summary

· List of acronyms

· Table of contents and lists of figures and tables

· Introduction

· Background

· Methodology

· Results (it should be organised as per study objectives)

· Discussion and conclusions with recommendations.

· References

· Annexes

The evaluation report should be prepared on A4 paper size in Calibri font size 11 and no longer than 35 pages (excluding appendices).

The production of the evaluation report will be the liability of the evaluator(s) covering all the evaluation questions, objectives and areas outlined in this ToR. Plan Bangladesh will be responsible for coordinating the evaluation exercise. During the research process, the evaluator will keep the Evaluation Management Team up to date and agree on changes to the methodology where appropriate. The evaluation report shall be produced in English language and should be simple in expression (jargon free). The Executive summary should include a brief description of the project, a brief section on methodology but with most text for conclusions and summary of recommendation. The evaluator(s) will be liable to submit an electronic version of the evaluation report in PDF Version by the agreed deadline and should also submit all survey data, transcripts of FGDs, KIIs, photographs etc. The final product should be fully referenced, with findings clearly linked to evidence.

The consultant/consulting firm shall produce the following deliverables:

· An inception report containing detailed proposed work plan and evaluation methodology based on consultations with the Plan team and inception meeting.

· An electronic version of the evaluation report in PDF Version by the agreed deadline.

· Study instruments pretested, finalised and printed both in Bangla and English.

· Draft report containing detailed findings, well blended qualitative and quantitative analysis on findings.

· Training of any staff or enumerators if required by the methodology

· Preparation of draft report, Validation workshop in country presenting initial findings, key recommendations and lessons to allow discussion and input from the team

· Final evaluation report including all of the above incorporating feedback from Plan along with all relevant appendices

· The final report should be delivered in acceptable English. The consultant will arrange for proof reading, if required to maintain the quality. The final report should be fully referenced, with findings clearly linked to evidence.

· Presentation of the key findings, lessons learnt and recommendations through a dissemination workshop online or in person as feasible

1. Mode of payment

The payment will be made in three instalments: upon contract signature, submission of the first draft report, and submission and acceptance of the final report.

1. Preparation of proposal

The proposal will be divided into two parts and should submitted by the 5^th of January 2019 in two separate files i.e. technical and financial. The technical part of the proposal should not exceed 7 pages and will contain the following:

· Detailed methodology of the study including interpretation of the TOR and initial sampling framework.

· Detailed timeframe (including dates for submission of first draft, dissemination of findings and final report).

· Account of experience of conducting survey and employing qualitative methods detailing how the meet the expected competencies.

· CVs of the team leader and key members of the study team which reflect relevant experience to conduct the study.

· A sample of an evaluation report for a similar project completed within the last 24 months (this will be treated as confidential and only used for the purposes of quality assurance) along with contact details for the responsible person from the organisation for whom the evaluation was done.

· Two references (including one from your last client/employer) to be contacted as needed

· Copy of VAT registration certificate (for consulting firm).

· Copy of valid TIN certificate and bank account detail.

The financial proposal should clearly identify, item wise summary of cost for the assignment with detail breakdown. The budget should not contain income tax as a separate head; it can be blended with the other costs as it will be deducted from the source. However VAT can be mentioned in the budget as per government regulation. The organisation will deduct VAT and Tax at source according to the GoB rules and deposit the said amount to government treasury. The consultant/consulting firm is expected to provide justified budget which is consistent with technical proposal. For non-Bangladeshi submissions, the financial proposal shall include details of ALL taxes liable to be paid or any exemptions.

Child Protection Policy

The individual shall comply with the Safeguarding of Children and Young People Policy of Plan International Bangladesh. Any violation /deviation in complying with Plan’s child protection policy will not only result-in termination of the agreement but also Plan will initiate appropriate action in order to make good the damages/losses caused due to non-compliance of Plan’s Safeguarding of Children and Young People Policy.

Annex 1: Framework for Questions of the Evaluation

The below are guiding questions which will be refined during the inception phase of the evaluation.

Effectiveness

· Have the activities been undertaken in a timely manner? Were objectives achieved on time?

· Quality of assessments: what was missing; what can we do differently next time?

· How appropriate and useful were the interventions and/ activities implemented?

· Were the beneficiaries, especially adolescent girls able to provide feedback throughout the different stages of programming? (CHS4)

· Was the feedback from beneficiaries able to be incorporated into the project design? Why/why not? and what were the enabling/ hindering factors for this? (CHS4)

· Have the alterations made during the project implementation had positive / negative effects to the achievement of the outputs & outcomes?

· What were the major factors influencing the achievement or non-achievement of the objectives?

· Has the provision of protection services followed the necessary quality standards and been delivered in gender and age appropriate ways?

· To what extent are staff expertise and competencies adequate to implement the response programme? Consider knowledge of the context, refugee rights and protection issues. (CHS8)

Efficiency

· Cost-benefit analysis / Has the scale of benefits been consistent with the cost?

· To what extent has the funding been utilized to directly assist beneficiaries - has project support and operational costs been reasonable (%) compared to entire budget?

· How well were the inputs (funds, people, materials and time) used to produce results?

· Were the strategies used, particularly the diverse mobile and static approaches, appropriate to achieve results intended?

Relevance

· Have the project objectives proven to be valid and appropriate in meeting the most pressing gender and age appropriate and priority needs of the affected population? If not, what should have been done instead? (CHS1)

· Has the project been consistent with the initial and changing needs and priorities of the intended beneficiaries (most vulnerable members of affected communities)?

· Has the project complemented and been compatible with government approach?

· Has the project approach or design changed to ensure continued relevance in and adaptation to a changing context? (CHS1)

· What factors have enabled/ hindered this adaptation to changes? (CHS1)

Sustainability

· Has the project managed to put in place systems to enable sustainability after donor funding ceases?

· What were the major factors which influenced the achievement or non-achievement of sustainability of the project?

· To what extent have long-term and inter-connected problems been considered when carrying out short-term activities? (CHS3)

· How has project impact on the environment been considered and how will it impact future opportunities in the area? (CHS3)

Impact

· What were the results achieved vs planned results and what was the reason for discrepancies?

· Has the project had any unforeseen positive and/or negative impacts (including on social, political, environmental and economic factors) which have influenced Plan or prompted changes in ways of working etc.? (CHS3)

· To what extent, and how, were negative impacts, including in all sectors from above, systematically anticipated, identified, and mitigated? (CHS3)

· Has the project been able to strengthen communities’ and local capacities? (CHS3)

· What are the differences the project has made to adolescent girls and boys and children targeted and the wider community?

· What was the influence of and complementarity with other factors?

Coordination and Communication

· How effective was the coordination/collaboration between Plan and coordination bodies, local stakeholders, government and other organisations?

· How gender, age and culturally appropriate and functional were Plan’s strategies for communicating with communities, community participation and closing feedback loops? Have they remained appropriate and functional through context changes? (CHS4)

· Did targeted young and adolescent girls, boys and the wider community feel safe and trusted the confidentiality when communicating with Plan and its relevant stakeholders? (CHS5)

· To what extent were the complaints mechanisms understood, relevant, trusted and appropriate to the context? (CHS5)

Coverage and targeting

· Was the selection criteria of beneficiaries appropriate? Was the selection criteria successfully and transparently implemented? Was the community involved in determining the selection criteria and well informed about the decisions? (CHS4)

· Did the project address the needs of intended beneficiaries in a consistent manner as per project design?

· Did the project consider age, gender, ability and culture appropriately in the project design and implementation?

Lessons, Replicability & Scale

· How has the project learned and used lessons across the team, response community and wider sector? (CHS7)

· What would be the main considerations and key lessons when replicating and scaling up/down this project design for future emergencies to ensure a better quality of response?

· To what extent has the programme applied previous learning of the Rohingya context, as well as lessons learnt in relation to refugee crises globally? (CHS7)

The technical and financial proposals should be submitted electronically to the email address: Planbd.consultant.hiring@plan-international.org with “**External Evaluation of Plan International UK’s DEC Funded Response to the Rohingya Refugee Crisis in Bangladesh**” as subject line**. Proposals submitted to any other email account except this and in hard copy will be treated as disqualified. Submissions after the deadline of the** 5^th January 2019** will be treated as disqualified. Two different folders i.e. technical and financial should be submitted into one zip folder with a covering letter. The proposals should be submitted in pdf format.

For questions or copies of the proposal logframe, please email Md. Towhidul Islam, Programme Monitoring and Evaluation Specialist, to the following email address: Towhidul.Islam@plan-international.org.