Review of the AusAID Cambodia Community Development Fund

FINAL REPORT

Rhonda Chapman

November 2007

CONTENTS

Executive Summary	iii

Introduction	1

Background	1

Approach and Methodology	5

Key Findings and Recommendations	6

Output 1	6

Output 2	10

Output 3	13

Conclusion	17

Annexes	18

Review Terms of Reference

CDF Funding Trends, 2004-2007

Program and Preparation for a Review Field trip to North-West Provinces

List of interviewees and interview notes

Review Workshop Program

Summary of Small Grant Schemes in Cambodia

Amended CDF Guidelines

Water and Sanitation Guidance Information and References

The Review Team

Team Leader	Rhonda Chapman, Independent Consultant

Team Advisor	Sun Boreth, Independent Consultant

Team Members	Lia Burns, Senior Program Coordinator, AusAID Cambodia

Le Minh-Nga, Program Assistant, AusAID Vietnam

Team Facilitator	Hieng Sokchea, CDF Program Coordinator.

Acknowledgements

This review is based on research and field work funded by AusAID Cambodia. Special thanks go to:

The staff of the NGOs, CCC, CDC and others who freely gave of their time to share insights, opinions and ideas about the CDF.

The review team for their assistance and support throughout the review and for many laughs over shared meals.

Mr Hieng Sokchea for logistical efficiency and for teaching us how to recognise a good rice crop.

Mr Yos Sinith for his ‘candid camera’ and keeping us safe on the roads.

ACRONYMS and ABBREVIATIONS

AER	[Royal Government of Cambodia] Aid Effectiveness Review
AusAID	Australian Agency for International Development
CBO	Community based organisation
CCC	Cooperation Committee for Cambodia
CDC	[Royal Government of Cambodia] Committee for Development Cooperation
CDF	Community Development Fund
CLTS	Community Led Total Sanitation
CPS	[AusAID Cambodia] Country Program Strategy
CSO	Civil Society Organisation
DFAT	Australian Department for Foreign Affairs and Trade
GPP	[CCC] Good Principles Project
HRGF	Human Rights and Governance Fund
NGO	Non-government organisation
INGO	International non-government organisation
LNGO	Local (Cambodian) non-government organisation
MRD	Ministry of Rural Development
MoWRAM	Ministry of Water Resources and Meteorology
M&E	Monitoring and Evaluation
NSDP	[Royal Government of Cambodia] National Strategic Development Plan
ODF	Open Defecation Free
PAF	[AusAID] Performance Assessment Framework
RGC	Royal Government of Cambodia
WSP	[World Bank] Water and Sanitation Program

About the Author

Rhonda Chapman is an independent consultant, specialising in NGOs, organisational and program development; monitoring and evaluation. At the time of the review, she was preparing her return home to Australia after an 18-month stay in Cambodia to conduct the field research for her PhD on NGO Effectiveness. During her time here she interviewed, met and worked with over 50 NGOs and CBOs working in many sectors and areas of Cambodia.

Executive Summary

The objective of the Community Development Fund Review mission was to consider and make recommendations on how the CDF can increase the effectiveness of its community development focus to strengthen the broader bilateral program with particular reference to the new Country Program Strategy and Performance Assessment Framework.

The review considered the CDF from the perspective of both AusAID Cambodia as the donor responsible for the management of a relatively small discretionary fund and how it can more effectively and strategically contribute to the achievement of the AusAID Cambodia Country Program Strategy.

It also considered the fund from the perspective of the organisations who receive funding from the fund. This included a mix of international and local non-government organisations that use CDF funding to complement and enhance larger projects, to fund small pilot initiatives and as a critical source of funding for core activities.

Recommendations 1 and 2 (p7-9) address the larger strategic and sectoral framework that informs the fund, using the draft AusAID Country Program Strategy as the main guide.

Recommendation 3 and 4 (p10-11) address the management of the fund – mechanisms to ensure the accountability and transparency of the NGOs receiving funding, as well as how the fund can be used to strengthen and support Cambodian civil society as part of AusAID’s broader good governance objectives.

Recommendation 5 (p12) addresses specific guidance for projects involved in water and sanitation projects, with a focus on enhancing community engagement and education as the basis for improving the effectiveness of such projects, and requirements to collaborate, coordinate or complement government initiatives in this area.

Recommendation 6 (p15) focuses on the monitoring and evaluation requirements for the CDF from the perspective of AusAID’s need to enhance the performance assessment of their programs. It recommends an approach to monitoring the CDF in order to collect information about its contribution to the AusAID Country Program Strategy that recognises the limitations of imposing new and complex processes or mechanisms onto NGOs and the capacity of AusAID to collect and analyse large quantities of information about small scale activities.

Underlying all the recommendations is the imperative to consider the CDF as a whole greater than the sum of its parts, and a fund that can have an impact on community development in Cambodia beyond its relatively small financial scale.

Introduction

The AusAID-Cambodia Community Development Fund is a small grants scheme for non-government organisations (NGOs) and public institutions in Cambodia managed as a discretionary fund by the Australian Embassy in Phnom Penh. Valued at approximately AUD500,000 per year, the CDF represents a small proportion of AUD33million country program managed by AusAID in Cambodia, however the fund’s reach and influence amongst the NGO sector in Cambodia is important.

Development agencies, governments and NGOs are increasingly concerned about understanding and assessing the effectiveness of their development activities. It is therefore timely to assess and review the CDF from the perspective of effectiveness and good development practice. The review is also an opportunity to reflect on progress made by the NGO sector in Cambodia in improving their standards of ethical and transparent practice, and to support the increasing capacity of the local NGOs as the building blocks of a strong civil society.

A purpose of the review was to assess the appropriateness of the guidelines, including eligibility and funding criterion, and the administration of the guidelines, in order to maximise the potential for the activities funded by the CDF to contribute to the strategic objectives of the AusAID Cambodia Country Program Strategy. It was also the purpose of this review to restructure the guidelines and provide guidance on the management of the fund, particularly in the way it is monitored and evaluated, in order to provide a framework for good development practice for the activities it supports. It was not the intention of this review to conduct a detailed financial audit or review of the management of project grants by recipients, nor an evaluation of the development outcomes of those projects. Nor was it the purpose of this review to conduct a detailed audit of the internal administration of the fund by AusAID Cambodia. The review also incorporated information about the Human Rights and Governance fund previously administered by DFAT and recently incorporated into the CDF.

At the time of the review the AusAID Country Program Strategy was itself under review. Therefore the recommendations made herein are based on draft indications of the final strategy objectives available at the time and may need to be reviewed in light of the approved final strategy. The recommendations have also been made based on retaining or enhancing current staffing arrangements for management of the CDF within AusAID Cambodia.

The CDF was last reviewed in 2000 and a range of recommendations were made about procedures and management. Since that review, a number of those recommendations have been successfully implemented – namely, the development of clear guidance for applicants, and developing a set of consistent procedures for the internal administration and management of the fund by AusAID.

Background

This section provides an outline of the context in which the CDF is managed and implemented. It is not the place of this report to provide an overview of the social, cultural and economic development context in Cambodia – that is addressed in other key documents such as the Royal Government of Cambodia National Strategic Development Plan 2006-2010 and the AusAID Cambodia Country Strategy Plan which inform the strategic framework recommended for the CDF.

The National Strategic Development Plan 2006-2010 (NSDP) is declared as the overarching, guiding and reference document for pursuing the prioritized goals, targets and actions of the Royall Government of Cambodia (RGC) across the range of sectors relevant to poverty and development.

The framework for RGC policy is the Rectangular Strategy for Growth, Equity, Employment and Poverty Reduction, adopted in 2004 and incorporated into the NSDP in 2006. AusAID’s Country Program Strategy 2003-2007 is well aligned with the strategy, particularly its commitment to increase the productivity and incomes of the rural poor, reduce vulnerability of the poor and strengthen the rule of law.

The principles of good governance provide a foundation for both the RGC NSDP and AusAIDs Country Program Strategy. Good governance is also one of the pillars of the Australian Governments White Paper on Foreign Aid.

RGC’s Decentralization and Deconcentration (D&D) program is an important element in public administrative reform aimed at devolving decision making to provincial and commune levels and so promote a more responsive to the needs of the community. This is relevant to AusAID activities in the area of governance.

Assessing progress towards achieving development priorities in Cambodia is becoming an increasing focus of development programs. The Cambodia Aid Effectiveness Report (AER) 2007 is the report prepared by the Technical Working Group (TWG) for the Government-Donor Coordination Committee and represents an attempt to introduce a more evidence-based and results-oriented approach to aid management in Cambodia (page ii).

Key results and findings in the AER are:

Despite efforts to establish programme-based approaches, development assistance is provided mainly in the form of projects, with approximately half of total aid in the form of technical assistance.

Broadly in line with the NSDP, the sectors receiving the most development assistance in 2006 were health, governance, education, transportation, and rural development.

NGOs contributed USD 50.2 million of their own resources in 2006 and displayed a disbursement profile broadly in line with NSDP priorities. In addition to this core funding, these NGOs reportedly receive USD 63 million from development partners, nearly half of which is directed to governance-related work, while at the same time, very little of NGO core funds are used for governance work (p17).

The report states that the significant level of support to NGOs for governance-related work reported by development partners does not appear to correspond with the data reported by NGOs (which could be explained by the fact that there are a number of prominent NGOs who do not appear on CDC records).

While the AER data on NGO support provides some indication of the level and focus of the contribution of NGOs to the development effort it states that, above all, Government and NGOs need to strengthen their partnership so that each gains an enhanced understanding of the other’s priorities and operations.

The AusAID-Cambodia country strategy has been in operation since 2003. It was extended to 2007 in order to allow a review of the country strategy to be undertaken as part of the development of the subsequent Country Strategy 2008 – 2011. The CPS was in the final stages of drafting at the time of writing, with the strategic objectives identified as:

Strategic Objective 1:	Royal Government of Cambodia more effectively financing, coordinating and monitoring the agricultural sector

Strategic Objective 2:	Increase in agribusiness activities and outputs (in selected provinces)

Strategic Objective 3 :	Improved health systems financing, service delivery and monitoring

Strategic Objective 4:	Improved capacity and commitment of courts and prisons to function effectively in the dealing of juvenile and other vulnerable groups

These strategic objectives have been framed under three of the four pillars described in the White Paper on the Australian Government’s Overseas Aid Program (2006):

Economic Development

Functioning and Effective States

Investing in People

The Mekong countries are identified within the White Paper as a regional priority with engagement identified around infrastructure, health and education, regional growth and integration, HIV/AIDS and access to water as key issues.

The White Paper also offers strategic possibilities to consider in light of the CDF Guidelines:

Supporting private sector led rural development: (p39) – specifically community driven, small scale rural infrastructure, safeguards for the rural sector and SME, with a focus on women’s participation.

Fostering functioning and effective states (p43): support for increased domestic demand for better governance – role of local civil society is critical and already supported in other areas direct support through relevant organizations.

Performance measurement and aid effectiveness is a key theme of the White Paper. There are some emerging lessons from global research and Australia’s own aid program that are pertinent when considering the CDF.

Economic growth is essential but not necessarily sufficient. Policies and institutions are also important as is well-targeted assistance (poor aid does harm – creates dependency, absolves governments of responsibility etc).

Building capacity may take generations (particular in post conflict country such as Cambodia), and for reform to be successful it has to be driven by local champions. The CDF can have significant, if long term, influence on encouraging and supporting local champions at the small scale.

Improved governance must be locally contextualized; best results are achieved where local people can express their own demands for better services and performance. This requires long-term, locally focused capacity building.

The recent (2007) Effectiveness Review of the AusAID Cambodia Country Program Strategy assessed the results achieved (planned versus actual) in terms of the country strategy performance assessment framework. Some of the key findings relevant to this review were:

That the rural focus of the country strategy was appropriate as 90% of poverty in Cambodia is in rural areas.

That the delivery of the program through contractors, NGOs and UN programs was justified and that the program has generally been managed efficiently.

That the strategy and results framework was probably over ambitious with the scale of the program insufficient to make an impact across all the equal priority areas identified in the strategy, especially where interventions are delivered as small one-off projects regardless of how effectively.

The key recommendations for the next strategy period include a stronger priority given to aid effectiveness, a policy focus on agriculture, and ensuring that objectives and indicators in the strategy results framework and monitoring are aligned to higher level outcomes not just localised project objectives.

The NGO sector in Cambodia has evolved rapidly since the influx of international NGOs in the early 1990s. The Aid Effectiveness Report by the RGC outlines the general profile of the role that NGOs in Cambodia fill in implementing and delivering development priorities in Cambodia, as well as analysis of current funding levels. However, the report offers few insights beyond this quantitative analysis. The author has drawn on her independent research with NGOs in Cambodia during 2006-07 to offer some reflections on the issues about NGOs that may be useful to consider in the context of the CDF.

When working to understand and enhance NGO effectiveness, it is important to consider the nature of the NGOs as organisations as much as the projects that they deliver. That is, their effectiveness cannot be understood only as the aggregate of what they do but is also about their values, policies and strategies; their organisational practices.

NGOs value and emphasise the role of relationships as central to the effectiveness of their work, regardless of sector. This relates to relationships at the community level, with government and with international development partners such as official and international NGO donors.

The centrality of this role of relationships is rarely recognised and reported on in documentation such as proposal and reports to donors.

NGOs, particularly local NGOs, are rarely given the opportunity to try innovative or more experimental approaches, in many cases actively discouraged from taking risks. One NGO stated that a donor said “working with them as a local NGO was risk enough”.

While donors are clearly interested in effectiveness and impact, current monitoring and evaluation of NGO activities reflects donor compliance reporting (implementing according to the agreed proposal and workplan), budget management and outputs. While NGOs indicate (to varying degrees) an understanding of the need to focus monitoring and reporting at the results and outcome level, the majority are struggling to achieve this. This appears to reflect the fact that donors (official and INGO) themselves are struggling with making this shift (Winter et al, 2007).

Compliance to donor reporting is a significant burden on their organisational resources (funds, time and personnel). Few, if any, particularly LNGOs, have resources or capacity left to develop their organisational systems and structures to systematically capture the information required to monitor and assess their work at the results and outcome level.

A large number of Cambodian NGOs are increasingly aware of and committed to increased organisational performance and governance measures – many guided by the principles outlined in the CCC Good Practice Principles Code of Ethical Practice and Minimum Standards. This should be encouraged and supported.

It is also essential to consider the Human Rights and Governance Fund (HRGF) and its role supporting small scale initiatives in Cambodia. This AusAID funding has, until recently, been administered by the Department of Foreign Affairs and Trade (DFAT) in Cambodia as a discretionary fund. The HRGF is administered similarly to the CDF in that it provides funding to NGOs for up to one year. The main difference with the CDF is the focus on human rights and governance, particularly activities that enhance space and opportunities for civil society in Cambodia.

DFAT have used the HRGF to fund activities that were realistic and achievable relative to the scale of the fund and consistent with DFAT objectives within Cambodia. They identify many similar issues regarding the fund that were identified for the CDF, for example the limitations of 12 months funding, as well as the limitations by DFAT to effectively monitor funded activities. While DFAT accept that incorporating the HRGF into the CDF offers strategic and administrative advantages, they are concerned that the flexibility of the fund is retained and that the definition of human rights and governance are not narrowed within the CDF guidelines.

The Implications for CDF

As a small grants scheme, it will require a conscious commitment by AusAID to consider the activities funded by the CDF in the broader strategic framework of the CPS if the CDF is to contribute to the achievement of its objectives.

A further challenge will be finding the balance between identifying and agreeing on a targeted scope for the program in order to maximise the potential to achieve outcomes that contribute to the country strategy, while not being so restrictive that it limits or discourages funding of community initiated projects and/or NGO initiatives.

Small grant schemes such as the CDF are an important source of development funding generally, and in the Cambodian context for local NGOs in particular. The small scale of the funding (in AusAID terms) offers AusAID an opportunity to support innovation and experimental programs that would otherwise be considered too risky under bigger schemes.

Scale is also an important issue when considering what NGOs and activities should or could be eligible for funding under the scheme. Given the small scale of the fund, a clearer focus on sectors and resisting the common tendency to be over-ambitious will be important in order to deliver results that more clearly contribute to the delivery of the country strategy objectives.

Due to the similarity of the two funds, the incorporation of the HRGF fund into the CDF will have minimal administrative and strategic implications beyond increasing the total funds available. The type of activities previously funded under the HRGF will now be eligible for funding under the CDF objective for Human Rights, Governance and Justice. The flexibility and broad definition of human rights and governance identified as strengths of the HRGF by DFAT are accordingly reflected in the amended CDF guidelines.

Given the focus on governance in both the RGC Rectangular Strategy and as the underlying basis for the Australia Cambodia Country Strategy, it is also worth considering the role of NGOs, in particular Cambodian NGOs. Civil society development in Cambodia is relatively recent, with NGOs becoming more predominant during the 1990s. NGOs and community based organisations (CBOs) are the foundations for broader civil society development, especially in the historical and post-conflict context of Cambodia and the CDF can play a significant role in supporting their development.

Approach and Methodology

The review was conducted as a formative evaluation and involved the examination of relevant documents, quantitative analysis of CDF funding trends and qualitative data collection with a range of people directly and indirectly engaged with the CDF. The terms of reference for the review, including a list of key documents and a detailed methodology and time frame, are included in full at Annex 1.

A desk review of key documents was conducted, including the documents referenced in the above context analysis as well as project proposals, project reports and the report of the previous review of the CDF conducted in 2000. Proposals, reports and guidelines of the former Human Rights and Governance Fund administered by DFAT were also reviewed.

An analysis of funding trends of the CDF for the years 2004-2007 was conducted based on financial data provided by AusAID. This is included at Annex 2.

The primary information was collected through semi structured interviews. A range of NGOs and project sites of current CDF projects were visited in Oddar Meanchey, Kampong Chhnang and Kampong Thom provinces, as well as the offices of a number of NGOs Phnom Penh. NGOs who had recently received funding under the former Human Rights and Governance fund were also interviewed. Other stakeholders such as CCC and CDC were also interviewed. Annex 3 outlines the program for a field visit and the preparation workshop, and Annex 4 provides a list of the NGOs and individuals interviewed and a copy of the interview notes.

Two workshops were held with AusAID staff – one with the review team as preparation for the visit to provincial project sites. The second was a workshop with the ten A AusAID staff to discuss some of the key findings of the review and seek their input into the main recommendations, particularly recommendations to identify specific sectors as priorities for CDF funding and the administrative implications of recommended changes to the CDF guidelines. The workshop programs are included at Annex 5.

Research was also conducted on the range and nature of other comparative small grant schemes available to NGOs in Cambodia and can be found at Annex 6.

Key findings and recommendations

The findings and recommendations are presented according to the outputs specified in the review Terms of Reference (Annex 1).

Output 1

Determine how CDF can best contribute to the Cambodia bilateral program in the context of the new Country Program Strategy. Develop for consideration, a goal and associated objectives, by considering priorities of i) a greater sectoral focus, ii) support for increased transparency and accountability of NGOs.

The timing of this review did not coincide with the finalisation of the new Country Program Strategy (CPS) as anticipated at the time of planning. However the development of the CPS had progressed sufficiently to enable the reviewers to be informed by its broad framework. The recommendations below may need to be reconsidered after the finalisation of the CPS.

The overarching goal of the CDF was considered in light of the attempts to align it more closely to the CPS, the integration of the Human Rights and Governance fund and the current development context in Cambodia. AusAID staff contributed to the development and consideration of the CDF goal at the Review Workshop (Annex 5).

The NGOs interviewed about the fund told a largely consistent story about the important role the CDF plays in supporting their community-based activities.

“We like AusAID CDF because the fund allows NGOs to implement our own plan, that they are more flexible.”	MODE, Stoeung Province, Kampong Thom

The majority of LNGOs interviewed confirmed the analysis of the proposals suggesting that the levels of funding, while small in the broader scheme, is substantial and important to their work. It is especially important for small NGOs (INGO and CNGO) and INGOs working in poorly serviced and remote areas such as Oddar Meanchey as well as locally based organisations with relatively limited capacity to implement larger and more complex funds. Many indicated that the CDF is one of the few funds that funds pilot initiatives, which has a degree of flexibility and is accessible to most NGOs.

“We’re doing things on the ground that they could never do at the central government level due to bureaucracy but we have developed models, and have worked with people who are moving up into government system so are influenced by models developed by these small programs on the ground.”	(Prison Fellowship)

The commitment to governance that underpins the NSDP and the CPS also provides an opportunity to use the CDF more strategically. Strengthening CSOs directly can be a legitimate activity as part of a holistic approach to governance, and this can be both implicitly and explicitly supported through a fund such as the CDF.

The objectives proposed in the recommendation below were broadly agreed upon at the review workshop after consideration of the review findings.

Recommendation 1

That the following goal and objectives be considered for the CDF fund:

Goal

The goal of the AusAID Community Development Fund (CDF) is to support Cambodian communities and civil society organisations to implement local development initiatives that will contribute to poverty reduction and improved quality of life.

Objectives:

To contribute to the development of Cambodian civil society through support for Cambodian non-government and community based organisations.

Contribute to the broader achievement of the objectives of the AusAID Cambodia Country Program Strategy, including funding innovative and pilot initiatives consistent with bilateral and national programs.

To be a responsive and flexible funding mechanism responding to and supporting local initiatives.

Output 1.1

Consider and make recommendations on how the CDF can contribute to the effectiveness of the aid program; through aligning more closely with priority sectors and geographic focus; eg, Health, Governance, including criminal justice, human rights

There are a range of elements that can either enhance or inhibit the effectiveness of activities funded under the CDF, and therefore the contribution the CDF can make more broadly.

Overwhelming, the review found that the main factor inhibiting the effectiveness of the activities supported by the fund was the funding period limitation of 12 months. The review, as well as the author’s own research, has also found that using a fund such as this to support small scale, community-based development initiatives is an important contribution to aid achievements more broadly. It is also time to reconsider the limitations placed on LNGOs as a whole by the CDF guidelines – the local NGO sector in Cambodia is on a path towards enhanced organisational practice and there are many well managed and structured LNGOs that are worthy of funding in their own right without the need to be refereed by an INGO as currently required. All these issues are addressed in recommendations for Output 2.

When considering priority sectors, it is useful to reflect on the funding trends for the previous 3 years (see Annex 2). The analysis of sectors indicates a strong emphasis on health related activities (17 of the 52 grants), with Rural Development and Agriculture representing 13 and 9 respectively. Given the consistency of these emergent priorities to the CPS, it was agreed to retain these sectors as priority areas. The recommendation includes merging Rural Development and Agriculture to more closely reflect the CPS outcomes for economic development.

Governance, while representing only 5 of the activities funded for the period, will be retained as a priority sector as per the CPS, as Human Rights and Governance. This reflects the intrinsic link between human rights and demand-led governance typically implemented by NGOs and CBOs, as well as capturing the human rights focus previously funded by the DFAT managed Human Rights and Governance fund.

Retaining governance as one of the priority sectors for the CDF will enable a range of governance activities to be funded. This could include supporting the grass-roots civil society activities traditionally implemented by CBOs and LNGOs and community based law and justice activities such as the prison support program implemented by Prison Fellowship. It will also enable the fund to contribute to good governance by supporting NGO and CBO strengthening explicitly, as well as implicitly through supporting the work of CBOs and LNGOs which meet minimum standards of practice.

Of the remaining sectors funded in 2003-2007, a total of sixteen activities were funded in four sectors that are inconsistent with the CPS as specific sectors: Vocational Skills Training, Education, Infrastructure and Environment. Due to the limited funding available for the CDF, it was agreed that these will no longer be considered as priority sectors for the CDF. This does not necessarily mean that proposals addressing such issues will no longer be accepted but that they will be required to demonstrate a contribution to priority sectors. For example, skills training or small scale infrastructure could be presented as strategic contributions in the rural development sector for example.

When looking at the overall dispersal of funding according to geographic location, almost half of all activities were implemented in just 3 provinces: Banteay Meanchey, Phnom Penh, Oddar Meanchey, with the remaining portion dispersed throughout 16 provinces. While it was clear that some provinces, such as Oddar Meanchey, have a substantial development need as a remote and recently integrated province, it is difficult to prioritise that need above other provinces. The NSDP states that the prevalence of poverty in Cambodia remains one of the highest in the region at 34.7% of the population, with 90% of poor people living in rural areas.

However, despite the difficulties of identifying geographic priorities for the CDF, there are strong management imperatives to encourage geographic clusters of activities. Supporting activities in geographic clusters would enable AusAID to visit more projects more often, thus enhancing the quality of monitoring and the quality of engagement with NGOs. It was agreed at the review workshop that clusters of funded activities seemed to be occurring anyway and that specifying geographic priorities may pose more limitations than advantages, especially if a clear need rapidly emerges outside of specified areas. The pattern of geographic disbursement will not be specified but is to be monitored by the CDF Programme Coordinator.

Recommendation 2

Priority Areas of Focus

CDF supports priority sectors consistent with the objectives outlined in the AusAID-Cambodia Country Program Strategy, focusing on community-based development. Proposals responding to or addressing the following areas will be given priority.

Agricultural development

integrated rice-based farming systems

market-oriented agricultural development and product diversification

capacity building of farmers, farmer organisations and rural business to increase incomes by increasing the value of rural products

small scale agricultural water management schemes

(Recommendation 2 cont)

Governance, Human Rights and Justice:

community based prisoner and family support programs

judicial and legal rights awareness raising and education

crime prevention and alternative livelihoods

community safety and awareness programs

promoting public sector accountability and transparency of government decision making and accessing information, particularly at commune council and district level.

encouraging democratic participation through raising awareness of electoral processes and opportunities for communities to influence policy making

strengthening national civil society by supporting organisational development of Cambodian non-government organisations and community based organisations.

Health:

improving health and hygiene standards through community-based water and sanitation activities, particularly in rural areas

basic health education and awareness, particularly of government reforms and programs

Output 1.2

Review the Cooperation Committee for Cambodia (CCC), Good Practice Principles project and consider ways in which the CDF could utilise this process to strengthen the overall governance of NGOs working in Cambodia.

According to the CCC, now is a good time to start incorporating GPP into guidelines as some NGOs are in process of certification. However it needs to be included within a broader context, i.e. as part of commitment to strengthening civil society and good governance.

Further to the finding it should not be assumed that all LNGOs require an INGO referee as part of their proposals to CDF, the GPP Code of Ethical Practice and Minimum Standards offers a pragmatic way for LNGOs to demonstrate their organisational integrity and development. It is proposed to invite LNGOs to demonstrate a commitment to becoming certified as part of the GPP as one of a number the ways to demonstrate this. The GPP is a relatively new initiative in Cambodia and as a voluntary process, it is not reasonable to expect broad participation in the program at this stage. While feedback to date indicates strong interest from LNGOs and many have applied for certification, it is important not to exclude LNGOs who can demonstrate their organisational integrity in other ways. Therefore, it is not recommended that participation in or certification by the GPP is a pre-requisite for LNGOs to access CDF funding, not least of which because it could also suggest that gaining certification leads to funding.

Establishing a clear link between the CDF and the GPP offers other opportunities. As an AusAID funded activity, the GPP Code is a clear statement of ethical principles and minimum standards of practice that AusAID can use as a benchmark for other NGOs applying to the fund, even if they are not a part of the process. Explicitly identifying such principles and standards in connection to the CDF strengthens the underlying commitment of the CDF to civil society strengthening as reflected in the Recommendation 1.

The other link with GPP discussed at the review workshop was the possibility that instead of the NGO representative on the CDF Assessment Committee be fulfilled by the Executive Director of CCC, that the Compliance Committee of the GPP be invited to nominate one of their members to act as the NGO representative each funding round. This would reduce the burden of the role being filled by one person, and provide a transparent and logical link with the GPP.

Recommendation 3

That the Cooperation Committee for Cambodia (CCC) Good Practice Principles (GPP) project be linked to the CDF in the following ways:

That a demonstrated commitment to certification with the GPP be accepted as one of a number of the ways that local NGOs demonstrate their organisational development and integrity. (Annex 7 - Amended CDF Guidelines, Section 2 – Proposal Requirements.)

That the GPP Code of Ethical Principles and Minimum Standards of Practice be attached to the CDF Guidelines for the reference of all NGOs to indicate the expectations of AusAID regarding NGO integrity and practice. (Annex 7)

AusAID proposes to CCC that the GPP Compliance Committee be invited to nominate one of their members to act as the NGO representative on the CDF Assessment Committee for each funding round.

Output 2

Review and make recommendations on how the overall management and administration of the Fund can be strengthened and streamlined. Areas for review include (i) guidelines and selection criteria (ii) requirement for appropriate technology ie latrines and wells.

The qualitative information collected through the informal interviews with NGOs produced some consistent and clear findings with regard to the management and administration of the CDF.

That all NGOs interviewed considered the guidelines of the fund to be clear, easy to use and appropriate to the nature and size of the fund.

That as identified above, all found the biggest limitation of the CDF was the 12 month funding period.

The levels of funding were generally considered appropriate.

While some NGOs interviewed referred to minor administrative issues, largely they stated that they found AusAID’s administration and management of the fund to be good. They generally described the Program Coordinator as ‘helpful, flexible and approachable”.

There was some inconsistency with regard to the approval process – some people said it was very quick, others said they had waited for some months without notification.

The LNGOs experience of working with referee INGOs varied, and in reality, rarely reflected the intentions of the referee relationship outlined in the guidelines.

The CDF Guidelines have been amended to reflect the findings above, as well as a review of the eligibility criterion. These have been amended to offer more opportunities for LNGOs to demonstrate their organisational development and integrity in their own right. The requirement for all LNGOs to have a referee organisation has been removed, although a partnership supporting organisational development remains as one of the options (Annex 7, Amended CDF Guidelines, Section 2).

The guidelines have also been amended to include increased requirements for INGOs to both demonstrate more explicitly their commitment to supporting civil society in Cambodia through supporting organisational development of partner LNGOs or CBOs,

Recommendation 4

That AusAID review the administration and management of the CDF as follows:

That the period of funding allowable be increased to accept proposals of 1 – 3 year duration.

That the CDF Guidelines invite proposals for periods up to 3 years, but state that the approval of funding for the 2nd and 3rd years be conditional on the satisfactory implementation and acquittal of the project in the 1st year.

That the funding limits be retained.

That the requirement for LNGOs to include a referee organisation be removed and a range of options be offered in order that the LNGO can demonstrate their organisational development and integrity.

That INGOs be required to more explicitly demonstrate their contribution to civil society strengthening through supporting LNGO or CBO partners in their proposal.

That the funding rounds be amended to occur twice yearly at specified dates, with these dates included in the CDF guidelines according to AusAID requirements.

That AusAID duly notify in a timely manner all applicants to each funding round that status of their applications, including those not shortlisted for funding.

That AusAID consider providing the rating and assessed by the Assessment Committee of their proposals to NGOs. It will be important for this information to be provided from the committee rather than AusAID staff and to ensure clarity to NGOs that the information is provided in order to facilitate learning rather than an opportunity to negotiate the decisions of the committee.

Output 2.1

Review and redevelop the CDF guidelines and selection criteria in line with findings from Output 1.

The CDF Guidelines have been reviewed and amended according to Outputs 1 and 2, and the associated recommendations.

The revised guidelines are included in full at Annex 7.

Output 2.2

Based on identification of priority sectors; consider the issues around provision of and use of appropriate technology by NGO’s. Areas for consideration include; latrines, water filters, wells, water testing etc.

Research was conducted on the current situation regarding Water and Sanitation issues in Cambodia, including an in-depth interview with Jan Willem Rosenboom, Country Representative of the WB Water and Sanitation Program (WSP). The notes from this meeting are included in full at Annex 8.

The challenges faced by water and sanitation programs do not appear to be directly related to a lack of availability of appropriate technology, although there are some market supply issues affecting latrines. According to this research, it appears that the major issues inhibiting the effectiveness of water and sanitation programs relate directly to community engagement and education leading to behaviour change. This relates to water supply programs involving the installation of wells and pumps, small scale irrigation activities and sanitation programs providing subsidies for latrine installation.

All agricultural water schemes come under the mandate of the Ministry of Water Resources and Meteorology (MoWRAM) and any activities addressing water management for agriculture must work within the framework and guidance as outlined by Circular No 1 issued by the ministry.

The most recent WSP report on Sanitation and Hygiene in Cambodia (March 2007) provides a useful analysis of the national situation as well as an outline of some approaches being trialled to enhance the community engagement and commitment to sanitation programs. Traditional programs of subsidised latrine installation mean that latrines are being installed by those who can afford the financial contribution and additional costs if they wish to upgrade. The rural coverage of latrine installation in Cambodia is currently 17% - the lowest in the region and “traditional water and sanitation programs that include subsidised latrine installation do not achieve health impacts while ever the bottom poorest do not have or use latrines.” (Jan Willem Rosenboom, WSP Cambodia).

Information on appropriate technology and methods for water and sanitation programs is available through the Department of Rural Health Care in the Ministry of Rural Development. This information is user friendly and specifically designed to be used with rural communities where literacy levels may be low. NGOs implementing water and sanitation programs are encouraged to collaborate with the Department of Rural Health Care and to design their activities around approaches currently being adopted to enhance community engagement and education regarding sanitation. This includes the department’s process to certify communities as ODF (open defecation free) and the CLTS (community led total sanitation) approach implemented by Plan and UNICEF.

There is also useful and very accessible information on the use and distribution of ceramic water filters available through WSP, including a DVD guide in English and Khmer as well as written materials and the International Development Enterprises – www.ide-cambodia.org has useful water related resources and links.

The CDF guidelines have been amended to reflect the recommendations below.

Recommendation 5

Organisations intending to implement agricultural water activities be referred to Ministry of WRAM for guidelines and resources, particularly approaches to enhance community engagement and ownership.

Organisations intending to implement health and sanitation activities involving latrine installation should contact the CDF Program Officer or the Ministry of Rural Development to access the ‘Informed Choice Manual on Rural Household Latrine Selection”.

Where a proposed project involves activities related to sanitation and hygiene (i.e. the construction of latrines), proposals addressing the following points will be more favourably assessed:

Demonstrated awareness of recent assessment of water and sanitation activities in Cambodia (available through the Department of Rural Development), including the significant issues related to subsidized latrine installation.

A clear link to Ministry of Rural Development activities at the district and/or provincial level.

Reference to and use of community-based education and behaviour change approaches such as the Community Led Total Sanitation (CLTS) approach or the ODF (Open Defecation Free) certification process implemented by DRD.

Use of the ‘Informed Choice Manual on Rural Household Latrine Selection” available the Department of Rural Health Care in the Ministry of Rural Development.

Output 2.3

Informed by findings from 2.1 and 2.2, provide recommendations to modify the CDF guidelines and management processes to improve the targeting, impact and likelihood of successful project implementation and to improve administration.

This output has been addressed in the previous outputs and recommendations.

Output 3

Develop options for a robust and practical CDF monitoring and evaluation framework for AusAID in Cambodia.

Monitoring the contribution of a fund such as the CDF to a country level strategy presents a number of challenges to both AusAID and the NGOs implementing activities. Due to the diversity of organisations funded and activities implemented, a structured framework based on the traditional use of indicators is not appropriate or helpful in general terms.

The current focus on aid effectiveness within AusAID (and more generally) adds to these challenges. As Winter, Hall and Ivatts (2007) state, this requires ‘the collection of quite different sorts of information and a different process of enquiry that asks not only ‘is what we do working?” but also “Are we getting the best possible results from the money and staff time we are investing in the country?”

The assessment of the CDF contribution to the country strategy necessarily requires a move from the project focus to accounting for results more broadly, which in turn, as Winter et al (2007) state, will put a spotlight on the quality of monitoring and evaluation at the activity level. They also state that ‘links to country strategy objectives are difficult to sustain because the stated ambitions of the country strategy fall well beyond the credible influence of the activities’.

If the CDF is to be managed in order to contribute more strategically to the Country Program Strategy, the collection of information that will tell AusAID (and the NGOs) whether this is being achieved will require a significant change in the management of the fund overall. The monitoring information collected will need to be enhanced, and the amount of time spent by AusAID staff analysing and understanding that information will necessarily increase if it is to be useful and systematic. This will take time for AusAID and the NGOs to adapt their current monitoring practices and to develop the capacity to collect and analyse information about results and outcomes.

This change from collecting activity information to understanding and identifying program level outcomes is a significant challenge for country program managers. Binnendijk (quoted in Winter et al) says that “organisational change that attempts to stimulate the use of performance information at higher levels of complexity has profound implications for the resources devoted to capturing and analysing it and for the nature of accountability in an organisation.”

The Performance Assessment Framework developed for the CPS provides limited guidance for monitoring the CDF. While it is relatively easy to argue that activities such as small scale rural livelihoods projects implemented by local NGOs could contribute to the country strategy objective “To contribute to ensuring household food security, increased incomes and improved livelihood for rural poor farmers” (draft Country Strategy 2010, PAF, 6 Nov) the objective indicators at the strategy level provide no useful guidance to identify the contribution small scale activities may have made.

A recent report from an M&E training of AusAID staff in the Pacific suggested caution about clarity of terms as used by AusAID in light of updated performance assessment frameworks and the focus on quality and effectiveness. This is particularly important for NGOs who often find the range of technical terms used by different donors unclear at best, sometimes confusing. As Paul Nichols (quoted in Winter et al) points out, “at the level at which an agency seeks to measure higher level indicators increases, indicators are less easy to rely on and need to be supplemented by a set of research questions that start with “How?” or “To what extent?”.

Contribution analysis (described by Winter et al, 2007) is useful concept to assist with the type of assessment of the CDF discussed here. It is described by John Mayne, Office of the Auditor General of Canada, 1999 as “Finding credible ways of demonstrating that you have made a difference through your actions and efforts to the outcomes.”

The framework recommended below is based on an attempt to address the inherent complexity involved in assessing performance and effectiveness at a programme or multi-activity level, the complexity of the diversity of NGOs and the range of activities they are implementing and an attempt to propose an approach appropriate to the scale and nature of the CDF. It is also an attempt to acknowledge and build on current capacities, integrate with current mechanisms and suggest an approach that can be developed iteratively.

Enhancing the quality information collected about CDF activities is also important when considering the recommendation to extend the funding period to one to three years. Collecting information about performance at a more complex level is more meaningful when collected over a period of time when iterative changes can be more clearly tracked.

Time is also an important consideration here: a balance needs to be found between investing the ideal amount of time to performance monitoring tasks and an acknowledgement of limited time available for such tasks by both AusAID and NGO staff.

The annual reflection and learning forum is offered as an attempt to maximise opportunities for dialogue and consultation with a range of NGOs in a timely manner; as a mechanism for AusID to engage with as many CDF funded NGOs without creating an unrealistic expectation of visiting all activities in any one year and to foster dialogue among and with the NGOs to enhance the quality of information collected. While organising such forums will require substantial AusAID staff input for a short period of time, the expectation is that there will be substantial benefits in terms of quality of information gained, as well as the opportunity to develop relationships (among NGOs as well as between AusAID and NGOs).

When considering the recommendations below, there are some implications for AusAID Cambodia staff to also consider:

The additional staff time consumed in not only collecting but analysing the information at a more complex level

Staff capacity of both AusAID and the NGOs to undertake this analysis.

What to do with the information once it is analysed – will it contribute to AusAID reporting against the Performance Assessment Framework for the CPS for example?

Is the extra effort worth it for a financially small component of AusAID’s Cambodia program?

Recommendation 6 – Monitoring and Evaluation Framework

That the following steps and mechanisms be considered and gradually introduced to the management of the CDF in order to progressively collect, synthesise and analyse information about the performance and effectiveness of the fund overall, and it’s ultimate contribution to country program strategy objectives. It also attempts to build on current processes and mechanisms as much as possible.

That three questions be added to the CDF report format and monitoring visit report:

What changes have occurred?

How do you know they occurred? (for example, direct observation, interviews with community members, return visits)

To what extent did the project contribute to these changes?

Review currently used internal monitoring systems and determine an appropriate mechanism for analysing the information currently (and potentially) collected in CDF reports and monitoring visits

That AusAID consider training in monitoring concepts such as Contribution Analysis to assist with assessing the CDF performance and contribution towards the CPS.

Conduct an initial monitoring and learning exchange forum in early 2008 with NGOs funded by CDF to seek their input about monitoring needs, ideas, and suggestions for practical ways to conduct such forums. Work with these NGOs to develop a ‘light touch’ monitoring plan for CDF for the coming 2 years, including an annual reflection and learning forum.

Review the above framework – questions, systems for analysis, mechanisms for engaging with NGOs – after one year and two years (possibly as part of the annual reflections).

Given the need for enhanced higher level performance monitoring coupled with the limited AusAID resources to effectively monitor all CDF activities, amend the procedures for the assessment committee to identify a number of activities at the conclusion of each appraisal process that are to be regularly and consistently monitored over the funding period.

The criterion to identify such activities should include those of higher than usual risk (e.g. pilot activities), activities with clear links to the CPS and/or bilateral programs where contributions to the CPS will enhance AusAID assessment of performance and activities implemented by LNGOs who would benefit from increased monitoring support and guidance (based on prior consultation).

The number of activities identified at each funding round should be at the discretion of the assessment committee according to AusAID guidance regarding capacity.

For proposals funded for 3 years, and particularly those identified at 6 above, provide the option to allow 5-10% of total project budget (3 year budget) to be allocated to conduct an independent evaluation of the project.

Output 3.1

Review the current monitoring and evaluation efforts through the study of selected Post monitoring reports, formal project reporting and selected field visits.

The review of written monitoring reports and project reports revealed an expected range of responses and reasonable to good quality of reporting. The report format itself does not offer much scope for performance assessment but rather focuses on activity monitoring and compliance to the original proposal and requirements. While this is useful and appropriate for activities funded for one year, enhancing the quality of information about changes over time will be more meaningful for activities implemented over 2-3 years.

The monitoring trip reports as well as the formal project reports offer some useful analysis and insights about achievements, lessons, approaches etc. However, it is not clear how, if at all, this information is systematically collected. Systematically synthesising some of the key changes and lessons embedded in these reports would offer a useful picture of the activities individually as well as allowing a picture of the CDF as a whole to begin to be formed.

Most NGOs interviewed said that the timing of the reporting was appropriate, with a couple of minor comments about some flexibility with reporting requirements for projects between 6-12 months for example. The responses were very mixed with regards to the length of the report – some said it was too short, others too long, thus it is probably just right, although maybe a range of length could be suggested, for example from 3 – 6 pages.

Monitoring frequency is a challenge - the NGOs interviewed consistently said they would appreciate more engagement with AusAID about monitoring, and that they prefer building relationships with donors who are interested in monitoring and visiting project sites. Most said that they would like more visits from AusAID, with some saying they had never had a project visit by AusAID staff only phone calls to check on progress. It is clear that having time and staff to regularly visit the range of CDF projects is a challenge for AusAID and it may be helpful to address these expectations up front, for example at the time of approving the project or at the reflection and learning forum.

Some said that their first report was difficult and that guidance on the precise information required for such a short report would be helpful. Some said their partners were helpful in interpreting guidelines and criteria, especially if they also had a relationship with AusAID. Some said that the “jargon and matrices” can be a challenge, especially for Khmer staff responsible for interpreting information as the concepts are very western – very ‘western’.

Output 3.2

Identify (i) strengths and weaknesses of project implementation; and (ii) risks to implementation including anti-fraud measures (linked to 1.2); and consider options to address these issues as part of a more comprehensive M&E framework.

While project sites and NGOs were visited and interviewed, the focus of the review focused on the guidelines and generally approach of the CDF rather than a detailed assessment of individual project activities. Given the variety of activities being funded in a range of contexts by a range of NGOs, it is difficult to make meaningful comment about the strength and weaknesses of project implementation beyond the general recommendations made above in this report.

Risks to implementation have also been addressed elsewhere in this report. The most significant risk identified during this review was the limit to 12 month funding – many NGOs indicated that the project would have been better quality and more holistic if they had been able to implement it over a longer time period and thus be more integrated in working with the community.

While it is not always possible to predict or prevent opportunistic fraud, the risk of systemic fraud within NGOs can best reduced through prevention at the front end of the application process. The recommendations about links with the GPP and the options offered to LNGOs to demonstrate their organisational capacity will provide useful information about NGO financial systems and organisational capacity. Combined with the financial reporting for the project itself, this information should offer sufficient indication of the risk of fraud. If there are concerns about an organisation’s financial management, financial statements may be requested, ideally audited although currently in Cambodia this is not always possible or realistic. Beyond these measures, the GPP code offers appropriate guidance for financial reporting.

While INGOs generally have more sophisticated systems and mechanisms for financial management and organisational integrity, this cannot be assumed. International NGOs are also often signatories to a range of voluntary Codes of Conducts or Ethics yet they are rarely asked to demonstrate their commitment to these principles. Recommendation 4 above requires INGOs to more explicitly demonstrate how they meet organisational standards of good practice.

Conclusion

The review of AusAID Cambodia Community Development Fund has found that this discretionary small grants scheme is greatly appreciated by the NGO community in Cambodia. While in-depth evaluations were not conducted on specific activities, the review and anecdotal evidence suggests a mechanism that has impact beyond the scale of its funding; one that supports local, small-scale and low cost activities that offer good value for money. The quality of implementation indicated good development practice, particularly community development (the 12 month funding limitation notwithstanding) and the quality of reporting by NGOs was found to be reasonable to good practice.

The recommendations about the strategic direction, management and administration and monitoring of the CDF offer some challenges to both AusAID and the NGOs in terms of assessing performance and understanding the effectiveness of the fund. The recommendations also reflect the finding that overwhelmingly, NGOs using the guidelines and format found them clear and easy to use so changes have not been made unnecessarily.

Changes have been made that recognise that local NGOs have been making good progress in their own organisational development. The new goal and objectives for the CDF reflect AusAID’s commitment to good governance and the importance of supporting indigenous civil society as a part of that commitment.

A balance between more strategic focus and flexibility has been sought, as well as a realistic assessment of the time and resources available to collect and assess performance information. The CDF is a fund of diverse stand alone activities but can also be regarded as a whole that has the potential for good quality community development beyond the scale of its funding.

ANNEXES

Review Terms of Reference

CDF Funding Trends, 2004-2007

Program and Preparation for a Review Field trip to North-West Provinces

List of interviewees and interview notes

Review Workshop Program

Summary of Small Grant Schemes in Cambodia

Amended CDF Guidelines

Water and Sanitation Guidance Information and References

see output 2.2 for guidance about NGOs implementing agricultural water projects

John Winter, Jo Hall and Susan Ivatts, (2007) “More than the sum of the parts: assessing country level effectiveness in the Australian international development program”, Australasian Evaluation Conference, Melbourne, Australia September 2007.

Paul Nichols, Praxis Consultants, personal correspondence, October 2007