The FUST instrument is subject to an external independent evaluation at the end of each phase. The evaluation is intended to provide advice to the Government of Flanders and UNESCO concerning the extension of the FUST agreement. While the evaluation of 2007 examined in detail ODINAFRICA (IOC), FRIENDNILE (IHP), SUMAMAD (MAB), and to a lesser extent FETWATER (IHP) and CAZALAC (IHP), the evaluation of 2013 put more focus on FETWATER II and SPINCAM. the general objectives of the evaluation are:
- Examine whether the supported programmes are meeting their objectives and expected results;
- Examine the effectiveness of the governing mechanism of the FUST and its supported programmes;
- Develop a forward-looking perspective on how the collaboration between the Flemish Government and UNESCO within the framework of the Trust Fund can further be strengthened.
The outcomes of the past evaluations (phase I-phase III) are summarized below.
3- THE FLANDERS UNESCO SCIENCE TRUST FUND (FUST) PHASE III - An External Evaluation Report (March 2008 - March 2013)
Prepared by Savithri Narayanan and Wouter Buytaert - July 2013
Download in PDF format (318 KB)
An evaluation was undertaken of the “Flanders/UNESCO Trust Fund for the Support of UNESCO’s Activities in the Field of Science” (FUST), to assess its relevance, effectiveness and areas requiring changes or improvements in order for the two partners to decide on its future directions. The FUST was the result of the recognition by both the Government of Flanders and the UNESCO on the importance of collaboration to achieve common global objectives, particularly in areas of data and information management, science in support of policy development and ecosystem management, and capacity enhancement in priority geographical areas of Africa and Latin America. It was established in 1999 between the UNESCO and the Government of Flanders and extended to 2013 in multiple phases. In the third phase, which is evaluated here, FUST funded selected projects of three UNESCO Programmes: International Hydrological Programme (IHP), the Man and the Biosphere (MAB) Programme, and the Intergovernmental Oceanographic Commission (IOC).
The evaluation process included a review of technical and financial reports, consultation with officers at UNESCO and Government of Flanders, and site visits to Oostende (Belgium), Guayaquil (Ecuador), Bogota and Santa Marta (Colombia), Pretoria (South Africa), and Windhoek (Namibia), to meet with the teams implementing two of the FUST programmes, SPINCAM and FETWATER. The visits allowed the reviewers to get input directly from the institutions and staff actually delivering on these programmes: relevance of these programmes to their regional and national programmes, impediments and challenges, and suggestions for way forward.
The following are high-level observations (additional observations are given under each project):
• The FUST projects portfolio is found to be highly relevant to both UNESCO and the Government of Flanders, in that it is consistent with the established priorities of both partners and complements their funded activities, and provides good value for money. FUST – funded training was found to be particularly relevant to enhance the scientific activities in the participating countries as they fill a void in the range of training that is typically available in these regions.
• Many of the large projects of the portfolio have been effective in leveraging funds from other sources as well as scientific and technical expertise, and successful in generating relevant and high- quality outcomes. Without the contributions from FUST and the intergovernmental coordination, these projects could not have been launched.
• Small-scale activities not only support the larger projects, but also enable a targeted approach to responding to opportunities and scientific requirements, and even to crosswalk between natural sciences and social sciences, particularly in the area of Human Rights including Children’s Rights.
• The visibility of the FUST-funded projects is in general good, particularly through the web, workshop and conference reports and others. However, some issues were identified with regard to access to project outputs and long-term sustainability of their availability. • Sustainability remains challenge, particularly due to a weakness in the regional data and information management infrastructure and database inter-operability.
• FUST has been found as a good model for strategic investments by Member States in UNESCO as it is based on long-term vision and objectives and has built-in modality to:
- support multi-phased projects at longer-term with ear-marked funding; - fund specific activities that fill the gap between larger programmes and help take advantage of other initiatives, through a faster approval process by the Government of Flanders; - leverage additional resources by attracting further co-funding;
- serve as a platform for involvement of Flemish academic community (through secondments, participation in research projects, internships by students etc.);
- enable secondments to UNESCO Headquarters to help coordinate the FUST activities and ensure smooth governance, particularly as the national and UNESCO’s policies can be quite daunting for scientific institutions.
- ensure a governance process including a Steering Committee, which plays an advisory role in the preparation, negotiation and proposal of projects, and reviewing and supporting the approved projects, and thus assist the Flemish Government to arrive at the final decision.
• Projects that explore enhanced interlinkage of the UNESCO programmes, and those that break through the traditional barrier between natural and social science (such as Human Rights, literacy and gender equality) may want to be considered to reap maximum benefits from FUST.
The following are the recommendations from the evaluation.
• General recommendations
Recommendation 1. As FUST has made a very important contribution to the global community towards responsible environmental stewardship by facilitating research, data and information management, capacity building and generation of useful products and publications, it is strongly recommended that the partnership between the Government of Flanders and UNESCO be continued and FUST funding be maintained at least at the current level.
Recommendation 2. It is recommended that the FUST funding modality be maintained in its current combination of large multi-year projects, small scale activities and secondments, and further adapted to including opportunities for scientists to be engaged in these projects without impacting on their research career, and reap benefits in terms of scientific publications, access to graduate students, and increased collaborations with their counterparts in other institutions.
Recommendation 3. Noting the success of the data and information efforts funded by FUST, and recognizing that not all regions have acceptable capacity levels for integrated data management, it is recommended that the investments in data and information management along with associated training be supported as priority for Latin America for the next period of funding.
Recommendation 4. It is recommended that the visibility of outputs and activities achieved in FUST funded multi-year projects is duly enhanced, including the contribution of other funding partners, by making better use of the web and social media as well as through focussed effort on communication nationally, regionally and globally. Such outcome is particularly important to ensure sustainability of the skills and the science in the participating countries.
Recommendation 5. It is recommended that FUST pay particular attention to enabling enhanced involvement of more junior scientists, including from Flanders (e.g., MSc students, PhD students and interns) to ensure the projects’ scientific quality. This may for instance be explored through the Flemish Internship programme.
Recommendation 6. To remove the bottlenecks for the involvement of academics, a stronger focus on secondments of scientific experts is recommended, potentially with a formal link to a Flemish academic institution. Such secondments would decrease the time pressure on involved academics, at the same time providing benefits to both the seconded scientific experts and the participating academics through co-authored scientific publications among others.
Recommendation 7. It is recommended that in the call for proposals for FUST funding, additional themes be included that explore (a) generation of education materials for enhanced literacy and science education; (b) Human Rights and Children's Rights; (c) Gender equality in Science and Policies; (d) transdisciplinary approaches to poverty alleviation and sustainability promotion.
• IHP
Recommendation 8. Put more emphasis on the long-term sustainability of the projects through, for instance, a programme website that can host outputs (e.g., training material) and document outcomes (e.g., course attendance) of individual projects after they have finished.
Recommendation 9. Enable more active involvement of Flemish stakeholders and academia (students and faculty) and further foster the scientific input in the project (e.g., through the recently established IHP Belgian committee).
• MAB
Recommendation 10. Encourage the formulation of new FUST projects of the type of SUMAMAD, given its excellent combined emphasis on (1) leveraging relevant scientific research; (2) strong ties to the local and regional policy level; (3) targeting vulnerable communities.
Recommendation 11. Give particular attention to target specific deliverables (e.g., peer reviewed publications with authors from the South) through flexible funding schemes to enhance the participation of Flemish academics.
• IOC
Recommendation 12. Extend the FUST support to IOC programmes building on the successes of past investments and aligning with the 2014-2021 medium-term strategies of UNESCO and IOC.
Recommendation 13. Develop an effective ODIN programme in Latin America under IODE to obtain maximum benefit from SPINCAM and other science projects, and to support regional needs and those of the member states.
Recommendation 14. Enhance the usefulness of the coastal indicators at the national, municipal and provincial levels by enabling the SPINCAM team to make the results available through various media including social networks, and through publications such as the State-of-the Coast reports.
Recommendation 15. Maintain and strengthen the interaction of SPINCAM participants with local national, and regional stakeholders.
Recommendation 16. Enable enhanced involvement of more junior scientists in marine sciences from Flanders (e.g., MSc students, PhD students and interns) to enhance the projects’ scientific quality and the visibility of Flanders.
Recommendation 17. Enable further enhancement of the visibility of the FUST funded projects, particularly those that concern policies relevant for both UNESCO and Flanders, such as SPINCAM.
2- THE FLANDERS UNESCO SCIENCE TRUST FUND (FUST) PHASE II An External Evaluation Report (March 2004- March 2007)
Prepared by: Lesley Rickards and Bisher Imam October 2007
Download in PDF format (552 KB)
Executive Summary
The evaluators found that Phase II of the Flanders UNESCO Science Trust Fund (FUST) has been largely successful both at individual program level and collectively. Overall, individual programs have been well managed and highly effective. All of the programs managed to achieve their desired results, deliver the expected outcome, and contribute to capacity building within their respective mandates. The evolution of these programs during phase II is consistent both with UNESCO’s mandate and with the strategic goals of the Government of Flanders Science and Innovation Policy. More specifically, phase II can be characterized by marked increases in regional scientific cooperation, strong involvement by Flemish counterparts, significant gains in regional and country-level capacity through training, and by the initiation and completion of many applied research projects. The program has been highly effective and it is accurate to say that the financial and scientific contributions of the Flemish community are well directed and visible. The reviewers believe that the success and the continuing evolution of the programs warrant the continuation of the current agreement.
______________________________________________________________________
1- The Flanders UNESCO Trust Fund (FUST 1999-2003) - Evaluation Report - August 2002
Prepared By: G.L.Holland and H.S.Wheater
Download in PDF
Executive Summary
The evaluators found that the first phase of the FUST Agreement has been successful, despite some delays in the completion of projects due to start-up difficulties in the transfer of funds. Individual projects under the Agreement have been well managed and the desired results have been achieved. Important contributions have been made to the general programme objectives, as well as to the specific project deliverables.
The evaluators agree that the success of the respective programmes warrants the continuation of the Agreement.
Background
The Flanders UNESCO Science Trust Fund (FUST) is the subject of an agreement between UNESCO and the Government of Flanders initially covering a period of five years (1999-2003). Each year FUST provides for an allocation of 1.116 million EUR to support the UNESCO mission, to contribute to peace and security, by promoting co-operation among the nations in the field of education, science, culture and communication. The Trust Fund is managed by a Steering Committee representing both UNESCO and the Government of Flanders. It supports programmes of the International Hydrological Programme (IHP) and the Intergovernmental Oceanographic Commission (IOC), respectively taking into account the UNESCO programme and the priorities accorded by Flanders. UNESCO and the beneficiaries contribute to the activity in a co-operative manner. Scientists from the Flemish universities and research institutes are able to participate in one or more of the proposals if they so wish. The agreement can be extended beyond 2003 on the basis of an evaluation of the co-operation between UNESCO and Flanders. This review contributes to that evaluation process.
The specific programmes evaluated under the Agreement are the Oceanographic Data and Information Network for Africa (ODINAFRICA) of the Intergovernmental Oceanographic Commission and four programmes of the International Hydrological Programme, namely those centred in Gaza, the Nile Basin, Southern Africa and the Latin America and Carribean region. The latter project is only just getting underway, however ODINAFRICA and the other three IHP projects are all founded on initiatives already established, and have a history of Flemish involvement.
Although the FUST Agreement on ODINAFRICA dates only from 1998, the foundation had been laid many years earlier, in an East Africa project developed jointly by IOC, Vrije Universiteit Brussel (VUB) and the Limburgs Universitair Centrum (LUC).
The Gaza project was conceived by a Flemish mission in 1995 and has the general aim of strengthening the Palestinian water sector institutions to ensure the long-term conservation of water resources. Presently the project is on hold, pending resolution of the local political situation.
In the Nile Basin over the last 35 years, a series of governmental initiatives has been put in place to pursue sustainable water management. The Nile FRIEND programme was established in 1996, with the support of UNESCO and forms the origin of the FUST Nile project, which supports a 4 year programme which started in November, 2001. A launch workshop for each theme, attended by Flemish counterparts, refined work plans and created a Project Management team reporting to the FRIEND/NILE Steering Committee.
The Southern African project originated from a request to UNESCO and WMO from the Department of Water Affairs and Forestry (DWAF), for assistance in the assessment of education and training needs. A Work Plan for capacity building and networking for water resources management was agreed for the period 2002-2005, with support from FUST, the Republic of South Africa and UNESCO.
The importance of water resources management in the arid and semi-arid regions of the world is widely recognised, and has been a priority for the IHP. Large areas of the Latin America and Caribbean Region fall into this category, and the establishment of a Water Resources Centre for the Region has been under discussion for some time. The general objective of the FUST project is to create a regional centre and Chile has been selected as both a natural choice and a willing host.
The primary purpose of the evaluation is to provide advice to the Government of Flanders and UNESCO concerning the extension of the FUST agreement. Visits to Brussels and to the UNESCO Headquarters were carried out jointly by the two consultants. The effectiveness and efficiency of the FUST projects within the International Hydrological Programme and the ODINAFRICA project within the Intergovernmental Oceanographic Commission, respectively, were evaluated using the annual reports and other documentation supplied. Separate field trips for the two consultants, to Cairo and to Zanzibar, respectively, completed the evaluation process.
Major findings
UNESCO and the Government of Flanders each have their own financial and administrative rules and regulations to respect and these are not necessarily easy to harmonise and co-ordinate. Some evidence of difficulties faced is apparent in the minutes of the Steering Committee and the members are to be commended on the resolution of potential issues.
In ODINAFRICA, regional networking, considered of immense importance to the development of an African renaissance in the future, is being promoted through the increased connectivity of data centres, through improved communication amongst regional scientists and through the distribution of scientific publications and journals. Similarly, networking within a country is essential for the development of a sustainable infrastructure. The activities funded by FUST in Tanzania were found to be well integrated into the national priorities and objectives. The funding and equipment provided by FUST have been catalytic in the transfer of capacity and in the development of national responsibilities in the context of coastal area management and the indigenous support has been real and sustainable. Progress in the modernisation of marine data management is having a beneficial impact on other national resource responsibilities in Tanzania.
One of the objectives of the FUST is the multiplication effect of its program. For the IOC, apart from the funds levered from the regular budget, ODINAFRICA, although widely acclaimed at governing body meetings, does not seem to have attracted other major partners from Member States.
FUST builds on an established history of support from Flemish academics for the IHP and has enabled several important initiatives to come to fruition. There has been strong support from the Flemish Universities for the 3 projects currently in place. The Gaza project has been running for some time and has been well managed and highly productive. Significant progress has been made in creating an effective Water Research Centre and a major programme of training and capacity building activities has been completed. In addition, major efforts have been made in the public awareness of water issues. The re-focussed Phase II programme, to develop a Palestinian Water Resources Network, is an appropriate response to changing needs and offers an important vehicle to develop collaborative research and training.
Before FUST support, little or no progress had been made in the Nile FRIEND project. The availability of financial support for project activities provoked a re-evaluation of management structures and facilitated the effective launch of the programme in November 2001.
In both the Southern African project and the establishment of the Water Resources Centre in Chile, FUST funding has been complementary to substantial resources provided by the host countries. In the Southern Africa project, preliminary commitments of support for the networking initiative have been achieved, at least from South African partners and stakeholders. The Chile Centre has yet to be created, but has the potential to make an important contribution to the Latin America and Caribbean Region. Substantial support and guidance will be required from UNESCO to build a truly regional Centre, and to establish effective links with other regional and global initiatives.
Lessons learned
The FUST Steering Committee has recognised the need for the Trust Fund to be able to accept complementary small-scale activities using the accrued interest on unexpended moneys. This suggestion has considerable merit.
Although many of the regional objectives in ODINAFRICA are common, there are significant differences in national priorities and how each participating national centre can best manage the allocated funds.
Communication is a particular source of difficulty for ODINAFRICA. Problems occur with data and information transfer and with the vulnerability to disruptions caused by electronic virus infections. These problems also impact on communications from the region to the outside partners. Language issues are apparent, but appear to be being slowly overcome.
The ODINAFRICA programme suffered a substantial delay on two separate occasions due to the late arrival of funds. There were too many differences between the planned budgets and those finally implemented, although overall objectives remained constant.
The participating countries in Sub-Saharan Africa are not at the same level of capability or capacity and training courses need to take these differences into account. As more countries join, the problem will be accentuated and future plans may need to be amended to allow for multi-tier training approaches. More use could be made of indigenous expertise for initial training purposes.
The Tanzanian visit demonstrated a lack of visibility for the Government of Flanders at the local level, which should be addressed as ample opportunities exist for co-operation. There seemed little knowledge of how to make contact with potential Flemish scientists.
The main constraint in the Gaza project is the current political situation. In Phase I, disruption to travel between the West Bank and Gaza was a major obstacle to joint Palestinian meetings. At present, it is not possible for staff in Gaza to travel to work. A correct decision has been made to put Phase II on hold, while continuing to support continuity of key staff at the Centre.
Progress in the Nile FRIEND project has demonstrated the importance of strong leadership and the need for flexibility to overcome political problems.
A general issue for the Nile and Gaza Phase II projects concerns the constraints on the use of FUST funds.
The Nile project relies on research manpower, but while money can be provided for equipment and travel, it cannot easily be used to support staff. Most of the theme contributors are from African universities, and do not have the resources to undertake the necessary work. Very modest amounts of support for research staff could alleviate this issue. However, there are ambiguities in what is meant by research funding, as strictly the FUST support precludes support for salaries.
A general issue for all projects is the support of Flemish counterpart expertise. The partners see this as essential in providing access to state-of-the-art methods, awareness of current research developments, and in supporting training.
Synopsis of Recommendations
General:
- Complementary proposals, below an agreed maximum amount, should be supported from the accrued interest on the Trust Fund.
- Part of the management fee for the Trust Fund, combined with the considerable expertise present in the UNESCO Bureau for Extrabudgetary Funding, should be used to find additional funding partners.
ODINAFRICA:
- The efforts to increase the participation of African States must be continued.
- The cost and advantages of satellite communication should be investigated.
- An effort should be made to increase the number of contributing institutes in developed countries.
- Regional managers and the ODINAFRICA Co-ordinators should be funded to organise regional activities and workshops and to prepare materials for basic instruction.
IHP:
- Review missions should continue to play an important role in the assessment of project progress and constraints and in the review of strategic directions
- Limited funding should be made available to support research assistants/students.
- The role of Flemish counterparts needs definition and agreement on a project basis. Some financial support for counterpart inputs is recommended.
- UNESCO should develop a vehicle to establish and maintain contacts between different Regional Centres with common interests and to involve regional centres in relevant global initiatives, particularly with respect to arid and semi-arid areas.