Volunteers Association for Bangladesh
Evaluation
Evaluation of the Programs of Volunteers Association for Bangladesh (VAB) aimed at achieving quality education in rural Bangladesh
EXECUTIVE SUMMARY
Introduction
Volunteers Association for Bangladesh (VAB, pronounced as VaaB – meaning “idea” or “feeling” in Bengali) is a Charity founded by non-resident Bangladeshis (NRBs), led by Dr. Abu T Rafiqur Rahman, in 1998 and run jointly by VAB USA and VAB Bangladesh. It raises funds in the USA, Bangladesh and elsewhere, mostly from individual philanthropists, and in certain cases, from institutions, and utilizes the funds for quality high school education in rural Bangladesh.
The founding NRBs have been led by the conviction that a good quality education at the secondary level is a prerequisite for human resource development that can take the youths of the country out of the poverty cycle and help them achieve a better life. In the process, it can be of use to the country in taking advantage of the green dividend and progressing towards further economic development. Poverty is severest and the need for human resource development direst in rural Bangladesh, and also the potentials of green dividend the highest. It has also been the idea of the founding NRBs to work in partnership with existing rural high schools within the framework of the country’s existing public and secular education system.
Thus, VAB is addressing poverty of rural families by enhancing access to and continuity in education, improving quality of education in rural areas, following a strategy of building on the existing infrastructure of rural schools and using the core of the academic framework of the public education system, and involving actual stakeholders through empowerment measures. Bearing in mind the constraints of resources, both at the level of the Charity itself and at the national level, at large, improvement within the system working with existing schools is the most pragmatic, in terms of cost-effectiveness, replicability and sustainability.,
In developing programs for improvement in the quality of education in a pragmatic approach, VAB follows a bottom-up method of active participation of the direct stakeholders – students, teachers, schools and communities -- in identifying solutions for critical needs. Starting in 2000 with only one program of awarding scholarships to needy rural students, by 2012, development of improvement programs came to a stage where all the programs needed to be placed in a cohesive integrated framework (this is dubbed as the VAB Model for Quality Education in Rural Bangladesh), taking advantage of the interrelationships among various individual programs.
During the period, 2012-2016, these improvement programs were being implemented in three groups of 5 schools in each group. These schools were in clusters, i,e., in geographical proximity of one another. The cluster approach has been taken by VAB because, in its view, this approach: (a) can achieve quality through cooperation and competition among the schools in the cluster; (b) can take advantage of any externalities that may exist; (c) can ignite interest and enthusiasm in the wider community, thus fostering improvement in quality of education; and (d) can minimize administrative costs of implementing programs and monitoring implementation. These schools are usually referred to as “cluster schools”.
By 2017, it was considered high time to assess whether VAB’s educational quality improvement programs in rural schools are achieving what they are aimed at, and/or whether there was any need to modify or enhance any effort for better results. An assessment was also essential because VAB has decided to direct its future work towards cluster schools. VAB also wishes to demonstrate accountability and result-orientation for retention and expansion of its donor base. Besides, VAB wishes to attract attention of governmental, non-governmental, and international policy and decision makers. This sets the stage for conducting the present evaluation study.
This is VAB’s first serious and comprehensive evaluation exercise commissioned by VAB USA with a clear purpose in mind. The evaluation is expected to respond to longstanding queries from the VAB Board and other stakeholders surrounding the relevance and effectiveness of the VAB program which evolved over time. Some of the significant questions in this context were:
Methodology and Collection of Data and Information
The evaluation study followed the internationally accepted evaluation criteria of: relevance, effectiveness, efficiency, impact, and sustainability. The analysis also addressed a number of cross-cutting issues impacted by the VAB programs. The methodology and data and information collection for the study were devised and implemented accordingly.
The evaluation followed a mixed method of both quantitative and qualitative data collection and analysis. The data collection and analysis was organized in a way that allowed comparison of certain key indicators/variables when VAB programs were implemented in the designated schools with the benchmark situation when there were no VAB programs in those schools. For convenience, this could be dubbed as a comparison of the “VAB situation” with the “pre-VAB situation”. It was decided that five-year periods would be taken to reflect each of these situations so that year-to-year variations are ironed out.
As mentioned before, in the cluster schools, VAB has been implementing its programs during the period, 2012-2016. Thus, logically, this period would be chosen to show the VAB situation. To show the pre-VAB situation, the preceding five-year period, i.e., 2007-2011, when no VAB programs were implemented in these schools, was selected.
The Evaluator worked closely with the VAB project team and volunteers to collect reliable data and information. The exercise involved several methods of collecting quantitative and qualitative data to fulfill the objectives of the evaluation. Related secondary literature was reviewed at the outset of the study. Primary data were collected from the field using questionnaires/checklists following methods, such as meetings/consultations, interviews, focus group discussions (FGDs), case studies, and, importantly, documentation of enrolment, attendance records, dropouts, exam results, etc. Data were collected from the teachers and the students of selected high schools as well as through meetings and exchanges with members of School Management Committees (SMCs), parents, VAB officials, VAB Advisory Board Members, etc. Concerned Upazila Secondary Education Officers (USEOs) were also consulted with a view to collecting requisite information in relation to the implementation of VAB programs. (These are government officials at the upazila (sub-district) level who have the responsibility of overseeing the secondary education situation in the upazila.) From the 15 schools in 3 clusters, in order to reduce the workload, a sample of six schools, two from each cluster, were selected.
In terms of methodology and data and information collection, it can be added that the evaluation study initiated attempts in three other areas, but the outcomes have been kept on hold pending further investigation. Methodologically, two areas are: (a) a comparison of performance of the students of class VI and class VIII in VAB-partnered schools with those of the government’s evaluation project called the Secondary Education Quality and Access Enhancement Project (SEQAEP); and (b) a comparison of the performance of the VAB schools with those of the so-called “control schools”. With respect to data and information collection, the area explored was: a comparison of the VAB and pre-VAB situations in (i) schools partnering with VAB for five to nine years; and (ii) schools partnering with VAB for ten years and more. This was in view of the fact that VAB’s work with rural schools in Bangladesh has been going on since 2000, albeit not in its most complete form as with the cluster schools.
Findings of the Evaluation
VAB interventions in the subject schools, i.e., the cluster schools that have been implementing the various measures under the VAB model have produced a significant increase in enrollment, a moderate decrease in dropouts, a remarkable improvement in pass rates in public exams and in performance of students in public exams. The changes do reflect a positive trend as anticipated. However, there was a slight decrease in attendance, a matter that has to be further investigated.
While the above results are very encouraging, the stakeholders were positive that the total effect of the VAB model in schools have been much more in terms of imparting quality education, acquiring overall institutional strength in terms of capacity to deliver quality education, and generating community support for quality education. For rural schools in Bangladesh, positive effects in all these three areas are remarkable. In this context, the VAB model can be said to have a cumulative impact on enhancing intrinsic qualities and attributes of students, teachers and school management for a far-reaching transformation for the coming years.
Such transformation becomes more significant when it is considered that the VAB model also provided a workable structure for enhancing capacities of schools with minimal external support initially and then working on its own in the future based on the strengthened structure and capacity, resulting from the measures under the model to enhance sustainability of quality education and self-reliance of the schools and the communities to continue in the quality path.
The evaluation study clearly shows that the VAB model along with its cluster concept is clearly gaining good traction on the ground and the multi-dimensional positive benefits are emerging slowly but surely. Examples of this are the initiatives of the schools, teachers and communities to compliment classroom learning of students with self-learning, to foster desirable character traits and citizenship qualities through different activity clubs, participation in different nationwide competitions, promotion of voluntarism among students, etc.
The focus in the VAB model on competence building, in computers and English language, adds a quality dimension that is not only significant for rural schools of Bangladesh, also contributes to a brighter future of the students themselves, and at the same time to progress in the country.
The findings of the evaluation demonstrate that overall, the VAB model is achieving the desired outcome. Perhaps the pace of positive change is not as rapid as would have been anticipated. However, VAB is aware that grass-root level changes in quality of education with daunting constraints, as is the case with rural schools of Bangladesh, require a longer gestation to achieve a noticeable momentum. What is important is a positive and gradually accelerated trend.
In view of the importance of the assessment of effectiveness of educational programs to the people at the policy and decision making levels (in Government, national and international agencies, donors, development practitioners and VAB itself) emphasis is placed on relevant findings on effectiveness of VAB programs. These are presented in detail below.
Indicators for measuring effectiveness The effectiveness of VAB programs was examined with regard to the following quantitative indicators:
Effectiveness was also investigated in terms of the following cross-cutting streams of outcomes, which are essentially of a qualitative nature:
Some detailed findings based on analysis of data, qualitative enquiries and interviews are stated below.
On an average, in a cluster school, every year 121 students enrolled in class VI in the VAB period. This contrasts with 82 students in the pre-VAB period. As for class X, on an average, every year 49 students enrolled in the VAB period, while enrollment in the pre-VAB period was 46.
The average number of students dropping out of class VI in the VAB period was 9 out of 120 students (8%), while in the pre-VAB period it was 7 out of 82 (9%). On the other hand, the average number of students dropping out from among the class X students was 4 out of 49 (8%) in the VAB period, while it was 8 out of 48 (17%) in the pre-VAB period.
In JSC, on an average, each year, in the VAB period, 64 students out of 71 (90%) passed, while in the pre-VAB period 53 students passed out of 66 (80%). In SSC, in the VAB period, 41 students passed out of 45 (91%), while in the pre-VAB period, it was 30 students out of 46 (65%).
As measured by “letter grades”, in the VAB period, as to the performance in JSC exams, on an average, 3 students obtained A+ (improved from 1 student in the pre-VAB period), 24 students obtained A (improved from 12 students), and 27 students obtained A- (improved from 12). A total of 54 successful students out of 64 (84%) obtained grades A- and above (improved from 25 successful students out of 53 (47%) in the pre-VAB period).
Also, in SSC exams, the improvements were: in the VAB period, 2, 12 and 11 students obtained grades A+, A, and A- , respectively, while in the pre-VAB period, 1, 7 and 8 students obtained grades A+. A, and A-, respectively. Thus, in the VAB period, 25 out of 41 successful students (61%) obtained grades A- and above, while in the pre-VAB period, 16 out of 30 successful students (53%) obtained grades A- and above.
The evaluation study also examined the situation regarding the exams at the end of Class VI, the first year of high school education in Bangladesh. The pass rate was virtually the same for the exam after class VI. In the annual exam of class VI, in the VAB period, on an average, each year, 103 students passed out of 108 (95%); however, in the pre-VAB period, it was 75 out of 78 (96%). As for performance of students in the class VI exams, absolute numbers improved in the VAB period compared to the pre-VAB period as follows: 8 students obtained A+ (compared to 5), 17 students obtained A (compared to 13), and 23 students obtained A- (compared to 19). A total of 48 students out of 103 successful ones (47%) obtained grades A- and above during the VAB period (compared to a total of 37 students out of 75 in the pre-VAB period (49%)). So, there was a slight decrease in the performance of students, percentage-wise. However, better-performing students in class VI exams improved in absolute numbers during the VAB period, although in relative terms, there was a slight decrease.
In a cluster school, average attendance in class VI was 84% in the VAB period while it was 88% in the pre-VAB period. Attendance in class X was 91%, on an average, in the VAB period, while it was 86% in the VAB period.
The effectiveness in terms of other factors can be summarized as follows.
Evaluation of VAB Programs by Other Criteria
Relevance Data collected from school records consistently support that VAB’s education model satisfies the needs of the beneficiaries and the stakeholders as well as adheres to the policies and priorities of the Government of Bangladesh and the Development Partners. It is thus found quite relevant, and in harmony with the political and social context of Bangladesh. The measures under the VAB model were found to have contributed to empowering the students, the teachers, the schools and the communities. For example, as part of empowering the students, VAB assists needy students financially with scholarships; provides tutoring support to overcome learning deficiencies at the entry level of high school students; provides assistance to organize co-curricular programs, like debate, cultural and sports clubs, and for participation in Math Olympiads and Science Fairs; trains them in computer skills and English language skills; facilitates voluntary work to benefit the school and the community enhancing the students’ civic responsibility, team spirit and contribution to the society. As another example, as part of empowering schools, VAB extends supports to equip Science and Computer Labs; provides books to the school library to create interest in reading among students and enhances reading habits; provides furniture to the classroom for minimizing seating difficulties; and facilitates creation and build-up of endowment fund to generate a steady resource for the present and the future with matching funds contributed by the community.
Efficiency Specifically speaking, efficiency is the ratio of the value of input resources against that of project results. If the value of project results is greater than that of input resources, it can be said that the project has high efficiency. The same can be said if the object of evaluation shows a better value ratio than alternatives. There are several types of efficiency, the most important of which are technical efficiency and distributive efficiency.
Technical efficiency refers to generating as much results as possible with a given amount of resources, or generating specified amounts of results with as little resources as possible. Technical efficiency may also compare the methods of the project with alternatives that can achieve the same purposes as those of the project. Distributive efficiency evaluates in a comprehensive sense whether added value is created by the project for the society.
Efficiency evaluation applies economic evaluation methods. Examples are cost-effectiveness analysis, cost-benefit analysis, and cost-utility analysis. However, in the present evaluation we kept our analyses quite limited and tried to assess cost effectiveness of the project. The evaluation in this regard necessarily relied on qualitative indicators. Such qualitative indicators were collected from Headmasters, Assistant Teachers (who teach English, General Math and General Science), members of SMC and VAB officials. These indicators suggest that the project was operated efficiently involving all stakeholders like: teachers, students, SMC members representing schools and parents representing communities. These data sources also indicated that there was no visible alternative to achieve the same quality and quantity of the results with a different method and a more limited budget. The evaluation study collected data on the expenditure of various components under the four sets of empowerment measures: student empowerment, teacher empowerment, school empowerment and community empowerment. Such cost data corroborate the above findings about maximum results from given amounts of resources. Such data also support the finding that there are no visible alternatives to achieve the same quantity and quality of results with a different method and a more limited budget.
Impact While evaluating impact of the VAB model in the rural high schools, data were collected from the study schools, the respective Headmasters, Assistant Teachers, members of the SMC, officials of VAB working with the program and the Advisory Board of VAB. We are aware that it is somewhat difficult to measure in-depth the impact of the program only from the comments of the respondents of the study. However, effectiveness and efficiency of the program can also provide a reasonable indication of the impact of the program. Therefore, the impact of the program needs to be considered in conjunction with the comments of the respondents as well as with the results in respect of effectiveness and efficiency of the program, as put forth above.
The impact of a program tells us about the long-lasting results of the interventions under the project. The respondents made a number of comments which address the impact of the VAB model. The overall development of the schools as a result of VAB’s programs is an area where all the respondents appeared to be in general agreement. The qualitative data obtained from Headmasters reveal that the VAB interventions contributed to the overall development of the schools that includes, among others, exam results of the students, involvement in co-curricular activities, etc. The Assistant Teachers corroborated this by stating that both the teachers and the students are now more aware of developing their own school. They also mentioned that the VAB interventions contributed to empower teachers and students, to conduct classes regularly, and to joint initiatives of teachers and SMC members, etc. The Headmasters added that the VAB interventions contributed to empower teachers, schools, community linkage, etc. The comments of the Assistant Teachers support this. They commented that VAB’s interventions contributed to the development of schools which include students’ empowerment, improved pass rate, involvement in debate and other cultural activities, improvement of community linkage, awareness of cleanliness and of demerits of early marriage, etc. The SMC members made similar comments. They were of the view that VAB intervention contributed to the development of schools, which includes increasing number of students in the schools, improving computer literacy, improving exam results, reducing dropouts, benefiting poor and brilliant students, improving community linkage, etc. VAB officials commented that the VAB interventions contributed to empower teachers, students, schools and communities as well. They added VAB interventions contributed to increase number of students in schools and their exam results, to equip science and computer labs, to improvement in co-curricular activities in the schools, and to improvement in quality of teaching through the provision of training to the teachers, etc.
The VAB interventions contributed to create a positive learning environment in the schools through providing training to the teachers, giving scholarship and support for tutoring to the students, equipping computer and science labs, etc., which contributed to improve exam results, reduce dropouts and increase eagerness among the students and the teachers for further development. Computer literacy among teachers and students has improved greatly. Participation of students voluntarily in different activities of the schools and the communities increased. Teachers and students are now more disciplined and they voluntarily come forward to organize and participate in different programs. There is also a growing interest among the teachers, students and SMC members for the development of the school. The teachers, students and community members participate in the design and implementation of the steps of VAB programs. A good governance system of the schools has been developed. In terms of sustainability, VAB’s programs, being suited to the culture of the schools combined with the development of a good governance system resulting from them, are considered positive factors. There are also some initiatives noted for raising funds from the locality as an effective measure of generating resources for future sustainability of the institutions. The school authorities are now more aware that they must be independent, and rely only on local support.
The Headmasters also added that the VAB interventions contributed to increase transparency, and participation of local people in schools’ welfare. They also added that the VAB interventions contributed to make the parents and the community more cooperative towards the schools. About the degree of contribution of VAB’s interventions to develop capacity and enhancing system of the schools, Headmasters commented that it was mostly high. However, some Assistant Teachers commented that it was moderate while others commented that it was high. In this regard the latter group cited examples that VAB provided teaching materials and multimedia tools, developed science labs with adequate equipment, contributed to improve relation between teachers and students and thereby ignited interest of students in learning.
While asked about whether the VAB interventions satisfy the priorities and demands of the teachers, students and the community, all the Headmasters commented positively. All the Assistant Teachers and the SMC members came to the same conclusion independently. While asked about whether teachers, students and the community support the project, Headmasters commented quite positively. They also commented positively while asked about whether the project was well suited to the social culture of the teachers, students and the community and whether recipients participated in the design and implementation steps of the project.
All the Headmasters commented that development that is now seen in the schools would not have been possible if there were no intervention of VAB. On the other hand, when asked the opposite question about what would have happened if there was no intervention from VAB, all the Headmasters commented that development that is now seen in the school would not be possible then. The Assistant Teachers, SMC Members and VAB officials also supported this.
It should be added that in terms of impact, there could be intended or unintended impacts, or in other words, positive or negative impacts in the particular area of intervention depending on the design and implementation process of the program. One of the main purposes of evaluation is to identify unintended impacts and analyze their reasons. The findings of the evaluation show that Headmasters, Assistant Teachers, SMC members, VAB officials and members of the VAB Advisory Board were in total agreement in commenting that there is no unintended impact of VAB programs either on the school or on the community, on the welfare of the parents and the community, on the system of the school, on the school environment, or on running and managing the project.
Cross-cutting issues There are mainly the gender issue and the environment issue that have been considered in the present study. The study reveals that both boys and girls are equally treated in the distribution of scholarships, in extending tutoring support, etc. As data reveal, there is also no negative impact of any component of VAB programs on the environment in general.
Recommendations of the Study
Many of the recommendations have been distilled from the findings of the study under various criteria. For convenience, the recommendations have been organized under certain themes: program related issues, operational issues, and budgetary matters.
Program related issues
Regarding programs, the recommendations deal primarily with better teaching and effective training of teachers.
Operational issues
Budgetary matters
Conclusions
VAB’s programs as embodied in the VAB Model for Quality Education in Rural Bangladesh is responsive to the requirements of rural students, teachers, schools and others who have direct and indirect stake in improving quality of secondary education in rural Bangladesh. Besides, VAB’s philosophy and volunteer approach have great appeals, and are to be nurtured continuously. Raising additional funds and expanding the application of VAB programs to cover more rural schools in the country will create greater positive impact in the field of secondary education in the country. VAB works only in rural areas with the most vulnerable schools where mainly poor children are enrolled. This gives VAB a unique strength and role in improving secondary education in rural areas of Bangladesh.
Quality education for the underprivileged in rural Bangladesh.
CONTACT US
USA: 122 Taxter Road
Irvington, New York 10533
Phone: (914) 231-7972
Email: mati_pal@yahoo.com
Email: mati_pal@vabonline.org
BANGLADESH: House 806, Road 3, Baitul Aman Housing Society, Adabor, Dhaka – 1207, Bangladesh
Phone: 910-4558
Email: alimdu@gmail.com
© 2021 / All Rights Reserved