Evaluation
*Jump to Evaluation Reports
Program evaluations embrace a diversity of methodologies used to appraise many aspects of programs in institutions. Generally evaluations involve careful collection and analysis of selected program information to inform a specific decision. There are many types of evaluation; for example needs assessments, process or outcomes based evaluations and cost benefit analyses. Organizations choose evaluation approaches depending on the kind of program insight they require and the purpose thereof.
SAIE seeks to affirm and enhance the impact of our entrepreneurial training products and services to our end users: primary, secondary and tertiary learners and underprivileged communities. We also seek to improve delivery methods, improving efficiency and reducing costs. In addition, evaluations help us to confirm if we are achieving what we intended to and compare what we think we are doing and what we are actually doing. Feedback with both flaws/weak spots/pitfalls and strengths give us the scope to interrogate, improve and fill gaps where they emerge. The rich insights (discoveries) from our evaluations provide feedback into our learning cycle. Thus, our evaluations lend themselves to capacity building as they enforce our product development and implementation of projects.
Unlike previous SAIE evaluations, we are moving away from exclusively quantitative methods. They tend to be restrictive as most conclusions revolve around statistical and methodological ‘accuracy’, ‘reliability’ and ‘validity’ at the expense of reflections and recommendations.
SAIE has thus embarked on the development and deployment of a comprehensive evaluation program called EntrEval. It is derived from the EvaluLead framework, an evaluation process that was jointly developed by the WK Kellogg Foundation and USAID on a foundation of Monitoring and Evaluation global best practice.
The parameters of analysis are designed to be comprehensive and to incorporate both quantitative and qualitative methodologies. This is achieved by breaking down the dimensions of impact into domains, result types and forms of inquiry.
The three result types (levels of change) are:
- Episodic: more predictable, tangible, short term results discernible immediately after program intervention
- Developmental: more open ended, less predictable and less controllable changes that occur over time and represented as sequences in a dynamic process. They show differences in pace or rhythm among different individuals.
- Transformative: fundamental shifts in values and perspective preceding fundamental shifts in behavior and or performance; often the prize which programs aim for.*
These changes occur in the three domains of individual, organizational and societal or community. The two types of inquiry used to collect evidence are the evidential (more quantitative) and evocative (more qualitative). The integrated impacts may be summarized in the result map (grid) shown below.
(This diagram originates from EvaluLead: A Guide for shaping and
evaluating leadership development programs, Grove, J.T., Kibel, B.M.
& Haas, T., Sustainable Leadership Initiative, 2005.) Although the
three by three (3x3) grid is a valuable results map, in practice it is
more of a guide and the final results may be captured in different
databases and spreadsheets.
Ultimately, the character of EntrEval enables SAIE to be intuitively sensitive to the relevance, productivity, effectiveness and potentials of our programs. The evidential components (episodic facts, developmental markers and transformative indicators) capture the traditional quantitative components of the ‘what happened’ type. This includes students’ entrepreneurial knowledge, skills and attitudes. The evocative components (episodic opinions, developmental stories and transformative reflections) embrace the hitherto marginalized qualitative components of the ‘why and how things happened’ type. Such qualitative dimensions shed light on a better understanding of the intimate dynamics of program implementation in the classroom, and the program effects, problems and challenges on learners and educators alike. These discoveries are captured as valuable feedback used for program refinement.
EntrEval is designed to be compatible with the SAIE Management Information System (MIS), allowing SAIE to pay systematic attention to evaluation design thereby safeguarding against substandard evaluations. It allows for the capturing of evidence during the life of a project and after – resulting in an evaluation report which is based on sound programme logic, efficient monitoring and empirical evidence of activities and impacts.
*EvaluLead: A Guide for shaping and evaluating leadership development programs, Grove, J.T., Kibel, B.M. & Haas, T., Sustainable Leadership Initiative, 2005.)
For more information about EntrEval and SAIE's Management Information System (MIS) please visit the Discoveries page.
