Last week I started EDTECH 505. This course explains the systematic process used to evaluate programs and products. The course uses the textbook called ABCs of Evaluation written by Boulmetis and Dutwin. Chapter 1 is titled What is Evaluation? and Chapter 2 is Why Evaluate? For the assignments I use principles and from book and explain how it connects with my evaluation report project. For other parts I answer questions stated in the book or asked by the course professor.
What is Evaluation?
Components common in all evaluations are objectives and a systematic process of collecting and analyzing data. All evaluations have objectives with varying purposes. These are to determine the efficiency, effectiveness, or impact of a program or product. They can also be used to make a decision to improve or fund a program. Collecting data plays a vital role because they provide the information on what’s happening within a program. The data will dictate how to proceed with a decision or make improvements. My evaluation project will evaluate a business English website which is used to help students learn vocabulary for the BEC exam. The objective is to improve the website to make it more effective and interesting for learning. In order to make the necessary changes, data collection is needed. The data will show the strengths and weaknesses of particular areas and show are changes are needed. The website owner/instructor and I discussed three possible categories to improve vocabulary learning. One was to get students to log in more. Second was to determine the existing activities on the website which are effective. Third was to determine the activities are meeting the objectives.
When considering the evaluation objectives, efficiency is the cost of program in relation to benefits received or the effectiveness of a program. Effectiveness examines if the objectives of the program has been achieved. These objectives usually refer to changes in knowledge, attitudes, or skills. A third objective of a program is the impact. It is similar to effectiveness but looks at the long term changes a program has had on the participants.
When thinking about the reasons for doing an evaluation, it is to determine if the program is doing what it was set out to do. The data will show if a program is meeting or is not adequately meeting its objectives.
In my evaluation report, I will focus more on justifying activities of an existing website used to help adult students learn business English vocabulary. The evaluation will pinpoint the website’s strengths and weaknesses to determine the activities that should be kept and those that require a change. An evaluation will allow me to carefully examine the product and make notes about the product to to understand its framework. Through regular discussions with the stakeholder we can reexamine the goals and objectives and decide methods for collecting data to determine its effectiveness.
A limitation in all evaluations is that it will not guarantee objective results. This could be based on how the sample was selected and how the data was collected. For example, in my evaluation report the instructor distributed the survey with the students knowing that it was about the website she created. This may prevent the students from being completely honest. Also the total population of small of 21 was small, and so, having a larger sample would provide a more accurate picture how the students are using the website.
The results from the evaluation project will show if the objectives are being met and to what degree. This will be done by collecting and analyzing the data. From the results, I’ll know the the activities that were effective and interesting and those that were not. By using the open-ended questions I also hope to generate new ideas for activities that were previously not considered. As an instructor of reading comprehension for second language learners, the results from this evaluation may help me to understand how plan activities for vocabulary acquisition.