Abstract
The purpose of this study was to determine the psychometric features (Factor structure, validity and reliability) of the Course Experience Questionnaire(CEQ) that have 43 items and developed by Wilson and et al(1997). This research is the sort of normative research carried out in the first semester of the academic year 2021-2022 at Gilan University. The statistical population of this research included all students accepted in the academic years of 2017 and 2018, which was conducted using a simple random sampling method on 332 third and fourth year undergraduate students of the Faculty of Humanities of Guilan University. Exploratory factor analysis elicited 7 factors which generally explained 71% of the total variance. Furthermore Confirmatory factor analysis results on the items showed that all the items have significant load factor on their own factors, and the seven-factor structure of the questionnaire was approved. Concurrent criterion validity coefficient between each of the items of Course Experience Scale With the criterion variables (the Course Satisfaction and Achievement) that were linked theoretically was also analyzed and approved. Altogether, findings of this study showed that scale of the Course Experience Questionnaire (CEQ) has adequate reliability.
Extended Abstract
Introduction
One of the most important concerns of all university systems is designing educational courses effective for students. Promoting the curriculum's quality and improving the educational program's quality is essential to attaining a high-quality education (Thien & Jamil, 2019). It is also crucial to student satisfaction (Griffioen, Doppenberg, & Oostdam, 2018). Nowadays there is a growing tendency to investigate students' satisfaction with the quality of educational courses (Yin, Gonzalez, & Huang, 2018). The significance of this issue has made the universities aware of the effectiveness of their programs and courses, and prioritizing the concept of evaluating their educational course has never been greater. The evaluation is exceedingly essential in that it measures the effectiveness of the curriculum elements and, at a greater level, the entire curriculum. (Wolstencroft & Main, 2021). Given these points, the present study tries to localize the experience scale of an educational course (Wilson et al., 1997) as an evaluation tool of the educational course done by its students.
Theoretical Framework
Evaluation or assessment refers to a neatly structured process for collecting and interpreting information that determines the intended goals and scale of the program (Seif, 2022). Evaluation can be used in countless ways, including determining the value, competency, importance, and esteem of an educational phenomenon for judging and making decisions in order to establish a system (program), continuity of activities, adjusting the system, validating the system, understanding the different dimensions of the system as well as supporting it (Bazargan, 2021). In this regard, institutes of higher education also know the value of evaluation and try to use it in the best way possible to improve the quality of their programs.
Evaluation in the university is done in different ways, such as evaluation by students, self-evaluation, and evaluation by the head of the department, colleagues, and school officials. One of the most common evaluation methods in most countries, including Iran, is an evaluation done by students. Students understand the curriculum through evaluation (Wolstencroft & Main, 2021). In other words, the assessment causes the higher education curriculum to be defined and reviewed from the student's point of view (Ramsden, 2003). It also significantly impacts their learning (Biggs, 2003). It leads to students' satisfaction with the educational course (Griffioen, Doppenberg, & Oostdam, 2018). Students are present in the teacher's classroom during one academic semester, so they get to know all of his capabilities and mannerisms. On the other hand, they can recognize better than any other person whether the professor has been able to directly or indirectly transfer the course material to them (Nasr et al., 2004). Therefore, student evaluation of the provided educational courses is the most influential factor in determining the effectiveness of these courses.
Most of the questionnaires used so far have focused more on evaluating students as a whole, rather than individual lecturers of courses. As mentioned by Marsh (1987), although these types of evaluations provide significant and helpful information such as feedback to faculty members and contrubuting the employee's decision-making, and are valuable for evaluating teaching, from the point of view of a higher education institute that tries to maintain and improve the quality of education, it is the far more appropriate to focus on the evaluation of the entire course (Richardson, 1994).
Achieving a standard for measuring the effectiveness of training courses has been accompanied by many efforts by those involved in higher education. The university managers are trying to improve their institution's performance based on the performance indicators. They consider these indicators as the guarantor of the quality of teaching in different fields and its achievement. Many efforts have been made in the field of compiling performance indicators. In this regard, the Educational Course Experience Questionnaire (CEQ) was developed as an authentic evaluation tool to meet students' needs for educational courses and implemented as a national test in universities in Australia about four decades ago. Because of the critical role of student's perception in evaluating the quality of the educational courses and the limitation of evaluating the country's most conducted research, which only focused on the professors' lesson presentation (Vakili et al., 2010), this current study tries to localize the Iranian student population's educational course experience scale (Wilson et al., 1997)
Methodology
The current research is the normative research field carried out in the first semester of the academic year 2021-2022 at Guilan University. The statistical population of this research included all students entering the academic years of 2017 and 2018, constituting a population of 3830 people. Based on Morgan's table, 350 people were identified as a sufficient sample size for this statistical population. With multiple follow-ups, 332 copies (94%) of the questionnaires were completed through sampling, which was randomly collected. The present research used the Educational Course Experience Questionnaire (Wilson et al., 1997). In this research, with the addition of the content appropriateness scale with four items, the educational course experience questionnaire had 43 items. Also, an additional item about overall satisfaction is included as a separate item along with the educational course experience questionnaire. The academic progress variable has also been measured using students' GPAs.
Discussion and results
In order to extract scale factors, the exploratory factor analysis method was used. The results of exploratory factor analysis with principal components method and final orthogonal rotation extracted seven factors that explain 71% of the total variance. The analyses performed on the fit indices have all been accepted for the model mentioned. All the values of the indicators are close to the suitability criteria and indicate the structural model's satisfactory and relatively acceptable adequacy. As a result, the 7-factor structure of the educational course experience questionnaire and the confirmatory factor analysis model have an acceptable fit and are approved. CFI (consistent fit index), NFI (normalized fit index), TLI (Tucker-Lewis's index), and IFI (incremental fit index) all indicate the acceptability of the model, and show the values greater than 0.90 which indicate the favorability of the model. The root means of the square error of estimation (RMSEA) index, which represents parsimonious fit, also shows a value of 0.050, indicating the fitting model's appropriateness. Finally, the GFI (Goodness of Fit Index), which represents the absolute fit, shows a value of 0.95 and indicates that the collected data is close to the acceptable range and that the developed model is confirmed.
The data shows that Cronbach's alpha coefficient of the educational course experience scale is 0.95 and satisfactory. In addition, the correlation of the mentioned scale items with the total score is significant (P=0.001), and the range of these coefficients varies from 0.51 to 0.83. In other words, all items of the experience scale of the educational course have the necessary homogeneity. Pearson's correlation coefficient results showed a high correlation between the seven subscales of appropriate teaching, clear goals, appropriate assessment, appropriate assignments, emphasis on independence, general skills, and appropriateness of content with the overall scale (training course experience) (P=0.001). In addition, the correlation coefficient between the educational course experience scale and all its components with the overall satisfaction scale of the course is significant (P=0.001). The correlation coefficient between the training course experience and its components with academic progress is also significant (p=0.001). These results indicate the concurrent criterion validity of the research results, in which the educational course experience scale and its components were correlated with the scales theoretically related to it (satisfaction with the course and academic progress.)
Conclusion
According to the findings of this research, the high validity and reliability of the educational course experience scale used in the current study can lead to the use of this scale as a valid and reliable tool to measure the quality of educational courses at the country's higher education level. This scale can also have many applications in teaching and learning research in higher education. The mentioned tool also reveals the appropriate performance indicators of university teaching quality so that it can be used to align the teaching quality evaluation of university professors with the global teaching quality evaluation standards. This tool can recruit and promote faculty members and their professional growth. In this regard, it is possible to improve their teaching competence by holding workshops and training courses for academic staff members.