體驗區
|
| 篇名 |
從理論到實踐：關係系統評鑑於評鑑能力建置之應用
|
| 並列篇名 |
From Theory to Practice: Applying the Relational Systems Evaluation to Evaluation Capacity Building
|
| 作者 |
林素卿(Su-Ching Lin)
、舒富男(Fu-Nan Su)
|
| 中文摘要 |
本研究旨在應用關係系統評鑑（Relational Systems Evaluation, RSE）作為評鑑能力建置（Evaluative Capacity Building, ECB）的模式並探究其影響。本研究的目的包括：一、瞭解參與者對評鑑能力建置方案的反應；二、參與者評鑑思維、文化能力和評鑑能力改變的情形；三、參與者應用關係系統評鑑觀點發展校本課程評鑑指標的情形及對關係系統評鑑的看法。關係系統評鑑模式源自於進化評鑑、關係發展系統和系統性思維等理論，其最大特徵是研究人員扮演促進者的角色，透過夥伴關係和以系統評鑑議定書作為步驟指南，提升教育實務工作者的評鑑知能。本研究的參與者包括一所都會型國中和一所原鄉國中的校長和該校自然、科技及藝術等領域之教師，共計八位。本研究採用質量混合設計，資料蒐集的工具包括量表、滿意度調查表及回饋單、e-portfolio專業成長檔案和訪談紀錄。研究發現，參與者對於評鑑能力建置的滿意度高，回饋意見正向；評鑑思維、文化能力和評鑑能力皆有顯著提升；所發展的校本課程評鑑指標能回應關係系統評鑑的觀點，展現校本課程的特性；參與者認為相較於傳統線性的評鑑模式，關係系統評鑑重視方案的動態性，能因時因地制宜，滾動修正；重視方案的情境脈絡，關注方案的特性能否符應學習者和在地的需求；能運用全方位的視角深入瞭解方案，選擇適合的評鑑方法。然在實際教學場域的應用上，仍有其限制，有待突破。
|
| 英文摘要 |
Program evaluation involves the systematic collection of information about a program to improve its quality (Rossi et al., 2004). Program evaluators may be program stakeholders rather than neutral third parties; this is particularly the case when educators are asked to design and implement curriculum reforms. In the case of Taiwan’s 12-year compulsory education curriculum, teachers are expected to design, implement, and evaluate the curriculum (the program) to ensure quality. This study used relational systems evaluation (RSE) as a model for evaluative capacity building (ECB) to equip teachers in different fields to evaluate interdisciplinary programs. Unlike a traditional evaluation model, which views the program as something set in stone to be implemented, RSE is based on a picture where a program and its implementation inform and shape each other for the program’s objectives to be achieved from the ground up (Bollinger, 2021). RSE is a cyclic process in which the local program context is evaluated in light of the relationship of program components to the whole. The changes to be effected, the scope of the evaluation, and the evaluative competencies to be developed are laid out in partnership with stakeholders using systems evaluation protocol (SEP). SEP, RSE, and ECB conceive of educators as both program practitioners and program evaluators who must thus have strong evaluative knowledge and skills (Trochim & Urban, 2021). RSE is derived from theoretical approaches such as evolutionary evaluation (EE), relational developmental systems theory, and systems thinking (ST). In EE, all programs are viewed as cycles, and evaluation is integral to each cycle. In evaluating a system, evaluators must select an appropriate evaluation model that fits the current stage of the program’s cycle (Trochim & Urban, 2021). In RSDT, a program’s environment drives development and change, and evaluators must consider whether the program requires adjustment based on the local environment or context (Bornstein, 2006). Finally, in ST, evaluators must consider the dynamic relationship between parts of the program to the whole program when implementing changes. Moreover, evaluators must establish program boundaries to determine the scope of the evaluation, identify the causes of strengths and weaknesses, and determine the critical outcomes to be measured. In RSE, all these processes are seen to build evaluative competencies in a collaborative SEP process (Chauveron et al., 2021; Urban, Archibald et al., 2021). ECB is a dynamic and intentional process for improving individual evaluation motivation, knowledge, skills, and attitudes; strengthening a team’s ability to conduct evaluations; continually optimizing evaluation quality; and making evaluation routine (Buckley et al., 2015; Clinton, 2014; Labin et al., 2012). This study analyzed three evaluative domains: evaluative thinking (ET), cultural competency (CC), and evaluative capacity (EC). Buckley et al. (2015) observe that ET is the core of ECB, which requires continual analysis and rethinking. Without ET, reviews stagnate, fail to get at essential qualities, and induce inappropriate or incorrect corrections. CC refers to the position and attitude of the evaluator toward the culture of the organization. CC recognizes that any decision made during the evaluation process reflects the values of the evaluator, from a statement of the purpose of the evaluation to the collection and analysis of data and interpretation and application of the results (Centers for Disease Control and Prevention [CDC], 2014; Frierson et al., 2002). EC refers to the basic competence of the evaluator in identifying stakeholders; establishing the intention, purpose, problem, and scope of the evaluation; selecting or developing assessment tools; collecting and analyzing evaluation data; and using evaluation results to improve a program (Frierson et al., 2002). Although several studies have discussed the implementation of ECB, empirical research on this topic is limited (Wandersman, 2014). Moreover, only two empirical studies have applied the RSE model to ECB (Chauveron et al., 2021; Urban, Linver et al., 2021). These studies have indicated that RSE is an effective model for enhancing participants’ ET and EC. However, no studies applying RSE have been conducted in Taiwan. This study had two primary goals: (1) To implement an RSE-ECB model to improve reforms of Taiwan’s 12-year compulsory education system, and (2) to translate the theoretical insights from these models into practical steps or programs through a series of workshops and self-directed learning modules. Through this process, the study will expand the literature on ECB by exploring its application to Taiwan’s unique cultural landscape of compulsory education reform. In ECB, the evaluation skills individuals acquire must be implemented in their workplace with the support of leaders in their organization (Chauveron et al., 2021; Labin et al., 2012); for teachers, these leaders would be their principal or department heads. Therefore, this study recruited a total of eight teachers in the disciplines of science and technology, art, and the humanities from one rural and one urban junior high school in addition to the principals of these schools. The study enabled these stakeholders to collaborate in employing ECB in the design, implementation, and assessment of a program of curriculum reform grounded in RSE theories. The secondary goals of this study were to (1) analyze participant responses to determine the effectiveness of the ECB program; (2) analyze changes in participants’ ET, CC, and EC; (3) analyze participants’ application of RSE perspectives in developing school-wide curriculum evaluation measures; and (4) analyze participants’ views on the RSE model. This study adopted a mixed-method design. This study collected data on ET, CC, and EC scale scores and from ECB workshop satisfaction questionnaires, feedback sheets, e-portfolio professional growth files, and interview records. The participants provided positive feedback about the ECB workshops, and their ET, CC, and EC scores significantly improved. The school-wide curriculum evaluation measures they developed reflected RSE perspectives and the unique demands of each school’s curriculum. According to the participants, compared with the traditional linear evaluation model, which only incorporated feedback at the end of the school year, the RSE model reflected the essential objectives of the curriculum and enabled real-time feedback for program revisions. Furthermore, the RSE model helped practitioners determine whether the curriculum met the needs of students and the community. In short, the RSE model provided a useful and comprehensible curriculum evaluation method for teachers and principals, although its implementation was not without challenges.
|
| 頁次 |
099-130
|
| 關鍵詞 |
文化能力
、評鑑思維
、評鑑能力
、評鑑能力建置
、關係系統評鑑
、cultural competency
、evaluative thinking
、evaluative capacity
、evaluative capacity building
、relational systems evaluation
、TSSCI
|
| 卷期 |
69:1
|
| 日期 |
202403
|
| 刊名 |
教育科學研究期刊
|
| 出版單位 |
國立臺灣師範大學
|
| DOI |
10.6209/JORIES.202403_69(1).0004
|
|
相關文獻 :
客服信箱