)) OECD ORGANISATION FOR ECONOMIC CO-OPERATION AND DEVELOPMENT lf ~f'; fH'l:~/lUIUIII fj(lf!JIIJf~~ O!rec;lo dos Servt~os de Educa~lo e Juventude ~ J'"1 ~ !J!, UNIVERSIDADE DE MACAU UNIVERSITY OF MACAU OECD Programme for International Student Assessment 2018 Macao-PISA 2018 Report: Assessment of reading, mathematical and scientific literacy performance of 15-year-old students from an international comparative perspective Educational Testing and Assessment Research Centre University of Macau Macao, People's Republic of China September, 2020 P/S1\ l'rc~jcct Cons ortium : Aus1ra l ian Council fo r h lu..:at iona l R..:s..:a rch ( AC l: R. Austra lia) cApStAn Linguist ic Quality Contro l ( lldgi um J lk utsch..:s lnst itut riir ln tcrnat ionail: l'iid,1gl1gisch..: Forschung ( DI 1' 1-:. ( icr111a 11 y ) h lucational Tl·sting Scrvic..: ( 1-:TS. lJ SA) Pearson ( LJ K ) Sta t ist ics Canad a ( Canada ) \\'.:stat ( lJSA)
-
Macao-PISA 2018 Report: Assessment of reading, mathematical and scientific literacy performance of 15-year-old students from an international comparative perspective Kwok-cheung Cheung, Pou-seong Sit, Soi-kei Mak & Man-kai leong Educational Testing and Assessment Research Centre University of Macau Macao, People 's Republ ic of China Septe1nber, 2020
Ti tl e : Macao-PI SA 201 8 Re port : Assess me nt of reading, mathematical a nd sc ientific literacy performance of 15-year-o ld students fro m an inte rna tiona l comparati ve perspcct i vc Autho rs : Kwok-chcung C he ung, Po u-scong Sit, So i-kci Mak & Ma n-kai Ico ng Publ ished by : Ed ucatio nal Testing and Assessment Research Centre, Un ivers ity of Macau Address : Room 2008, E33 , Faculty of Educatio n, University o f Macau, Avenida da Uni vcrs idade, Taipa, Macau, C hina Published Date : 2020.9 - First editi on Printed by : Designed by : Maxi Design and Prod uction Compan y Ltd. Maxi Design and Production Compa ny Ltd . ISBN 978-99965- 1- 141 -7
Foreword Andreas Schleicher, commonly named as "Father of PISA", is OECD's Director for Education and Skills. He believes that education can change society and fate of an individual. One reason for organizing PISA is that through scientific research and international comparison participating countries/economies are able to understand the strengths of each other and grasp the international trend globally. Countries/economies can devise appropriate education policies, promote quality of education, improve children's education, and elevate the overall economic scenarios of their countries/economies. Macao SAR government participated in PISA in 2003, and since then has participated in six cycles of the international assessment. Macao students' literacy performance in PISA 2018 are encouraging. Amongst the 79 participating countries/economies, Macao ranks third in the league tables of the three literacies, i.e. reading, mathematical and scientific literacy. The performance scores are only lower than China (B-S-J-Z) and Singapore. In addition, percentage of Macao students not reaching the baseline level (level 2) is very low, and it is only second to China (B-S-J-Z). Besides, OECD reported that Macao is the only economy whose education quality is continuously fast-improving in the past decade, and the growth in literacy performance is especially impressive in recent years. It is heartening to see that amongst the PISA 2018 sampled students grade retention is no longer a serious issue, as the percentage of students who are grade repeated is the lowest of all cycles of Macao-PISA assessment. Furthermore, compared with PISA 2015, male students' PISA 2018 reading literacy performance have improved a lot. There are more than ten schools having impressive growth in literacy performance. In the past decade, DSEJ has analyzed in-depth Macao students' literacy performance in PISA and engaged in a series of scientific research. Focusing on th~ implications regarding strengths and weaknesses of Macao's education, Macao SAR government is able to devise a number of policies continuously to raise schools' quality and develop students' abilities. For instance, there are policies such as continuous increase in educational investment, establishment of education development fund to optimize conditions of teaching and learning, assisting students of different levels of abilities to excel to new heights or guarantee for meeting minimum educational standards, helping students to read and learn effectively, promotion of well-being, creative thinking, critical thinking and problem-solving ability, as well as setting a platform for nurturing students' inner potentials and all-round development. DSEJ implements curriculum reform, provides opportunities to teachers and staff of the more senior ranks to engage in professional development and academic exchanges.
·~·lt·:·~--~- \ .. ~ .. ~, Macao students' brilliant results in PISA 2018 depend not only on the great support of the PRC China, Macao SAR and the participating schools, but also guidance by Macao SAR's Chief Executive and Secretary for Social Affairs and Culture. I would like to express my heartfelt thanks to them. Gratitude is also expressed to schools, principals, teachers, parents and students. Because of their dedicated efforts and unfailing cooperation, Macao is able to progress to new heights. Last, commendations and acknowledgement are due to Rector Song Yonghua of University of Macau and the research team of Macao-PISA Centre headed by Prof Cheung Kwok Cheung. Macao-PISA assessment can be launched and completed successfully in every cycle of PISA assessment. I must admit that their serious professional spirits and scientific attitudes are absolutely essential. Without doubt, PISA assessment results are of paramount importance to the development of Macao education. Through analysis of the data and information gathered, Macao SAR government is able to plan for the direction regarding the development of next ten years ' basic education. It is my sincere hope to have the continued support and cooperation of educational practitioners and parents to join hands to foster a reserve of talents so much needed in Macao's societal development. It is my fervent aspiration that Macao's basic education can be improved continuously, and that PISA helps build a long-term mechanism for achieving educational excellence and educational equity. Mr. Lou Pak Sang Director of Education and Youth Affairs Bureau (DSEJ) of the Macao SAR Government Macao-China PISA Governing Board (PGB) Member II
Figure 3.1 Gender gap in mathematical literacy performance of PISA 2018 · · · · · · · · · · · · · · · · · 42 Figure 3.2 Trend of the gender difference of Macao students' mathematical literacy performance .......................... . .. .. .. . ..... . ..... • • ........ • • ... • ... • • • • • • 43 Figure 3.3 Students ' proficiency levels on the mathematical literacy scale across all participating countries/economies in PISA 20 18 .... . .. . ......................... 45 Figure 3 .4 A comparison of the distribution of Macao and OECD student mathematical literacy proficiency level in PISA 2012 and PISA 2018 (broken down by gender) • · · · · · · · 46 Figure 3 .5 Trends of the distribution of Macao student performance at designated percentiles of the mathematical literacy scale (PISA 2003 to PISA 20 18) .... .... 4 7 Figure 3.6 Trends of the distribution of Macao student mathematical literacy proficiency level (PJSA 2003 to PISA 2018) · · · · · · · · · · · · · · ·. · · · · · · ... · · · · · · · · .. · · · · · · · ·. · · · · · · 48 Figure 3. 7 A comparison of the distribution of Macao (PISA 2003 to PISA 2018) and OECD (PISA 2018) student mathematical literacy proficiency level · · · · · · · · · · · · · 49 Figure 3.8 Changes in the distribution of Macao student performance for PISA 2012 and PISA 2018 at designated percentiles of the mathematical literacy scale (broken down by modal grades) · · · · · · .. · · · · · · · • · · · · · · · · · · · · · · · · · · · · · · · · · .. · · · · · · · 51 Figure 4.1 PISA 2018 gender gap in PISA 2018 scientific literacy performance • • . ..... • . ... 59 Figure 4.2 Trend of the gender difference of Macao students' scientific literacy perfonnance • · • • • • • • · · • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 59 Figure 4 .3 Students' proficiency levels on the scientific literacy scale across all participating countries/economies in PISA 2018 · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · 62 Figure 4.4 A comparison of the distribution of Macao and OECD student scientific literacy proficiency level in PISA 2015 and PISA 2018 (broken down by gender) · · • • • · · · · · · • · · · · · · • · · · · • · • · · · · · • • · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · 63 Figure 4.5 Trends of the distribution of Macao student performance at designated percentiles of the scientific literacy scale (PISA 2003 to PISA 2018) • • • • • • • • • • • • • 65 Figure 4.6 Trends of the distribution of Macao student scientific literacy proficiency level (PISA 2003 to PISA 20 18) · ·. · · · · · · · . . · · · · . ... .. · · · · · .... · · · · · · · · . · · · · · · ... . · · · · · 66 Figure 4. 7 A comparison of the distribution of Macao (PISA 2003 to PISA 2018) and OECD (PISA 2018) student scientific literacy proficiency level · · · · · · · · · · · · · · · · · · 67 Figure 4. 8 Changes in the distribution of Macao student performance in PISA 2015 and PISA 2018 at designated percentiles of the scientific literacy scale (broken down by modal grades) · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · 69 VII
Figure 5 .1 The three facets ofreading engagement .. • .. • • • • • • • • • • • • • • • • • • .. • • • • .. • · · .. · · • • · · 73 Figure 5.2 A comparison of item ratings of Macao students' reading enjoyment between PISA 2009 and PISA 2018 · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · 75 Figure 5 .3 Item ratings of "Enjoyment of reading" of Macao and referenced countries/ economies in PISA 2018 (% of students replying "Agree" or "Strongly agree") .. 77 Figure 5.4 Scatterplot of "Enjoyment of reading" and reading literacy performance of Macao and referenced countries/economies in PISA 2018 • • • • • • • • · · • • · · · · · · · · · · · · 78 Figure 5.5 Selected item ratings of "Understanding and memorization strategy" of Macao and referenced countries/economies in PISA 2018 (% of Macao students ranking strategies as "Very effective") · · • • • · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · 81 Figure 5.6 Scatterplot of "Understanding and memorization strategy" and reading literacy performance of Macao and referenced countries/economies in PISA 2018 · · · · · · · 82 Figure 5. 7 Item ratings of "Summarization strategy" of Macao and referenced countries/ economies in PISA 2018 (% of students replying "Effective" or "Totally effective" ) . ......... . ....... .. ......... .. • • • • • • • • • • • • • • • • • • • • · · • • • • • · · · · · · · · · · · · · 84 Figure 5.8 Scatterplot of "Summarization strategy" and reading literacy performance of Macao and referenced countries/economies in PISA 2018 • • • • • • • • • • · · · · · · · · · • · · · 85 Figure 5.9 Item ratings of "Evaluation of information quality and credibility strategy" of Macao and referenced countries/economies in PISA 2018 (% of students replying "Appropriate" or "Totally appropriate") ..... . ....... • .. .. . .. ... • • • • • • • • 87 Figure 5.10 Scatterplot of"Evaluation of information quality and credibility strategy" and reading literacy performance of Macao and referenced countries/economies in PISA 2018 . . · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · ·· · · · · · 88 Figure 5 .11 Item ratings of "Diversity of reading" of Macao students in PISA 2009 and PISA 2018 (% of students replying "Several times monthly" or "Several tin1es weekly") ............ . . . ...... • • • • • • • • • • • • • • • • • • • • • • • • • • • • · · · · · · · · · · · · · · · · · · 90 Figure 5 .12 Item ratings of "Online reading engagement" of Macao students in PISA 2009 and PISA 2018 (% of students replying "Several times daily" or "Several times weekly" • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • · · · • • • • • • · • · · · · · · • • • · · · · · · · · · · · · · · · · 91 Figure 5.13 Item ratings of "Reading self-concept: Perception of competence" of Macao and referenced countries/economies in PISA 2018 (% of students replying "Agree" or"Strongly agree") .... . .. .. .. . ... . .. . .... . ... ... • • • • ... • • • • • • • • • • • • • • • • 92 Figure 5 .14 Scatterplot of "Reading self-concept: Perception of competence" and reading literacy performance of Macao and referenced countries/economies in PISA 2018 · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · 93 VIII
Figure 5.15 Item ratings of " Reading self-concept: Perception of difficulty" of Macao and referenced countries/economies in PISA 2018 (% of students replying "Agree" or "Strongly agree") · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · 94 Figure 5.16 Scatterplot of "Reading self-concept: Perception of difficulty" and reading literacy performance of Macao and referenced countries/economies in PISA 2018 · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · 95 Figure 5.17 Item ratings of "Teacher stimulation of student reading engagement" of Macao and referenced countries/economies in PISA 2018 (% of students replying "Most lessons" or "Al l lessons") • • • • • • • • .. • .. • • • • • • • • • · • · · · · · · · .. · · · · · · · · · · · · .. · · 97 Figure 5. 18 Scatterp lot of "Teacher stimulation of student reading engagement" and reading literacy performance of Macao and referenced countries/economies in PISA 2018 · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · 98 Figure 5. 19 Item ratings of "Teacher stimulation of student reading engagement" of Macao students in PISA 2009 and PISA 2018 (% of students replying "Most lessons" or "All lessons") • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 99 Figure 5.20 Item ratings of "Classroom climate" of Macao and referenced countries/ economies in PISA 2018 (% of students replying "Some lessons" or "Never or almost never") • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 100 Figure 5.21 Scatterplot of "Classroom climate" and reading literacy performance of Macao and referenced countries/economies in PISA 2018 · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · 101 Figure 5.22 Item ratings of "Teacher support" of Macao and referenced countries/economies in PISA 2018 (% of students replying "Most lessons" or" All lessons") • • • · • · · · · · 103 Figure 5 .23 Scatterplot of "Teacher support" and reading literacy performance of Macao and referenced countries/economies in PISA 2018 · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · 104 Figure 5 .24 Item ratings of "Parental support" of Macao and referenced countries/economies in PISA 2018 (% of students replying "Agree" or "Strongly agree") • • • • • • • • • • • • • 105 Figure 5.25 Scatterplot of "Parental support" and reading literacy performance of Macao and referenced countries/economies in PISA 2018 · · · · .. · · .. · · · · · · · · · · · · · · · · · · .. · 106 Figure 5.26 Comparison of reading engagement of Macao students (PISA 2009 vs PISA 20 18) · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · 108 Figure 5.27 Score-point difference in reading performance due to indicators of reading engagement, reading self-concept or parental support • · · · • · · · · · · · • · · · · · · · · · · · · · 109 lt1: IX . 1'' • . : ... ~ . ',;:~"
Chapter 1 Project Design and Overall Performance
1.1 Overview of PISA 2018 The Programme for International Student Assessment (PISA) is a worldwide study by the Organisation for Economic Co-operation and Development (OECD) intended to evaluate 15-ycar-old students near the end of their compulsory education and assess how prepared they are for opportunities and cha llenges in real life. PISA is conducted every three years with specific focus on the knowledge and skills essential for modern societies, mainly covering reading, mathematics, and science. PISA 2018 focused on students' reading I iteracy, supplemented with mathematics and science. The test design helps researchers monitor the periodic variation in li teracy through seven rounds of PISA tests (PISA 2000, PISA 2003, PISA 2006, PISA 2009, PISA 20 12, PISA 2015 and PISA 2018). About 600,000 students were selected randomly to participate in PISA 20 18. The test sample represented over 32 million 15-year-olds in the schools of the 79 participating countries/economies (OECD, 2019a), including 3 7 OECD member countries and 42 non-OECD member countries/ economies (see Table 1.1 ). As in previous surveys, PISA 2018 explored to what degree 15-year-old students have acquired some of the skills that are essential for them to fully participate in social life, and collected information related to their reading and study, such as background information concerning students, parents, schools, teachers, famil ies and societies. 1
2 Australia Austria Belgium Canada Chile Colombia Czech Republic Demnarlc Estooia Finland FrllllCC Germany Greece Table 1.1 Countries/Economies pa rticipating in PISA 201 8 OECD colllltries Partner countries aad economies Huogary Norway Albania Hong Kong-China Panama Iceland Poland Azerbaijan (Baku) Indonesia Peru Ireland Portugal Argentina Jordan Philippines Israel Slovalc Republic Belarus Kazakhstan Qatar Italy Slovenia Bosnia&. Henegovina Kosovo Romania Japan Spain Brazil Lebanon Russian Korea Sweden Brunei Darussalam Macao-Clain a Saudi Arabia Latvia Switzerland Bulgaria North Macedonia Scotland Lilhuama Tarkey China (B-S-J-Z) Malaysia Serbia Luxembourg United Kingdom Costa Rica Maha Singapore Mexico United States Croatia Moldova Chinese Taipei The Netherlands Dominican Republic Montenegro Thailand New Zealand Georgia Morocco Ukraine United Atab Emirates Uruguay Viet Nam Note I: B-S-J-Z is an abbreviation of China's four provinces and cities pa rticipating in the PISA 2018. 8 stands for Beijing, S for Shanghai Municipality, J for Jiangs u Province, and Z for Zhej iang Province. Note 2: Scotland participated in PISA 20 18 as a partner economy. Macao-PISA 2018 reviewed the achievements of basic education for all 15-year-old students born in 2002 in Macao, and its findings are of great significance to Macao. From the perspective of international and comparative education, PISA helps the Government of Macao understands the trend and pace of global education development based on educational big data, and allows educational policymakers to review whether Macao's education has reached international standards according to the overall development, advantages and features of the basic education system, as well as the problems encountered, so as to adjust and formu late education policies. Therefore, the report will focus on the reading literacy of Macao students and relevant influencing factors. The report will also present the changes in students' literacy from PISA 2003 to PISA 2018, hoping to provide an appropriate starting point and reference for educational policymakers when they formulate and improve policies.
1.2 Features of PISA 2018 PISA assessment aims to understand how prepared 15-ycar-old students are for challenges in societies. In addition to students ' learning of school curricula, it also focuses on their ability to apply knowledge and skills to deal with and solve problems in daily lives. Therefore, main features of PISA project (OECD, 2019a) include: (1) Policy orientation : This links data on student learning outcomes with data on students' backgrounds and attitudes towards learning, and on key factors that shape their learning in and outside of school; this exposes differences in performance and identifies the characteristics of students, schools and education systems that perform well. (2) Innovative concept of " literacy" : This refers to PISA's emphas is on students' overall understanding of key concept rather than the understanding of specific subject knowledge in the schoo l curriculum. This is reflected in the fact that PISA not on ly emphasizes students' capabi lity to apply knowledge and skills in key areas, but also pays attention to their abil ity to analyze, reason and communicate effectively as they identify, interpret and solve problems in a variety of situations. (3) Relevance to lifelong learning: The acquisition of literacy is a lifelong learning process , where the key is learning how to learn. The concept of "literacy" put forward in the OECD Learning Framework 2030 refers to the integration of knowledge, skills, attitudes and values (OECD, 2019a), which indicates that PISA not only assesses the students' cognitive ability and life skil ls , but also attempts to understand their experience and affection in life from various aspects. It also pays attention to the cultivation of non-cognitive process such as feelings, attitudes and values, and investigates students' learning motives and their belief in self and learning strategies. The education concept of "literacy" can provide reference for the education reform of schools, and is also applicable to everyone's lifelong learning. ( 4) Continuous regular monitoring: PISA is conducted every three years to allow participating countries/economies to continuously and regularly monitor the effectiveness of all education policies, as well as the progress of key education goals. 3
4 (5) Universality and cooperation in geographic area: The number of countries/economies participating in PISA grew from 43 in PISA 2000 to 79 in PISA 2018, and there will be more countries/economies participating in PISA 2021. The continuous increase in the number of participating countries/economies makes the programme more representative, and facilitates participating countries/economies to identify suitable benchmark and learning targets from various education systems. In addition, by analyzing in-depth the factors affecting literacy in different education systems, it also helps the participating countries/economies learn from and promote each other. Decisions about the scope, nature of the assessment and the background information of the PISA study are jointly made by the ministers of education of all participating countries/economies as well as international leading experts. The concerns of each government are also taken into account to form consensus and formulate policies. Based on the previous research results, PISA 2018 continued to promote and improve the computer-based assessment, including the two key components on tests and questionnaires. The assessment took 120 minutes to complete, including 60 minutes of reading and 60 minutes of mathematics and/or science questions. The questionnaire took approximately 45 minutes to complete. As the focus of the PISA 2018 is reading literacy, reading items were organized into test units with one or several texts, charts or problems encountered by students in real life, and presented in a simulated web environment. In addition, on the basis of PISA 2015 computer-based assessment, PISA 2018 further advanced the methodology and implemented the multi-stage adaptive testing design (Figure 1.1) in its reading assessment. Unlike traditional classroom testing method, the PISA test delivery system automatically selected the most appropriate test units for students depending on their responses to the earlier reading test units, the response time required for the preset test units, as well as allocation congruent with cognitive demands of the test units, thus more accurately estimating the students' competency (Yamamoto, Shin, & Khorramdel, 2018; OECD, 2019a, 2019b).
Stage 1 H: Test units with high level of difficulty M: Test units with medium level of difficulty L: Test units with low level of difficulty . . . . \ \ar. err. \ •••• -----• ... . -~ .. . . ... ~ . .. . . .. . . .. . ... \ . ·····..••...•...... . . . ···• ...... : II?.. \ ~ '-llll ····· ··· ·······: '-lllll Stage 3 Figure 1.1 Design principle of multi-stage adaptive testing procedures 5
1.3 Description of PISA 2018 Score PISA assessment generally consists of four 30-minute clusters of test material, with each cluster containing several test units. The time limit for completing all the questions is 120 minutes. As PISA 2018 adopted the multi-stage adaptive approach in reading assessment, reading material was organized into blocks instead of clusters. The reading assessment was composed of a core stage followed by stage 1 and stage 2. PISA student delivery system first estimated the students' reading ability according to their responses to the reading items in the core stage. In stage 1 and 2, students were assigned test units of either greater or lesser difficulty, depending on their performance in earlier stages. For mathematics and science test units, test clusters were adopted, where one to two mathematics or science test clusters were combined for students to answer. Each student was randomly assigned one of the 36 test booklets. Students' performance in reading, mathematics and science in PISA 2018 was anchored on the trend line of PISA 2000, PISA 2003, PISA 2006, PISA 2009, PISA 2012 and PISA 2015. After scaling, the OECD average is approximately 500 points with a standard deviation (SD) 100 points. Because the scores are normally distributed approximately two-thirds of students across OECD countries scored between 400 and 600 points (OECD, 2019a, 2019b). As reading literacy was the major domain in PISA 2018, reading questions accounted for the highest proportion in the test delivery system. In addition to the combined reading scale, results were also presented according to three reading cognitive process subscales: locate information, understand, evaluate and reflect; and two reading textual form subscales: single and multiple. In order to better illustrate the abilities of students when attaining certain scores, PISA describes the specific performance of students by the proficiency level. The proficiency level for reading literacy in PISA 2018 was divided into nine levels, with 80 points difference between each level. It calibrated the students' ability and question difficulty through the item response theory (IRT) to understand the knowledge and skill s that students can master at a certain level according to the position of students and questions on the proficiency scale. In other words, the proficiency level of each literacy was determined based on the majority of test questions at that proficiency level that students were most likely to answer successfully. If the test questions focused only on simple tasks involving familiar scenarios, or displayed obvious information in the question stem, the questions were usually easy questions with low cognitive demands; on the contrary, if the test questions required students to execute complex tasks under unfamiliar scenarios, they were difficult questions with high cognitive demands. Figure 1.2 demonstrates the relationship between the literacy proficiency level of students and the difficulty of test items in PISA assessment. 6
literacy scale Items wtth relatively high difficulty Items with moderate difficulty Items with relatively low difficulty Item V • • Uem II • • Data source: OECD, 2019b Student A, with 1relatively high proficiency Student 8, with moderate proficiency Student C, with relatively low proficiency Figure 1.2 We expect student A to successfully complete items I to V, and probably item \If as well We expect student B to successfully complete items I and II, and probably item Ill as well;. but not items Vand VI, and probably not item IV either We expect student C to be una b1le to successfully complete any of items II to VI, and probably not item I either Re lationship between stude nt abili ty and item difficulty on the performance scale 7
1.4 Sampling Design of Macao-PISA 2018 Macao-PISA Centre commenced the PISA 2018 main survey on 21 April, 2018. Within one month, all the 15-year-old students in the 45 secondary schools were assessed. According to school type, study programme, and language of instruction, Macao-PISA Centre used the sampling software KeyQuest provided by the PISA Consortium to devise a list of sampled students. The target students were all born in 2002. When assessed in April/May 2018, their ages were between 15 years 3 months and 16 years 2 months. After the PISA 2018 Consortium validated the sampling information against Macao's census statistics and verified the list of sampled students provided by the participating schools, a total of 3811 students were approved to participate in the main survey. Finally, 3775 students were assessed with a commendable 99.1% response rate. Table 1.2 presents characteristics of the schools and the 15-year-old students sampled and tested in PISA 2018. The statistics are broken down by school type, study programme, and language of instruction. Table 1.3 presents the number of students (males/females) sampled and tested in PISA 2018. 8 Table 1.2 Characteristics of schools sampled and tested in Macao-PISA 2018 Stratifying variable Number of Number Number of Number of schools of schools students students sampled tested sampled tested School Type Government 4 4 133 133 Private-In-Net 36 36 3274 3247 Private 5 5 404 395 Study Programme Grammar-International 40 40 3693 3657 Technical-Prevocational 5 5 11 8 11 8 Language oflnstruction Chinese 31 31 2765 2745 English 7 7 480 473 Portuguese 1 l 37 34 Chinese & English 5 5 462 456 Chinese & Portuguese I 67 67 Total 45 45 3811 3775 Note l : All sampled schools offer basic education courses for 15-year-old students. A total of 46 schools met the PISA sampling requirements, among which one school was excluded for having no 15-year-old students during the test period, and a total of 45 schools were sampled and tested in the end. Note 2: All 15-year-old students sampled were born in 2002. Note 3: PISA 2018 was essentially a census as a ll el igible schools and students were sampled for assessment.
Table 1.3 Number of students sampled and tested in Macao Number of 15-year-old secondary school students Number of students sampled Number of students tested Response rate 3811 ( 1930 males; 1881 females) 3775 ( 191 3 males; I 862 females) 99. 1% Table 1.4 shows the grade distribution of Macao's 15-year-old students tested in the Macao-PISA 2018. Of note is that 29. 7% and 57 .9% of the Macao sample are studying at two modal grades (i.e. Grades 9 and 10) respectively, whereas 1.9 % and 9.4% of sampled students were studying at the lower Grades 7 and 8. The largest number of l 5-year-old students participating in Macao-PISA 2018 was attending Grade 10, accounting for nearly 60% of all students tested, while the percentage of students in Grades 7 and 8 continued to drop to 11.3%. Compared with the grades of 15-year-old students participating in the previous PISA studies (Figure 1.3), the percentage of students attend ing Grade 10 or above gradually rose from 22.2% in PISA 2003 to 58.9%, while the percentage of students attending Grades 7 and 8 gradually decreased from 43.7% to 11.3%, indicating that significant progress has been made in the education system of schools in Macao for lowering the retention rate. The distribution of grades of 15-year-old students may affect tbe average score of Macao and the literacy performance of schools. It is obvious that a sampled school's literacy performance is expected to be affected negatively when the proportion of students studying at the lower grades (i.e. Grade 7 and 8) are sign ificantly higher than the corresponding figures in the Macao sample. Similarly, a sampled school's literacy performance is expected to be affected positively when the proportion of students studying at the higher grades (i.e. Grade IO and 11) are significantly higher than the corresponding figures in the Macao sample. Table 1.4 Distribution of grades of Macao 's 15-year-old students tested Grade Number of students(%) Number of males(%) Number of females(%) 7 (Form 1) 71 (1.9) 49 (2.6) 22(1.2) 8 (Form 2) 356 (9 .4) 23 1 (12.1) 125 (6.7) 9 (Form 3) 1121 (29. 7) 577 (30.2) 544 (29.2) 10 (Form 4) 2187 (57.9) I 036 (54.2) 1151(61.8) 11 (Form 5) 39 (1.0) 20 ( 1.0) 19 (1.0) 12 (Form 6) l (0.0) 0 (0.0) 1 (0.1) Total 3775 (100.0) 1913 (100.0) I 862 ( 100.0) 9
% 70 PISA 2003 60 PISA 2006 58.9 • PISA 2009 50 • PISA 2012 • PISA 2015 40 • PISA 2018 30 20 10 0 Gr.,de 7 Gr.,de s Gr.ade9 Gr.,de 10 ••d •bove PIS.A.2003 15.3 28.4 34.1 22.2 PIS.A.2006 S.2 21.2 33.4 37.2 • PIS.A.2009 6.7 19.2 34.9 39.2 a PJS.A2012 5.3 16.3 33.2 45.l • PJSA2015 2.9 12.2 29.7 SS.2 • PISA2018 1.9 9.4 29.7 SS.9 Figure 1.3 Distribution of grades of 15-year-old secondary students in Macao (PISA 2003 to PISA 2018) 1.5 Performance of the three basic literacies in Macao-PISA 2018 Under the framework of global economic integration, the competition between countries or regions has become increasingly fierce, as governments have gradually realized that education, technology and innovation are the basis and key elements for the sustainable national competitiveness, and that the improvement of international competitiveness has become an urgent issue. In the long run, if a country's economy is built up rested on primary and general production factors, its competitiveness is usually unstable; on the contrary, countries with rich high-quality human resources and innovation capacity will gain advantages in future international competition. Since human resources are the most important cornerstone of social and economic development, the investment in education can increase the international competitiveness of a country/economy. In today's knowledge-based economy led by high-tech industries, talents with global vision and the ability to integrate cross- disciplinary knowledge and use information to solve complex problems will stand out in the highly competitive global economy. Therefore, to maintain its advantages in regional competition, Macao must seek development from education for a talent pool to serve the changing needs of the society. 10
In recent years, the Macao SAR Government has vigorously supported the development of education , and improved the quality of Macao's overall education system by adjusting and increasing the proportion of education expenditure, so as to facilitate the transformation of Macao's economy from relying on material and human resources input to depending on technology and knowledge, thus realizing the sustainable development of Macao (Cheung et al., 2018). Meanwhile, the rise of quality education systems around the world has also provided favorable conditions for the development of education, and put forward higher requirements for the quality of education. Therefore, education policymakers must further review the regional education system, and plan the interchange, cooperation and sharing wi thin and outside the region with higher standards and broader visions, so as to satisfy the requirements of social development and regional functional positioning. The main purpose of PISA is to improve education policies to ensure that students are best prepared for life and work in the future. Therefore, special attention should be paid to whether students can flexibly apply the knowledge and skills acquired to so lve the real issues in different scenarios. These issues include not only problems under the network of their personal life and school education, but also problems in their daily life and work. PISA defines students whose li teracy performance reaches level 5 or above as high-performing. Those students who are high-performing in all three basic literacies, i.e. the all-rounders, are especially valued, since only a small percentage of the population exist in most countries/economies. Therefore, by using the PISA results to preliminarily assess Macao's talent reserve and compare wi th the situation in other countries/economies, it will facil itate policymakers to better plan the future development of education. Table 1.5 shows the average scores of countries/economies ranking top-15 in the three core literacy domains in PISA 2018. As shown in the table, 15-year-old s tudents in Macao performed very well in reading literacy (525 points), mathematical literacy (558 points), and scientific literacy (544 points). Compared with the results of the previous Macao-PISA studies (see Figure 1 .4), after ten years of continuous efforts and teaching reform, students' reading literacy has gradually improved from its low point in PISA 2009 to a point significantly higher than the average of OECD countries in PISA 2018. The improvement of students' overall reading proficiency is equivalent to the progress of one PISA schooling grade, which shows that Macao has made sound progress in its basic education system with the efforts of a ll parties. 11
12 Table 1.5 Performance of the top- 15 countries/economies in reading, mathematical and scientific literacies in PISA 201 8 2 Singapore 549 Singapore 569 Singapore 3 Macao (China 525 Macao (China 558 Macao (China) 4 Hong Kong (China) 524 Hong Kong (China) 551 Estonia 5 Estonia 523 Chinese Tai ei 531 Ja an 6 Canada 520 Japan 527 Finland 7 Finland 520 Korea 526 Korea 8 Ireland 518 Estonia 523 Canada 9 Korea 514 Netherlands 519 Hong Kong China 10 Poland 512 Poland 516 Chinese Taipei 11 Sweden 506 Switzerland 515 Poland 12 New Zealand 506 Canada 512 New Zealand 13 United States 505 Denmark 509 Slovenia 14 United Kingdom 504 Slovenia 509 United Kingdom 15 JaQan 504 Belgium 508 Netherlands Portugal 492 Portugal 492 Portugal OECD average 487 OECD average 489 OECD average S90 S10 !! S50 0 " ,,, ., " i S30 E ~ C. SIO 490 470 450 PISA 2003 PISA2006 PISA2009 PISA2012 PISA2015 - Reading (Macao) 498 492 487 509 509 --Mathematics (Macao) 527 525 52S S38 544 - science (Macao) 52S SIi 511 S21 529 - • - Reading (OECD average) 494 492 493 496 493 - • Mathematics(OECD average) 500 498 496 494 490 - • - Science (OECD average) soo soo 501 SOI 493 Figure 1.4 Trends of Macao 15-year-olds in reading, mathematical and scientific literacy (PISA 2003 to PISA 20 18) 551 544 530 529 522 519 518 517 516 511 508 507 505 503 492 489 558 544 S2S PISA 2018 525 558 S44 487 489 489
1.6 Ratio of High Performer in Macao-PISA 2018 Figure 1.5 and Table 1.6 respectively compare the percentage of students from different countries/economies with high performance in three basic literacies in PISA 20 18. Figure 1.5 ranks the countries/economies by their ratios of students with high read ing literacy performance and all-round students from high to low. The blue bar on the right shows the percentage of students with high reading performance, with the dark blue area indicating that the students from such countries/ economies not only present high performance in reading, but a lso perform we ll in mathematics and science. The gray bar on the left reflects the percentage of students from participating countries/ economies with high li teracy performance in mathematics and/or science, but not in reading literacy. As indicated by the data, over one quarter (25.8%) students from Singapore, one fifth (21.7%) students from China (B-S-J-Z), and one seventh (13.8%) students from Macao are reading high performers. It is notable that the proportion of high performers of China (B-S-J-Z) is the highest, with nearly half of the students demonstrating high performance in at least one of the three literacies. Those from Singapore took up 43.3%, while those from Macao (32.8%) and Hong Kong (32.3%) accou nted for one third (see Table 1.6). In addition, China (B-S-J-Z) also has the highest proportion of all-round students with high performance in all three literacies, reaching 17.7%; Singapore ranks second with 14.7%. Macao ranks third with 7.4% in the world; meanwhile, Macao had a significant increase in its percentage of a ll-round students compared with PISA 20 15, one of the few regions with s ignificant growth. 13
• Mathematics only • Science only • Mathematics and science Singapore B-5-J-Z (China) Canooa Hong Kong (China) Anand Estonia Macao (Ctina) United States sweden Korea New Zealand Australia Poland Ireland United Kingdom Germany Norway Chinese Taipei Israel Japan Belgium France Netherlands OECD average Denmark Czech Republic Swltiertand Slovenia Luxemboll"g Austria Portugal Iceland Hungary Russian Federation Italy Malla Lithuania LaMa United Arab Emirates Croalla Slovak RepLllllc Belarus Greece Ukraine Ttrtey Oatw Chile 5erl>la Bulgarta Brazil Cyprus Uruguay Romania Brunel Oarussal..-n Maoova Coloml>la Peru Montenegro Mexico Le~non Argel1lna Costa Rica . Malaysia Kazakhstan Albania Repul>lk: or North Macedonia Joroan Georgia Panama Bosnia and Herzegovina Thailand Baku (Azerbaijan) Saudi Arabia Domlncan Repu~lc Indonesia Phlllppines Morocco Kosovo % 30 25 20 15 Percentac• of Top performers • Reading only • Reading and mathematics or science • Reading, mathematics and science ---.C --- ..... ----·------ ---"''" ... _ .. .. - 1o ,b h, -~ -~ -~ -~ -. . --' 10 0 10 15 20 Data source: OECD, 2019b, Figure I 8. 1 (PISA 2018 Database, T.TOPLOW.tp.) Figure 1.5 Percentage of top-performers in the three basic literacies in PISA 201 8 14 25 30 %
Table 1.6 Percentage of Macao 's top-performers in the three literacy domains for comparison with the top- 15 PISA 201 8 countries/economies Countries/ Economics China (B-S-J-Z) Singapore Macao (China) Hong Kong (China) Korea Chinese Taipei Canada Japan Estonia Netherlands Poland Finland New Zealand Switzerland United Kin dom OECD average 9.6 3.1 14.9 2.7 1.4 13.4 2.8 1.9 14.3 1.8 0.4 15.4 3.1 1.3 9.0 3.6 0.8 10.7 1.6 1.9 5.6 3.7 2.2 7.2 1.6 1.8 5.3 3.0 1.6 8.1 1.6 1.2 6.2 1.3 2.5 3.0 1.4 2.2 3.5 2.1 1.0 8.6 1.7 1.7 4.5 1.3 I.I 4.6 Percentage of top performers(%) R &M R only OR R &S 0.8 3.2 3.0 8.1 1.6 4.8 2.6 6.9 2.9 3.8 1.4 2.8 4.5 5.1 1.6 2.7 3. 1 4.2 1.0 2.7 3.0 3.9 4.2 4.8 3.8 4.3 I. I 2.9 3.2 3.7 2.5 2.9 17.7 14.7 7.4 5.4 6.4 6.7 5.5 5.9 6.6 5.3 5.3 5.2 5.0 4.1 4.6 3.4 R only ORR+ IL OR R+2L 21.7 25.8 13.8 14.8 13.1 10.9 15.0 10.3 13 .9 9.1 12.2 14.2 13.1 8.1 11.5 8.7 Note: M=mathematics; S=science; R=reading; L=literacy At least IL is top-perfom1ing 49.3 43.3 32.8 32.3 26.6 26.0 24.1 23.3 22.5 21.8 21.2 21.0 20.2 19.8 19.4 15.7 The results of PISA 2018 recognize the progress made in Macao's basic education system with the support of the country and the Government of Macao, and also reflect the limitations and deficiencies in Macao. By participating in the PISA study, Macao has always hoped to continuously monitor and improve the quality and status of its basic education from an international macro perspective via large-scale education survey and research, so as to promote schools in Macao to establish self-improvement curricula and teaching accountability mechanisms. Though there is some progress for Macao to educate students who are top-performing in the three literacies there is still much room for improvement. In this regard, Macao can learn from practices of the two top-performing education systems, i.e. China (B-S-J-Z) and Singapore. While cultivating outstanding professional talents, especia lly emphasizing on the balanced development of students' key capabili ties and core literacies, Macao's education system also needs to thoroughly discuss and formulate education policies and specific measures for talent cultivation from the macro perspective of global economic development. 15
Chapter 2 Reading literacy performance of Macao 15-year-old students
PISA 2018 focuses on the reading literacy of 15-year-old students. Same as PISA 2000 and PISA 2009, reading literacy was the major domain again in the seventh PISA assessment, whereas mathematical and scientific literacy were assessed to a minor extent. PISA 2018 defines that "Reading literacy is understanding, using, evaluating, reflecting on and engaging with texts in order to achieve one's goals, to develop one 's knowledge and potential and to participate in society." (OECD, 2019a, p. 28). PISA targets the reading literacy of 15-year-old students near the end of compulsory education, therefore, it must focus on reading literacy ski lls that include finding, selecting, interpreting, integrating and evaluating information from the fu ll range of texts associated with situations that extend beyond the classroom (OECD, 2019a). Due to the rapid social and economic development, the quantity and variety of information are increasing, and we need to process and apply these materials with new and increasingly complex methods. In the past, schools used to pay more attention to the collection and retention of information, but in the era of knowledge economy, the concept of knowledge has been broadened. If a person fully participates in social life, then he/she must be well-equipped with the basic ability to ascertain, acquire, understand and reflect on a ll kinds of information. Meanwhile, the development of technology also fundamentally changed the way people read and exchange information. Knowledge no longer dominates, and the new generation must possess the abil ity of information transmission, as well as data analys is, processing and application in order to survive in the society. Also, the Internet is casting impact on every aspect of life from study within and outside schools, workplace to personal affairs. According to the survey by the International Telecommunication U nion, only 1.7% of the world's population used the Internet in 1997, but by 2014, the number has gone up to 40.4%, equaled to 3 billion people. Between 2007 and 2013, the number of mobile phone subscriptions doubled, and the number of mobile broadband users increased to 2 billion (International Telecommunications Union, 2014). Personal and professional development is a lifelong endeavor and the students of tomorrow will need to be skilled with digital capacity in order to successfully manage the increased complex ity and quantity of information available (OECD, 2019a). This requires the update and extension of the concept of reading literacy, which should include both the basic reading ability, and the digital reading skills of higher level, while realizing that the concept of reading literacy will be constantly changing with the changes in new technology and society. Therefore, reading literacy is not only a foundation for achievement in other subject discipline within the educational system, but also a prerequisite for successful participation in most areas of adult life (Cunningham & Stanovich, 1997; OECD, 2013; Smith et al. , 2000). 17
The typical feature of the 21 st century is the Internet and informatization. As the medium through which we access textual information moves from print to computer screens and to smartphones, the structure and formats of texts have changed. The ability to understand and interpret multiple parallel texts, including literary texts, is still important, but so are the strategies to process complex information, including the analysis, integration and interpretation of relevant information from multiple sources or multiple texts. In addition, productive students will need to use information from across domains, such as science and mathematics, and employ technologies to effectively search, organize and filter a wealth of information (OECD, 2013, 2019a). Meanwhile, as the quality and form of information vary, productive readers should also be able to identify and screen reading materials, integrate cross-domain knowledge, and conduct evaluation and reflection. These will be the key skills necessary for full participation in the labor market, in further education as well as in social and civic life in the 21'1 century (OECD, 2019a). In other words, reading based on the rapid communication and information exchange under globalization requires readers to not only simply read and understand, but also construct knowledge, skills and strategies under various circumstances through interaction with peers and social groups, and proactively extract important information and concept to construct meanings (Chen & Chen, 2018; Ruddell, 2008; Snow & Sweet, 2003). 2.1 PISA 2018 reading literacy assessment framework PISA 2018 reading literacy assessment framework is based on the understanding of contemporary "extensive reading" concept as well as the requirements for obtaining and utilizing information in daily life scenarios. It is mainly reflected in three aspects: the definition of reading literacy, reading process, and text type. The changing nature of reading literacy Based on PISA 2009, the "evaluating" of texts has been added as part of the reading literacy in the PISA 2018 reading framework, and the word "print" has been removed. PISA's emphasis on "evaluating" lies in the further understanding of the diversified, interactive and constructive nature of reading emphasized by the various cognitive theories on reading (Kintsch, 1998; McNamara & Magliano, 2009), where readers employ the knowledge acquired and a series of texts and contexts in social life, use all means, skills and strategies to identify information, monitor and maintain their understanding of texts, and evaluate the relevance and validity of information to construct the meaning of the texts to themselves. The "evaluating" emphasizes readers' ability to identify and screen texts. During the identification and screening, the "evaluating" is to reflect the readers' "goal orientation" in reading, so that readers can accurately and effectively judge the relevance of the 18
author 's opinions and their own reading purposes. In the era of digitalization, the ability of evaluation and reflection has become an important indicator of whether people can become excellent readers. The removal of the word "print" reflects the extension of reading li teracy in PISA 2018 based on the develop·ment of the digital era. In PISA 20 18, the phrase "texts" is meant to include all language as used in its graphic form: handwritten, printed or screen-based. It is worth-noting that, in PISA 2018, in addition to written symbol, "texts" also inc lude visual displays such as diagrams, pictures, maps and tables. In other words, these visual texts can exist either independently, or they can be embedded with in larger texts (OECD, 2019a). Reading process The "text processing" in PISA 20 18 refers to the cognitive reading process. Readers need to complete three reading processes including information locating, understanding, as wel l as evaluation and reflection, to realize the full understanding of texts. Regardless of the language proficiency level, through PISA's reading literacy assessment, it is expected that readers shou ld demonstrate their level of ability at different reading levels. In PISA 2018, reading processes are divided into text processing and task management, and text processing is further divided into three reading cognitive process, including information locating, understanding, as wel l as evaluation and reflection, connected by reading fluency throughout. Additionally, task and context management is divided into two levels in PISA 2018, including goal setting and planning, as well as monitoring and adjustment, with particular emphasis on the assessment of metacognitive reading strategies for text processing (see Figure 2. 1 ). ln other words, if the task is to ask readers to extract or retrieve information, locating information in the text can achieve the purpose ofreading; however, ifreaders need to reflect on and evaluate the information, then simply locating information or understanding texts will not complete the purpose of such reading task regardless of how well they perform. Based on the concept of lifelong learning, PIS A's reading li teracy has a very clear functional purpose. The reading assessment is a component of reading literacy based on the thinking ability of reasoning, integration, interpretation, analysis, questioning, reflection and evaluation in the cognitive reading activities. The ability to change reading strategies according to reading goals is the abi lity to monitor and adjust, which is the metacognitive reading strategy. Therefore, the reading process in PISA 2018 is reflected not on ly in the multi-level text processing, but a lso in the multi-perspective task management. 19
Processing textual Information Read fluently Text types Locate information Access and retrieve information within a text Search and select relevant text Understand Represent lirteral meaning Integrate and generate inferences evaluate and reflect Assess quality and credibility Reflect ,on form and content Figure 2.1 PISA 2018 reading literacy assessment framework Based on PISA 2009, the word "print" has been removed from the definition of reading literacy in PISA 2018, due to the fact that under the background of informatization, digital reading has gradually become the mainstream reading method, leading to changes in the unit of texts. Based on the digital reading environment, the concept of "static text" and "dynamic text" are put forward in PISA 2018. Specifically, "static texts" are typically one or several screen pages arranged linearly with a s imple organization and low density of navigation tools, wh ile "dynamic texts" feature a more complex, non-linear organization and a higher density of navigational devices. In addi tion, text units are also divided into "single text" and "multiple text" in PISA 2018. Single texts refer to texts written or published by an author (or a group of authors) at a certain time that are independent of other texts; multiple texts refer to texts that are written by different authors, published at different times, with different titles or reference numbers, or composed ofrelatively independent texts, which can be continuous texts or non-continuous texts, and the relationships or views between texts can be loose, inconspicuous, or even contradictory (OECD, 2019a). The above two text types can be interrelated, as s ingle texts can appear in dynamic texts and multiple texts can appear in static texts. In addition, the reading tasks of single texts often refer to traditional reading cognitive capabilities, while the reading tasks of multiple texts refer to the ability to quickly extract information, integrate and understand the texts, which to some extent increases the difficulty of completing the PISA reading tasks. 20
The design of PISA features the emphasis on the abi li ty to solve problems and the integration of interdisciplinary knowledge domains. It pays attention to high-level thinking, focuses on finding useful clues to solve problems during the reading process, requires readers to state their own points or v iews according to the texts while providing appropriate reasons to support their statements, and emphasizes the ability to express and communicate, as well as the combination of daily life scenarios and reading tasks. It proposes that students should properly guide themselves under the cross-text and multi-text reading environment, manage reading tasks and scenarios, set targets and monitor the entire reading process. It is notable that in order for students to become productive readers, they must also utilize digital skills and reading strategies to effectively search, locate and organize the large amount of information. These abil ities are the most basic literacies of future citizens. On this basis, under the development of "globalization", a sound reading literacy foundation can provide students with the ability of lifelong learning, which is also the best competitiveness of a country in the future. The in-depth reading policy is one of the most important education policies for all governments to cultivate their modem citizens with global perspective. 2.2 Macao students' overall reading literacy performance and its gender gap The average score of reading literacy performance of Macao 15-year-old students is 525, w ith a standard deviation of 92. Comparing Macao's 15-year-old students' reading literacy performance with that indicated by the PISA 2015 study, there is a significant improvement of 16 score points. The data also shows that the improvement in PISA 2018 is more visible than in PISA 2009 by 3 8 score points. Moreover, Macao's PISA 2018 reading literacy performance is the highest of all previous cycles of assessment, indicating that there is great progress in reading literacy performance of our I 5-year-old students. Macao ranked the third among 79 participating countries/economies in terms of the average reading literacy score in PISA 2018, significantly higher than the score of OECD average (= 487, see Table 2.1 for details) statistically. In decreasing order of the mean of the reading literacy score, the countries/economies higher than Macao are China (B-S-J-Z) and Singapore, while Hong Kong, Estonia, Canada, and Finland are slightly lower than Macao. Ofnote is that Macao's reading literacy performance is not statistically significantly different with that of Hong Kong, Estonia and Fin land. 21
Rank Table 2.1 Ranking, mean in reading literacy performance: A comparison of Macao and referenced economies Countries/ Econom ics Combined scale Mean SD Reading process subscales Locate Understand information Evaluate & re fl ect Text-source subscalcs Single Multiple -.r -·. Chlna_. .. .s-i~z 1· •• ·sss ·· · .- -_,._-:::-.-.:.: ;..-:• si:•_;,, .- _:•-/s~2 :·. ·, >-s~s. ·"•::· --~·-: ss6 ·.-: s64.• · ." · 2 Singapore 549 109 553 548 561 554 553 3 Macao China S2S 92 S29 S29 S34 S29 S30 4 Hong Kong (China) 524 99 528 529 532 529 529 s Estonia 523 93 S29 526 521 S22 S29 6 Canada 520 100 517 522 7 Fhiland S20 100 52§ ,20 8 Ireland 518 91 517 9 t<:orea 514 102 S2S 10 Poland 512 97 1:1 S06 108 12 New Zealand 506 509 13 sos sos 14 United Kingdom 504 51 1 508 15 Japan S04 502 506 24 Portugal 492 96 489 489 494 487 494 ECO average 487 99 487 487 489 485 490 Females around the world enjoyed a clear advantage in reading. The gender difference between male and female students varied between 10 points and 65 points (Figure 2.2). Just like other countries/economies participating in PISA, females in Macao outperformed males in average (514 for males, 536 for females), but the gender gap between male and female students in Macao was also narrowed from 34 points in PISA 2009 to 22 points (Figure 2.3). Regarding this achievement, Macao males show considerable improvement in reading literacy performance. The male students ' mean score is not only above the OECD average, but a lso even higher than females in most of the countries/economies (Figure 2.2). Comparing PISA 2009 and PISA 2018, there is improvement in reading literacy. The results show that male students can progress in performance closer to that of the female students. 22
-All ltudents • loyt • Gk1t .. Data source: OECD, 2019c, Figure II.7.1 (PISA 2018 Database, Tables 1.GENDMEANS.read and I.MEANS.read) Figure 2.2 Gender gap in reading literacy performance of PISA 2018 23
540 520 500 480 460 440 420 400 - Boys (Macao) - Girls (Macao) Boys(OECD) Girls (OECD) .._Macao average 504 Sil PISA 2003 PISA 2006 PISA 2009 Figure 2.3 536 527 SIS 478 PISA 2012 PISA 2015 PISA 2018 Trend of the gender difference of Macao students' reading literacy performance 2.3 Macao student performance and gender difference in three reading processes Both PISA 2018 and PISA 2009 selected reading literacy as the major research domain. This section analyzes the gender difference of Macao students in three reading processes in PISA 2018 and PISA 2009. In Figure 2.4, the darker color of the same color scheme indicates the data of PISA 2018, while the lighter color indicates that data of PISA 2009. By comparing the results of the two PISA assessment with focus on reading literacy, it can be seen that both male and female students have made significant progress in terms of the three reading subscales, while male students showed greater progress than female students in the three reading processes, especially in the evaluation and reflection subscale, with 63 points up in PISA 2018 compared with PISA 2009. The significant progress of males has narrowed the gender gap between male and female students by 50% in terms of evaluation and reflection, dropping from 42 points in PISA 2009 to 22 points in PISA 2018. With regard to information locating, the corresponding access and retrieve subscale, as well as understanding and the corresponding integrate and interpret subscale, the gap between male and female students has also narrowed by one in third, with both scales dropping from 31 points in PISA 2009 to 20 points in PISA 2018 (Figure 2.5). 24
560 540 ~ g :::l 520 .., g 500 C. s· (IQ rn 480 C t;T rn (") c!. 460 ~ rn (") 0 .., 440 ~ 420 400 PISA 2009 Boys - PISA 2018 Boys PISA 2009 Girls - PISA 2018 Girls ....,_PISA 2009 Macao average - • - PISA 2018 Macao average Understand Locate Information (Access & Retrieve) (Integrate & Interpret) 477 473 519 519 509 504 538 539 493 488 529 529 Figure 2.4 534 ----------481 Evaluate & Reflect (Evaluate & Reflect) 460 523 502 545 481 534 Mean scores of three reading process subscales of Macao students in PISA 20 18 and PISA 2009 (broken down by gender) 25
45 --~ r----------I n 40 I PISA 2018 I ----~ i:r IIO •----------I Jg 35 -- -~ II> r, 30 Q ri II> 25 er --- 22 ~ 20 20 ~ IIO 20 --s-(IQ 15 II> Locate Information = Understand Evaluate & Reflect er II> r, --IIO - Access & Retrieve Integrate & Interpret Evaluate & Reflect ~ II> 15 -~ ... 20 ~ ~ i 25 ; 30 t::d 31 ~ 35 31 - r----------I 40 : PISA 2009 : L---------- 42 45 Figure 2.5 Gender gap of three reading process subscales of Macao students in PISA 20 18 and PISA 2009 26
2.4 Overall and gender difference in Macao students' distribution of proficiency levels of the reading literacy PISA's reading literacy scale is divided into nine proficiency levels from below 1 c to 6. Each proficiency level specifically describes the typical tasks and abi li ties that students reaching this level can do (OECD, 20 19a, p55). PISA defines students at literacy level 5 and level 6 as hi gh-performers, and those below level 2 as low-performers. As shown in Figure 2.6, nearly 90% of Macao students have reached the PISA reading literacy benchmark (level 2), mainly at level 3 and level 4, accounting for 56% of all students sampled in Macao. According to the data, apart from China (B-S-J-Z), Macao has the largest proportion of students reaching the baseline level 2, and the number of low-performing students in Macao has also been decreasing, with the percentage of students below proficiency level 2 remaining at about 11 %, indicating that students with low li teracy who cannot participate in today's economic and socia l activ ities only account for a small portion at the moment. Also, the percentage of high-performing students in Macao at level 5 and level 6 has increased to about 14%. However, the proportion of Macao students with high reading literacy performance was still quite low compared with neighboring countries/economies, and there was still a gap between the number ohop students (students at level 6) in Macao and that in Singapore and China (B-S-J-Z) in terms of reading . In addit ion, the percentage of males failing to reach the PISA reading literacy benchmark (level 2) was higher than that of females in Macao (13.6% for males and 7 .9% for females), while the percentage of females at reading literacy level 5 and level 6 was higher than that of males (12. 1 % for males and 15 .5% for females). Interestingly, both males and females in Macao have performed quite well in high-level thinking, and the percentage of males and females reaching a high level of evaluation and reflection subscale (above level 5) was higher than that of the other two reading process subscales (see Figure 2 .7). 27
China l~~{n~l Estonia Singapore Ireland Hong Kong (China) Finland Canada Poland Korea Denmark Japan United Kingdom Chinese Taipei Slovenia Sweden New Zealand United States Norway Australia Portugal Germany Czech Republic France Belgium Croatia Russia Latvia OECD average Italy Belarus Austria Switzerland Netherlands Lithuania Hungary Turkey Iceland Luxembourg Greece Israel Slovak Republic Chile Malta Serbia Uruguay Costa Rica United Arab Emirates Montenegro M exico Malaysia Bulgaria Colombia Brazil Qatar Brunei Darussalam Albania Bosnia and Heaegovina Peru Thailand Azerbaijan (Baku) Kazakhstan Panama Georgia Indonesia Morocco Kosovo Dominican Republic Philippines % 100 IJS ... --= -= i-= --= a-,;; -= --== -= --,;;a --== Students at Level la or below ~ i--aaa i-·-• -= -- --- , I ~ I ·, J l I ·, I I I I _I I I I I I I 80 60 40 20 -= -I ·-', I==! 1-= - - --l 1-= = :~ ,_ :~ 1 •= 1 1~ 1 I ,_ I : ~ I :~ 1 : I '~ 1 1= 1 ' --------, i ' - I 1~ 1 •-~ I •= 1 !~ 1 ' = 1 l==-e,a 1~ 1~ I==-1---== 1= 1=-- 1=-' ~ .:---3 0 20 40 Data source: OECD, 2019b, Figure I.5.1 (PISA 2018 Database, Table I.LEVELS.read.) Figure 2.6 ·- l::J Students at Level 2 or above I I • Level 6 • Level 5 • Level 4 • Level 3 Level 2 Level la Level lb • Level l e • Below Level le I I 60 80 100 % Students' proficiency in reading across all participating countries/economies in PISA 20 18 28
Combined 21. reading scnlc 17.0 7.9 Locate 11.S information 16.9 6. Understand 17.1 7.3 Evnluate & rcnccl 16.1 7.0 Multiple text 16.2 7 1 Single text 20.7 ff'i,~ Jt,U _, 1}0 ~~jip.:,; 16.8 7.1 • Level 1 a and below Level 2 • Level 3 • Level 4 • Level 5or •hove • Level 5 or above • Level 4 • Level 3 Level 2 • Level In Rnd below Boys Glrb Figure 2.7 PISA 20 18 Macao st1.1dents' proficiency in reading (broken down by gender) 2.5 Trend of Macao student reading literacy performance (PISA 2003 to PISA 2018) Figure 2.8 presents the changes in the distribution of the percentiles of Macao's student reading literacy score. The reading literacy scores of students at the same percentile from PISA 2003 to PISA 2018 (six PISA cycles in total) were compared in the figure. Except in the 5th to 10th percentiles of the reading proficiency scale of which students in PISA 2003 scored higher, the scores of Macao students showed positive growth at all other percentiles, i.e. 25 th to 95th percentile in PISA 2018. It can be seen that students s ituated at the higher percentile their performance are better. In other words, students at the higher end of the literacy performance distribution (75'h, 90th and 95th percentiles) made more significant progress than their corresponding counterparts in pervious PISA cycles. In terms of top students (students at 95 th percentile), they scored 62 points higher than students at the same percentile in PISA 2009, and 35 points higher than students at the same percentile in PISA 2015, the progress made by students in PISA 2018 was equivalent to approximately 1.6 and 1.0 PISA grade levels of secondary schooling respectively. However, students situated at the lower percentile, i.e. 5th and 1 oth percentiles, did not make s ignificant progress. These low-performing students with slow progress in reading literacy were mainly attending lower grade levels (Grade 7 or Grade 8). Therefore, while improving the overall literacy performance of students, the education system should also adopt targeted education policies to cultivate high-performing students and assist low-performing students in the future, so as to take into account the development needs of all students with different progress and capabilities, which is also a key task that highlights Macao's educational equali ty. 29
700 ~ 650 ~ ~~ --_, ... - ... ~ 600 ---;::~ 550 ~ 500 PISA2018 ~ ? -•- PISA2015 ····• ··· PISA2012 450 7 ~ - - • PISA2009 ,, , - • - PISA2006 400 , _., - PISA2003 ~ 350 300 5th 10th 25th 50th 75th 90th 95th - PISA2018 365 403 464 530 590 641 670 -•- PISA2015 365 399 456 514 566 610 635 --·• --- PISA2012 366 400 457 513 566 611 637 - - •PISA2009 357 388 437 489 540 582 608 - • - PISA2006 359 394 445 496 545 587 610 -PISA2003 381 409 455 501 544 583 601 Figure 2.8 Trends of distribution of Macao student performance at designated percenti !es of reading proficiency scale (PISA 2003 to PISA 20 18) Figure 2.9 displays the trends in the distribution of Macao's students reading proficiency levels. According to the results of the pervious PISA cycles, the overall reading literacy of Macao students has been continuously improving over the years. Relatively speaking, the percentage of sh1dents at level 4 and above in PISA 2018 has significantly increased compared to PISA 2009 which also took reading li teracy as the major domain, with a large proportion of high-performing students reaching level 5 and 6. By contrast, the results are opposite for students assessed below level 2. The data reflects that the improvement of Macao students' reading literacy was mainly benefiting from the increase in the percentage of medium- and high-performing students as well as a lower percentage of low-performing students in its student population. As seen in Figure 2. 10, the distribution of OECD reading proficiency levels of PISA 2018 was a lso analyzed and compared. It shows clearly that the proportion of Macao students with medium and high reading literacy proficiency levels was significantly higher than the OECD average. 30
% 45 40 35 30 25 20 15 10 5 0 32 PISA 2003 • PISA2006 • PISA2009 • PISA2012 • PISA2015 •PISA2018 22.6 13.8 Level la, lb, le and Level 2 Level 3 Level 4 Level 5 and above below le Figure 2.10 A comparison of the distribution of Macao (PISA 2003 to PISA 2018) and OECD (PISA 2018) student reading literacy proficiency level
In Figure 2.11 and 2.12, the three corresponding reading subscale proficiency levels of male and female students in Macao in PISA 2009 and PISA 2018 were compared respectively. As shown in the figures, male and female students have both made significant progress in each reading process subsca le , whi le male students had a large increase especially in terms of the proportion of high-performing students. The increase in the three reading process subscales in PISA 2018 made by male students was 2.7, 5.9 and 12.3 times that of 2009, while the increase made by female students was 2.1, 3.5 and 4.7 times. With regard to low-performing students, in general, the proportion of male students was still higher than that of female students, but fortunately the decline in male students was also larger than that of female students, especially in terms of the indicator for the high-level thinking ability of evaluation and reflection subscale. The percentage of male students failing to reach PISA 2018 baseline proficiency level has dropped 12.1 % compared with PISA 2009, while that of female students dropped 3 .1 %. The figures revealed that the Government of Macao, schools and all education stakeholders have effectively promoted a series of tasks to improve students' reading literacy over the past decade, and these policies have successfully achieved sound results for Macao's basic education system. PIS:\ 2009 Girls Access & 11.0 24.I retrieve ln1egrate& 10.0 27.7 i111erpre1 Evaluate& 10.1 27.2 rellec1 3S O ~(J - \ 1 P ISA 2018 Girls ! I j ~t11, • 3,J.9 I·. .. . ~ . .:- . 1, ' _•s ~ • 30h .. I • ' . J, ''I h :!8,8 16.9 17.1 7.3 Understand 16.1 7.0 Evaluate& ronoct Level la and below Level 2 • Level 3 • Level 4 • Level 5 ornbovc • Level 5 or above • Level 4 • Level 3 Level 2 Level la and below Figure 2.11 Change between PISA 2009 and PISA 2018 in reading process subscales of Macao females Access& 21.9 retrieve In1egra1e& 20.1 interprel Evaluate& 25.0 reflect PISA 2009 Boys PISA 2018 Boys 28.S 33.0 33.9 :!9 2 111 r, I : 19.1 Locate 11.5 information 12.6 Understand 12.9 Evaluate& reflect Level la and below • Level 2 • Level 3 • Level 4 • Level 5 or above • Level 5 or above • Level 4 • Level 3 Level 2 Level la and below Figure 2.12 Change between PISA 2009 and PISA 2018 in reading process subsca les of Macao males 33
2.6 Changes in the distribution of Macao student performance at designated percentiles of the reading literacy scale (broken down by modal grades) Figure 2.13 presents the trend of reading literacy scores of the modal grade (i.e. Grade 9 and Grade 10) students situated at different percentile in PISA 2018 and PISA 2009. By comparing the student reading literacy performance at designated percentiles (5 th, 10th, 2511\ 50th, 751\ 90th and 95th), the changes in the scores between the two Macao's modal grades can be observed visibly. Firstly, if we compare Grade 10 students' reading performance between the two PISA cycles, we would find a steady progress from tbe lower end to higher end of the reading proficiency continuum (except for those at the lowest percentile, i.e., the 5th and 10th percentiles). Moreover, the percentile higher, the progress greater. Tbe distribution of percentiles of Grade 9 students' reading literacy scores had the similar pattern. Secondly, if we further compare the two modal grades, it is clear that Grade I 0 students situated at all percentiles scored higher than Grade 9 students. The gap between the Grade 10 and Grade 9 students was almost equivalent to 1 PISA grade level of secondary schooling. As a result, if a student might study in a grade that is matched for his/her age (for example, without grade repetition, 15-year-olds in Macao should be Grade 10 students), it is expected that he/she will perform better in reading literacy. The findings so far indicate the significance of paying attention to the low-achievers. This is because irrespective of the Grade 9 or l O students low-achievers (at the 5th and 10th percentiles) were performing at a level lower than that a decade ago than their corresponding percentile counterparts. 34
2.7 Summary and reflection of the PISA 2018 reading literacy assessment results There are three key findings of PISA 2018 reading literacy assessment that are worthy for Macao educators to reflect. First, the reading literacy was improved to 525 score points since both of Macao's males and females have made a substantial progress. The results revealed the promotion of reading education policies and the various educational investment have been implemented successfully and properly in the past decade. Moreover, based on the visible increase of the percentages of medium- and high-performing students, it also indicated that some students in Macao have not only laid solid ground for their lifelong learning, but also preliminarily possessed the high-level thinking ability. In fact, schools in Macao still need to tailor the teaching and learning strategies and create favorable environment to support all students to master the lifelong learning ability as well as equip young people with the capacities for their future career. Second, the gender gap between males and females in reading literacy performance was narrowing in Macao. Even the phenomenon of gender difference in reading literacy is a common issue around the world and female students usually enjoy the advantages of gender difference in reading literacy. However, comparing with PISA 2009, Macao's males have made considerable progress in PISA 2018 and such improvement has contributed to narrow down the gender gap of reading literacy from 34 points in PISA 2009 to 22 points. Besides, Macao male students' reading performance is even higher than the female counterparts of most of the OECD member and partner countries/economies. The finding provides meaningful insights for educators to consider how to enhance students reading literacy. If the teachers are aware of the learning characteristics of male students and their interests, devise gender-inclusive reading instruction and intervention to enhance students reading engagement that may likely reduce the gender gap in reading literacy and help ensuring all students have equal learning opportunities to acquire reading capacity. Given the PISA research results, improving the males' reading literacy does not mean that this would hurt the progress of females made. In fact, it is not a reciprocal relationship between the males' and females' learning because females will also benefit from the reading enhancement policies. Therefore, it is worthwhile for Macao Government, schools and education stakeholders to continuously implement the follow-up plans or educational programs aiming to cultivate male students' reading literacy. 36
Third, it is notable to recognize the relationship of the grades and literacy performance of students. According to the results of previous rounds of PISA assessments, the reading literacy performance of Grade 10 students is significantly higher than that of Grade 9 students. In general, if a 15-year-old student studied in a grade which is matched his/her age, it is expected that he/she will perform better in the academic achievement. When comparing the distribution of grades of 15-year-old students in previous PISA studies, nearly 60% of the sampled students are studying in Grade 10, and the percentage of students of Grade 10 and above gradually increased from 22.2% in 2003 to 58.9% in 2018. The data shows that significant progress has been made in the education system for lowering the grade repetition rate. However, it is necessary to point out that PISA cannot directly explain the reasons for students' grade repetition, the data can at least reveal that grade repetition does not help with students' literacy development or improvement. In fact, the mean score of Macao or a sampled school's reading literacy performance is expected to be affected by the distribution of grades of 15-year old students. As a result, it is necessary to consider how to provide appropriate guidance and teaching support in order to avoid students' grade repetition, as well as to help them catch up with their peers. Scientific debates about the schooling accountability in Macao require schools and the education system to further investigate and reflect on their grade repetition policies, and to come up a consensus on the mechanism for guaranteeing education quality to solve this long-term issue that is bothering the education system in Macao. It is believed that the educators can formulate strategies and directions at last, and that a complete inclusive and fairness education environment and bright future can be built for all students in Macao. 37
Chapter 3 Mathematical literacy performance of Macao 15-year-old students
A minor focus of the PISA 2018 is on the assessment of mathematical literacy. Mathematical literacy is defined as "an individual 's capacity to formulate, employ and interpret mathematics in a variety of contexts. It includes reasoning mathematically and using mathematical concepts, procedures, facts and tools to describe, explain and predict phenomena. It assists individuals to recognise the role that mathematics plays in the world and to make the well-founded judgements and decisions needed by constructive, engaged and reflective citizens" (OECD, 2019a, p77). Mathematical literacy assessment is highly embedded in challenges and problems in the real world. In the test booklets, many PISA problems seek to measure not just the extent to which students can reproduce mathematical content knowledge, but also assess how well they can generalize from what they know and apply their mathematical knowledge in a variety of contextual situations. The focus on real-life contexts is also reflected in the usage of mathematical tools to solve problems. The word "tools" refers to the physical and digital equipment, as well as computer software and calculators used at home and in the workplace. Using these tools require a certain degree of mathematical reasoning that PISA 2012 is well-suited to measure. Well equipped with mathematical literacy is central to a young person 's preparedness for his/ her daily life in an ever-advancing digital society. Mathematics as an important academic subject of study is valuable for individuals to analyze information in environments bombarded with the massive and messy data. Specifically, students need to mathematize the problems encountered in everyday settings, formulate the problems set in contexts into mathematical models, and solve them to make appropriate decisions. 3.1 Macao students' overall mathematical literacy performance and its gender gap Amongst the 79 participating countries/economies, Macao students' overall mathematical literacy performance (mean= 558, SD= 81) is statistically significantly above the OECD average (mean = 489), and it ranks 3rd on the mathematical literacy proficiency scale (see Table 3 .1 ). That SD is less than 100 indicates that the performance is very homogeneous. In decreasing order of the mean of the mathematical literacy score, countries/economies higher than Macao are: China (B-S-J-Z) and Singapore, whereas countries/economies comparable in performance with that of Macao is Hong Kong. Furthermore, the data shows that students studying in East Asian countries/ economies consistently are achieving at high level of performance in mathematics. Seven East Asian countries/economies stand high up in the PISA 2018 performance league table. Estonia which comes next in the 8th position outperforms all other Western countries/economies. 39
Table 3.1 Ranking, mean, standard deviation and gender difference in mathematical literacy performance: A comparison of Macao and selected countries/economies Countries/ Males' Females' Diff Economics Rank Score SD (F - M) average average Macao-China 2018 3 558 81 560 556 -4 Macao-China 2015 3 544 80 540 548 8 Macao-China 2012 6 538 95 540 537 -3 Macao-China 2009 12 525 86 531 520 -11 Macao-China 2006 8 525 84 530 520 -10 Macao-China 2003 9 527 87 538 517 -21 China (B-S-J-Z) 1 591 80 597 586 -11 Singapore 2 569 94 571 567 -4 Hong Kong-China 4 551 94 548 554 6 Chinese Taipei 5 531 100 533 529 -4 Japan 6 527 86 532 522 - 10 Korea 7 526 100 528 524 -4 Estonia 8 523 82 528 519 -8 Netherlands 9 519 93 520 519 -1 Poland 10 516 90 516 515 -1 Switzerland 11 515 94 519 512 -7 Canada 12 512 92 514 510 -5 Denmark 13 509 82 511 507 -4 Slovenia 14 509 89 509 509 -1 Belgium 15 508 95 514 502 - 12 Portugal 28 492 96 497 488 -9 OECD' average 489 91 492 ~- '487 ··_;-,,: '· ; Sf;' .'t:: - •, .- ,,. Compared with PISA 2015, Macao's 15-year-old students performed very well with an improvement of 14 score points in mathematical literacy in PISA 2018. Macao's performance is the best amongst all the previous cycles of PISA assessment. This result shows that Macao students have been improving visibly in mathematical literacy performance. 40
On average across all participating countries/economies in PISA 2018, the gender gap in mathematics performance is narrower than that observed in reading, and the largest difference in mathematics performance between males and females is 24 score points (Figure 3.1 ). Broken down by gender, both males and females have improved in mathematics performance in a short duration of three years. A comparison of results in mathematics performance between 2018 and 2015 reveals that males' performance in mathematics has increased by 20 score points whereas fema les' performance just increased by 8 score points. The magnitude of males ' progress is 2.5 times of the fema les'. Unlike in PISA 2015 in which Macao females outperform males in mathematics performance, because of the impressive improvement of males than females in PISA 2018, there is no statistically significant gender difference in mathematics performance observed (Males = 560; Females = 556). Compared with the previous cycle of PISA 2015 assessment, although Macao females ' progress in mathematics was slightly less than the males, the mean score of Macao females still performed excellently among the participating countries/economies. On average, Macao females not only outperformed her peers of the same gender, but also the males of the 76 countries/economies. To sum up, the findings of PISA 20 18 reveal that Macao' rise in mathematical literacy performance can be attributed to the substantial progress by males and females. Identifying the gender disparities in learning may help narrow or close the gender gap in student perfonnance. Enhancing the males' reading ability can bring benefits to their learning, particularly helpful for nurturing their mathematical logic judgement. Since reading is the key competency for one to learn well, a recent study has shown that after control ling the gender effect of reading performance, males outperformed fema les in the perfonnance of mathematics and science across all the PISA participating countries/economies (Mak et al., 2019). In other words, a refreshed commitment in schools targeting at turning the underachieving males into proficient readers will potentially benefit males. As discussed in the earl ier section, all the data presented above show that while males read well they usually have good mathematics and science literacy performance. Thus, sustained approaches and strategies that focus on supporting males to acquire the reading competency are fervently needed. Indeed, the gender gap issue is certainly complex and there is no one-size-fits-all solution. One thing for sme is that we have to act to ensure all our students fulfil their potentials. Hence, schools need to make efforts to enhance the females' mathematical literacy. In this regard, educationa l practitioners can glean insights by studying the experiences of China (B-S-J-Z) and Singapore. It is hoped that in the long n m, males are able to join hands alongside with females to raise the overall quality and equity of Macao's basic education. 41
580 - Macao Boys - Macao Girls 560 OECDBoys 560 551 OECDGirls 556 -+-Macao average 540 538 530 520 500 S06 4~ 4112 480 487 460 440 PISA 2003 PISA 2006 PISA 2009 PISA 2012 PISA 2015 PISA 2018 Figure 3.2 Trend of the gender difference of Macao students' mathematical literacy performance 3.2 Overall and gender difference in Macao students' distribution of proficiency levels of the mathematical literacy There are 7 proficiency levels in the PISA 2018 mathematics proficiency scale, from below 1 to 6. As seen in Table 3.2 and Figure 3.3, 95% of Macao students have reached the PISA baseline level 2, and they are mainly situated at level 3 and 4, comprising a total of 55 .1 % of the 15-year-old sampled students. The result shows that apart from China (B-S-J-Z), Macao has the largest proportion of students reaching the baseline level, while the number of low-performing students has declined to 5%. Additionally, the percentage of high-performing students in Macao at level 5 and 6 has increased to 27.7%. The proportion of high-performing students is 2.5 times of that in OECD countries/ economies. In other words, on average one out of four students is high-performing in Macao. The data shown reveals that Macao's rise in mathematics performance is due to the education system reducing the share of low- performing students, as well as increasing the proportion of students at the middle to high proficiency levels. Compared with countries/ economies in neighboring regions such as China (B-S-J-Z), Singapore and Hong Kong, Macao is still soberly deficient in top-performing students in mathematical literacy. In a globalized world with intense competition, there is an urgent need to initiate and launch programs aiming at raising high-performing elite students for meeting the challenges of transformation as well as upgrades of the digital economy and industry in the future. 43
China (8-S-J-Z) Macao (China) Singapore Hong Kong {China) Estonia Japan Chinese Taipei Denmark Poland Finland Korea Ireland Netherlands canada Slovenia Switzerland Latvia Sweden Norway United Klf"€dom Belgium Czech Republic Iceland Austria Germany France RU$.1lan Newzealand AurtraNa Patugal Italy OECO average Spain Slovak Republic Lithuania Hungary United Slates Luxembourg Belarus Malta Croatia Israel Greece Ukraine Turkey Cyprus Serbia Malaysia Albania Bu1aria Untted Arab Emirates Montenegro Romania Brunel Darussalam Kazakhstan Moldova u,,.uay Azerbaljan(Baku) Chle ThaUand Qatar Mexlto Bosnia and Herzegovina Jordan Lebanon Cosla Rica Peru Republlc al Nath Macedonia Georgia Colombia Brazil Argentina Indonesia Saudi Arabia Morocco Kosovo Philippines Panama Dominican Republic " 100 Students at Level 1 or below - -~ -80 60 40 II ---------------I I I J w , __ 20 0 20 40 Data source: OECD, 201 9c, Figure I.6.1 (PISA 201 8 Database, Table I.LEVELS.math.) Figure 3.3 -= ==i Students at Level 2 or above I I • Level 6 • Level 5 • Level 4 • Level 3 Level 2 Level 1 • Below Level 1 I I 60 80 Students ' proficiency level s on the mathemati cal literacy scale across all participating countries/economies in PISA 201 8 100" 45
Figure 3 .4 presents distribution of Macao and OECD mathematical literacy proficiency level in PISA 2012 and PISA 2018, broken down by gender. The result indicates that both Macao's male and female students have improved greatly in PISA 2018. The share of high-performing students is larger among males than among females , as there are 30% of males and 26% of females are high-achievers respectively. By contrast, the share of low-achieving males and females has dropped down to 5%. Contrasting the mathematics performance between PISA 2018 and PISA 2012, it is found that there is a significant decrease in the share of low achievers in Macao. Relatively speaking, the Macao's mathematics literacy improvement is better at the low end than the high end of the proficiency scale. Finally, across the OECD countries the proportion of students scoring below level 2 in mathematics increased whereas the proportion of students scoring at or above level 5 decreased. Compared with the average of the OECD countries, Macao students in PISA 2018 indeed demonstrates a significant progress in mathematical literacy. PJSA 2012 Mna,o Boy, . 7 7 .8 15.5 OfCD Dey, 7.9 14.2 21.3 - ----- It' 7 IO, , M11a10 Girb 2.7 7.3 17.5 ~" <. 1, ,. ' O~CD G~b 8.1 IS.8 236 PJSJ\ 2018 21.S , , 1•1 I 'l I \ 23.0 J ,.,::i 14 'T 9.2 150 9.0 De low level 1 Level 1 • Level 2 • Level 3 • Level 4 • Level S • Levol 6 • Level 6 • Level 5 • Level 4 • Level 3 Level 2 • Level 1 Uelow level I Figure 3.4 A comparison of the distribution of Macao and OECD student mathematical literacy proficiency level in PISA 2012 and PISA 20 18 (broken down by gender) 3.3 Trend of Macao student mathematical literacy performance (PISA 2003 to PISA 2018) Figure 3.5 presents the trend of the distribution of the percentiles of Macao 's student mathematics literacy score. Based on the six cycles of PISA assessment from 2003 to 2018, the performance score at each of the percentiles is compared. In general, students of PISA 2018 situated at the low to the high end of the literacy performance distribution (i.e. from 5th to 95th percentile) perform better than their corresponding counterparts in the previous rounds of PISA assessment. Compared with PISA 2012, except students at the 90th and 95th percentiles whose performance levels are comparable in magnitude, those at other percentiles are performing better in PISA 2018. 46
Additionally, when each round of PISA assessment was considered separately, the PISA 2018 findings show that the performance score at the same percentile is higher than PISA 2015, and the progress is equivalent to one third of one PJSA grade level of secondary schooling. Moreover, comparing PISA 2018 and PISA 2012, considerable improvement exists between students situated at the lowest to the higher of the literacy performance distribution (i.e. at the 5th to 75th percentile). However, there is no significant difference between the two groups of mathematics top-performers (i.e. at the 90th and 95 th percentile). In comparison with PISA 2012, in PISA 2018, mathematics top-performers maintain excellent performance. Although considerable student progress occurs m PISA 2018 in mathematical literacy performance, there are still concerns that warrant scrutiny. Compared with PISA 2012, no apparent growth takes place in the percentage of top-performing students. In the future, Macao 's education system, in attempting to raise the overall literacy level of the students, needs to ensure that every student is equipped with core literacies in preparation for a productive life. More attention is required for the education of e lite students. Through continuous, sustained improvement in education quality, students will be stimulated to learn better and have better results. 700 650 ~ A ig ..-, -7-w ,. " 600 ~y 550 £ :}1/ - PISA2018 - • - PISA 2015 500 /';J/ . ..... PISA 2012 .•· --- PISA 2009 . 450 , . o/ -•- PISA2006 , --PISA 2003 , .. .. ... 400 350 300 5th 10th 25th 50th 75 th 90th 95th --PISA2018 420 452 505 561 613 659 686 - • - PISA 2015 408 439 491 547 599 643 669 . .... . PISA 2012 379 415 476 542 605 657 685 --- PISA 2009 382 415 468 527 584 634 663 - PISA 2006 384 416 467 526 585 632 660 --PISA2003 382 414 467 529 587 639 668 Figure 3.5 Trends of the distribution of Macao student performance at designated percentiles of the mathematical literacy scale (PISA 2003 to PISA 2018) 47
Figure 3.6 presents the distribution of Macao's student mathematics proficiency level. Based on the six cycles of PISA assessment from 2003 to 2018, the percentages of students at each of the proficiency level are compared. The findings suggest that student mathematics performance is continually improving. There are higher percentages of students with proficiency level 4 or 5 than the corresponding ones in previous rounds of PISA assessment. Compared with PISA 2012, in PISA 2018, the percentage of students with proficiency level 6 is maintained. Also, percentages of students with proficiency level 2 or below are clearly smaller than the ones in the previous cycles. This finding shows that Macao's rise in mathematics performance is due to the greater percentage of medium-performing students, as well as the sustained lower percentage of low-performing students in its student population. Last, but not the least, the distribution of mathematical proficiency levels of PISA 2018 is also analyzed and compared with previous cycles (Figure 3. 7). It shows clearly that the proportions of Macao students with medium and high mathematical literacy proficiency levels are significantly higher than the OECD average. 48 %35.0 .--------------------------- --30.0 -1-----------------.A- --------~ PISA 2018 15.0 +-------- .J=-_.,_,.-1----------~- ,-4,---- --•--PISA 2015 --•--PISA 2012 10.0 +------ -#-,;.....,e~ '---------------".l,,-,...,~ -- --•--PISA 2009 -- -- PISA 2006 5.0 +-__ _,,_::...._~ .,_----------------~- - --PISA 2003 0.0 -!--------~------------------Below ..,._PISA 2018 -..&--PISA 2015 --•--PISA 2012 --•-· PISA 2009 --•·· PISA 2006 --PISA 2003 level 1 Level 1 Level 2 Level 3 Level4 Level 5 Level 6 1.0 4.0 12.3 24.8 30.3 20.0 7.7 1.3 5.3 15. 1 27.3 29. 1 16.9 5.0 3.2 7.6 16.4 24.0 24.4 16.8 7.6 2.8 8.2 19.6 27.8 24.5 12.8 4.3 2.6 8.3 20.0 27.3 24.2 13.6 3.8 2.3 8.8 19.6 26.8 23.7 13.8 4.8 Figure 3.6 Trends of the distribution of Macao student mathematical literacy proficiency level (PISA 2003 to PISA 2018)
% 45 40 35 30 25 20 15 10 5 0 PISA 2003 • PISA2009 • PISA2012 • PISA2015 • PISA2018 23.8 Level 1 and below Level 2 30.3 Level 3 Level 4 Level 5 and above Figure 3.7 A comparison of the distribution of Macao (PISA 2003 to PISA 2018) and OECD (PISA 2018) student mathematical literacy proficiency level 49
3.4 Changes in the distribution of Macao student performance at designated percentiles of the mathematical literacy scale (broken down by modal grades) Figure 3.8 presents the trends of mathematical literacy scores of the modal grade (i.e. Grade 9 and Grade 10) students situated at different percentiles in PISA 2018 and PISA 2012. By comparing the student mathematical literacy performance at designated percentiles (i.e. 51\ 101\ 251h, 501\ 751\ 90th and 95th ) , the changes in the scores between the two modal grades/cycles can be examined. There are a few observations. Firstly, if we compare Grade 10 students' mathematical performance between the two PISA cycles, we will find a steady progress at the lower end (i.e., the 5th and 25th percentiles) but a slight drop at the higher end (i.e., the 75th and 95th percentiles) of the mathematics proficiency continuum. By contrast, the distribution of percentiles from 10th to 75th of Grade 9 students' mathematical literacy scores are higher than their counterparts in PISA 2012. Students of PISA 2018 situated at the higher end of the literacy performance distribution (above the 75th percentiles) perform lower than PISA 2012. Secondly, if we compare the two modal grades, it clearly shows that Grade 10 students situated at all percentiles scored higher than Grade 9 students. The gap between the Grade 10 and Grade 9 students on average is over 40 score points. It is almost equivalent to one PISA grade level of secondary schooling. Moreover, the two groups of elite students of the two modal grades (situated at 90th and 95th percentiles) perform lower than their counterparts in PISA 2012. 50
750 700 650 600 550 500 450 400 •.. · 350 300 5th - PISA 2018 Grade 10 464 - PISA 2018 Grade9 409 - • - PISA 2018 Macao average 420 -PISA 2012 Grade 10 458 - PISA 2012 Grade 9 408 · .... . PISA 2012 Macao average 379 .. •·· ... •· ... 10th 491 440 452 485 434 415 25th 535 489 505 533 477 476 Figure 3.8 - PISA 2018 Grade 10 - PISA 2018 Grade 9 - • - PISA 2018 Macao average --PISA 2012 Grade 10 - PISA 2012 Grade 9 ··• ·· PISA 2012 Macao average 50th 75th 90th 95th 585 633 674 699 537 585 630 655 561 613 659 686 586 636 68 1 706 529 584 634 663 542 605 657 685 Changes in the distribution of Macao student perfo rmance for PISA 201 2 and PISA 2018 at des ignated percentiles of the mathematical literacy scale (broken down by moda l grades) 51
3.5 Summary and reflection of the PISA 2018 mathematical literacy assessment results The definition of PISA mathematical literacy is an individual's capacity to formulate, employ and interpret mathematics in a variety of contexts. Particularly, student with proficient mathematical literacy can always demonstrate that he/she is used to engaging with and showing sensitivity to mathematics in daily life, as well as thinking logically. Based on the analysis of the PISA 2018 mathematical literacy assessment results, one can find that Macao's 15-year-old students continue to excel with flying colors in international comparison, and at the same time breaks its own records to rise to new heights. Strong evidences of empirical research are available to suggest that Macao students are able to integrate mathematics in their daily lives, and that teachers should embark on mathematics curriculum reform with a focus on the solving of real- life problems. There are also evidences calling for schools to promulgate policies helping students to achieve at comparable levels of their same-age peers, and trying their best to lower the grade retention rates (Sit et al. , 2015). On average across OECD countries, 23.8% of students do not reach the baseline level 2 in mathematics. Regrettably, these students are only able to identify information and carry out routine procedures according to direct instruction in explicit situations. By contrast, some Eastern Asian countries/economies have the smallest shares of students who score below level 2 in mathematics, i.e. Singapore (~ 10%), Hong Kong (~ 10%), Japan (11.5%), China (B-S-J-Z) (2.5%) and Macao (5%). The data presented above indicate that it is possible for students to attain success in learning if we can identify students who struggle with acquiring basic knowledge and skills in schools, know what problems they face in learning and provide them appropriate guidance and support. It is hoped that the high expectation of the public on Macao's education system can be realized by appropriate adoption of education policies. Traditionally, males outperformed females in mathematics. Comparing with PISA 2015, the magnitude of Macao males' progress is 2.5 times of that of the females and the percentage of high-performers is in favor of males. Nearly 30% of Macao males attained level 5 and 6. Although Macao females' progress in mathematics is slightly behind the males, the mean score of Macao females indicates that they still perform very well amongst the PISA 2018 participating countries/ economies. In fact, the statistics shows that Macao females not only outperform her peers of the same gender but a lso the males of76 countries/economies participating in PISA 20 18. The findings of PISA 2018 highlight that Macao' rise in mathematical literacy performance is attributable to the substantial progress made by students of the two genders. Identifying the gender disparities in learning may help narrow or even close the gender gap in student performance. In particular, data analysis of PISA data shows that helping males enhance their reading competence and this in turn 52
can bring benefits to their learn ing in science and mathematics, possibly because of its helpfulness for initiating logic reason ing and mathematical judgement. In a nutshe ll, narrowing gender gap favoring females in reading does help c lose the gender gap favoring Macao females in mathematics in PISA 2015 and beginning to see Macao ma les outperform females again in PISA 20 18. With this understanding, education policies promulgated need to pay due attention to the education of male students . As mentioned earlier, commitment in schools empowering the underachiev ing males into proficient readers is an essential step to raise education quality (Mak et al., 2019). At the same time, there are needs to increase performance of the home-disadvantaged students, and increase the percentage of the top-performing students . Regarding this, when designing education system, educational practitioners should consider in what ways community resources, learning environment, learning attitudes cast effects and overcome obstacles on student learning. Hopefully Macao's policy of education quality and equity can be improved gradually. Alec Ross (2016), an A merican who is expert in innovations and creative productions, once asserted that in the agricultural era the raw materia l is land, in the industrial era is iron, and in the information age is data. In the future, those people who possess data, grasp data and understand the meaning of data are at an advantage of reaping the benefits of commercial opportunities. If Macao students can capitalize on their strengths in mathematical literacy and invest their efforts developing themselves in the information technology domain, then mighty power can be exploded to propel Macao 's economy. There w ill be tens of thousands of working opportunities, facilitating Macao's sustained societal development in the coming years. 53
Chapter 4 Scientific literacy performance of Macao-15-year-old students
A minor focus of PISA 2018 is the assessment of scientific literacy. In the PISA 2018 assessment framework, scientific literacy is defined as the ability to engage as a reflective citizen with science-related issues and with the ideas of science (OECD, 2019a, p 100). PISA emphasizes that a science-literate person is one who uses that knowledge to navigate through today's world, and that all of us sometimes need to "think like a scientist" to weigh evidence and come to a conclusion. Also, we have to understand that scientific "truth" may change over time once new discoveries are made, particularly when we engage with science-related issues (OECD, 2016a). When reading about or discussing science-related topics, students must be able to separate science from spin, recognize misrepresentations of findings, and evaluate the level of uncertainty or the credibi lity associated with a specific claim. This requires not only knowledge of scientific facts, but also of the nature and origin of scientific knowledge. PISA assesses scientific knowledge and ski lls that is not restricted to the common aspects of participants' national curriculum. Rather, it begins from the identification of the scientific phenomena frequently occurred in daily lives. Therefore, the students who commit to a conclusion require an understanding how the data is presented and justified. The process highly relies on the correct interpretation and demonstration of charts and data. Thus, PISA scientific assessment typically depends on the evidence of the successful use of the competencies by looking for patterns in the data, perhaps through constructing simple tables or graphical visual izations. For instance, many science tasks in the PISA assessment target precisely at the ability to figure out appropriate conclusions from data, perform and evaluate scientific enquiries, and reflect on the uncertainty of measurements when interpreting data (OECD, 2016b). In contemporary societies, scientific literacy is essential for the persons whose careers depend on the understanding of science and of science-based technology directly. In addition, it is important for the citizen who wishes to make informed decisions related to the many controversial issues under debate today. For instance, a wide variety of personal to local topics, such as maintaining a healthy diet or managing waste in big cities, and even the global and far-reaching issues, such as the costs and benefits of genetically modified crops, how to prevent and mitigate the negative consequences of global warming on physical , ecological and social systems, a ll are involved the scientific literacy (OECD, 2019b). Thus, a well science-literate person can address these problems accurately and make judgement independently (Hsiao, 2011). 55
4.1 Macao students' overall mathematical literacy performance and its gender gap Amongst the 79 participating countries/economies, Macao students ' overall scientific literacy performance (mean = 544, SD = 83) is statistically significantly above the OECD average (mean = 489), and it ranks 3rd on the scientific literacy proficiency scale (see Table 4.1 ). In decreasing order of the mean of the scientific literacy score, countries/economies higher than Macao are: China (B-S-J-Z) and Singapore. It is heartening to see that Macao students' scientific literacy performance has been continuously improving since 2006. The scientific literacy performance score increases from 511 in PISA 2006 to 544 in PISA 2018. The increase of 33 score points is equivalent to approximately one PISA grade level of secondary schooling. Apart from China (B-S-J-Z), Singapore and Macao, the remaining top-10 high-performing countries/economies in PISA 2018 are: Estonia, Japan, Finland, Korea, Canada, Hong Kong and Chinese Taipei. All these countries/economies are also top-performing in PISA 2015, showing their leading position in science education globally. 56
Table 4.1 Ranking, mean, standard deviation and gender difference in scientific literacy performance: A comparison of Macao and selected countries/economies Countries/ Ma les' Females' Diff . Rank Score S D Econom ics average average (F - M) Macao-China 2018 3 544 83 543 545 2 Macao-China 2015 6 529 81 525 532 8 Macao-China 2012 17 521 79 520 521 Macao-China 2009 18 511 76 510 512 2 Macao-China 2006 17 511 78 513 509 -4 Macao-China 2003 7 525 88 529 521 -8 China (B-S-J-Z) 590 83 596 584 -12 Singapore 2 551 97 553 549 -4 Estonia 4 530 88 528 533 5 Japan 5 529 92 531 528 -3 Finland 6 522 96 510 534 24 Korea 7 519 98 521 517 -4 Canada 8 518 96 516 520 3 Hong Kong-China 9 517 86 512 521 9 Chinese Taipei 10 516 99 516 515 -1 Poland 11 511 92 511 511 0 New Zealand 12 508 102 509 508 -2 Slovenia 13 507 88 502 512 10 United Kingdom 14 505 99 506 503 -2 Netherlands 15 503 104 499 508 8 Portugal 26 492 92 494 489 -5 OECD average 489 94 488 490 2 Compared with PISA 2015 and PISA 2006, Macao 's 15-year-old students performed very well in PISA 20 18 with an improvement of 15 and 33 score points respectively in scientific literacy. Macao's performance in PISA 2018 is the best amongst all the previous cycles of PISA assessment. This result shows that Macao students have been improving visibly and continuously in scientific literacy performance. 57
Amongst the many issues in science education under study by the researcher, gender difference in science is an important topic of study needed deliberation by the educational practitioners. Findings of large-scale sample survey, such as TIMSS and PISA, revealed that gender gap in scientific literacy is narrowing in recent years (Neuschmidt, Barth, & Hastedt, 2008; OECD, 2016a, 2016b). On average across all participating countries/economies in PISA 2018, there are seven countries (i.e. Cyprus, Malta, Finland, United Arab Emirates, Saudi Arabia, Jordan and Qatar) whose gender gaps in science performance are greater than 20 score points (Figure 4.1). Most other countries/ economies male students ' performance is comparable to that of the female students ', and hence the gender difference in scientific literacy performance is not statistically significant. Also, there is a small number of countries/economies in which female students outperform the male students. In Macao, there is no statistically significant difference in scientific literacy performance between the two genders. Of note is that in PISA 2018, in Macao, both males and females have improved in science performance in a short duration of three years. A comparison of results in science performance between 2018 and 2015 reveals that males' performance in science has increased by 18 score points whereas females' performance increased by 13 score points. The magnitude of males' progress is 1.4 times of females'. Unlike in PISA 2015 in which Macao females outperform males in science performance, because of the impressive improvement of males than females in PISA 2018, there is no statistically significant gender difference in science performance observed (Males= 543; Females = 545). The progress of the male students is very apparent. This may be attributed to their improvement in reading literacy. Upon closer scrutiny it is found that one key characteristic of the PISA tests is the large amount of information needed read by the examinees. Very often the problem task is phrased implicitly to be embedded in contexts of students' everyday life. The main goal of PISA assessment is to examine whether students are able to apply the knowledge and skills acquired in real-life situations. Thus, mastery of language or reading abilities to an appreciable extent is required for successful solving of the problem task. As explained in chapter 3, improvement of male students' reading literacy is helpful for their correct understanding and interpretation of scientific issues, and amplification of their strengths in mathematical logical thinking. The outcome is the narrowing of the gender gap in science witnessed in the results of PISA 2018. In summary, compared with the previous cycle of PISA 2015 assessment, although Macao females ' progress in science was slightly less than the males, the mean score of Macao females still performed excellently among the participating countries/economies. On average, Macao females not only outperformed her peers of the same gender, but also the males of the 76 countries/ economies. 58
f l l r a ;;-s. ! a i: ;!-3 ;;-5 I 3: ,. .. ::, 8 ii 0 ::, ,o ; ,o f JO l<) 10 ·10 ·lO ••• • • • • • •• • . •• • • •·.· .· • • • • f •••••• • • f f f -All 1tud•nt1 • &oy1 • Glr11 f f • f • t f f t t t t ••' t Data source: OECD, 20 I 9c, F igure ll. 7.3 (P[SA 20 I 8 Database, Tables l.GENDMEANS.scie and I. MEANS.scie.) 580 S60 S40 520 S00 480 460 440 Figure 4.1 PISA 201 8 gender gap in PISA 20 18 scientific literacy performance PISA 2003 - Boys (Macao) - Girls (Macao) Boys(OECD) Girls (OECD) -+-Macao average PISA 2006 PISA 2009 PISA 2012 PISA 201S Figure 4.2 543 545 PISA 2018 Trend of the gender difference of Macao students' scientific literacy performance 59
60 4.2 Overall and gender difference in Macao students' distribution of proficiency levels of the scientific literacy There are 8 proficiency levels in the PISA 2018 science proficiency scale, from below level 1 b to 6. As seen in Table 4.2 and Figure 4.3, around 94% of Macao students have reached the PISA baseline level 2, and they are mainly situated at level 3 and 4, comprising a total of 63. l % of the 15-year-old sampled students. Same as reading and mathematical literacy, the result shows that apart from China (B-S-J-Z), Macao has the largest proportion of students reaching the baseline level, while the number of low-performing students has declined to 6%. Additionally, the percentage of high-performing students in Macao at level 5 and 6 has increased to 13.6%. The proportion of high-performing students is 2 times of that in OECD countries/economies. In other words, on average one out of eight students is high-performing in Macao. The Macao PISA 2018 results show that elevation of scientific literacy performance is due to the increase in percentage of the medium- to high-performing students, as well as decrease in percentage of the low-performing students. Admittedly, the number of high-performing students in scientific literacy, compared with China (B-S-J-Z), Singapore and Estonia, is still inadequate.
China (B-S·J.Z) Macao (China) Estonia Sln1apore Japan Hon1 Kon1 (China) Fin land Canada Poland Korea Slovenia Chinese Taipei Ireland u nlted Kin ad om New Zealand Latvia United States Denmark Czech Republic Australia Sweden Portu1al Germany Belalum Netherlands Switzerland France Norway Russia Spain Austria OECD averare Lithuania Hun1arv Belarus Iceland Turi
Figure 4.4 presents distribution of Macao and OECD scientific literacy proficiency level in PISA 2015 and PISA 2018, broken down by gender. The result indicates that both Macao 's male and fema le students have improved greatly in PISA 2018. The share of high-performing students is larger among males than among females, as there are 14.6% of males and 12.7% of females are high-achievers respectively. By contrast, the share of low-achieving males and females has dropped down to 7% and 5% respectively. Contrasting the science performance between PISA 2018 and PISA 2015, it is found that there is a decrease in the share of low-achievers in Macao, and the percentage is more for the males than females. However, there is an increase in the share of high-achievers in Macao, and the percentage is comparable between the two genders. Finally, across the OECD countries , irrespective of the gender of student there is a decrease in the percentage of high-performing students. However, percentage of male low-achievers increases and the corresponding percentage for the female low-achievers remains rather stable in magnitude. PISA20ll PISA 2018 1.7 1.0. lncoo Boys 8 -I 21.1 ----~____....:.,;.,._..;..::.,a 177 d.O 6.6 0.6 0.7 1 .. 1 0.8 167 ,U 0.8 0.7 OECD Girls S.I 1$.S 2:J.? 262 Its ,_; Lcn:llb&bclow Lc\'clln • Lc\'cl 2 • Lc..-cl.1 • Le-.·cl-1 • Lcrcl5 • Lc..-d 6 • Lcn~I G • Lcvc! S • lcn:14 • Lc\'cl.l • Lc,..cl 2 • Lcn:lln Lc\'cllb & bclow Figure 4.4 A comparison of the dis tribution of Macao and OECD student scientific literacy proficiency level in PISA 20 15 and PISA 2018 (broken down by gender) 63
4.3 Trend of Macao student scientific literacy performance (PISA 2003 to PISA 2018) Figure 4.5 presents the trend of the distribution of the percentiles of Macao's student scientific literacy score. Based 011 the six cycles of PISA assessment from 2003 to 2018, the performance score at each of the percentiles is compared. Ill general, students of PISA 2018 situated at the low to the high end of the literacy performance distribution (i.e. from 5th to 95th percentile) perform better than their corresponding counterparts in the previous rounds of PISA assessment. On average, the growth is 13 to 18 score points. Comparing PISA 2018 with PISA 2006, progress is even more apparent. The PISA 2018 findings show that as the percentile increases from the low to high end of the science proficiency distribution student performance increases progressively resulting in a difference of 0.6 to approximately one PISA grade level of secondary schoo ling. Although cons iderable student progress occurs in PISA 2018 in scientific literacy performance, there are still concerns that warrant scrutiny. Attention should be paid to the large performance gap amounting to 112 score points between the two groups of students at the 25th and 75'h percentiles of the scientific literacy distribution. This gap amounts to 1.5 PISA proficiency level in scientific literacy, or approximately 3 PISA grade levels of the participating schools in PISA 2018. 64
Figure 4.6 presents the distribution of Macao's student scientific proficiency level. Based on the six cycles of PISA assessment from 2003 to 2018, the percentages of students at each of the proficiency level are compared. The findings suggest that student science performance is continually improving. There are higher percentages of students with proficiency level 4 or 5 than the corresponding ones in previous rounds of PISA assessment, whereas that of level 6 the increase is very slight in magnitude. Also, percentages of students with proficiency level 2 or below are clearly the smallest amongst the ones in the previous cycles. This finding shows that Macao's rise in science performance is due to the greater percentage of medium-performing students, as well as the lower percentage of low-performing students in its student population. Last, but not the least, the distribution of scientific proficiency levels of PISA 2018 is also analyzed and compared with the average of OECD countries in PISA 2018 (Figure 4. 7). It shows clearly that the proportions of Macao students with medium and high scientific literacy proficiency levels are significantly higher than the OECD average. 66 40.0 % 35.0 , .. , ' ,' f\ 30.0 25.0 20.0 15.0 ~ PISA2018 -i---------,!ft.'T--r- --------......._..r~------ -o- PISA 2015 10.0 ··• ··PISA 2012 +------~ 1- ----------..-'r'!--'r------ · ·•· ·PISA 2009 ···~··· PISA 2006 5.0 +---~~~ -------------~~.....-.~ --- --PISA 2003 0.0 _._PISA 2018 --PISA 2015 ·--·· PISA 2012 -... ··PISA 2009 ···•···PISA 2006 --PISA 2003 Level lb and below Level la Level 2 Level 3 Level 4 Level 5 Level 6 level lb 0.9 5.1 17.2 32.3 30.8 11.9 1.7 1.2 6.9 20.6 34.2 28.0 8.3 0.9 1.4 7.4 22.2 36.2 26.2 6.2 0.4 1.S 8.1 25.2 37.8 22.7 4.5 0.2 1.4 8.8 25.9 35.7 22.9 5.1 0.3 1.6 8.4 22.2 31.S 25.8 9.1 1.4 Figure 4.6 Trends of the distribution of Macao student scientific literacy proficiency level (PISA 2003 to PISA 2018)
% PISA 2003 45 - PISA 2006 - PISA 2009 40 - PISA 2012 35 - PISA 2015 - PISA 2018 30 -a-OECD average (2018) 25.8 25 20 15 10 5 0 Level la, lb and below lb Level 2 Level 3 Level 4 Level 5 and above Figure 4.7 A comparison of the distribution of Macao (PISA 2003 to PISA 201 8) and OECD (PISA 201 8) student scientific literacy proficiency level 67
4.4 Changes in the distribution of Macao student performance at designated percentiles of the scientific literacy scale (broken down by modal grades) Figure 4.8 presents the trends of scientific literacy scores of the modal grade (i.e. Grade 9 and Grade 10) students situated at different percentiles in PISA 2018 and PISA 2015. By comparing the student scientific literacy performance at designated percentiles (i.e. 5111, 1011\ 2511\ 50th, 7 5th, 90111 and 95th ) , the changes in the scores for the two modal grades/cycles can be examined. There are a few observations. Firstly, if we compare Grade IO students' scientific literacy performance between the two PISA cycles, we will find no apparent progress at the lower end (i.e., at the 5th and 10th percentiles) but positive growth in scientific literacy score at each of the percentiles from the low to high end (i.e. from the 25th to the 95th percentile) of the percentile distribution. The trend is that as percentile increases the growth in scientific literacy score increases. Thus, one can say that in the student population of 15-year-old students in PISA 2018 there is apparent growth in scientific literacy for the Macao students situated at the middle- or high-end (i.e. 50111, 751h and 9Y11 percentile) of the percentile distribution. For those high- or top-performing (i.e. 75 th, 90th and 95 111 percentile) students, their scientific literacy performance are even better than their PISA 2015 counterparts by at least 15 score points, which is equivalent to approximately 0.4 PISA grade level of secondary schooling. Secondly, the situation of Macao 's Grade 9 students are similar to the Grade 10 ones. One main difference is that there is a small growth in scientific literacy performance at the low end of the percentile distribution (i.e. 5th and 10th percentile). 68
750 700 650 600 550 - PISA 2018 Grade 10 500 --PISA 2018 Grade 9 450 _ ,._ PISA 2018 Macao average 400 - PISA2015 Grade 10 --PISA 2015 Grade 9 350 ·· • ·· PISA 2015 Macao average 300 5th 10th 25th 50th 75th 90th 95th - PISA 2018 Grade 10 435 467 517 570 619 662 687 - PISA 2018 Grade 9 392 423 474 527 580 624 650 - .. - PISA 2018 Macao average 402 434 489 547 601 648 674 - PISA 2015 Grade 10 434 463 509 559 606 646 670 --PISA 2015 Grade 9 385 413 461 515 565 607 632 .. .. .. PISA 2015 Macao average 389 420 474 532 586 630 656 Figure 4.8 Changes in the distribution of Macao student performance in PISA 2015 and PISA 20 18 at designated percentiles of the scientific literacy scale (broken down by moda l grades) 69
4.5 Summary and reflection of the PISA 2018 scientific literacy assessment results PISA scientific literacy assessment seeks to examine 15-year-old students on whether they possess various scientific abilities and attitudes to take the challenges in their future studies and the workplace. Great attention is paid on the use of scientific knowledge on daily lives. The scientific literacy concept proposed by PISA includes two components, namely science and science-based technology. Science-based technology, in essence, aims at finding optimal solutions to human problems, and very often there is no one solution to such problems. From the vantage point of PISA, the science is not just patent of the scientists. In the information age, people are bombarded with information every day. Hence, it is important that every citizen is empowered to "think like a scientist". When citizen is able to think like a scientist, he/she can weigh evidences to arrive at conclusions, understand that scientific truths may be falsified as time passes, increases in one's ability and understanding regarding the power of nature and scope of technological inventions, and ultimately engages in rational discourse and decision making in the society one lives. The findings of the PISA 2018 scientific literacy assessment reveal that the average of scientific literacy performance of Macao's 15-year-old students, like their achievement attained in reading and mathematics, breaks record to become top-3 in the world. This is indeed a remarkable achievement in the history of Macao's PISA literacy assessment. In fact, this acclaimed outcome is not the honor of just one person, but are due to the joint efforts invested by the educational practitioners and researchers aiming at improvement of Macao 's basic education. Upon entering the 21st century, there are higher level of demands on basic education quality by our society. Around the world countries/economies are required to meet opportunities and confront challenges amidst the waves of the scientific and new industrial revolution. To propel economic development talent cultivation in preparation of the revolution in areas of mathematics and science are fervently needed. According to the research findings of Hanushek and Woessmann (2012), student achievements in mathematics and science in international assessments are good predictors of the economic growth GDP of a country. In a world in which everything is changeable, and the future is not that certain, there are grave needs for the educational practitioners to ponder the contents and processes of education of our school children, i.e. what to learn and how to learn it. Many countries/economies are attempting to find out the correct or right mix of the recipes through conduct of education and curriculum reforms. 70
An important finding of the PISA 2018 scientific literacy assessment is that one in eight of the Macao 15-year-old students are high-achievers, and this ratio is double of the average of the OECD countries. These top-performing students are able to solve the most challenging scientific problem tasks, and in the problem-solving process think in ways like a scientist. At the same time, only second to China (B-S-J-Z), Macao is an economy having the least percentage (6%) of students not reaching the baseline level. By contrast, in OECD countries one in five students do not meet level 2 of the baseline in the PISA assessment. In a few participating countries/economies, this ratio is even one in two, signifying the situation is far from satisfactory. These low-performing students are not able to use very basic scientific knowledge in daily lives and to arrive at valid conclusions from simple data, which are basic literacy expected for every citizen in the globalized society of the 21 s t century. One characteristic of the PISA science questions is that reading a large quantity of textual information is a prerequisite for an adequate understanding the context of the problem tasks, which often are embedded in everyday scenarios in order to find out whether the examinees are able to apply the knowledge and skills in daily life situations. As afore-mentioned in chapter 3, Macao male students' improvement in reading literacy help their correct understanding and interpretation of scientific issues, as well as their expression of rational and logical thinking in mathematics and science studies. The consequence is that there is a narrowing of gender gap in scientific literacy. Compared with PISA 2015, Macao male students' progress in scientific literacy is 1.4 times of the female students' . In PISA 2018, Macao male students' mean level of scientific literacy is on par with the female students, and the slight gender difference observed is not statistically significant. Additionally, as at 2018, Macao female students' scientific literacy performance is also highly satisfactory, since the mean score is not only better than female students of other 76 countries/ economies, but also the male students of these 76 countries/economies. In summary, the PISA assessment results uncover that there is not a small reserve of high-achievers in mathematics and science in Macao. These talents should be cherished and educated to prepare them to work competently for the advances in science and technology. Last, but not the least, same as what is persisted in PISA, student in the future may not become a scientist but it is still necessary for them to participate in the realistic world that is imbued with science and technology. School should serve as a platform to convey constructive messages to their students, i.e. through learning science they are interested in exploring the things su1Tounding them, and in these explorations find interests in their school learning. 71
Chapter 5 Examination of relationships between reading literacy and student reading engagement
A large quantity of empirical studies testified the s ignificance of reading engagement contributory to the cultivation of student reading abilities (Clark & Douglas, 201 1; Guthrie, 1996; Guthr ie et al. , 1996; Guthrie & Soloman, 1997; OECD, 2019a; Wigfield & Guthrie, 1977). When PISA assesses students' reading literacy, reading engagement is highly emphasized comprising examination of degree of involvement in reading, as well as awareness of usage of appropriate metacognitive reading strategies in the processing of different types of textual information (Mak, 2013; OECD, 2009, 2019a). PISA not only envisages reading engagement as an important factor of reading literacy acquisition but also delineates it to consist of three facets, i.e. enjoyment of reading, diversity of reading, and strategies of reading (see F igure 5.1). Diversity of reading Reading Engagement Enjoyment of reading Strategies of reading Figure 5.1 The three fac1ets of reading engagement I 73
Speaking in general, extent of reading involvement depends very much upon individuals. This is because it entails reading motivation and attitudes. Of note is that when a student finds interest in reading, internal motivation increases resulting in more time spent in reading. Additionally, diversity of reading materials and classroom experiences affect student reading performance a lot (Cheung et al., 2009; Pang et al. , 2003). In the literature, there is no shortage of research reporting that competent readers possess favorable reading motivation. They are able to develop effective reading strategies from ample reading experiences so as to increase reading effectiveness (Baker, 2008; Dole, Brown, & Trathen, 1996; Garner, 1987; Guthrie, Wigfield, & Vonsecker, 2000; Lau, 2010, 2011; Lau & Chan, 2003; Paris, Lipson, & Wixson, 1983; Perfetti, 1985; Shell, Covin, & Bruning, 1995; Wang & Guthrie, 2004). During the process of instructing students to master reading strategies, teachers need to mobilize the motivational dynamics of students. This mobilization increases students confidence in reading, enhances student reading self-efficacy, elevates student reading interest, and the end results are better reading engagement and voluntary usage of reading strategies (Borkowski, Weyhing, & Carr, 1988; Chan, 1996; Clark, 1996; Dole et al. , 1996; Guthrie & Alao, 1997; Lau, 2004; Paris & Paris, 2001; Pearson & Fielding, 1991; Pressley et al., 1998; Schuder, 1993). In PISA, reading researchers make use of a conceptual model of reading engagement, and base on it to survey student participation in reading activities. Through administering a digital questionnaire, PISA collects students' home background information, learning and psychological dispositions, as well as views about their teachers and schools. Reading literacy is the major domain of PISA 2018. There are a lot of factors affecting reading engagement, as well as relationships between reading engagement and reading performance. Important factors examined in PISA 2018, amongst others, are: (1) Enjoyment ofreading; (2) Deployment of metacognitive reading strategies; (3) Diversity of reading and online reading engagement; ( 4) Reading self-concept; (5) Teacher stimulation of reading engagement and classroom climate; (6) Teacher and parental support to student. Below is a detailed discussion on how these factors play out on the reading processes of students in Macao and some referenced countries/economies. It is hoped that this analysis will help perfect the design of the reading curriculum, promote reading engagement, and elevate reading performance. 74
5.1 Enjoyment of reading Enjoyment of reading is about whether a student enjoys reading, is fond of reading, or is able to derive satisfaction through engaging in it. Research shows that intrinsic motivation is better than extrinsic motivation to stimulate an individual engaging in reading activities (Ryan & Deci, 2000). According to Macao's findings of PISA 2009, there is a very strong association between student reading performance and enjoyment of reading. Regarding this, Mak (2013) stated that only when student likes reading and treats it as a life-long habit will his/her reading performance be enhanced to new heights. Generally speaking, those students who enjoy reading are willing to spend more time on it. Because of the experiences accumulated, they will have more opportunities to develop their strategic reading skills and know how to access information efficiently. The PISA index of "Enjoyment of reading" comprises 5 items from the student questionnaire. The response scale is 4-point Likert-type (i.e. "Strongly Disagree", "Disagree", "Agree" and "Strongly Agree"). Figure 5.2 presents a comparison of item ratings of Macao students' reading enjoyment between PISA 2009 and PISA 2018. I read only to get information that I need.• I read only if ! have to.* 90.0 80.0 70.0 60.0 ... -···::~--·'·• .. , \'\ \ ( ... ··:··· ~:: I \ : \ \ 1 I \ \ \ \ \ \, 0.0 r I I \ ., . , \.___, I ,\ ! '_\\ __ Ji \ \ :, \ : ' \ \ i \~-', j \. I I I I I I I I , I I ; I I I I / / , I I I I I , I / I / I Reading is one of my favourite hobbies. For me. reading is a waste of ;_' .. _···_···_·· --------time.* '1 like talking about books with other people. ~PISA 201 8 ··PISA 2009 Note: * represents percentage of students replying "Disagree" or "Strongly disagree" . Figure 5.2 A comparison of item ratings of Macao students ' reading enjoyment between PISA 2009 and P ISA 201 8 75
When comparing the percentage of Macao students' magnitude in reading enjoyment in PISA 2018 with PISA 2009, more than half of the sampled students in PISA 2018 view reading as one of their most favorite hobbies. This is an increase in magnitude compared to PISA 2009. Of note is that "Reading is one of my favorite hobbies" is a key item of the PISA index of "Enjoyment of reading". Higher score on the index is associated with higher level of reading performance. The mean value of the PISA 2018 "Enjoyment of reading" index is 0.26, which is 3.3 times of that in PISA 2009 (=0.08). One unit increase of the index results in 32.4 score points increase in reading performance. The PISA 2018 results show that Macao students' progress in reading performance is very much due to fostering of student interests in reading in the last decade. Making reference to the index ratings of selected countries/economies, researchers are able to propose recommendations for the directions of Macao's future education. Figure 5.3 and 5.4 respectively present the item ratings of"Enjoyment of reading" of Macao and referenced countries/ economies in PISA 2018, and the scatterplot of "Enjoyment of reading" and reading literacy performance of Macao and referenced countries/economies in PISA 2018. 76
I read only to get information that I need,* For me, reading is a waste of time.• I like talking about books with other people. Reading is one of my favourite hobbies. I read only if I have to.• • B-S-J-2 (China) • Singnpore • Finland • Chinese Taipei 00 Hong Kong (Chinn) • Macao (Chinn) Enjoyment ofreudlng IO.O 200 30.0 -!0.0 lOO (,(1 ,0 700 80,0 I rend only if I hnve Reading is one of my I like tnlking nbout For me. rending is n books with other to.• favourite hobbies, people. wnscc of time.• 88.1 83.1 75,7 94,7 54.3 48,9 41.1 78.0 49.3 25,3 30.7 62,3 50.3 60.2 53.3 77.4 53.9 59.3 48.6 78.8 53,3 56,8 45,9 81.6 • B-S-J-2 (China) • Singapore • Finland • Chinese Taipei Hong Kong (Chinn) • Mncno (Chinn) 90 0 1000 I rend only to get infomrn1ion that I need.• 72,6 52.2 48.1 51.7 53.4 49.6 Note: * represents percentage of students replying "Disagree" or "Strongly disagree". Figure 5.3 Item ratings of "Enjoyment of reading" of Macao and referenced countries/ economies in PISA 201 8 (% of students replying "Agree" or "Strongly agree") 77
As shown in Figure 5.3, Macao students' enjoyment of reading are inferior to China (B-S-J-Z) but better than Finland. Nevertheless, it is comparable with Singapore, Chinese Taipei and Hong Kong. 78 650 600 550 500 450 400 Enjoyment of reading xReading literacy lligh performu I + ,,: Hil "gh perlf""an~ I Low JOYREAD : ______ High J°IYREAD CHN 1- China hLu_Z) l - --MAC_ Maeao1(~) ! HK - Hong Kong (China) i CHN . AP , - Chine~ Taipei CND : EST MAC1------'---- • EST - Estonia FIN I SGP I- Singapore • AUS • KORHK _!lOR - _Portugal ~ER -fnc • • U~A ._. TAP'.-----+----4---, ---~~=-ii'm~:!'st;t~--7---____ ,__oirn t -• - POJLJ~'\r_ - - -- -- -- -- ---- -- - - - --UK - United Kingdom _______ : I - GER _ Gennan:y---i :-FIN - Finland I : AUS - Australia : JAP - .Japan._ ' KOR - Korea 1 Low performance 350 " JOYREAD _J Low perfol'lllllce High JOYRLI\D - l.O -0.8 -0.6 -0.4 -01 0.0 0.2 0.4 0.6 0.8 Figure 5.4 Scatterplot of "Enjoyment of reading" and reading literacy performance of Macao and referenced countries/economies in PISA 2018 1.0
As shown in Figure 5 .4, Macao is an economy of "High reading performance & High reading enjoyment". Countries/economies having similar categorization with that of Macao are: Hong Kong, Korea, Japan, Chinese Taipei, Singapore, Portugal and Estonia. It is note-worthy that although China (B-S-J-Z) and Macao both belong to the "High reading performance & High reading enjoyment" category, China (B-S-J-Z) is having significantly higher level of reading enjoyment than Macao. Comparing the ratings of the items in the "Enjoyment of reading" scale, it is found that China (B-S-J-Z)'s are all higher than Macao's. Mak et al. (2017) pointed out that enjoyment of reading not only is associated positively with reading performance, but also is able to mediate the effect of gender on reading performance. Research results suggest that when males enjoy reading they will be keen on finding reading resources from their schools and community, resulting in higher opportunities to acquire better reading performance or come close to their gender counterparts. In this regard, Macao may make reference to China (B-S-J-Z) to inform how to enhance Macao students' enjoyment of reading. 79
5.2 Deployment of metacognitive strategies Metacognition refers to learners' awareness, monitoring and evaluation of their knowing processes, entailing a series of higher order thinking, as well as knowledge, experiences and strategies associated with metacognition. Learners can autonomously apply these knowledge and strategies to monitor, manage and control the cognitive process in learning (McCormick, 2003; Veenman, Van Hout-Wolters, & Affierbach, 2006). Metacognitive strategy is an important factor influencing student reading literacy (OECD, 2019a). To examine the relative efficacy of different forms of metacognitive reading strategies, PISA adopted the expert rater-scoring system to process the student responses to the metacognitive reading strategy questionnaire items. The method compares student's responses with that of the experts for consistency to determine to what extent student deploys effective reading strategies acknowledged by the experts to engage in the reading comprehension process. That is to say, for certain set of metacognitive reading strategies, when student ranks the relative effectiveness or appropriateness of the strategies the PISA index of "Deployment of metacognitive strategies" will be higher when the rank order matches to the greatest extent with that of the experts (OECD, 2012, 2019a). Through understanding student's deployment of metacognitive reading strategies, educational practitioners will have a better idea of the learning situations so as to devise curricular activities for the development of student metacognitive reading abilities . There are altogether three types of metacognitive reading strategies for examination in PISA 2018, namely: (1) Understanding and memorization strategy; (2) Summarization strategy; and (3) Evaluation of quality and credibility strategy. (1) Understanding and memorization strategy This index is about the extent a student recognizes which strategies help understand and memorize the information or content of the texts during the reading process. Reading experts think that the three strategies, namely: ( 1) "I underline important parts of the text"; (2) "After reading the text I discuss its content with other people", and (3) "I summarize the text in my own words", are effective understanding and memorization strategies. Percentage of Macao students ranking these three strategies at 5 or 6 level (i.e. "Very effective") are 26.0%, 39.6% and 35.7% respectively (see Figure 5.5). The higher the score of the PISA "Understanding and memorization strategy" index, the higher the level of student reading literacy performance. The mean value of the index for Macao students is -0.08. For each of one unit increase of the index, student reading performance will increase by 24.8 score points. 80
Making reference to the index performance of selected countries/economies, researchers are able to propose recommendations for the directions of Macao 's future education. Figure 5.5 and 5.6 respectively present the item ratings of "Understanding and memorization strategy" of Macao and referenced countries/economies in PISA 2018, and the scatterplot of "Understanding and memorization strategy" and reading literacy performance of Macao and referenced countries/ economies in PISA 2018. Mctncognitivc strntegy: Undcrstnndlng nnd mcmorlzntlon I Hummnrisc lhc lcxt in my own word.w. I underline irnportunt port• of the text. After rending the text. I di•cu•• ilM content with other people, • B-S•J•Z (Chinn} • Singnporc • Finlnnd • Chine•• Tnipci • Hong Kong (Chino) • Mocoo (Chinn) 0.0 After rending the text. I discus• ilM content with other people, B.8 n.1 31.-1 27.0 27.2 26.0 • B-S-J-Z (Chiun) • Singnporc • Finlnnd • Chine•• Tnipei • Hong Kong (Chinn) • ~lnono (Chinn) 10.0 20,0 30.0 -10.0 so.o 60.0 70.0 80.0 I underline importnnt ports of the text. I 1111nmnrise the text in my om, word>. 68.0 6-1.0 S4.4 -12.8 41.S 39.2 38.4 38.3 36.2 33.8 39.G 3S.7 Figure 5.5 Selected item ratings of "Understanding and memorization strategy" of Macao and referenced countries/economies in Pl SA 2018 (% of Macao students ranking strategies as "Very effective") As shown in Figure 5.5, Macao students ' metacognitive understanding and memorization strategy are inferior to China (B-S-J-Z), Singapore and Finland. Nevertheless, it is comparable with Chinese Taipei and Hong Kong. 81
650 I High performance I + : I.ow ; REM_ I i 600 - e - Chim (B-S-J~il 1---1---1--M.! C- SBChim) : HK - H~g ong_ ( . . ) TIN scr 550 _ AP - Chine Taipe1 - - 1 t 1 .-CHN ESir - ~ • I J I(,{A,(:' SGP - S:m~e f • 1 EST 500 l POR - Portugal KOR • tlS~ J.i\P UK.- G - CND-=-Canada T. • • 1 ' • ----usA- =-- ~-Statts- - - - - - - - llS- -- - t-t_PO.R _ _ _ _ _ _ _ _____ _ UK\ - United Kingdom pECD GER - Germany ---1-- J_ ~ - Finland AUi; - Australia r~ - Japan 400 - KOR - Korea 450 T I Low performance Low performance JSO Low UNDREM High~REM -1.0 -0.8 --0.<5 -0.4 -0.2 0.0 0.2 0.4 o., Figure 5.6 Scatterplot of "Understanding and memorization strategy" and reading literacy performance of Macao and referenced countries/economies in PISA 2018 1.0 As shown in Figure 5.6, Macao is an economy of "High reading performance & Low understanding-memorization strategy". High-performing countries/economies having higher index values than Macao are: Germany, United Kingdom, Estonia, China (B-S-J-Z), Japan and Portugal. 82
(2) Summarization strategy This index is about the extent a student recognizes which strategies help summarize a piece of text or write an abstract during the reading process. Reading experts think that the two strategies, namely: (I) "I carefully check whether the most important facts in the text are represented in the summary" ; and (2) "I read through the text, underlying the most important sentences, then I write them in my own words as a summary" are effective summarization strategies. Percentage of Macao students ranking these two strategies at 5 or 6 level (i.e. "Very effective") are 36.4% and 42.1 % respectively. There are also reading experts who think that the two strategies, namely: (1) "I write a summary. Then I check that each paragraph is included in the summary, because the content of each paragraph should be included"; and (2) "Before writing the summary, I read the text as many times as possible" are mediocre effective summarization strategies. Percentage of Macao students ranking these two strategies at 5 or 6 level (i.e. "Very effective") are 17 .6% and 28.9% respectively. Last, but not the least, there are reading experts who think that the strategy, namely, " I try to copy out accurately as many sentences as possible" is an ineffective summarization strategy. Percentage of Macao students ranking this strategy at 1 or 2 level (i.e. "Very ineffective") is 46.9% (see Figure 5.7). The higher the score of the PISA "Summarization" index, the higher the level of student reading literacy performance. The mean value of the index for Macao students is -0.23. For each of one unit increase of the index, student reading performance will increase by 30.1 score points. 83
84 J try to COi))' 0 111 nccurotcly lHI 1111111)' HCU h!llCCS nit p,,11sible, • Before writing thu summnry, I rtnd th~ lt:-XI Oft mnny timcti nH po.~~iblc. r write n 11u111111n11•. Then 1 check thru cnch pnrngrnph iH covcr~d in the Hummciry. bccnuJl'.c the content of cnch porngrnph •houicl bo included. I rend through the text, 11ndcrlinin11 tho mo;t important ,wntc11cc11. Then I write lhem in my own word11 RH n Rumnrnry. I cnrcl\tll)' chock whether the moHI importnnt foctH in !he tcxl ure rcprcHcnlcd in the sumnmry. Mctncognitlvc strategy: Summnrizntion 0.0 I0,0 lO.O ;o,o 40.0 ,o.o 60.0 70,0 80.0 I write n summnn1 • I coreli11ly check \'•1:cl thtt'· the Then I check th~I ,whether the 1110~1 11,:~,;~~: i~: 1~~~~11 cnch porngrnph i, Before writing the I Ir)' to cop)' 0111 unportnnt fncltt 111 Henh:nccN, I'rhcn I co\lcrcd in the sununnry, I rend uccurutcly ns • B-S-J-Z (Chinn) • SinJlllporc • l'inlnnd • ChineHc Tnipci • 1101111 Kon11 (Chinn) • Mocoo (Chinn) lhc ICXI nre '1 ti I Hlllllllllll')', bccnuHe lhc ltJXI IIH 11111ll)1 1111lll)' Rtnle-nccH CIH rcpreH-Cnted in !he ' ~':\'~ ,,~~~~1,'~!'~' the conlcnt of ench timeH n11 poHNiblc. pott~ihlc. • ,tmunury. pnrngroph ,hould HllllllllUI')', be inchKled. 67.2 74.0 47. 1 59.4 ,9.0 59.4 Go.G 26.1 27.9 46. 7 54.0 47.:1 34.4 22.:l 6:l.8 40.2 :18.4 22,0 26.8 :18.8 ,H ,5.s 2:i.s 21.s no 36.4 42.1 17.6 28.9 46.9 • D-S-J-Z (Chinn) • Slngnporc • Finlnncl • Chinoac Tni1>ei • Hong Kong (Chinn) • Mncno (Chinn) Note: * represents % of students replying "Not that effective" or "Totally ineffective" Figure 5.7 Item ratings of "Summarization strategy" of Macao and referenced countries/economies in PISA 201 8 (% of students replying "Effective" or "Totally effective" )
As shown in Figure 5.7, Macao students' ratings on most items of metacognitive summarization strategy are inferior to China (B-S-J-Z), Singapore and Finland. Nevertheless, it is comparable with Chinese Taipei and Hong Kong. 650 600 550 500 450 400 High pedormance Low METASUM Summarization strategy x Reading literacy t High performance High METASUM CHN - Chma (B-W.Z) MAC- Macao (China) 1 HK - Hong Kong (China: t CHN • ~~p TAP - Chin~ Taipei MAC ~ EST EST - Eslorua j HK • • CND SOP - Singapore 1 • •• POR - Portugal ~ ------ ---- __ :~-~---~ -~::s-~ _,J!,,~~ ~--~~~ ---t:~ =~ --_ J Low performance Low METASUM f OEC'D POR UK - United Kingdom GER - Germany FIN - Finland 1 AUS - Australia JAP - Japan KOR - Kor~ 350 -1.0 -0.8 -0.6 -0.4 -0.2 0.0 0.2 0.4 Low performance High METASUM 0.6 0.8 1.0 Figure 5.8 Scatterplot of "Summarization strategy" and reading literacy performance of Macao and referenced countries/economies in PISA 2018 As shown in Figure 5.8, Macao is an economy of "High reading performance & Low summarization strategy". Amongst the 14 referenced countries/economies, except Hong Kong and Chinese Taipei, the rest of the 12 countries/economies all have higher index values than Macao. 85
(3) Evaluation of quality and credibility strategy Coiro and Dobler (2007) pointed out the importance of effective and specific self- regulatory strategies to the fostering of reading goals and devising ofreading plans during online reading. Due to the increasing on line reading activities, PISA 2018 capitalizes on the characteristics of digital reading to pay attention to another significant metacognitive reading strategy index, i.e. evaluation of quality and credibility strategy. This index is about the extent a student recognizes which strategies help evaluate the information quality and credibility during the online reading process. "Fictitious task" method is adopted to develop the questionnaire item. The problem context of the assessment task is: "You have received a message in your inbox from a well-known mobile phone operator telling you that you are one of the winners of a smartphone. The sender asks you to click on the link to fill out a form with your data so they can send you the smartphone." Five strategies are listed for the student to respond (see Figure 5.9). The higher the score of the PISA "Evaluation of quality and credibility" index, the higher the level of student reading literacy performance. The mean value of the index for Macao students is -0.13. For each of one unit increase of the index, student reading performance will increase by 35.3 score points. 86
Metncognirive strategy: Evaluation of information quality and credibility Answer lhc email und ask for more infoml:11ion nboul the smortphone. • Click on the link lo fill out lhc fonn as soon ns possible• Check the sender's email address. Check the website of lhc mobile phone operator to sec whether the s11111rtphonc otli;r is mentioned. Delete the emnil \\i thout clicking on lhe link. 0.0 10.0 Check the website of Delete the email the mobile phone \\ithout clicking on operntor to see whether the U1e link. smnrtphone offer is mentioned. • 8-S-J-Z (Chino) 31.1 42.7 • Singapore 34.6 SI.I • Finland 29.0 5 1.8 • Chinese Taipei 38.1 27.7 • Hong Kong (China) 28.4 39.3 • Mucuo (Chino) 23.0 41.0 • B-S-J-Z (China) • Singapore • Finland • Chinese Taipei • Hong Kong (Chino) • Maeno (China) 20.0 30.0 ~0.0 lO.O 60.0 70.0 80.0 Click on the link 10 Answer the emnil Check the sende(s fill out 1he form ns and ask for more cmuil nddrcss. information ol>oul soon as possible• the smortphone. • 57,7 74.9 5 1.5 57.3 67.2 48.1 52.8 70.0 53.4 38.1 55.4 44.6 39.3 56.2 44.3 38.5 56.8 42.6 Note: * represents % of students replying "Not that appropriate" or "Totally inappropriate" Figure 5.9 Item ratings of "Evaluation of information quality and credib ility strategy" of Macao and referenced countries/economies in PISA 2018 (% of students replying "Appropriate" or "Totally appropriate") 87
As seen in Figure 5.9, reading experts think that the two strategies, namely: (1) "Delete the email without clicking on the link"; and (2) "Check the website of the mobile phone operator to see whether the smartphone offer is mentioned" are the most appropriate strategies. Percentage of Macao students ranking these two strategies at 5 or 6 level (i.e. "Very appropriate") are 23.0% and 41.0% respectively. There are also reading experts who think that the strategy "Check the sender's email address" is of mediocre appropriateness, or even inappropriate. Percentage of Macao students ranking this strategy at 5 or 6 level (i.e. "Very appropriate") is 38.5%. Last, but not the least, there are reading experts who think that the two strategies, namely: (1) "Click on the link to fill out the form as soon as possible" and (2) "Answer the email and ask for more information about the smartphone" are very inappropriate strategies. Percentage of Macao students ranking these two strategies at 1 or 2 level (i.e. "Very inappropriate") is 56.8% and 42.6% respectively. Macao students' ratings on metacognitive evaluation of quality and credibility strategy are inferior to China (B-S-J-Z), Singapore and Finland. Nevertheless, it is comparable with Chinese Taipei and Hong Kong. Evaluatwn of.information quality and credibility strategy x Low l\lETASP~ :___ High l\lETASPAN 650 High performm~ I +-Re4g Uteracy Hip performance 600 r CHN - China (.B..s.J-Z}- : MAC- ~o (China) : I ~ = ~!~China) ~ SG 550 E$Y-- Esfpcia MAC ] • ~--1----i S(}P - Singapore I FIN I POR - Portugal I • , R CNtj•• EST . I 500 ~-- ---~ - -~Sta~ - ~- ~ -- - -~~~ ~ ~otl!~GIJ{g _____ --t--------------UK - UnitedKingdom OF.dD 4S0 GER-~y : - ~ -Tmiand T AUS -~a 400 ,-!~;r: Low METASPAN 3-50 --+----¼--~~------1.0 -0.8 -0.6 -0'.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0 Figure 5.10 Scatterplot of "Evaluation of information quality and credibility strategy" and reading literacy performance of Macao and referenced countries/economies in PISA 2018 As shown in Figure 5 .10, Macao is an economy of "High reading performance & Low quality-credibility evaluation strategy". Amongst the 14 referenced countries/economies, except Hong Kong, Korea and Chinese Taipei, the rest of the 11 countries/economies all have higher index values than Macao. 88
5.3 Diversity of reading and online reading engagement Apart from school textbooks and learning materials, teacher should encourage students to read widely. This is especially so when online reading nowadays is becoming very popular. Additionally, teacher should understand students' online reading activities, including whether they read emails, online news, and search the web for online learning resources, as well as the frequency of which they embark on these activities. Through reading different magazines students can in contact with information and their reading interests can be developed in tandem. Of note is that teacher should pay attention to the time their students spend on comic books, magazines, online chat and discussion forum. Those who are addicted should be counselled accordingly (Cheung et al. , 2013; Cheung, et al., 2019). There are five questionnaire items loading on "Diversity of reading" (see Figure 5.11). There are 5 response categories for each item, i.e. "Never or almost never", "A few times a year", "About once a month", "Several times a month" and "Several times a week". There are six questionnaire items loading on "Online reading" (see Figure 5.12). There are 5 response categories for each item, i.e. "J don't know what it is", "Never or almost never", "Several times a month", "Several times a week" and "Several times a day". Figure 5.11 presents item ratings of "Diversity of reading" of Macao students in PISA 2009 and PISA 2018. Of note is that compared with PISA 2009, the distribution of diversity of reading of Macao students in PISA 2018 is more homogeneous. Specifically, there is an apparent decrease in the percentage of students reading newspapers or magazines several times a month and several times a week. On the contrary, in PISA 20 I 8, 45.6% of students get into habit of reading fiction (novels, narratives, stories) several times a month and several times a week. Compared with the corresponding figures in PISA 2009, there is a large increase of 13.7%. In fact, reading fiction involves deployment of a variety of reading strategies and slcills, and this deployment facilitates enhancement of high-ordering reading skills. This is probably one main reason that there is an improvement in student reading literacy in PISA 2018. 89
Newspapers Magazines 80.0 70.0 60.0 50.0~ .. - ' ..... - "40.0 ', .... ' ,.,-' 30.0 ', ..... ' "", ~--- ', ' ' ',, ",, ',,, Non-fiction books (informational, docwnentary) Figure 5.11 ~ PISA 2018 - • - PISA 2009 Comic books Fiction (novels, narratives, stories) Item ratings of "Diversity of reading" of Macao students in PISA 2009 and PISA 2018 (% of students replying "Several times monthly" or "Several times weekly") Figure 5.12 presents item ratings of"Online reading engagement" of Macao students in PISA 2009 and PISA 2018. In PISA 2018, it is very common that Macao students engage in online reading and chat, and that nearly 90% of the sampled students expressed that they do these several times a week or several times a day. This engagement pattern is similar to PISA 2009. Of note is that in PISA 2018 there is 44.1 % of students indicating that they search information online to learn about a particular topic several times a week or several times a day. Compared with the corresponding figures in PISA 2009, there is a large increase of 16.9%. Also, percentages of students reading emails or searching for practical information online (e.g. schedules, events, tips, recipes) are declining, indicating more students are inclined to go for online learning. This is probably another main reason that there is an improvement in student reading literacy in PISA 2018. 90
Searching for practical information on1ine (e.g. schedules, events, tips, recipes) Taking part in online group discussions or forums Reading emails 90.0 80.0 70.0 60.0 50.0 Searching information on1ine to learn about a particular topic Figure 5.12 Chat on line ( e.g. Whatsapp®, Messenger®) -+-PISA 2018 -•- PISA 2009 Reading on1ine news Item ratings of "Online reading engagement" of Macao students in PISA 2009 and PISA 2018 (% of students replying "Several times daily" or "Several times weekly" 5.4 Reading self-concept Reading self-concept is about knowledge and evaluation of one's reading characteristics, abilities and related performance. Generally speaking, to have a fair and unbiased evaluation of one's reading characteristics, one is required to recollect prior experiences and outcomes achieved in the past. Of note is that there is a positive relationship between reading self-concept and student reading performance (Tse et al., 2006). Those students who are more confident in themselves possess more positive reading self-concept (Bandura, 1997). In the student questionnaire, there are 6 items on reading self-concept. After scaling of the student item responses, two subscales are formed, namely: (1) Reading self-concept: Perception of competence; (2) Reading self-concept: Perception of difficulty. Figure 5.13 to 5.16 present item ratings of the two reading self-concept subscales, as well as scatterplots of the two reading self-concept subscales and reading literacy performance, of Macao and referenced countries/economies in PISA 2018. Shown in Figure 5.13 and 5.15 are reading self-concept items on a 4-point Likert-type scale of four response categories ranging from "Strongly disagree", "Disagree", "Agree" to "Strongly Agree". 91
Reading self-concept: Perception of competence I read fluently. • B-S-J-Z (China) • Singapore • Finland I am able to understand difficult texts. • Chinese Taipei • Hong 1:ong (China) • Macao (China) I am a good reader. 0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0 100.0 • B-S-J-Z (China) • Singapore • Finland • Chinese Taipei • H ong 1:ong (China) • Macao (China) I am a good reader. 73.7 66.5 7:,,7 63.0 60.3 50. l I am able to understand difficult te)l.'is. 55.3 62.0 70.l 43.7 56. 1 45.3 Figure 5.13 I read fluently. 90.9 76.7 84.9 68.7 80.3 74.3 Item ratings of "Reading self-concept: Perception of competence" of Macao and referenced countries/economies in PISA 201 8 (% of students replying "Agree" or "Strongly agree") Macao students ' self-concept ratings on perception of competence are inferior to China (B-S-J-Z), Singapore, Finland and Hong Kong. Nevertheless, it is comparable with Chinese Taipei. 92
650 600 550 500 450 400 Perception of competence x Reading lit,eracy High performance Low S0RE.ADCOMP l High performance : High SCREADCOMP ( CHN - China (B-S-J-Z) : M..\C - Macao (China) ' HK - Hong Kong (China) S~P ~ TAP - Chinese Taipei Ali\C HK : EST - Estonia • 4t ._ ESf • FIN • CND SOP - Singapore • KOK I s' POR - Portueal • JAP • TAP , GER •• • U ~ CND _ Canada -------~--------------~-----• -fOR ---+-----AtJ~Ulf- ----usx--~-UfuleoS~fes ___ _ l Low perform.ance Low SCREADCOMP OECD I UK - United Kingdom GER - Germany F1N - Finland AUS - Australia JAP - Japan KOR - Korea 350 -LO -0.8 ~0.6 -0.4 -01 0.0 0.2 0.4 Low performance High SCREADCOMP 0.6 0.8 1.0 Figure 5.14 Scatterplot of "Reading self-concept: Perception of competence" and reading I iteracy performance of Macao and referenced countries/economies in PISA 20 18 As shown in Figure 5.14, Macao is an economy of "H igh reading performance & Low competence perception" . Because Macao students ' ratings of the three items of this subscale is on the low side, the mean value of the index for Macao students is -0.41. Amongst the 14 referenced countries/economies, except Japan, the rest of the 13 countries/economies all have higher index values than Macao. The findings uncover Macao students ' low self-concept of their reading ability and this is an indication of a lack of self-confidence in reading. There is positive association between self-concept of reading ability and student reading performance. For each of one unit increase of the index, student reading performance will increase by 32.8 score points. 93
Reading self-concept: Perception of difficulty I find ii difficult to answer questions about n text. I have to read a text several limes before completely understanding ii. I hnve always had difficulty with reading. • B-S-J-Z (China) • Singapore • Finland • Chinese Taipei • Hong Kong (China) • Macao (China) 0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 I haYe to read a text I fmd it d ifficult to I have a lways had se,·eral times before answer questions about cli.fficulty wiU1 reading. completely understanding it. a text. • B-S-J-Z (China) 19.3 55.G 29.4 • Singapore 18.2 53.5 37.l • Finland 17.5 34.3 24.9 • Ch inese Taipei 20.0 37.8 22.0 • Hong hong (Ch ina) 19.6 4 1.3 34.6 • IV!acao (China) 22.4 45.5 29.5 Figure 5.15 Item ratings of "Reading self-concept: Perception of difficulty" of Macao and referenced countries/economies in PISA 2018 (% of students replying "Agree" or "Strongly agree") As seen in Figure 5 .15, Macao students' self-concept ratings on perception of reading difficulty are comparable with China (B-S-J-Z), Singapore, Finland, Hong Kong and Chinese Taipei. Nevertheless, it is noteworthy that one in five of the Macao students frequently encounter difficulty in their reading, and this ratio is a bit higher than that of other referenced countries/economies. As far as awareness of reading difficulties is concerned, Macao students ' ratings on the three items of the reading self-concept subscales tend to load on the high side. The mean value of the index is 0.21. 94
650 600 550 500 450 400 Perception of difficulty x Reading lit.eracy High perlo.nna.nce Lov SCREADDIFF I I I I I CHN - China (B-S-J-Z)" ! f M>\C - Macao(Chtca) : HK - Uong Kong (China) : C'HN SGP TAP - Chinese Taipei : ---• e EST - ~tonia EST F1N tND HJ • MAC High performance High SCRLIDDIFF SGP - Smgapore • • :e • KOR - POR - Portu~ G~ . d l --==-'- USA J.i\P. - -f- ------ --~ -~-~!f st~te;------>- - -~!~t-~ s POR Ulf - - - - - - _ 1 _ ------i•-------. UK - U.mted Kingdom , ' 1 GER - Germany- - - -FIN - firuand I _.\US - Australia JAP - Japan. lx,oR - ~orea Low performance Low SCREADDIFF 350 -1.0 -0.8 -0.6 -0.4 -0.2- 0.0 0.2 0.4 Low performance High SCREADDIFF 0.6 0.8 1.0 Figure 5.16 Scatterplot of "Reading self-concept: Perception of difficulty" and reading literacy performance of Macao and referenced countries/economies in PISA 2018 As shown in Figure 5. 16, Macao is an economy of "High reading performance & High difficulty perception". It is noteworthy that amongst the referenced countries/economies under study Macao, with the exception of Singapore and Japan, its students are experiencing higher level of reading difficulty. This indicates that Macao students lack self-confidence in reading. There is negative association between reading se lf-concept of reading difficulty and student reading performance. For each of one unit increase of the index , student reading performance will decrease by 25 .2 score points. Amongst the 14 referenced countries/economies, only four, namely Chinese Taipei, F inland, Estonia and Germany whose mean value of the diffic ulty subscale are lower than average of the OECD countries. Macao may make reference to the reading policy of these four education systems to think about how to lower students ' perception of reading difficulty. 95
5.5 Teacher stimulation of student reading engagement and classroom climate (1) Teacher stimulation of student reading engagement Role of the teacher is very important in the fostering of reading engagement. In classroom situation, there are many ways for teacher to stimulate students to participate actively in reading activities, thus facilitative of their development of reading interests and improvement of reading skills. For example, teacher can encourage students to express their opinion about a text, relate the stories they read to their lives, show the information in texts builds on what they already know, and pose questions that motivate students to participate actively. Figure 5.17 to 5.18 present item ratings of the teacher stimulation of student reading engagement scale, as well as scatterplot of this scale and reading literacy performance, of Macao and referenced countries/economies in PISA 2018. Shown in the Figure 5 .17 are 4 items on a 4-point Likert-type scale of response categories ranging from "Never or almost never", "Some lessons", "Most lessons" to "All lessons". 96
Teacher stimulation of student reading engagement The teacher poses questions that moti,·ote students to participate actively. The teacher shows students how the informotion in texts builds on what thev already know. · The teacher helps students relate the stories they rend to their lives. The teacher encourages students to express their opinion about o text. The lcochcr help, The teacher encourage! i ludcntM relnlc the lltud~n_ts to expreu lbeu 1lorh~11 they rend 10 their oprn1on nbout n text. li\'e.1. • B-S-J-Z (Chino) 71.8 71.2 • Singapore S9.S S0J • Finlnnd SU 30.8 • Chine11c Tnipei 49.4 S7.8 • Hong Kong (Chino) Ss.J S2.9 • ~locno (Chinn) 48.2 4H The tenchcr 1how1 studcntlJ how the infonnnlion in texts build., on whnt they nlrcndy know. 69J 61.0 39.4 S6.2 S6.S 48.7 Figure 5.17 • B•S-J-Z (Chinn) • Singnporc • Finlnnd • Chi.nete Tnipci Hong Kong (Chino) • Mncno (Chinn) The lcnchcr posci! que8tion1 thnt moth·n1c HtudcnlJI lo pnrtioipnte ncli\'cly. 69.8 63.8 SS.2 S4. t 60.6 46.1 Item ratings of "Teacher stimulation of student reading engagement" of Macao and referenced countries/economies in PISA 201 8 (% of students reply ing "Most lessons" or "All lessons") As seen in Figure 5 .17, Macao students' ratings on items of the teacher stimulation of student reading engagement are inferior to China (B-S-J-Z), Singapore, Chinese Taipei and Hong Kong. Nevertheless, it is comparable with Finland. There is positive association between teacher stimulation of student reading engagement and student reading performance. The mean value of the index for Macao students is -0.06. For each of one unit increase of the index, student reading performance will increase by 11.8 score points. 97
T,eacher stimlllation of stude,rt reading engagement xReading literacy 650 600 550 High performancJ Low STil\fREAD --CHN - China..(R-SJ"--MAc- Macao (Chim) HK - Hong Kong (C TAP - Chinese Taipei EST - soru~ SGP - Singapore - ~--Canaaa 500 f PO~ - Portlfgal -- -- -- -- us'p;: ---= --l:forted-3tarer UK - Unifed K.ingd GER - German__y FIN - Fin.land 450 ALTS - Australia _I i-) I : I I -, SG MAC , • : HK e FIN I ES' : JAP ~. -'-_______ GER. ___ ___ __ .,_ J OECD : POR I I I I I I I JAP - Japan 400 - 1':0R _ K~----+------4--Low ped ormance I I T I JSO Low STIMREAD I I I I --'--'•CND OR ' UK High pertprmance High STIMREAD I t CHN -4-------1 I l ~~:i--J--------------- =..J'AP --~,-----+---~ r L
The teacher poses questions that motiYate students to participate actiYely. The teacher encourages students to express their opinion about a text. 50.0 The teacher shows students how the infonnation in texts builds on what they already know. Figure 5.19 --PISA 2018 -· · PISA 2009 The teacher helps students relate the stories they read to their liYes. Item ratings of "Teacher stimulation of student reading engagement" of Macao students in PISA 2009 and PISA 2018 (% of students replying "Most lessons" or "All lessons") (2) Classroom climate School is an important place which influences an individual's growth and Ii teracy development. Within school, classroom climate is a quality education indicator assessed in every cycle of PISA assessment (OECD, 2019c ). If the classroom climate of lessons is not well and discipline of students is poor, then teacher will find it very difficult to engage in effective teaching and students are not able to immerse in learning. The consequence is that student academic performance and emotional development are affected adversely. These are not facilitative of students' all-round personal development. Therefore, educational practitioners should pay attention to the classroom climate and its effects on student learning. 99
Figure 5.20 and 5.21 respectively present the item ratings of classroom climate of Macao and referenced countries/economies in PISA 2018, and the scatterplot of classroom climate and reading literacy performance of Macao and referenced countries/economies in PISA 2018. Shown in the Figure 5.20 are 5 classroom climate items on a 4-point Likert-type scale of response categories ranging from "All lessons", "Most lessons", "Some lessons" to "Never or almost never". There is positive association between classroom climate and student reading performance. The mean value of the index for Macao students is 0.10. For each of one unit increase of the index, student reading performance will increase by 13 .6 score points. 100 Classroom climate Students don't start working for a long time after the lesson begins. Students cannot work well. The teacher has ro wait a long time for students to quieten down. TI1ere is noise and disorder. Students don't listen to what the teacher says. • B-S-J-Z (Ch ina) • Singapore • Finland • Chinese Taipei • Hong Kong (China) • Macao (China) 0,0 10.U 20.0 30,U -l-0.0 ,o.O 60.0 70.0 Students don't . . The.teacher h:18 to listen to what the There.is noise and wait a long tune Students cannot t I disorder. for students to work well. eac ier says. quieten down. 89.1 91. 1 92.3 91.0 73.3 66.9 74.2 86.2 70.1 63.5 72.5 84.5 77.4 79.5 81.1 82.2 80.7 81.0 83.9 83.2 77.2 83 .6 86.3 86.3 Figure 5.20 • B-S•J-Z (Chino) • Singapore • Finlnnd • Chinese Taipei Hon~ Kong (Chino) • Mneno (Chinn) K0.0 90.0 100.0 Students don't start working for a long ti me after the lesson begins. 90.9 81.7 74.8 78.8 78.7 80.4 Item ratings of "Classroom cl imate" of Macao and referenced countries/economies in PISA 2018 (% of students replying "Some lessons" or "Never or almost never")
As seen in Figure 5.20, Macao students' ratings on items of classroom climate are inferior to China (B-S-J-Z), slightly better than Finland. Nevertheless, it is comparable to Singapore, Chinese Taipei and Hong Kong. Classroom climate x Reading literacy 650 High performance f ~ : Low DISCLIMA : 600 CfL'i - Chitta (B-S-J-Z) f ~c - ~o (China) : HK - Hong Kong (China) : SGP IAP - Chin~ Taipei CND 1 + • 550 High performmce High DISCLIMA CHN I --1 EST - Estonia I MAC HK SGP - Singapore FIN " : • • I I 500 ~ = ~= AUS , GERpi~ • • _1 ~ EST -t-• JAP e KO~ ----~ -~-g:rj ~~- -- ->-oicor--\ t usA--- y------- -- ---t---____ T ___ 450 GER - Geq:nan_y ~- l UK mt -Finland : AUS. - Auitralia l 1.-\P - Japan ----+---: 400 K.OR - K~ 7· · ]-Low performance : I Low DISCLIMA : 350 1 -1.0 -0.8 -0.6 -0.4 -0.2 0.0 0.2 Figure 5.21 0.4 0.6 Low performance High DISCLll\fA 0.8 1.0 1.2 Scatterplot of "Classroom climate" and reading literacy performance of Macao and referenced countries/economies in PISA 2018 As shown in Figure 5.2 1, Macao is categorized as an economy of "High reading performance & High classroom climate". It is noteworthy that although Korea, China (B-S-J-Z) and Japan are of the same category as Macao, their classroom climate are higher than Macao's. Hong Kong, Estonia, Chinese Taipei, United States, Singapore and United Kingdom are education systems having comparable level of classroom climate with that of Macao's. 101
5.6 Teacher and parental support to student That teacher and parents join hands to create a conducive learning atmosphere at home can support student learning to read (Finn, 1998; Darling, 2011 ). When parents communicate frequently with children and understand their activities at school, their children's academic performance are influenced greatly (Ho & Willms, 1996; Ho, 2010; Schulze & Lemmer, 2017). When parents pay attention to education, their children's learning generally are also better. In Macao, home reading resources and provisions are shown to have indirect effects on 15-year-old students' reading literacy performance (Cheung, Sit, & Mak, 2011 ). It is noteworthy that both home and school are important factors affecting physical and psychological health of the students. When student feels the support and concern from their teachers at school, and concomitantly gets the support and care from their parents/guardians, there will be constructive influences upon student's development psychologically, cognitively and socially. All in all, every individual meets challenges and encounters drawback every day. It is of paramount importance for student to have understanding, care and respect from teachers. Equally important is to have their unfailing support in order to overcome difficulties. In PISA 2018 through student questionnaire, it is possible to assess student's awareness of the support rendered by their teachers and parents. Figure 5.22 to 5.25 present the item ratings of teacher and parental support to students, and the scatterplot of these support and reading literacy performance, of Macao and referenced countries/ economies in PISA 2018. Shown in the Figure 5.22 are 4 teacher support items on a 4-point Likert-type scale of response categories ranging from "All lessons", "Most lessons", "Some lessons" to "Never or almost never". Shown in the Figure 5.24 are 3 teacher support items on a 4-point Likert-type scale of response categories ranging from "Strongly disagree", "Disagree", "Agree" to "Strongly Agree". 102
The teacher continues teaching until the students understand. The teacher helps students with their learning. The teacher gives extra help when students need ii. The teacher shows an interest in every student's learning. Teacher support 00 100 200 ,0 0 40.0 soo 600 The teacher gives extra The teacher helps students The teacher shows an interest in every student's learning. 83.2 80.4 73.6 57.0 73.4 69.7 help when students need it. with their learning. • B-S-J-Z (China) 85.5 85.7 • Singapore 83.9 87.6 • Finland 85.3 87.2 • Chinese Taipei 79.0 8 1.8 • Hong Kong (China) 74.3 77,9 • Macao (China) 72.3 75.4 Figure 5.22 • B-S-J-Z (China) • Singapore • Finland • Chinese Tai~i • Hong Kong (China) • Mocao (China) 70.0 3(10 900 1000 The teacher continues teaching until the students understand. 76.2 78.4 74.8 70.3 70.3 68.4 Item ratings of "Teacher support" of Macao and referenced countries/economies in PISA 2018 (% of students replying "Most lessons" or "All lessons") As seen in Figure 5.22, Macao students ' mean ratings on all the 4 items of the "Teacher support" scale are lower than China (B-S-J-Z), Singapore, Finland and Hong Kong. Compared with Chinese Taipei, amongst all the 4 items only "The teacher shows an interest in every student's learning" receives higher endorsement by the Macao students. 103
Teaclter support x Reading li,uracy 650 High perfornwu,e High TEAC'HSIJP l~~t~.=i 7+i 600 CHN - China (B--S.J-Z) :----+---..---MAC - Macao(China) : +-+ HK - ~~ong ( China) : SGP .rlllY I 550 T _ · Taipei -- l,{A 1 .'-Ill" I EST - Estoniaj l JIK I SGP - Smgapqre EST.. KOR• FIN I 500 450 7:1-FuuandL -I - Australia I - Japan 400 KOR.'--K«~ -----1----1-----!-------'-~ !Low~~~;; 350 ~ 1--1.0 -0.8 -0.6 -0.2 0.0 0.2 0.4 Figure 5.23 ~ performance High TEACHSUP 0.6 0.8 Scatterplot of "Teacher support" and reading literacy performance of Macao and referenced countries/economies in PISA 2018 LO As shown in Figure 5 .23, Macao is categorized as an economy of "High reading performance & Low teacher support". Amongst the 13 referenced countries/economies, except Estonia and Germany, the rest of the eleven are having higher index value of teacher support than Macao's. 104
My parents encourage me to be confident. My parents support me when I am facing difficulties at school. My parents support my educational efforts and achievements. • B-S-J-Z (China) • Singapore • Finland • Chinese Taipei Hong Kong (China) • Macao (China) 70.0 My parents support my educational efforts and achievements. 93,7 92.6 90.5 9 1.3 88.4 88.4 Parental support • 13-S-J-Z (China) • Singapore • Finland • Chinese Taipei • Hong Kong (China) • Mncao (China) 75.0 80.0 85.0 90.0 95.0 IOO.O My parents support me when I My parents encourage me to be am facing difficulties at school. confident. 91.8 84.2 86.6 89.5 84.4 81.2 Figure 5.24 91.8 87.4 87.4 86.4 85. 1 81.8 Item ratings of "Parental support" of Macao and referenced countries/economies in PISA 201 8 (% of students reply ing " Agree" or "Strongly agree") As seen in Figure 5.24, Macao students ' mean ratings on all the 3 items of the "Parental support" scale are lower than China (B-S-J-Z), Singapore, Finland and Chinese Taipei. Compared with Hong Kong, amongst all the 3 items only "My parents support my educational efforts and achievements" receives comparable level of endorsement by the Macao students. 105
650 High performance LowEMOSUPS 600 550 Parental support x R eading literacy MAC High performance High EMOSUPS CHN -- China (B-S-J-Z) , MAC -- Macao (China) CHN HK - Hong Kong (China) SGP I ; TAP - Chinese Taipei EST : EST - Estonia HK • 7 J __ lfOR SGP - Singapore 500 .JAP• • TAP ~KW e AUS POR - Portugal ·---------------------------------------------------OECD·t~ t-~;~•-.P-OK ___ :.A---=--~::!~:~~-----GER -- Germany FIN - Finland AUS -- Australia JAP -- Japan KOR -- Korea 450 400 Low performance LowEMOSUPS 350 -1.0 -0.8 -0.6 -0.4 -0.2 0.0 Figure 5.25 0.2 0.4 Low performance High EMOSUPS 0.6 0.8 1.0 Scatterplot of "Parental support" and reading literacy performance of Macao and referenced countries/economies in PISA 2018 As shown in Figure 5.25, Macao is categorized as an economy of "High reading performance & Low parental support". Amongst the 13 referenced countries/economies, except Hong Kong's parental support is comparable to that of Macao, the rest of the twelve economies are having higher index value of parental support than Macao's. There is positive association between teacher support and student reading performance. The mean value of the index for Macao students is -0 .08. For each of one unit increase of the index, student reading performance will increase by 6.1 score points. Concomitantly, there is positive association between parental support and student reading performance. The mean value of the index for Macao students is -0.34. Amongst the referenced countries/economies examined, Macao students' parental support are low in magnitude. For each of one unit increase of the index, student reading performance will increase by 10.8 score points. Based on the above research results, Macao teachers and parents should join hands to establish conducive learning environments for their children. They should provide sufficient support to enhance adolescents' reading literacy development. 106
5.7 Summary Reading needs long-term development to establish relationships. It really takes time to foster and cannot only depend on the sheer passion of storytelling by parents or merely increasing of quantity of reading in school. Ko (2006) noted that there is a need to lead children to learn to read at an early age, based on sound theories of reading instruction. Besides, scholars who research into reading strategies emphasize a lot on the so-called "strategies" and have it differentiated with knowledge and skills taught in traditional classrooms. They think that deployment of reading strategies is the ability that should be flexibly adapted to the needs of an individual, which entails complex and well-thought-out cognitive processes. The key rationale of reading instruction is to enable students to learn to read, and empower them construct meanings of the texts to read for understanding. Also, students are able to solve problems encountered in the learning process, and they need not rely on their teachers or textbooks so as to interpret the meanings of the text read (Dole et al. , 1991; Lau, 2004; Rosenshine & Meister, 1997; Stahl, 1997). Mak (2013) remarked that reading engagement, effective metacognitive reading strategies, and becoming a proficient reader, are in fact mutually interdependent, entailing an ongoing cyclic development process in the development of reading literacy. Reading engagement is an important component of reading literacy. In Figure 5.26, reading engagement of Macao students in PISA 2009 and PISA 2018 are compared. As seen in the figure, due to a decade of strenuous efforts invested by Macao schools, there are considerable progress of student reading performance between the two cycles of PISA assessment. Worthy of mention is that there is considerable increase in sh1dent reading enjoyment in PISA 20 I 8. Because teachers in lessons consciously stimulate students to engage in reading activities, there are also considerable progress in student metacognitive reading ability. Admittedly, compared with China (B-S-J-Z), Singapore and Hong Kong, Macao judged on these indices are still lacking behind and in this regard, schools still have ample room for improvement in the future. 107
Teacher's stimulation of reading engagement Enjoyment of reading 0.30 Metacognitive strategy: Swnmariz.ing Figure 5.26 -+-PISA 2018 - • - PISA 2009 M etacognitive strategy: Understanding and memorizing Comparison of reading engagement of Macao students (PISA 2009 vs PISA 20 l 8) As elaborated before, the reading engagement indicators are valuable to foster in students reading literacy, and readers should be aware that their effect sizes are somewhat different (see Figure 5.27). The most important implication regarding find ings of PISA 2018 is as follows: Teacher should not only develop students' interests in reading, but also develop in them habits of reading a variety of textual materials. In reading instruction teacher should not omit teaching students how to read skillfully in orderly classroom atmosphere. Apart on the cognitive side of learning, it is equally important that the affective side such confidence in reading should also be paid attention to and supported by teachers as wel l. With a ll these done effectively student reading performance should be improved accordingly. 108
40 3S 32.4 30 2S 20 IS 10 0 35.3 32.8 30.1 24.8 13.6 11.8 I 10.8 I I 6.1 I Figure 5.27 Score-point difference in reading performance due to indicators of read ing engagement, reading se lf-concept or parental support When devising reading policy and making recommendations, there is a need to create opportunities for teachers to understand the reading characteristics of their students, as well as the rationale underlying the reading policy. Then adequate support should be rendered. All these practices and actions should facilitate fostering of adolescents' development in reading literacy. In sum, PISA studies seek to help participating countries/economies to understand their countries' /economies' gap in educational quality, equity and efficiency after an objective comparison with the international average. In the past decade, Macao-PISA Center had made use of the empirical assessment data to validate the effects of reading engagement on student reading performance, particularly students of different demographic characteristics. Specifically, secondary data analyses are conducted and discussions are held to find out reading engagement learner characteristics in the promotion of reading performance for informed reading policy making (Cheung et al., 2016; Mak et al., 2017). With these endeavors, Macao's education policy is able to tie in well with the weaknesses unveiled in the PISA studies, resulting in the building of a firm foundation of basic education in the 2 l st century. 109
Chapter 6 Conclusions and recommendations
In the second decade of the 2P1 century, Macao's basic education system has been highly endorsed by a world-renowned international education organization. Specifically, the PISA research conducted by the OECD every three years, pointed out that Macao is the only country/economy with sustained and rapid progress in education quality (OECD, 2019b ). The equality of Macao's education has won international recognition and enjoyed its worldwide reputation. Macao's progress in PISA is inseparable from the national education policies that have long facilitated the development of basic education in Macao, the full support of the Macao SAR Government for Macao's basic education system, and the unremitting efforts made by educators in talent cultivation. Under the leadership and support of the Education and Youth Affairs Bureau (DSEJ) of the Macao government as well as scholars and experts from the University of Macau, the Macao-PISA Centre has successfully conducted and implemented the PISA 201 8 study (see Appendix 1 for details of the list of experts from the University of Macau in PISA 2018). The results of PISA 2018 was released globally on December 3, 2019, and this report serves as an extended report on the PISA 2018 results of Macao, China. 6.1 Significance of Macao's participation in PISA studies PISA studies are increasingly valued by more countries around the world. It has become an important indicator to reflect the quality, equality and efficiency of education systems of countries/ economies, affects the education reform and development around the world. PISA's feature of "policy orientation" in serving educational po licy formu lation and curricular reform is demonstrated visibly in two major research aspects: "excellent education quality" and "inclusive education equity". OECD stresses that, during the basic education stage, it is not necessary to deem "quality and equal ity" as contradictory policy objectives as they are highly compatible. In fact, PISA's research results show that some countries have indeed ensured the quality of education service to students while guaranteeing education equity. The Macao SAR Government has participated in six cycles of PISA studies, utilized the international exchange and cooperation platform created by PISA, guided basic education schools in Macao to refer to the successful experiences of various countries/economies in promoting education reform, and continuously conducted self- improvement and teaching intervention to gradually improve the overall level of Macao's education and facilitate students in their successful learning. 111
PISA's assessment of literacy focuses on the application of functional knowledge and skills, pays attentions to whether students can proactively participate in social activities, and emphasizes that individuals should not only passively complete tasks but also actively improve the quality of decision-making. Different from the tests that aim at talent selection and entrance examination, as well as the TIMSS (Trends in International Mathematics and Science Study) tests, PISA is not based on the curriculum standards for compulsory education of any country/economy, nor is it focused on subject knowledge. It does not only test the so-called "ability/capability" but also measures affection, attih1de and value judgment. Macao's education sector has benefited from its participation in PISA by referring to PISA's interpretation of literacy, incorporating relevant important indicators for ability cultivation and assessment into the education perspective of schoo ls, and thoroughly analyzing and discussing relevant factors affecting the literacy performance of Macao students. The argumentation and practice of any education reform are gradually carried out with focus on research, assessment and policy analysis with academic basis. Twenty years after Macao's return to China, under such guidance, Macao has overcome the difficulties and obstacles in its education reform step by step, and gradually improved the overall level of Macao's educational provision. In the summary of progress made by Macao's basic education system in PISA 2018, as usual, there are many factors that greatly affect the literacy performance, and their effects vary according to the context background of schools (Cheung, 2015; Cheung, 2017; for details of Macao-PISA 2018, please refer to Figure 5.27 in Chapter 5 of this report). It should be pointed out that the improvement of literacy cannot be achieved immediately upon the partial or superficial policy promulgation or teaching intervention. It must rely on the long-term and intensive cultivation of the education system to plan a comprehensive and systematic intervention to achieve results . PISA's Secretary General Andreas Schleicher once said that "without data, all you have are just opinions" . Reasonable, orderly and effective education policies cannot be formulated by opinions alone. PISA test has always been based on the collection of educational data from multiple aspects and the systematic information analysis. By comparing with the performance of students from other countries/economies, Macao will observe and inspect the quality and practical issues of its basic education provision more objectively and rationally, so that each project and its various components in the formulation, implementation and evaluation of education policies, such as the resource input at different key stages of education, multiple meanings of education results, conditions for achieving education objectives, cost for obtaining education performance, and education opportunities for all stakeholders, can be established based on solid research foundation to provide important reference for the scientific administration of the Macao SAR Government. 112
6.2 Reasons for Macao's progress in PISA 2018 Targeting at the issues and weaknesses in Macao's basic education system reflected in the PISA research, the Macao SAR Government and researchers have effectively linked various components in education policy, education practice and education research through empirical research and plentiful discussions, thus systematically promoting the reform of Macao's basic education system. In particular, educators conducted observation from two aspects, namely, input of education resources and teaching guidance based on empirical research, to understand how Macao can continuously monitor and improve the quality and equity of its basic education from an international macro PISA perspective, so as to promote schools in Macao to improve their accountability mechanisms for curriculum and teaching. 6.2.1 Effective input of education resources Whether the government's investment in basic education and its measures to benefit the people can achieve the expected results depends on whether these measures can be directly converted into high-quality public education services. In terms of the government's resource allocation and ensuring the education equity ofchildren, high-efficiency education systems are characterized by the effective utilization of resources, professional teacher teams and students actively engaged in study. Through the effective school education resource allocation policy, the Macao SAR Government has provided favorable conditions for schools from various aspects. Taking the basic education as an example, in terms of fund guarantee, according to the statistics of the DSEJ from 2006 to 2016, the government's public expenditure for non-higher education has increased by 3 .2 times in 2016 compared with 2006; its average expenditure for students' public education per capita increased by 3.8 times; in terms of the construction of teaching staff, in 20 I 6, 99.5% of kindergarten teachers, 98.5% of primary school teachers and 91.3% of secondary school teachers in Macao have received trainings on professional instructional skills (DSEJ, 2019). So far, these numbers are still growing. Meanwhile, the DSEJ has also provided financial support for schools to change their structures to further improve the learning environment and conditions of students, such as providing various subsidies and incentives to reduce the retention rate, reduce the staff-student ratio, reduce the class size, establish science labs , and renovate or reconstruct schools through fund assistance, providing equipment, dispatching professional reading promoters and IT personnel, and supporting students to participate in international exchanges to broaden their perspectives. In addition, to further improve the professional capacity of teachers, according to the impact of teaching quality of human resources (i.e., teachers and principals) on the learning effectiveness of students, the DSEJ has also requested national education departments to send outstanding teachers from Mainland China to Macao to improve the teaching quality of local schools, as well as guide and strengthen the teaching skills and efficiency of in-service teachers; it also cooperated with education departments of universities and colleges to provide on-the-job trainings for teachers on relevant teaching methods, assessment strategies and classroom management to consolidate the professional capacity of teachers in Macao. The aforementioned education policies have positively promoted the sound development of Macao's education system, and fully demonstrated the positive significance of placing education resources in key positions. 113
6.2.2 Evidence-based pedagogy and instruction Education is a science that involves theory and practice. It requires the guidance of scientific theory and relies on the support and promotion by education research. The OECD conducts PISA assessments every three years to provide transnational and comparative evidence of educational effectiveness for participating countries/economies, and advocates that the formulation of education policies should be evidence-based. In fact, the feature of evidence-based education is to provide scientific, data-based and systematic answers to practical education issues that require solutions, so as to develop sound education policies and guide educators to improve their work quality and efficiency. McKay (2017) pointed out that evidence-based education should be implemented in three phases: first, methods that have been proven effective should be identified, which means that empirical research results should be integrated through theories and transformed into knowledge with easy accessibility to users. Second, a support system should be set up during the implementation phase to ensure that the evidence-based education is able to identify and solve on-site issues, and to establish further support or adjustment through more practice, i.e., increase the external validity of evidence-based education. Third, outcomes of practices should be further promoted, including evidence-based education with external val idity as well as support and verification required during the practice process. According to McKay's viewpoint, the key for the eventual formulation and implementation of evidence-based policies is to collect relevant evidences with cautious attention paid to the relationships between "scientific analysis" and "decision improvement". The advocated policies should be based on the understanding of comprehensive and forward-looking evidences in order to improve the efficiency and effectiveness of the decision-making processes by administrative departments. The Macao-PISA Centre is a research and development institution under the Educational Testing and Assessment Research Centre (ETARC) of the University of Macau. Since its establishment in 2006, the Centre has undertaken all technical projects of the PISA test, and conducted analysis on the outstanding issues and principal contradictions reflected in the PISA research results. It utilizes PISA data to find solutions from the aspects of academic theory, data analysis and teaching practice, and carries out a series of systematic research tasks, including: 114 (i) Argumentation on issues concerning Macao's grade retention system; (ii) Argumentation on teaching guidance aiming to improve the reading literacy of male students; (iii) Argumentation on the cultivation of information navigation ability in the era of big data from print-based to adaptive digital testing. These argumentations will be elaborated in the following sections.
(i) Argumentation on issues concerning Macao's grade retention system The assessment results released from PISA 2003 to PISA 20 I 2 have all shown that the percentage of 15-year-old students in Macao who have been detained in the same grade in primary school and/or secondary school is one of the highest among the participating countries/economies. There have always been two oppos ing views in the education sector on taking the retention as an accountability means by schools. In order to persuade schools to reduce the retention rate, the Macao-PISA Centre has explained the devastating influence of Macao 's school-retention system on students ' psychology, classroom management and learning opportunities with the PISA data, pointing out that the effectiveness of courses guaranteeing minimum education standards provided by some schools for their retained students urgently needs to be improved; researchers suggested schools to replace grade retention with supplemented teaching support to al low students to attend classes with their peers of the same age, so that schools would understand that providing supplemented teaching support can better help improve students' core literacy performance (Sit et al, 2015; Mak & Cheung, 2016). Meanwhi le, in terms of education policy, in order to help and encourage schools to reduce the retention rate, the DSEJ has also provided strong guidance and financial support for schools. According to the results of PISA 2018, Macao has a higher percentage of 15-year-old students attending appropriate grades (i.e., Grade 9 or Grade 10), which has increased from 56% in 2003 to 88% with an increase of over 30% (see Figure 1.3 in Chapter l). It is one of the reasons for the improvement of Macao students' performance in the three core literacies. (ii) Argumentation on teaching guidance aiming to improve the reading literacy of male students Reading literacy is an important foundation for school education and lifelong learning of citizens. The reading literacy of Macao students has gradually increased from its low point in PISA 2009 to a level significantly higher than the average of OECD countries in PISA 20 I 8, which was mainly due to the improvement of reading literacy of male students. According to the researchers ' analysis of PISA 2009 data, female students in Macao performed well in reading, but not male students; the three reading engagement indicators Uoyful reading, extensive reading and strategic reading) are not only the key elements to improve Macao 's overall reading literacy level, but also an important measure to narrow the gender differences among students (Mak, 2013). In particular, in the analysis that explains the mediating mechanisms involved in the gender differences in reading at the student level and across-level from student to the school, it shows that male students attending coeducational schools can benefit from the co-learning environment, especially at the two reading engagement of "joyful reading" and "strategic reading" (Mak et al, 2017). When improving the reading literacy performance of male students, schools in Macao do not need to teach male and female students separately, as both of 115
them will improve their reading literacy performance as long as teachers assist students in "joyful reading" and "strategic reading", and gender differences will also be narrowed in tandem. The research results prove that the gender differences in reading literacy can be narrowed through nurture of individually configured education. It is suggested that Macao teachers can change the reading engagement model of students through three levels of response to intervention according to the learner characteristics of their students' reading engagement model, and refer to the relevant research results of the Macao-PISA Centre for specific measures (Cheung et al. , 2016). In PISA 2015, female students in Macao performed better than male students in all three literacy domains, which shattered the stereotype that males were better at mathematics and science than females. However, through the in-depth analysis, it was found that if the level of reading literacy performance is controlled statistically (that is, if the reading abilities of a male and a female are similar), male students in Macao will outperform female students in science literacy with a gap of about 1/3 to 1/2 grades of schooling, and so as in mathematics literacy. It can be deemed that males are better at mathematics and science than females , which is likely due to genetic advantages. Female students in Macao performed better than male students in all three literacy domains in PISA 2015, because the mathematics and science literacy questions set in the PISA test involved a large amount of content requiring reading comprehension and reflection, and the further improvement offemale students' reading literacy also helped with their ability to solve mathematics and science questions; on the contrary, the low performance of male students in reading literacy has affected their performance in solving mathematics and science questions. Therefore, before the data processing and statistical analysis, male students had almost no advantages over female students, or were even worse than them in terms of mathematics and science in the PISA literacy assessment results in recent years. In other words, if males could reach the same reading level as females, males would continue to maintain and fully utilize their advantages in mathematics and science. The similar results were observed not on ly in Macao but also in many other countries/economies participating in PISA 2015 (Mak et al, 2019). In short, improving the reading performance of males will help improve their mathematics and science performance which is generally lagging behind females in academic studies as evidenced around the world in recent years. Therefore, it is of great import for teachers and parents to consciously cultivate students' interest in reading to acquire reading strategies in their extensive reading experiences. However, for schools, different education systems have their own ways of investing in learning opportunities, human resources, materials and education expenditures (Gamoran, Secada, & Marrett, 2000). The study shows that human resources (teachers and principals) can better explain the variation in students' performance than materials and financial resources (Buchmann & Hannum, 2001; Fuller, trainings to better educate their students. Therefore, in the past ten years, the Faculty of Education of University of Macau and the DSEJ have intensified many pragmatic tasks for reading promotion 116
and classroom teaching, especially for the theory and teaching content of reading engagement. In chapter 5, development of Macao students' reading enjoyment and deployment ofreading strategies has been reported in great details, partially explaining why Macao students excel in reading literacy in PISA 2018. (iii) Argumentation on the cultivation of information navigation ability in the era of big data from print-based to adaptive digital testing With the popularization of the Internet and the development of cloud computing, the acquisition and analysis of big data has fundamentally changed and inspired people's understanding and operation models in many areas, including the education sector. Based on the PISA 2015 digital test, PISA 2018 further improved the assessment technology, as the multi-stage adaptive testing design was introduced in the reading literacy test. The PISA test system automatically selected the most appropriate questions for students according to their responses to the question clusters, the response time required for the preset reading questions, as well as the cognitive demands of the test units. In such tests based on the development of information technology, students need to challenge the computer systems step by step to prove their information navigation ability, and consciously monitor and manage web page navigation during the process to better demonstrate their ability. When we look back at the development of PISA, we find that digital test was first introduced in PISA 2009, in which the research result showed that Macao students enjoyed the digital reading test. However, when compared with countries/economies with high-performing reading literacy, Macao students lacked the ability to navigate between web pages and evaluate the sources and quality of information, which was why their digital reading performance was not improved (Sit et al. , 2014). At the same time, it also revealed the urgent need to enhance the students' ability to evaluate and reflect on the authenticity of information in school education, as the digital reading ability to read in multiple parallel texts and effectively search, organize and filter the large amount of information is the most basic literacy of future citizens. The assessment frameworks of PISA 2012 and PISA 2015 were revised to meet the development needs of the times, further requiring students near the end of their compulsory education to have: (I) the ability to interpret data generated during the process of problem solving and virtual experiments, and (2) the ability to read charts and tables when employing appropriate data to perform high-level thinking. The Macao-PISA Centre has conducted several PISA tests, and used research data to compare the quality of students' answers and methods in digital tests in Macao and other countries/economies (Jeong et al., 2016; Jin, 2018; Jin, Cheung, & Sit, 2016), which was conducive to explaining the importance of improving the digital navigation efficiency and effectiveness of students in Macao. The Government of Macao has vigorously promoted the IT education policies and provided financial support for the IT equipment of schools, which also facilitated the progress of Macao students in the PISA 201 8 digital test. 117
These education reforms based on empirical research all proposed that schools must inspect their education objectives and adjust their curriculum according to the development of the society in order to further improve the education system in Macao. 6.3 Prospects and suggestions New breakthroughs can only be made when there are forward-looking and practical reform strategies in the education system. Macao's progress in PISA 2018 has demonstrated that its basic education system has reached a new level and entered a critical period of development, which means that, in its development of educational dispositions, instead of simply constructing and investing in hardware, it should also pay attention to issues such as cultivation objectives, cultivation models and cultivation mechanisms, so as to conduct more rational thinking and response from the perspectives of the comprehensive development of students, the construction of favorable dispositions of schools, and people-oriented education. In the future, while the government guarantees the 15 years of quality education for children, schools in Macao should also continue to make progress by promoting curriculum innovation, optimizing teaching process, building a harmonious relationship between students and teachers, and enhancing the education of parents. According to the assessment results of Macao-PISA 2018, Macao's basic education should be continuously improved in the following three aspects : 6.3.1 Increase the proportion of 15-year-old students in Macao attending the appropriate grades PISA's sampling targets include all secondary school students aged between 15 years and 3 months old and 16 years and 2 months old during the test period. Most of the students in Macao sampled in PISA 2018 were attending Grade 9 and Grade I 0, accounting for 88% of all the tested samples. The data indicated that significant progress has been made in the education system of schools in Macao for lowering the retention rate. However, compared to China (B-S-J-Z), Singapore, Hong Kong and Chinese Taipei where respectively 96.9%, 98.4%, 92.1 % and 99.9% of students were attending Grade 9 and Grade l 0, schools in Macao still need to work harder to allow more 15-year-old students to attend age-appropriate grades (Grade 9 and Grade 10), should adopt more flexible teaching models and assessment methods according to students ' learning differences, as well as provide proper teaching guidance and learning support to facilitate students' successful learning. 118
6.3.2 Make utmost efforts to cultivate outstanding talents with balanced development PISA defines students with a literacy proficiency level of 5 or above as high-performing students; all-round students who score high literacy performance in all three domains only account for a small proportion, and they are the excellent talents valuable to the society. With 33% of students having high performance in one literacy domain and 7.4% of all-round students performing well in all three literacy domains, Macao is one of the few regions with significant growth. However, compared with the top two education systems, namely, China (B-S-J-Z) and Singapore, there is still a considerable room for Macao's improvement. Only 2.8% of talents in Macao were good at both mathematics and science, which may bring potential risks to the future digital economy or industrial upgrading. While cultivating outstanding professional talents, especially emphasizing on the balanced development of students' key competence and core literacies, Macao's education system also needs to thoroughly discuss and formulate education policies and specific measures for talent cultivation from the macro perspective of global economic development. 6.3.3 Enhance the teaching guidance for reading engagement Reading engagement is the most important part ofreading literacy. After ten years of hard work, Macao students have made certain progress in their reading engagement. In particular, the pleasure of reading of students participating in PISA 2018 has improved greatly, as teachers have consciously intensified and encouraged students to participate in reading activities in the classroom, and there was also considerable progress in the students' metacognitive reading strategies. It should be also noted that, as far as the universality of reading materials is concerned, the proportion of fiction has increased among students participating in PISA 2018, which has helped them conduct in-depth reading and develop structured reading methods. However, students still need to be introduced to a wider variety of reading materials. It should be acknowledged that, compared with China (B-S-J-Z), Singapore and Hong Kong, Macao still lags behind in terms of the above indicators, and schools in Macao still need to continue their efforts in the future. When cultivating the students' interest in reading, schools need to take into account not only their reading preferences but also their reading abilities to extend the topics, complexity in order to consolidate acquisition of reading strategies. 119
6.4 Outlook Many countries are attempting to identify the solution and momentum for education reform based on what children living in an ever-changing world really need to learn and how they should learn to embrace the uncertain future. The concept of "literacy" that integrates knowledge, skills, attitudes and values was put forward in the OECD Learning Framework 2030, which can provide reference for the education reform of schools, and is also applicable to everyone's lifelong learning. The purpose of sustainable development proposed by the United Nations aims to satisfy the development needs of the current generation without damaging the ability of future generations to meet their own needs. To realize the sustainable development, all countries in the world should coordinate the three core elements: economic growth, social inclusion and environmental protection. These elements are interrelated and crucial to the well-being of all individuals and the society. In particular, the acquisition of high-quality education is the basis for improving the people's lives and achieving sustainable development (United Nations, 2016). China's Education Modernization 203 5 and the Implementation Plan for Accelerating Education Modernization (2018-2022) issued by the State Council provided a blueprint for the country's education development in the future, emphasizing that China must realize the high-quality and balanced compulsory education, and establish a modern education system of lifelong learning to serve all citizens (Central People's Government of the People's Republic of China, 2019a & 2019b ). Education modernization is the overall improvement of level of popularization, quality, equity and structure. Since its return to China, Macao has made remarkable progress in its educational business under the support of the state. In the planning for non-higher education in the next ten years, we still need to continuously adopt appropriate measures to complete the unfinished missions. According to the results of PISA, like all other countries and regions in the world, schools in Macao still need to improve in many areas. For example, although the cultivation of students' cognitive ability is still very important, it is no longer the only value in the pursuit of educational development. In addition to further increasing the proportion of high-level students and reducing the proportion of low-level students, schools should also pay attention to students' mental health, help students develop their curiosity, imagination, adaptive capacity and self-regulating ability, guide them to respect and appreciate the ideas, opinions and values of others, teach them to cope with failures and rejections, lead them to march forward in adverse situations, and ultimately cultivate them to become lifelong learners who "value well-being, have foresight, practice reflection, and put words into actions". The Macao-PISA Centre has always adhered to the original intention of participating in the PISA research project, collected data from multiple levels and multiple perspectives for educators, education appraisers, education policymakers and other stakeholders in the previous assessments, and provided the public with many suggestions based on research evidence for them to ponder upon. As the analysis of PISA data becomes more thorough, it can be expected that the improvement and development of education will also be clearer and more operable. 120
From the perspective of international and comparative education, the Educational Testing and Assessment Research Centre (ETARC) of the University of Macau has collated and reported the test results of Macao-PISA 2018, hoping to assist education researchers, education administrations and education institutions to understand the progress of Macao 's basic education, present the objective statistics to reflect the quality and equity of education, and guide educators to further reflect on and clarify the educational problems identified in the PISA test results, adjust and formulate education policies, actively improve teaching quality, closely integrate learning and assessment, thoroughly implement the concepts of "assessment for learning" and "assessment as learning", and make joint efforts to cultivate talents with global competitiveness in Macao. In his poem The Book, Zhu Xi, the famous Neo-confucianist of the Southern Song Dynasty wrote: "There lies a glassy oblong pool, where light and shade pursue their course. How can it be so clear and cool? For water fresh comes from its source" (translation by Xu Yuanchong). A mind can stay as clear as Zhu Xi 's with the nutrients provided by the continuous flow of knowledge. Where are the nutrients for the development of Macao's educational enterprise? We should "take the PISA data as reference, guide teaching based on research, and determine the education policies and reform progress according to the results of international assessments". It is hoped the Macao-PISA Centre and all educators will continue to make joint efforts in this regard. 121
Appendix Macao-China PISA Studies PISA Government Board (PGB) Member: Mr. Lou Pak Sang, Director of Education and Youth Affairs Bureau (DSEJ), Macao SAR Government Macao-PISA Project Manager (NPM): Prof. Kwok-cheung Cheung, Educational Testing and Assessment Research Center (ETARC), Faculty of Education, University of Macau PISA 2018 experts and scholars from University of Macau Honorary Advisors: Chair Professor Song Yonghua; Chair Professor Lionel Ni Ming-Shuan; Distinguished Professor Chuang Wang Research Team Members: Professor CHEUNG Kwok-cheung; Associate Professor SIT Pou-seong; Associate Professor CHENG Chun-wai; Associate Professor WEI Bing; Associate Professor Matilda Ming Wai WONG; Associate Professor ZHOU Yisu; Associate Professor KAM Chun Seng; Assistant Professor JIANG Chunlian; Assistant Professor SHIEH Jin-jy; Assistant Professor LAM Kin Wing; Assistant Professor SUNG Min-chuan; Assistant Professor Emily Pey Tee OON; Assistant Professor WANG Yan; Ms. GAO Lili; Dr. MAK Soi-kei; Dr. IEONG Man-kai; Ms. FONG In-fan; Mr. HO Sek-pio; Mr. Lam Chi-chio Acknowledgements: Successful completion of the Macao-PISA 2018 Study was contingent on: • Financial support and leadership by DSEJ of the Macao SAR Government; • Guidance and resource support by University of Macau authority, especially Research Services and Knowledge Transfer Office (RSKTO); • Academic and technical support by ETARC, Faculty of Education, University of Macau; • Cooperation of secondary schools and their coordinators participating in the PISA 2018 Study; • Active participation of teachers, students and their parents in responding to PISA 2018 tests and questionnaires; • Cooperation of University of Macau and Colegio de Santa Rosa de Lima (English Secondary) providing facilities and serving as testing venue. 122
References Baker, L. (2008). Metacognition in comprehension instruction. In C. C. Block & S. R. Parris (Eds.), Comprehension instruction: Research-based best practices (pp. 65-69). New York: Guil ford Press. Bandura, A. (1997). Self-efficacy: The exercise of control. New York: Freeman. Borkowski, J. G., Weyhing, R. S., & Carr, M. (1988). Effects of attributional retraining on strategy-based reading comprehension in learning-disabled students. Journal of Educational Psychology, 80(1), 46-53 . Buchmann, C., & Hannum, E. (2001). Education and stratification in developing countries: A review of theories and research. Annual Review of Sociology, 27, 77-102. Chan, L. K. S. (1996). Combined strategy and attributional training for seventh grade average and poor readers. Journal of Research in reading, 19 (2), 111-127. Chen, H. H., & Chen, C. C. (2018). [ ~~¥5L,, ~amt (2018) o 1i~i:p1J\&ffe!:rjfj!f:l~tHffl~ffM!A tt~Ilili'.~~ 0 il-~~lVJWf~ , 13 (I) , 1-50 ° J Cheung, K. C. (2015) . Factors affecting mathematical li teracy performance of 15-year-old students in Macao: The PISA perspective. In: B. Sriraman, J. Cai, K. H . Lee, L. Fan, Y. Shimuzu, C. S. Lim, & K. Subramaniam, (Eds.) The first sourcebook on Asian research in mathematics education: China, Korea, Singapore, Japan, Malaysia and India (pp. 91-110). Charlotte, NC: Information Age Publishing. Cheung, K . C. (2017) [ ~lit$ (2017) 0 PISA2015 B
Cheung, K . C., Mak, S. K. , Sit, P. S ., & Jeong, M. K. (2018). Why Macao is commended as an economy of high-performance and high-equity amongst PISA 2015 participating economies? An explanation from an insider perspective. In: C. Magno & A. David (Eds.). Philippine and global perspectives on educational assessment (pp. 49-71 ), Philippines: Philippine Educational Measurement and Evaluation Association. Cheung, K. C., Mak, S. K., Sit, P. S., & Sob, K. C. (2016). A typology of student reading engagement: Preparing for response to intervention in the school curricu lum. Studies in Educational Evaluation, 48, 32-42. Cheung, K. C., Sit, P. S., & Mak, S. K. (2011). Parental engagement and home support for the betterment of adolescents' reading li teracy in Macao. Curriculum and Instruction Quarterly, 14(4), 93-116. Cheung, W. M. , Tse, S . K. , Lam, J. W. I. , & Loh, E. K. Y. (2009). Progress in International Reading Literacy Study 2006 (PIRLS): Pedagogical correlates of fourth-grade students in Hong Kong. Journal of Research in Reading, 32(3), 293-308. Clark, C. H. (1996). Keys to successful cognitive and affective intervention with challenging students. Reading and Writing Quarterly: Overcoming Learning Difficulties, 12(3), 265-290. C lark, C., & Douglas, J. (2011). Young people's reading and writing: An in-depth study focusing on enjoyment, behaviour, attitudes and attainment. London: National Literacy Trust. Coiro, J ., & Dobler, E. (2007). Exploring the online reading comprehension strategies used by sixth-grade skilled readers to search for and locate information on the Internet. Reading Research Quarterly, 42(2), 214-257. Cunningham, A., & Stanovich, K. (1997). Early reading acquisition and its relation to reading experience and ability 10 years later. Developmental Psychology, 33(6), 934-945. Darling, S. (2011). Strategies for engaging parents in home-support of reading acquisition. The Reading Teacher, 58(5), 476-479. Dole, J. A., Brown, K. J. , & Trathen, W. (1996). The effects of strategy instruction on the comprehension performance of at-risk students. Reading Research Quarterly, 31(1), 62-88. 124
Dole, J. A., Duffy, G. G., Roehler, L. R., & Pearson, P. D. ( 1991). Moving from the old to the new: Research on reading comprehension instruction. Review of Educational Research, 61(2), 239-264 Education and Youth Affairs Bureau, Macao SAR Government (2019) [ ~r~¥-J~1Jffif:)l:i&~1f ~ wfJ::mi (2019) 0 ~l=}l~~1U1ftt~-.ffi~ 2019 ° ey El https://portal.dsej.gov.mo/webdsejspace/ internet/Inter _main _page.jsp?id=85 l 4] Finn, J. D. (1998). Parental engagement that makes a difference. Educational Leadership, 55(8), 20-24. Fuller, B. (l 987). What school factors raise achievement in the third world? Review of Educational Research, 57(3), 255-292. Gamoran, A. , Secada, W., & Marrett, C. (2000). The organizational context of teaching and learning: Changing theoretical perspectives . In M. T. Hallinan (Ed.), Handbook of the sociology of education (pp. 37-63). New York: Kluwer Academic/ Plenum Press. Garner, R. (1987). Metacognition and reading comprehension. Norwood, NJ: Ablex Publishing Co. Greenwald, R. , Hedges, L. , & Laine, R. (1996). The effect of school resources on student achievement. Review of Educational Research, 66(3), 361-396. Guthrie, J. T. ( 1996). Educational contexts for engagement in literacy. The Reading Teache1·; 49(6) , 432-446. Guthrie, J. T. , & Alao, S. (1997). Designing contexts to increase motivations for reading. Educational Psychologist, 32(2), 95-105. Guthrie, J. T., & Soloman, A. ( 1997). Engagement in reading for young adolescents. Journal of Adolescent & Adult Literacy, 40(6), 438-447. Guthrie, J. T., Meter, P. V., McCann, A. D., Wigfield, A ., Bennett, L., Poundstone, C. C., Rice, M.E. , Faibisch, F.M., Hunt, B. , & Mitchell , A. M. (1996). Growth of literacy engagement: Changes in motivations and strategies during concept-oriented reading instruction. Reading Research Quarterly, 31(3) , 306-332. 125
Guthrie, J. T., Wigfield, A., & Vonsecker, C. (2000). Effects of integrated instruction on motivation and strategy use in reading. Journal of Educational Psychology, 92(2), 331-341. Hanushek, E. A., & Woessmann, L. (2012). Do better schools lead to more growth? Cognitive skills, economic outcomes, and causation. Journal of Economic Growth, 17(4), 267-321. Ho, E. S. C. (20 10). Family influences on science learning among Hong Kong adolescents: What we learned from PISA. International Journal of Science and Mathematics Education, 8(3), 409-428. Ho, S. C., & Willms, J. D. (1996). The effect of parental involvement on the achievement of eighth grade students. Sociology of Education, 69(2), 126-141. Hsiao, F. Y. (2011) [ ifj'jc; (2011) 0 H~~'tf~,fh,~:i~H~Jt O :;l(r~&t O I{;;(.§ https://www. cw.com.tw/article/article.action?id=5011898] Ieong, M. K., Sit, P. S., Mak, S. K., & Cheung, K. C. (20 16) [ :r~Jt,f! ' i§f j!yt/il ' ~fi'/tl:FJt ' ~~l~lfl (2016) 0 ~:@l. Log ~tf:J,1Hfr~~r"'~~flilt,1;lcff~ : ~PISA20 12 1¥J01ffl~~ffLl O m1J~~ flj '63 (3)' 153-178 °] International Teleco1mnunications Union (2014). Key 2005-2014 JCT data for the world, by geographic regions and by level of development. Retrieved from http ://www. itu.int/en/ITUD/ Statistics/Pages/publications/mis2014.aspx. Jin, S. L. (2018). Factors discriminating between problem-solving experts and novices: Analysis of PISA 2012 student digital problem-solving responses from the big data perspectives (Unpublished doctoral dissertation). Macao: University of Macau. Jin, S. L., Cheung, K. C., Sit, P. S. (2016) [ ~tIJffl, ~wil#, fil~friil (2016) 0 liJffl logJtti ~:j;fj- PISA 2012 ~~fOltI~HJtrtlHEI ~ffltf.?~•5¥•{,,~129:J: 0 j~~~flj , 63 (4) , 227-252 ° ] Kintsch, W. (1998). Comprehension: A Paradigm for Cognition. Cambridge, MA: Cambridge University Press. Ko, H. W. (2006) [ :l'riJ¥~ (2006) 0 ~1±\lfflffi}J O F-:i'it: :;l(ri!Ut O ] 126
Lau, K. L. (2004) [ ~LlmI:E0- (2004) 0 i::l:tJt™Jfil"I~~B'-:J1frnht-l}[J~rU~m~~&t~filUiEr-J~~~~!~,®. • ~~~ff ~ffl ' 13 (2) ' 1-16 °] Lau, K. L. (2010) [ ~Ll~J0- (2010) 0 tsuttlft-it filLlr:p~!\-Jt~Bi1i:i'£1ffl~fi~~Er-JWJlffi.l?-f1Jfj~;J-j5:ts<:JB51: ~: 0 ~f[,wf ,r:~ffl ' 25 (2) ' 287-323 ° ] Lau, K. L. (2011) [ IU~~ (2011) 0 {f£iH~~:i:tf1iiiHIU (PISA) EfJ*t*&,'lJ,wm~!tJt~ti, ~*~~:±Er-J lffl~fi~~ 0 ~~~~~*flj ' 14 (4) ' 49-72 ° ] Lau, K. L., & Chan, D. W. (2003). Reading strategy use and motivation among Chinese good and poor readers in Hong Kong. Journal of Research in Reading, 26(2), 177-J 90. Mak, s. K. (2013) [ ~Jlm:ijt (2013) 0 ~r~+.li~~:±:lfflITTl~•Er-Jti)j1J~~fiUI : lfflffi~~B
OECD (2009). PISA 2009 assessment framework: Key competencies in reading, mathematics and science. OECD Publishing. OECD (2012). PISA 2012 technical report. Paris: OECD Publishing. OECD (20 13). OECD Skills Outlook 2013: First Results from the Survey of Adult Skills. Paris: OECD Publishing. OECD (2016a). PISA 2015 Assessment and analytical framework: Science, reading, mathematics and.financial literacy. Paris: OECD Publishing. OECD (2016b ). PISA 2015 results (Volume I): Excellence and equity in education. Paris: OECD Publishing. OECD (2018). The future of education and skills: Education 2030. Paris: OECD Publishing. OECD (20 19a). PISA 2018 assessment and analyticalframework. Paris: OECD Publishing. OECD (20 19b). PISA 2018 report (volume I). Paris: OECD Publishing. OECD (20 19c). PISA 2018 report (volume 11). Paris: OECD Publishing. Pang, E. S., Muaka, A., Bernhardt, E., & Kamil, M. L. (2003). Teaching reading. Brussels: International Academy of Education. Paris, S. G., & Paris, A. H. (2001). Classroom applications of research on self-regulated learning. Educational Psychologist, 36(2), 89-101. Paris, S. G., Lipson, M. Y., & Wixson, K. K. (1983). Becoming a strategic reader. Contemporary Educational Psychology, 8(3), 293-316. Pearson, P. D., & Fielding, L. (1991). Comprehension instruction. In R. Barr, M. L., Kamil, P. B. Mosenthal, & P. D. Pearson (Eds.), Handbook of reading research, Vol. 2 (pp. 815-859). New York: Longman. 128
Perfetti, C. A. ( 1985). Reading ability. New York: Oxford University Press. Pressley, M. , EI-D inary, P. B. , Wharton-McDonald, R., & Brown, R. (1998). Transactional instruction of comprehension strategies in the e lementary grades. In D. M. Schunk, & R. J. Zimmerman (Eds.), Selj:regulated learning: From teaching to self-reflective practice (pp. 42-55). New York: The Guilford Press. Revkin, S., Hanushek, E., & Kain, J. (2005). Teachers, schools and academic achievement. Econometrica, 73(2). 417-458. Rosenshine, B., & Meister, C. (1997). Cognitive strategy instruction in reading. In S. A. Stahl & D. A. Hayes (Eds.), Instructional models in reading (pp. 85-106). Mahwah, NJ: Lawrence Erlbaum. Ross, A. (2016). Jndustries ofthefiilure. NY: Simon & Schuster. Ruddell, M. R. (2008). Teaching concept reading and writing (5th ed.). Hoboken, NJ: John Wiley & Sons. Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55, 68-78. Schuder, T. (1993). The genesis of transactional strategies instruction in a reading program for at-risk students. The Elementary School Journal, 94(2), 183-200. Schulze, S., & Lemmer, E. (2017) . Family experiences, the motivation for science learning and science achievement of different learner groups. South African Journal of Education, 3 7( 1 ), 1-9. Shell, D. F., Covin, C., & Bruning, R. H. (1995). Self-efficacy, attribution, and outcome expectancy mechanisms in reading and writing achievement: Grade-level and achievement-level differences. Journal of Educational Psychology, 87(3), 386-398. Sit, P. S., Cheung, K. C., Cheong, W. C., & Mak, S. K. (2014). The effects ofwebpage navigation on digital reading performance. The International Journal of Assessment and Evaluation, 20(3), 22-36. 129
Sit, P. S., Cheung, K. C., Sob, K. C., Mak, S. K., Cheong, W. C., & Ieong, M.K. (2015). What studying problems are faced by the adolescent grade repeaters in Macao: Uncovering underlying mechanisms based on evidences from the PISA 2012 Study. Asia Pacific Education Review, 16(3), 367-377. Smith, M. C., Mikulecky, L., Kibby, M. W., Dreher, M. J. , & Dole, J. A. (2000). What will be the demands of literacy in the workplace in the next millennium? Reading Research Quarterly, 35(3), 378-383. Snow, C. E., & Sweet, A. P. (2003). Reading for comprehension. In A. P. Sweet, & C. E. Snow (Eds.), Rethinking reading comprehension (pp.1-11). New York: The Guilford Press. Stahl, S. A. (1997). Instructional models in reading: An introduction. In S. A. Stahl, & D. A. Hayes (Eds.), Instructional models in reading (pp. 1-30). Mahwah, NJ: Lawrence Erlbaum. The Government of People's Republic of China (2019a) [ i:r¥Al~~J=!J11J~i:p;k;A~il&Wf (2019a) 0 i:p ~~ffffl~,ft 2035 ° ]f;Z 13 http://www.gov.cn/zhengce/20l9-02/23/content_53 67987.htm] The Government of People's Republic of China (2019b) [ i:r¥A~~flJ~i:p;k;A~il&Wf (2019b) 0 )]1]•~1l:ii~Wm~1t•:tiffi1J~ ( 2018-2022 ~) 0 IIR § http://www.gov.cn/xinwen/2019-02/23/ content_ 5367914.htm] Tse, S. K., Lam, R. Y. J. , Lam, J. W. I., Chan, Y. M., & Loh, E. K. Y. (2006). Attitudes and attainment: A comparison of Hong Kong, Singaporean and English students' reading. Research in Education, 76, 74-87. United Nations (2016) [ ~~v-g-~ (2016) 0 2030~~ff : 1=]11~§"-lllffilfJtl!~ 0 I!R § https:// unesdoc. unesco.org/ark:/48223/pf000024565 6 _ chi] Veenman, M. V., Van Hout-Wolters, B. H., & Affierbach, P. (2006). Metacognition and learning: Conceptual and methodological considerations. Metacognition and Learning, 1, 3-14. Wang, J. H., & Guthrie, J. T. (2004). Modeling the effects of intrinsic motivation, extrinsic motivation, amount ofreading, and past reading achievement on text comprehension between U.S. and Chinese students. Reading Research Quarterly, 39, 162-186. 130
Wigfield, A., & Guthrie, J. T. (1997). Relations of chi ldren's motivation for reading to the amount and breadth of their reading. Journal of Education Psychology, 89(3), 420-432. Yamamoto, K., Shin, H. J., & Khorramdel, L. (20 I 8). Multi-stage adaptive test designs in international large-scale assessments. Educational Measurement: Issues and Practice, 3 7( 4), 16-27. 131
