Exploring Opportunities And Challenges For PISA In Bhutanese Context: Essay

Importance of PISA-D

Discuss about the PISA Examination Based On Bhutanese Educational Context.

There is increased focus towards international education system to help solve problems facing the current century and future generations. The global convergence of education has been widely as reported by Wiseman (2010) as a way of increasing efficiency in measuring students’ competencies as well as saving cost. Barro & Lee (2013) describes various changes in education system of developed and developing countries necessitating teachers and schools to offer learning that meets national and global testing system. PISA is increasingly becoming one the best international testing system in education (Jerrim, 2011). This study focusing on PISA on Bhutanese context will help inform various possible school reforms to help meet international education standards to enhance its competitiveness.

Since year 2000 over 70 countries have participated in PISA-D examination. The study was conducted by OECD (Organization for Economic Co-operation and Development). Bhutan conducted its first preliminary examination for PISA-D on 30 March, 2017.In this paper, I present a preliminary literature review on PISA- D and its benefits on students’ academic performance and national policy. Following the review, I present the research questions I intend to include in the study. This preliminary literature review explores the concept of PISA-D and its benefits by drawing on international literature. Since, it has not been long after Bhutan participated in PISA, no literature could be traced, which indicates the existence of knowledge gap.

Many researches have reported significant impact of decentralization of testing and assessments on education system globally (Kamens & McNeely, 2009). Wiseman (2010) argued that convergence in education system worldwide can be achieved through consensus of various countries on goals steering, managerialism, choice, privatization, decentralization, accountability, evaluation and competition. The PISA meet the such characteristics and is therefore used as baseline for attaining global education standards. OECD reports significant increase in acceptance of international students in higher institution of learning from countries participating in PISA (Grek, 2009). The Bhutan can therefore carry out reforms to enhance participation in PISA and improve its education system to address socioeconomic challenges and contribute to economic development.

The increase in proficiency of reading, mathematics and other sciences in European countries is attributed to reforms attributed to PISA (Kuenzi, 2008). The arguments in favor of PISA in Bhutan is attributed to improved educational quality and increased efficiency in school resulting in positive student outcomes as reported by Alacac? and Erba?, 2010). Publishing the PISA test results will put pressure on Bhutanese schools to improve measured outcomes. Sweden and Croatia reported improvement in schools’ results, efficiency and education quality as a result of participating in PISA (Adams et al., 2010). Furthermore, the parents will be able to make informed decision based on PISA relating to choice of schools for their children based on competitive results.

Decentralization and Convergence in Education System

Since PISA started in 2017 in Bhutan, there is shortage of information regarding challenges and opportunities of such international test on Bhutanese context. Study by Winther and Achtenhagen (2009) reported huge flows on international literacy assessments. Contrarily, Breakspear (2012) reported beneficial effects of PISA on educational policy. This conflicting ideas offers interesting area to observe in this study. The globalizing connects between local and global forces due to educational context call for explorations on how the PISA can change the Bhutanese education system to be better. There is little information regarding rationale for Bhutan to participate in global assessment programs. To understand the extent to which PISA is mediated by economic, historical, political and cultural dynamics, this study will be conducted to offer appropriate inquiry and contribute to enhance understanding of PISA and perception of literacy on Bhutanese context.

To access the opportunities and challenges for PISA in Bhutanese context.

1. What are the Bhutanese teachers and students’ perceptions about the recently conducted PISA-D Examination in Bhutan?
2. What are the challenges that BCSEA faced in conducting PISA-D examination for the first time in Bhutan?
3. What are the benefits of Bhutan’s participation in PISA-D examination?

The PISA contributes to consequential accountability that act as merit for increasing remunerations for the teachers to improve on the student’s performance. Furthermore, the increased competition brought about by the PISA will put more pressure on the schools and teachers to perform well thus lifting the student’s achievement. This is because many countries who have introduced PISA in their educational models have had positive effects on the students’ performances. Additionally, the arguments and findings of this study will help inform the development of schooling policies that establish incentive system to improve on the student learning outcomes. Conclusively, better student performance has ripple effect on country’s future economic and social performance. The criticism for current Bhutanese education system is that it is placing more focus on test results thus narrowing scope of teaching to areas likely to be tested. However, PISA encourages standardization of education system that stretches curriculum to meet broader international education’s goals.

Organization for Economic Cooperation and Development’s (OECD) Programme for International Student Assessment is a global assessment tool conducted in member and non-member nations. It measures the academic performance of 15 years old students in subjects such as mathematics, science and reading (OECD, 2014). The test measures how students are able to apply their knowledge in both inside and outside classroom. The first test was conducted in 2000 with few countries. Since then it has become a popular yardstick to measure the quality of education and the number of member countries increased (Breakspear, 2014).

The data that is generated provides possibility for comparing the standard of education with other countries and improve the education policies and outcomes (Breakspear, 2014). This test also provides information about the areas of improvement after comparing the achievements of high-performing countries. In many countries, it has served as basis to conduct research on best practices and initiate learning processes from similar countries. For example, Hallstrom (2015) conducted a study in Sweden based on the mathematical scores in PISA-D and was able to recommend the importance of increasing teacher’s wages to attract high quality teachers which will have a bearing on students learning and performance in the test. Additionally, PISA-D collects extensive information about the characteristics of young people, schools, and education systems enabling the formulations of guidelines on policy actions (OECD, 2014).

Impact of PISA-D on European Countries

The areas of learning that are assessed in science subject are students’ ability to engage with science and the ideas of science. Specifically, evaluate and design scientific research, and interpret data and evidence in a scientific manner included in the test (OECD, 2014).  In the reading component students’ ability to understand, use, reflect and relate to written texts, develop their knowledge and potential to actively participate in the society are included in the test. In the mathematics, subject students’ ability to formulate, use and interpret mathematics in variety of contexts is included. It also includes mathematical reasoning and the use of mathematical concepts, procedures, facts, and tools to describe, explain and predict phenomenon (Anderson, Lin, Treagust, Ross &Yore, 2007). Bhutan joined PISA-D in 2017. The organization that is responsible for the conduct of PISA-D examination is Bhutan Council for School Examination and Assessment (BCSEA). Under the coordination of BCSEA, students did the first PISA-D examination in the same year.

Several studies show that PISA-D has beneficial effect on students’ academic motivation, academic achievement and self-beliefs. Ross (2008) explored the relationship between achievement motivation and academic motivation in two distinct cultures: Western (Canada, the Unites States and the United Kingdom) and Asian (Hong Kong-China, Japan, and Korea). Intrinsic motivation predicted an increase in scores for all of the Asian country models, but results were inconsistent for the Western country models. Self-efficacy predicated increased scores for all models for all countries. The positive relation between the student’s motivation and academic achievement is attributed to the better performance in the PISA-D examination. According to OECD 2013 indicated that as per the 2012 PISA results, countries/economies with a mean performance/ share of top performers above the OECD average were from the eastern part of the world.

In the study by Milford (2009) where the relationships between science self-beliefs and academic achievement in science across all nations who participated in PISA 2006 was investigated, reveled that students with both higher science self-concept and higher science-efficacy tended to achieve higher academically. Studies also show that data from PISA influence policy decision and reform. Breakspear’s (2014) study on the impact of PISA on national policies on 37 countries showed that PISA has become a reliable instrument for benchmarking student performance worldwide, and influencing the policy reform in the majority of participating countries/economics. Although there are studies demonstrating the benefits of PISA in many member countries, there are no studies conducted in Bhutan exploring the benefits. Since, Bhutan recently became a member country and participated in the first test in 2017, a study determining the benefits, challenges and perceptions of its participation in PISA need to be conducted.

Challenges and Opportunities of PISA-D in Bhutanese Context

The schools have long been focusing on academic skills, however the rapid changes in technology is causing changes to education system to allow students gain social and emotional skill to thrive in current century. The psychologists are emphasizing on the need to equip the students with social skills to enable them be connected to increasing workforce diversity, be resilient in face modern social challenges and diversity and also have higher hope and aspirations for their future (Kivunja, 2014). Some of the qualities targeted by modern education system include cognitive, physical, psychological and social qualities that enable students live fulfilling live as they grow into adulthood. Bybee and McCrae (2011) reports that PISA is incorporating test on student’s wellbeing in the mentioned areas as they build relationship with teachers, parents, siblings, peers and people they meet outside their homes or schools.

The youths are facing huge social problems including terrorism, crime and drug abuse. The adolescent students not satisfied with their life are likely to develop irresponsible behavior that put stress and financial strain on parents, community and country as a whole (Kivunja, 2014). The PISA results can therefore be used to test level of satisfaction to inform different strategies to be developed in countries to help motivate the youth increase their satisfaction with life. This may involve fostering student relations with parents and teachers to foster support psychologically and academically to improve overall wellbeing of the students.

The education is a weapon for bettering the lives of individuals as it is reported to be an equalizer for the rich and poor. The existing literature demonstrate that most education system around the globe is putting more focus on the skill and knowledge though there has been paradigm shift to incorporate wellbeing. Many researches have indicated that the youths can thrive well beyond professional and academic success thus necessitating schools to teach skills for better relationship, more engagement, finding more meaning in life and positive emotions. Kidd (2008) argued that there is great desire to look for opportunities of improving safety, moral development, health and education progress of the youths. Durlak et al. (2011) suggested that the schools are conducive for these opportunities and thus should expand their focus to include promotion of wellbeing and character beyond the academic learning.

The existing literature present narrow attention on youth wellbeing. The adolescent is critical developmental stage of the youth that present major challenges and opportunities on equal measures. However, the education is putting more focus on the academic performance while undermining personal character. The existing models offers multidimensional constructs of attaining individual’s wellbeing. Some of the domains that can be used to define youth’s wellbeing according Seligman’s wellbeing theory (Seligman, 2012) include flow, purpose, positive relationship, positive emotion and achievement. Various indices have been developed for defining quality of life and include income, community, environment, life satisfaction, work-life balance, housing, jobs, education, governance, safety and health. Recent studies have indicated need for measuring youth wellbeing to help identify individual’s strength and weaknesses. PISA offers great opportunity for incorporating metrics of assessing wellbeing to provide information to the schools and councilors on areas that need improvement.

Areas of Learning Assessed in PISA-D

The issues of education policy have been major debates of many countries across the world. Generated heat debates surrounding education policy is attributed to increased globalization that is pushing more countries to improve education sector as one of the pillars of achieving economic developments. Programme for International Student Assessment (PISA) is arguably one of the best measure for obtaining evidence-based results for the policy advice (Auld & Morris, 2016). Since Bhutan recently participated in the PISA, it is interesting to know how the Bhutanese students preform compared to those of participating countries. According to Gorur (2014), there is increased interest among researches on PISA relating to scoring and ranking of countries necessary for policy recommendations. PISA is contributing significantly to international policies that are recommended by various international organization such as world bank and OECD to help attain quality education system across the world (Gorur, 2014).

The international education policies are increasingly being adopted by various countries around the globe to increase competitiveness. Some of the school policies proposed by Lingard (2010) include standardization of education, increased global focus on literacy and accountability. PISA is one of the contributing factor for the standardization of education in that it allows external examination of students from different countries. Additionally, PISA is reported to increased setting of performance and testing standards of participating countries thus contributing to significant educational reforms that increase learning (Coe, 2009). The global education reforms are based on PISA due to its ability to test basic knowledge and skills regardless of economic, social or political status of participating countries (Radaelli, 2009). The PISA can therefore be used to inform school policies regarding inspecting, punishing, promoting or rewarding teachers and schools. Consequently, globalization of education policy result in increased efficiency of school system across the world.

There is growing concern on the use of PISA data by various researches. Given that PISA is growing influence internationally, Hanushek (2014) raises concerns that analyzing PISA performance and comparing country wide may not be of great help if results and recommendations are not used by government and various educational organization to formulate policies. One major criticism of PISA is the issue of cultural difference among the participating countries (Fischbach et al., 2012). Duru-Bellat (2011) survey indicates that students from different cultural backgrounds tend to react uniquely to common questions. This create the need for the PISA designers to design strategies to iron out cultural differences and put into consideration such cultural differences when interpreting the PISA outcomes. Secondly, the test is conducted in different languages necessitating translation or at times the student may do exam in different language other than their native one. there is therefore need to determine effect of language used in test on the student’s performance.

Benefits of PISA-D on Students’ Academic Performance and Self-Beliefs

Takayama (2008) criticized PISA for its sampling survey for the students to participate in test. The author argued that some schools may prefer choosing individuals with high learning abilities to participate in test leaving those with learning difficulties. The use of cross-sectional survey design in PISA is also under scrutiny for its limitation to infer correlation among different measures (Fuchs & Wößmann, 2008). The information can therefore not be used to infer causality as it is wrongly used by media and government to imply causality. There is also limitation of PISA disregarding national curricula due to its emphasis on providing answers using common sense rather than knowledge of particular national curriculum. Additionally, the PISA has narrow focus on the science, reading and mathematics thus underestimating other subjects such as art, geography, foreign language, history, music and civics.

This study will adopt the descriptive survey resign study. This design is chosen because Kuria et al. (2012) suggested that the data collected from the technique provide answers to questionings relating status of the study phenomena. Berry et al. (2012) further argued that descriptive survey is best when one wants to know more about situations, condition or people. These arguments formed basis for choice of this design. In this research, both qualitative and quantitative methods will be used whereby the qualitative constructs focus on understanding respondents’ perceptions while quantitative focuses on prediction. The questionnaires will be used to collect primary data from the chosen participants

There are 20 political divisions of Bhutan organized into districts. Bumthang district is chosen for the study has relatively higher number of schools. The district is important trade center and transportation hub for Bhutan. The school from this district will be the source of participants.

The population refers to the target individuals who will participate in the study. The target population of current study will comprise the students who have completed PISA previously. Furthermore, principals and teachers from PISA participating schools dealing with subjects being examined that is science, reading and math will be incorporated into the study. The total population will be 300 comprising of 200 youth students and 100 adult teachers.

The stratified random sampling techniques will be adopted to recruit about 800 individuals to ensure that all strata are represented in the eventual sample. This will be followed by random sampling to select participants from each strata and reduce population size to 300 people.

Table 1: Sampling frame

Initial sample (by stratified sampling)		Final study population (random sampling)
Strata	Number	Strata	Number
Principals	100	Principals	20
Subject teachers	300	Subject teachers	100
Students	500	Students 500	200

Influence of PISA-D on Policy Reform

The structured questionnaire will be used for data collection. Such questionnaires will be distributed to participants at their respective schools. The questionnaires are chosen since they are reported to provide high degree of standardization. The questionnaire will be designed then revised by the supervisor. There will be two parts in questionnaire, first part will comprise of close-ended questions to collect factual information like demographic characteristics. The second part aims at seeking respondent’s perception regarding the study variables. Therefore, the statements for variable of study in part 2 will be ranked based on 5-point Likert scale (1=strongly disagree, 2=disagree 3=neutral, 4=agree and 5= strongly agree).

The pilot study will be conducted in one school that will not participate in final study. The pilot study will help in determining the reliability and validity of the questionnaires thus assisting in improving items in the research instrument.

The letter permitting research will be obtained from the university. The questionnaires will be administered to the participants using drop and pick method. A period of 2 weeks will be given to respondents to fill questionnaires. Research assistants will be trained to help assist in dropping and picking the questionnaires.

The raw data in questionnaires will be coded into computer software called Statistical Package for Social Sciences (v.24) and cleaned for any inconsistencies. The reliability will then be determined using Cronbach’s Alpha method and data subjected to correlation and regression analysis to determine the relationship between the study variables. The inferential and descriptive statistics will then be used to explain the findings.

The researcher will discuss with research participants the nature of research and provide full information. The informed consent will then be sought before the study and no one will be forced to participate in the study. Furthermore, the research participants can withdraw anytime during the study. Lastly, personal information or the identity of the participants will not be revealed to the general public.

	May  2018	June 2018	July 2018	August 2018	September 2018	October 2018	November 2018
Proposal writing/Literature review
Proposal submission and defense
Pilot study
Data collection and analysis
Report Writing
Report presentation
Project  submission

Item	Resource / Items	Quantity	Cost per Item (AUD)	Total Cost (AUD)
Stationery	-Spring files	5	2	10
	-Flash discs	1	10	10
	-Box file	4	4	16
	-Ream of photocopying paper	3	20	60
	-Pens	8	0.5	4
	-Pencils	5	0.5	2.5
Proposal writing	-Internet service/charges	50 per month	6 months	300
	-Binding (spiral for drafts/final)	6 copies	2	12
Piloting	-Photocopying questionnaires	140 pages	0.2	28
	-Travel expenses & subsistence	4 trips	200	800
Data collection	-Photocopying questionnaire -	300 copies	2	600
	Travel expenses & subsistence	5 days	200	1000
	-Per diem for 2 research assistants	10 days	200	2000
Report writing	-Photocopying of drafts	3 copies	10	30
	-Binding (spiral drafts)	3 copies	1	3
	-Binding (final report)	3 copies	1	3
	-Publication fee	2 publications	500	1000
Sub-total				5878.5
Contingencies 10% of the total				587.85
Grand total				6466.35

 

References

Adams, R., Berezner, A., & Jakubowski, M. (2010). Analysis of PISA 2006 preferred items ranking using the percent-correct method.

Alacac?, C., & Erba?, A. K. (2010). Unpacking the inequality among Turkish schools: Findings from PISA 2006. International Journal of Educational Development, 30(2), 182-192.

Anderson, J. O., Lin, H.-S., Treagust, D. F., Ross, S. P., &Yore,L. D. (2007). Using large-scale assessment datasets for research in science and mathematics education: Programme for International Student Assessment (PISA). International Journal of Science and mathematics Education, 5(4), 591-614

The Need to Equip Students with Social Skills for Future Success

Auld, E., & Morris, P. (2016). PISA, policy and persuasion: Translating complex conditions into education ‘best practice’. Comparative Education, 52(2), 202-229.

Barro, R. J., & Lee, J. W. (2013). A new data set of educational attainment in the world, 1950–2010. Journal of development economics, 104, 184-198.

Berry, P. A., Gillespie, G. L., Gates, D., & Schafer, J. (2012). Novice nurse productivity following workplace bullying. Journal of Nursing Scholarship, 44(1), 80-87.

Breakspear, S. (2012). The policy impact of PISA: An exploration of the normative effects of international benchmarking in school system performance. OECD Education Working Papers, (71), 0_1.

Breakspear, S. (2014). How does PISA shape policy making? Why how we measure learning determines what counts in education, Centre for Strategic Education, Victoria, Australia.

Bybee, R., & McCrae, B. (2011). Scientific literacy and student attitudes: Perspectives from PISA 2006 science. International Journal of Science Education, 33(1), 7-26.

Coe, R. (2009). School improvement: Reality and illusion. British Journal of Educational Studies, 57(4), 363-379.

Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students’ social and emotional learning: A meta?analysis of school?based universal interventions. Child development, 82(1), 405-432.

Duru-Bellat, M. (2011). From the appealing power of PISA data to the delusions of benchmarking. In PISA under examination (pp. 157-167). SensePublishers.

Fischbach, A., Keller, U., Preckel, F., & Brunner, M. (2013). PISA proficiency scores predict educational outcomes. Learning and Individual Differences, 24, 63-72.

Fuchs, T., & Wößmann, L. (2008). What accounts for international differences in student prformance? A re-examination using PISA data. In The economics of education and training (pp. 209-240). Physica-Verlag HD.

Gorur, R. (2014). Towards a sociology of measurement in education policy. European Educational Research Journal, 13(1), 58-72.

Grek, S. (2009). Governing by numbers: The PISA ‘effect’in Europe. Journal of education policy, 24(1), 23-37.

Hallstrom, T. B. (2015). The Factors Behind Success in PISA. Bachelor of Science in Business and Economics Thesis. Lulea University of Technology. Sweden.

Hanushek, E. A. (2014). Why the US results on PISA matter. Education Week, 33(15), 20-21.

Jerrim, J. (2011). England's" plummeting" PISA test scores between 2000 and 2009: Is the performance of our secondary school pupils really in relative decline (No. 11-09). Department of Quantitative Social Science-UCL Institute of Education, University College London.

Kamens, D. H., & McNeely, C. L. (2009). Globalization and the growth of international educational testing and national assessment. Comparative education review, 54(1), 5-25.

Kidd, B. (2008). A new social movement: Sport for development and peace. Sport in society, 11(4), 370-380.

Kivunja, C. (2014). Do you want your students to be job-ready with 21st century skills? Change pedagogies: A pedagogical paradigm shift from Vygotskyian social constructivism to critical thinking, problem solving and Siemens’ digital connectivism. International Journal of Higher Education, 3(3), 81.

Kuenzi, J. J. (2008). Science, technology, engineering, and mathematics (STEM) education: Background, federal policy, and legislative action.

Kuria, S., Alice, O., & Wanderi, P. M. (2012). Assessment of causes of labour turnover in three and five star-rated hotels in Kenya. International journal of business and social science, 3(15).

Lingard, B. (2010). Policy borrowing, policy learning: Testing times in Australian schooling. Critical studies in education, 51(2), 129-147.

Milford, T. (2009). An investigation of international science achievement using the OECD’s PISA 2006 datasetUnpublished doctoral dissertation, University of Victoria, Victoria, British Columbia, Canada.

OECD. (2014). Education at a Glance 2014-OECD indicators.

Radaelli, C. M. (2009). Measuring policy learning: regulatory impact assessment in Europe. Journal of European Public Policy, 16(8), 1145-1164.

Ross, S. P. (2008). Motivation correlates of academic achievement: Exploring how motivation influences academic achievement in the PISA 2003 dataset. Unpublished doctoral dissertation, University of Victoria, Victoria, British Columbia, Canada

Seligman, M. E. (2012). Flourish: A visionary new understanding of happiness and well-being. Simon and Schuster.

Takayama, K. (2008). The politics of international league tables: PISA in Japan’s achievement crisis debate. Comparative Education, 44(4), 387-407.

Winther, E., & Achtenhagen, F. (2009). Measurement of Vocational Competencies–A Contribution to an International Large-Scale-Assessment on Vocational Education and Training. Empirical Research in Vocational Education and Training, 1(1), 85-102.

Wiseman, A. W. (2010). The uses of evidence for educational policymaking: Global contexts and international trends. Review of research in education, 34(1), 1-24.