Fritz Schiltz, Paolo Sestito, Tommaso Agasisti and Kristof De Witte
The added value of more accurate predictions for school rankings
School rankings based on value-added (VA) estimates are subject to prediction errors since VA is defined as the difference between predicted and actual performance. We introduce the use of random forest (RF), rooted in the machine learning literature, as a more flexible approach to minimize prediction errors and to improve school rankings. Monte Carlo simulations demonstrate the advantages of this approach. Applying the proposed method to Italian middle school data indicates that school rankings are sensitive to prediction errors, even when extensive controls are added. RF estimates provide a low-cost way to increase the accuracy of predictions, resulting in more informative rankings, and more impact of policy decisions.
Daniel Horn, Hubert János Kiss
Which preferences associate with school performance? – Lessons from an exploratory study with university students
Success in life is determined to a large extent by school performance so it is important to understand the effect of the factors that influence it. In this exploratory study, in addition to cognitive abilities, we attempt to link measures of preferences with outcomes of school performance. We measured in an incentivized way risk, time, social and competitive preferences and cognitive abilities of university students to look for associations between these measures and two important academic outcome measures: exam results and GPA. We find consistently that cognitive abilities (proxied by the Cognitive Reflection Test) are very well correlated with school performance. Regarding non-cognitive skills, we report suggestive evidence for many of our measured preferences. We used two alternative measures of time preference: patience and present bias. Present bias explains exam grades better, while patience explains GPA relatively better. Both measures of time preferences have a non-linear relation to school performance. Competitiveness matters, as students, who opt for a more competitive payment scheme in our experimental task have a higher average GPA. We observe also that risk-averse students perform a little better than more risk-tolerant students. That makes sense in case of multiple choice exams, because more risk-tolerant students may want to try to pass the exam less prepared, as the possibility of passing an exam just by chance is not zero. Finally, we have also detected that cooperative preferences—the amount of money offered in a public good game—associates strongly with GPA in a non-linear way. Students who offered around half of their possible amounts had significantly higher GPAs than those, who offered none or all their money.
Koen Declercq, Joris Ghysels, Júlia Varga
Gender differences in applying for STEM programs in higher education:
evidence from a policy shift in Hungary
We study how admission policies in higher education affect enrollment decisions of men and women and the decision to apply to STEM programs. More specifically, we investigate how an increase in the relative acceptance probability for STEM programs affects these decisions. We apply our analysis to Hungary and we evaluate a policy reform that limited access to subsidized non-STEM programs. We find that this change in the selectivity of the admission system differently affected application decisions of men and women. After the reform, fewer students applied to higher education and the reform especially discouraged the participation of women. After the reform, more men and women applied to STEM programs or non-subsidized non-STEM programs in which they have to pay tuition fees. This last effect is stronger for women. As the reform affected the chance to be admitted to higher education, we estimate a structural model to analyze how the responsiveness to admission probabilities in application decisions differs between men and women. We find that women are more sensitive to admission probabilities. Finally, we use the model to simulate the impact of alternative admission policies on enrollment in STEM programs. We find that an open access policy in STEM programs would stimulate more men and women to apply to these programs.
Zoltán Hermann – Marianna Kopasz
Educational policies and the gender gap in test scores:
A cross-country analysis
Girls tend to outperform boys in reading tests, while they usually lag behind boys in math. However, the size of the gender gap varies to a great extent between countries. While the existing literature explains these differences as being mainly due to cultural factors, this paper explores whether this cross-country variation is related to educational policies like tracking, grade retention, and individualised teaching practices. The gender test score gap is analysed in math, reading and science using the PISA 2012 dataset. Multilevel models are used in the estimation. The results suggest that the extent of the gender gap is indeed associated with certain characteristics of the various education systems. First, applying a difference-in-differences estimation method, it was found that early tracking has a direct effect on the gender gap in test scores, in favour of girls. Second, suggestive evidence shows that more student-oriented teaching practices also benefit girls relative to boys, both between and within countries, and within schools. Finally, grade retention is correlated with the gender gap, though there is further evidence suggesting that this correlation is very unlikely to represent a causal effect.
Fritz Schiltz, Deni Mazrekaj, Daniel Horn and Kristof De Witte
Does It Matter When Your Smartest Peers Leave Your Class?
Evidence from Hungary
Elite schools in Hungary cherry pick high achieving students from general primary schools.
The geographical coverage of elite schools has remained unchanged since 1999, when
the establishment of new elite schools stopped. We exploit this geographical variation in
the immobile Hungarian society and estimate the impact of high achieving peers leaving the class on student achievement, behaviour, and aspirations for higher education. Our estimates indicate moderate but heterogeneous effects on those left behind in general primary schools.
Thomas Wouters – Zoltán Hermann – Carla Haelermans
Demand for secondary school characteristics
Evidence from school choice data in Hungary
We estimate preferences for school tracks in upper secondary education in Hungary.
We consider travel time, school SES composition, school level (in terms of peer quality) and school quality (in terms of added value). We find that students have stronger preferences for school SES composition and school level, rather than school quality (which may be harder to observe). Furthermore, these preferences vary between high and low ranked schools, indicating students use heuristics in the process of compiling their ranked preference list.
Ágnes Szabó-Morvai – Dániel Horn – Anna Lovász – Kristof De Witte
Human Capital Effects of Kindergarten and School Enrolment Timing
Using instrumental variables approach this paper studies the effect of kindergarten starting age jointly with that of school starting age. We show that estimating the effect of kindergarten or school enrolment timing on later human capital outcomes separately, without taking their inter-relatedness into account, may confound the two effects and produce endogenous results. The instruments originate from exogenous birthdate-related enrolment cutoffs in kindergarten and school admissions. Using a rich Hungarian database, we show that both earlier kindergarten enrolment and later school enrolment have a significant and non-negligible positive effect on standardised test scores in grade 6, 8, and 10, class marks given by the teacher, aspirations for higher education, and track choice. These effects tend to decrease over time and are heterogeneous across mother’s education, as earlier kindergarten enrolment age seems to matter only for the children of low educated mothers.
Tamás Hajdu, Gábor Kertesi and Gábor Kézdi
Health Differences at Birth between Roma and Non-Roma Children in Hungary Long-Run Trends and Decompositions
This paper uses birth records linked to census data to document health differences at birth between Roma and non-Roma children in Hungary between 1981 and 2010. It focuses on differences in average birth weight and average gestational age, as well as the likelihood of low birth weight and the likelihood of preterm birth. The paper shows large gaps in all indicators over the 30 years, with a small narrowing of the gap in absolute terms but not in relative terms. Roma mothers are twice as likely to give birth to babies with low birth weight and before the 37th week. Standard decompositions show that around 80% of the gap is explained by socioeconomic factors, and education alone explains more than half. Despite significant changes in society, the explanatory power of education and other factors remains constant. Narrowing the gap in educational attainment, especially at higher levels, may have the highest potential to improve the relative health of Roma births.
Tamás Hajdu, Gábor Kertesi and Gábor Kézdi
Parental Job Loss, Secondary School Completion and Home Environment
This study examines the effect of parental job loss on children’s completion of secondary school and the mediating role of home environment in that effect. It uses rich survey data from Hungary on adolescents age 14 through 21, with detailed measures of parental employment and home environment. The study replicates the average negative effect found in the literature. No effect is found for families with a history of providing a cognitively stimulating home environment, but the negative effect is strong for other families. Home environment matters more than initial income in mediating the effect. The results highlight the protective nature of a cognitively stimulating home environment.
This paper employs a large-scale, individual-level, panel dataset to analyse the effect of EU accession on the probability of out-migration on the part of Hungarian physicians and dentists between 2003 and 2011. The study uses event history modelling and competing risk models. The results show that EU accession did not at the time affect the probability of out-migration while after the end of the transitional period of restrictions on the free movement of labour from the new EU member states to Austria and Germany, the probability of doctors’ migration increased considerably. Relative wages and peer pressure also exercise a significant role in the out-migration decisions of young medical doctors. We also find that more than half of those medical doctors who left the country during the observation period returned some time later. The data furthermore suggest a massive flow of doctors to domestic jobs outside the health care system.
Joris Ghysels, Zoltán Hermann, Iryna Rud and Melline Somers
The effect of increased general education in vocational schools – Evidence from a Hungarian vocational school reform
This paper aims at the evaluation of the reform of vocational education introduced in 1998 in Hungary. The reform extended the duration of education by one year, and increased teaching time spent on non-vocational subjects. The reform affected two of the three tracks in upper-secondary education in Hungary, vocational secondary school and vocational school. We estimate the effect of the reform on educational attainment, employment and wages in a comparative interrupted time series (CITS) framework, using the academic track and secondary school drop-outs as control groups. The results suggest that the reform has had heterogeneous effects. First, we detect no effect for the vocational secondary track, while the reform has improved labour market outcomes of vocational school students. Second, in the vocational school group the reform has increased men’s wages, while not affected their employment. For women we found a positive employment effect, while wages have increased only for the younger cohorts.
Hungarian legislation provides firms with financial incentives to train apprentices from vocational training schools. In line with these incentives, it is observed that firms increasingly train apprentices over the period 2003-2011, in particular, in the sectors manufacturing, construction, wholesale and retail and hotels and restaurants. However, at the same time, it is observed that firms decreasingly retain the trained apprentices in these four sectors. This finding leads to the hypothesis that apprentices are not profitable in the long run. The formulated hypothesis is known in the previous literature as the ‘substitution strategy’. This recruiting strategy is particularly observed among firms that replace their low-skilled labour with apprentices in order to reduce the cost of wages. For these firms it is not beneficial to hire an apprentice after accomplishing his training, because then he becomes a low-skilled worker paid at higher wages. This paper investigates the effect of the share of days worked by apprentices on productivity and gross profits of Hungarian firms by using a unique matched employer-employee dataset. Different approaches that allow us to estimate the effect are discussed among which fixed effects first-difference models and system GMM. The results indicate that apprentices decrease productivity and gross profits of Hungarian firms. These negative effects on firm performance were more prominent and robust before (2003-2007) than after the financial crisis (2008-2011).
Anna Lovász, Ewa Cukrowska-Torzewska, Mariann Rigó, Ágnes Szabó-Morvai, Andrea Kiss
One Size Fits All? Gender Differences in the Effect of Subjective Feedback
The effect of objective feedback on performance is often studied, while subjective feedback is largely neglected in the economics literature. We estimate the impact of positive subjective feedback – encouragement and praise – on effort and performance, and compare the effect by gender. We use a computer game, during which players are randomly chosen to be given either no feedback (control) or positive subjective feedback (treatment), and analyze the treatment effect on effort (clicks) and performance (score). Based on previous economic and psychology theories, we test the pathways through which subjective feedback can have an impact: on (1) effort, due to the updating of expected performance or direct (dis)utility from the feedback, or (2) marginal productivity. The results point to significant differences in the mean effects of subjective feedback by gender. For women, encouragement has a significant positive effect while praise has a significant negative effect on performance, while men are less responsive to subjective feedback in general. Gender differences are mostly explained by different confidence distributions, while there are no gender differences in treatment effects if confidence level is held fixed. The effects are mostly realized through changes in effort. These results suggest that better targeted supervisory communication in schools or workplaces can improve the performance of lower-confidence individuals and thereby decrease the gender gap in performance.
Fritz Schiltz, Chiara Masci, Tommaso Agasisti and Daniel Horn
Using Machine Learning To Model Interaction Effects In Education: A Graphical Approach
Educational systems can be characterized by a complex structure: students, classes and teachers, schools and principals, and providers of education. The added value of schools is likely influenced by all these levels and, especially, by interactions between them. We illustrate the ability of Machine Learning (ML) methods (Regression Trees, Random Forests and Boosting) to model this complex ‘education production function’ using Hungarian data. We find that, in contrast to ML methods, classical regression approaches fail to identify relevant nonlinear interactions such as the role of school principals to accommodate district size policies. We visualize nonlinear interaction effects in a way that can be easily interpreted.
Ágnes Szabó-Morvai and Anna Lovász
Childcare and Maternal Labor Supply – a Cross-Country Analysis of Quasi-Experimental Estimates from 7 Countries
Evidence from single country studies suggests that the effect of subsidized childcare availability on maternal labor supply varies greatly by institutional context. We provide estimates of the childcare effect around age 3 of children for 7 EU countries, based on harmonized data and the same quasi-experimental methodology, and evaluate their cross-country variation in light of key institutional factors (leave policies, labor market characteristics, cultural norms). The identification of the childcare effect utilizes birthdate-based kindergarten eligibility cutoffs specific to each country in an instrumental variables approach. We combine data on mothers from the EU-LFS, eligibility cutoffs gathered from country experts and verified using further datasets, and country-level institutional characteristics from various sources. We discuss the role of the context, timing, and the point of estimation. The results suggest that the childcare effect is the highest in CEE countries, where at this child age, maternal participation is still relatively low compared to that of mothers with older children, and leaves with job protection are just ending. We find less evidence of an impact in Southern EU countries, where leaves end at a much earlier age, and maternal participation at older child ages is low. Western EU countries also show some impact, despite the already high maternal participation rates prior to this age. Specific policy implications are derived from the results in light of the EU Barcelona targets for childcare expansion under age 3.