ECOS3997 Semester 1 2023
Instructions for report
Individual assessment
Weight of final grade: 60%
Due on 5th of May (Friday) 11.59pm (online submission via Canvas)
Must be submitted as a word document (no PDFs) and include the word count
1. Background
According to the Australian Government Department of Education, students from disadvantaged socioeconomic background have low participation in STEM (Science, Technology, Engineering, and Mathematics) education. The average 15-year-old adolescent from a low socioeconomic background is 3 years behind their high socioeconomic background peers in math and science, is less likely to aspire to a STEM career, and is less likely to end up working in a STEM occupation as an adult.
Studying STEM subjects is important for one’s future career trajectory. It is estimated that 75% of future occupations require STEM-related skills and wages tend to be higher in STEM occupations (Panizzon et al., 2018). The socioeconomic gap in STEM in school is contributing to the persistent earnings gap between individuals who grew up in advantaged versus disadvantaged socioeconomic environments.
To address the problem, the University of Sydney in partnership with the NSW Department of Education, implemented a program to lift students’ interest and confidence in studying math and science, and increase the number of socioeconomically disadvantaged students pursuing STEM-related careers in the future.
References
Australian Government Department of Education. https://www.education.gov.au/australian-curriculum/national-stem-education-resources-
toolkit/i-want-know-about-stem-education/which-school-students-need-stem-education/students-low-socio-economic-areas
Panizzon, D. L., Geer, R., Paige, K., O'Keeffe, L., Schultz, L., Zeegers, Y., & Brown, L. (2018). Exploring the ‘hard facts’ around STEM in Australia: Females, low socioeconomic status and absenteeism. International Journal of Innovation in Science and Mathematics Education, 26(8).
2. Program features
The program was conducted in 2018 and targeted students enrolled in Year 10. Through the program, 3rd year University of Sydney students in STEM fields visited Year 10 classrooms each month for the entire school year. During each visit (which lasted 3 hours), they conducted one or more of the following activities:
1. Science workshops: demonstration of scientific concepts through experiments. Workshops were educational, fun and sparked Year 10 students’ curiosity in learning more about science.
2. Future studies and career mentoring sessions: sharing their lived experience and advice on how to prepare for post-Year 12 studies in a STEM field.
3. One-on-one tutoring sessions: supporting Year 10 students with homework and preparing for assessments in STEM-related subjects.
The program was designed to help students build confidence in their ability to succeed in STEM, encourage them to consider a STEM career and support them in taking the required steps to succeed in a STEM career. The university students involved in the program delivery were role models to Year 10 students, as they were highly motivated to improve outcomes of socioeconomically disadvantaged students and shared with them a similar socioeconomic background.
3. Program implementation
Due to limited resources and considering the importance of rigorously evaluating the impact of the program, it was implemented as a randomised controlled trial. First, the Department of Education asked several principals of low SES schools in Greater Sydney if they would be interested in implementing the program in their school, to which 30 school principals responded positively. Then, of these 30 schools, 15 were randomly assigned to the treatment group and received the program. The remaining 15 schools were assigned to the control group and did not receive the program.
4. Outcomes for the evaluation & dataset
Immediately prior to the start of the program, a survey was conducted with Year 10 students in all 30 schools that were interested in taking part in the program. The survey asked students about their intentions with regards to future studies and jobs. At the end of the program, the same survey was conducted with the same students. A final survey was also conducted 4 years after the program to find out, if after leaving high school, the students ended up undertaking further education in STEM (to encourage responses, everyone responding received a $20 gift card).
The dataset for the evaluation of the program includes the following variables: a) Treatment assignment and school characteristics
treatment: takes value 1 (program) if the student is in a school that received the program and 0 (control) otherwise
icsea: the school’s Index of Community Socio-Educational Advantage (ICSEA) (see https://docs.acara.edu.au/resources/About_icsea_2014.pdf). The higher the value of the index, the higher the level of socioeconomic advantage at the school level. The reference value (national average) is 1000; a school with a ICSEA above 1000 is a school that scores above the national average in terms of socioeconomic advantage; a school with a ICSEA below 1000 is a school that scores below the national average in terms of socioeconomic advantage.
b) Pre-program outcomes (data collected immediately before the start of the program) b.1) School outcomes
y9mathgrade: Math grade in Year 9 (scale: A-F, with A being the highest grade and F the lowest grade)
y9sciencegrade: Science grade in Year 9 (scale: A-F, with A being the highest grade and F the lowest grade)
b.2) Intentions and aspirations
preprog_y12intention: takes values 1 (yes) if the student answered they intended to complete Year 12 and 0 (no) otherwise
preprog_furtheredu: takes values 1 (yes) if the student answered they intended to pursue further education and 0 (no) otherwise
preprog_furtheredustem: takes values 1 (yes) if the student answered they intended to pursue further education in a STEM field and 0 (no) otherwise
preprog_occupationstem: takes values 1 (yes) if the student answered they aspired to have an occupation in STEM in 10 years and 0 (no) otherwise
c) Post-program outcomes (data collected after the completion of the program) c.1) School outcomes
y10mathgrade: Math grade in Year 10 (scale: A-F, with A being the highest grade and F the lowest grade)
y10sciencegrade: Science grade in Year 10 (scale: A-F, with A being the highest grade and F the lowest grade)
y12graduation: takes value 1 (yes) if the adolescent graduated from high school and 0 (no) otherwise (dropout out before Year 12 or failed Year 12)
c.2) Intentions and aspiration
postprog_y12intention: takes values 1 (yes) if the student answered they intended to complete Year 12 and 0 (no) otherwise.
postprog_furtheredu: takes values 1 (yes) if the student answered they intended to complete further education and 0 (no) otherwise.
postprog_furtheredustem: takes values 1 (yes) if the student answered they intended to complete further education in a STEM field and 0 (no) otherwise.
postprog_occupationstem: takes values 1 (yes) if the student answered they aspired to have an occupation in STEM in 10 years and 0 (no) otherwise.
c.3) Long-run (post-Year 12) outcomes
posty12_edu: takes value 1 (yes) if the adolescent is pursuing further education after Year 12 and 0 (no) otherwise
posty12_edustem: takes value 1 (yes) if the adolescent is pursuing further education in STEM after Year 12 and 0 (no) otherwise
3. Your task
You are an economist at the Australian Department of Prime Minister and Cabinet, in charge of advising government about social policy to reduce socioeconomic inequality in youth. Your task is to write a report to the Minister for Education and Youth, whose top priority is to fund effective programs across Australia to decrease inequality in adolescent outcomes between high and low SES schools. Your report will be based on your analysis of the impact of the STEM program.
Your report must include the following sections:
I. Executive summary (~ 300 words)
II. Introduction (~ 1400 words)
III. Data analysis and results (~ 500 words)
IV. Discussion of the results (~ 600 words)
V. Policy recommendations (~ 200 words)
VI. References (not included in word count)
Writing concisely is a key skill that develops with practice. The maximum number of words for the report is 3000 (excluding references). If the report exceeds 3000 words, I will only read the first 3000 words.
Guidelines for each section:
II. Introduction [Total: 35 out of 100 points] a. Background information [20 points]
Explain why it’s urgent to address socioeconomic inequality among youth in Australia, by:
• presenting statistics on the degree of socioeconomic inequality in adolescent outcomes (can be in relation to participation and achievement in STEM but does not necessarily have to),
• discussing past or current policies / programs that relate to the one being evaluated (can be large programs ran in many schools, or local programs implemented by one or few schools) and what we know about their effectiveness,
• discussing implications of the problem if left unaddressed. It must include a discussion of the implications for individual well-being, the society, economic growth and social policy.
Examples of valid sources: ABS data, government reports, journalistic articles, commentaries from experts, etc.
b. Literature review [12 points]
Present research findings on the impact of programs, related to the one you are analysing (from Australia or other countries), to address socioeconomic inequality in youth.
In your literature review, for each article refer to the population being studied, the key research questions, methods and findings. It is very important that you carefully explain how each research article relates to your report.
Sources: research articles (from economics or other disciplines like psychology, education, health, etc). Do your search on google scholar (https://scholar.google.com.au/) using keywords.
How many articles? There is no minimum or maximum number, but quality is better than quantity. You can aim referring to 3 highly relevant articles.
c. Brief statement about what you will do in your report [3 points]
• Briefly summarise the program you are reporting on.
• Give a roadmap for what comes next. Should not be longer than 2-3 sentences.
III. Data analysis & results [20 points] This section must include:
a. Analysis of adolescent outcomes and intentions / aspirations, prior to the program. This must include a comparison of adolescent outcomes and intentions / aspirations, prior to the program, across schools that received the program and schools that didn’t receive the program.
b. Analysis of the impact of the program on school outcomes, intentions / aspirations and long-run outcomes.
In each subsection a-b, you need to:
• Report on the statistical and economic significance of the gaps
o Statistical significance: is the gap statistically different from zero?
o Economic significance: is the gap large enough so that it is meaningful?
Remember statistical significance ≠ economic significance!
• Report on your analysis using figures and briefly comment on the statistical significance of the differences using t-test
• Number and label each figure (hint: you can use command “graphcombine” in Stata to combine all figures about pre-program outcomes into one single figure, and all figures about post-program outcomes into one single figure)
IV. Discussion of the results [20 points]
In this section, you need to:
• Interpret and discuss the results: explain your findings and their implications, stressing the economic significance of your results (that means, comment on whether the effects small or large)
• Discuss strengths and weaknesses of your analysis. Examples of aspects to discuss: o what aspects of the program implementation and data give credibility to your
results/conclusions?
o what additional analysis would you like to conduct to learn in more detail about the
impact of the program but would require data that is not available to you?
• Discuss the implications of your results for social policy
V. Policy recommendations [10 points]
• Make a maximum of 3 brief policy recommendations based on your results
• Present the recommendations in bullet point format
• Remember, this is what the Minister reads! Must be accurate, clear and concise
VI. References
• Reference all your sources of information
• Follow the guidelines for adequate referencing style, as shown here:
https://libguides.library.usyd.edu.au/c.php?g=508212&p=3476096
[15 points for writing / presentation style and correct referencing]
