SOST30/70172 Quantitative Evaluation of Policies, Interventions and Experiments

This is the final assignment for the module SOST30/70172 for the academic year 2022/2023. This project will be worth 30% of your final mark. The submission deadline is 2pm on the 22nd of May 2023

Goal

You will complete this work as part of a team. Your team will produce a short video presentation summarising published evidence regarding the causal effect of a specific policy, innovation or technology. The theme of the report and the presentation is:

Near-winners, near-losers: Regression Discontinuity and Party Politics.

Distribution of Marks

You will receive a global mark for the project between 0 and 100.

Beyond these, additional marks will be awarded to individual team members who have contributed the most to the joint effort of the team.

Overview

Abundant research has tried to estimate the extent to which partisan politics affect our lives. In theory, when in power, political parties of different ideologies should have distinct influence on their domains. However, answering this question is complicated, due to Selection Bias. During the lectures, we studied Lee (2001), an innovative paper which suggested that the effect of incumbency on re-election could be estimated via a Regression Discontinuity design with threshold located at the 50% vote share. This 50% design, has been used repeatedly since Lee’s work to study many other aspects of party politics. This literature is often referred to as the “close-election” literature. Examples are:

• Boas, Taylor C., et al. “The Spoils of Victory: Campaign Donations and Government Contracts in Brazil.” The Journal of Politics, vol. 76, no. 2, 2014, pp. 415–29, https://doi.org/10.1017/s002238161300145x. Copy

• BROLLO, FERNANDA, and TOMMASO NANNICINI. “Tying Your Enemy’s Hands in Close Races: The Politics of Federal Transfers in Brazil.” The American Political Science Review, vol. 106, no. 4, [American Political Science Association, Cambridge University Press], 2012, pp. 742–61, http: //www.jstor.org/stable/23357707.

• De Magalhaes, L. (2015). Incumbency Effects in a Comparative Perspective: Evidence from Brazilian Mayoral Elections. Political Analysis, 23(1), 113-126. doi:10.1093/pan/mpu012

Your task in this project is to provide a review of the literature using the close-election Regression Discontinuity Design

1

Where to look for papers.

The topic is of broad interest, and therefore you will find abundant literature out there. This is good (you’ll find 3 papers quickly) and bad (which papers are worth reading and which are not?).

There are two rules of thumb you can use to detect important contributions: Journal quality and number of citations. These are noisy signals of academic excellence.

You can check journal quality at: https://www.scimagojr.com/journalrank.php

Citations are normally listed in Google Scholar or PubMed, next to the papers it might return in your searches.

How to handle the literature.

The point making a compilation of published papers is that your team should discuss these papers, assess them critically and produce a classification of results. To assess the papers critically, you have to focus on the “identification strategy” (e.g. is it IV, Regression Discontinuity, Diff-in-Diffs, etc) and the credibility of the identifying assumptions within the context of each paper. Specifically, you need to understand what the authors are saying about the credibility of the identifying assumptions AND you need to form your own opinion about whether the authors’ arguments are credible/sensible (here, different people might disagree with you, which is okay).

If you form a good opinion about what you’ve read, you and your team will be better at finding a way of summarising the literature and crafting the report and presentation.

Remember, all papers rely on SUTVA; then each specific identification strategy has its own critical assumptions. All these assumptions must be credible in the context of each specific application.

Structure of the final report.

You are free to structure your presentation as you wish, however it must include the following sections: • An anonymised title page.

• An introduction

• One or more sections discussing your findings

• A section explaining the policy implications of your findings • A conclusion.

• A bibliography using Harvard referencing system.

• If needed, an appendix.

In addition to this, your presentation must include the following information:

• A description of the main identification challenges and, in particular, how the selection bias arises in this literature.

• For the methods you encounter in the selected bibliography, a good description of the assumptions underlying the designs (e.g. unconfounded assignment, SUTVA, etc), how to interpret these assumptions in general AND in the context of the application, and the extent to which they might/might not be violated (and why). Again, do not provide a review of 15-16 different strategies in 15-16 different papers; rather provide global overviews of assumptions and methods.

• Different papers might estimate different treatment effect (ATE, ATT, LATE, etc): different methodolo- gies estimate different parameters (i.e. LATE is estimated in IV papers; etc). How should we interpret these treatment effects and (if different from each other) how should we compare the results of the papers? Which results are more generally applicable to whole populations? Which are specific to a sub-population?

2

• Critical: The policy implications of your findings and your views regarding how these papers contribute to advance the debates around education, including the extent to which the results of these papers can be extrapolated (the external validity of the results)

WARNING!!!!

The final work must not be a discussion of 15/16+ papers in isolation; it must present a coherent story of what has been done in the literature, which methods have been used (and how have these methods been justified), what are the main conclusions or sets of conclusions and which body of evidence (if any) you believe to be the most credible.

The presentation

Your team’s presentation should be a 5-7 minute video summarising the main findings of the report. You can use any technology you wish. One simple method of doing this would be a Power Point presentation with voice-over. You have free rein regarding how you structure the presentation.

Team members’ contributions

STUDENTS WILL EVALUATE TEAM MEMBERS CONTRIBUTIONS ANONYMOUSLY. Each team member will submit, via Blackboard, a ranking the other team members by contribution (excluding themselves). In each team, students voted as

• The First contributor, will receive 8 additional marks in the report • Second contribution, will receive 5 additional marks in the report • Third contribution, will receive 3 additional marks in the report

In case of TIES,

• Team members tied in first place will receive 7 additional marks in the report

• Team members tied in second place will receive 4 additional marks in the report • Team members tied in third place will receive 2 additional marks in the report.

3