PP346 Program Evaluation
Problem Set 1 Due Thursday, October 8, by 11:59 pm For this assignment, provide a write-up where you answer the questions below. Now and for all the problem sets in this class, whenever you are asked for estimates, be sure to construct a table that is readily readable. It doesn’t have to be beautiful, but the reader must be able to look at it quickly and understand the main results. Page after page of regression output will not do. Some of the tables that you are asked to replicate provide a useful guide, although I’d generally rather see standard errors than stars. Be concise in your write-up; excess wordiness will be penalized. Finally, submit a log file that includes all commands and results for your entire analysis. Study motivation In the mid-1990s, the US dramatically reformed its welfare system, first at the state level, then at the federal level. Florida was one of the first states to implement time limits, which soon thereafter became one of the most controversial components of the federal welfare reform law. Built into its reform program was an experimental evaluation carried out in Escambia County. The treatment group was subject to the new program, called FTP, while the control group operated under the old rules. The four-year follow-up report appears in Bloom et al (2000); Grogger and Michalopoulos (2003) reanalyzed the data in a manner designed to distinguish the effects of the time limit from the other features of FTP. Specifics MDRC, the contractor that carried out the official evaluation, provides a public-use version of the data for researchers to use. For the first problem set, you will familiarize yourself with the data and generate some initial results. For the second problem set, you will implement a number of experimental and non-experimental estimators. We may revisit these data in the last problem set as well, so keep track of your work! Questions: (A) How did FTP change the behavior of welfare recipients? (B) What can we say about the effects of the time limit as opposed to other features of the program? Data Data for this assignment are in the files ftp_ar.dta and ftp_srv.dta on Canvas. Analysis:
(1) You will need to start by reading at least pp. 1-20 of “FTP PUF Documentation_start here_.pdf.” (2) Open the file ftp_ar.dta. Can you verify that it has the correct number of observations? The correct numbers in the treatment and control groups? (3) Verify the data by replicating the control-group means in the file “Annotated AR table 1_U.xls.” Are there any discrepancies? While you are at it, comment on the levels of employment and earnings among the people in the study population. (4) Compare your results to those in Table B.1 of Bloom et al (2000). Why are they different? How do you expect these differences to affect your results? (5) Test for balance. Find all pre-RA (random assignment) variables on AFDC receipt, Food Stamp receipt, and employment. How many variables are there? Conduct two simple tests for balance. First, regress each variable on the treatment dummy. How many times do you reject the null hypothesis at the 5% level? Roughly how many rejections would you expect under the null of random assignment? Second, regress the treatment dummy on all the variables together and conduct a joint test for balance. Do your tests concur? What do you conclude about the overall balance of the sample? What other condition(s) do you need to conclude that random assignment identifies the treatment effects of interest? (6) Returning to the post-RA data, estimate simple treatment effects such as those shown in “Annotated AR table 1_U.xls.” Can you replicate those results? (7) In their publications, MDRC reported not raw, unadjusted estimates such as those that you just computed, but rather regression-adjusted estimates that controlled for a number of characteristics. Those characteristics are discussed beginning on p. 90 of “FTP PUF Documentation_start here_.pdf.” Find the relevant variables and compute regression-adjusted treatment effects. Can you replicate the estimates that appear in “Annotated AR table 1_A.xls?” (8) Discuss how the adjusted estimates differ from the unadjusted estimates. Are the differences ever greater than one standard error of the estimate, in either direction? Do the signs of the estimates change? How frequently? Among the coefficients that change sign, how many were significant in the first place? (9) How do the differences you just discussed accord with your expectations? Explain. (10) Summarize how FTP affected welfare receipt, employment, and income over the first four years of the program. This calls for a discussion of magnitudes as well as general trends.

欢迎咨询51作业君