bright
bright
bright
bright
bright
tl
 
tr
lunder
Recent Content of Educational Evaluation and Policy Analysis
This site designed and maintained by
Prof. Glenn Fulcher

@languagetesting.info

runder
lcunder
rcunder
lnavl
Navigation
lnavr
navtop
lnavs
rnavs
lnavs
rnavs
lnavs
rnavs
lnavs
rnavs
lnavs
rnavs
bottomnav
 
 

 

 
Titles and Abstracts
     
 
Educational Evaluation and Policy Analysis

  • Do Principals Fire the Worst Teachers?
    by Jacob, B. A.

    This article takes advantage of a unique policy change to examine how principals make decisions regarding teacher dismissal. In 2004, the Chicago Public Schools (CPS) and Chicago Teachers Union signed a new collective bargaining agreement that gave principals the flexibility to dismiss probationary teachers for any reason and without the documentation and hearing process that is typically required for such dismissals. With the cooperation of the CPS, I matched information on all teachers who were eligible for dismissal with records indicating which teachers were dismissed. With these data, I estimate the relative weight that school administrators place on a variety of teacher characteristics. I find evidence that principals do consider teacher absences and value-added measures, along with several demographic characteristics, in determining which teachers to dismiss.



  • The Impact of Tuition Increases on Enrollment at Public Colleges and Universities
    by Hemelt, S. W., Marcotte, D. E.

    In this paper we review recent increases in tuition at public institutions and estimate impacts on enrollment. We use data on all U.S. public 4-year colleges and universities from 1991 to 2006 and illustrate that tuition increased dramatically beginning in the early part of this decade. We examine impacts of such increases on total enrollment and credit hours, and estimate differences by type of institution. We estimate that the average tuition and fee elasticity of total headcount is -0.0958. At the mean, a $100 increase in tuition and fees would lead to a decline in enrollment of about 0.25 percent, with larger effects at Research I universities. We find limited evidence that especially large tuition increases elicit disproportionate enrollment responses.



  • Assessing the Effects of Teachers' Reading Knowledge on Students' Achievement Using Multilevel Propensity Score Stratification
    by Kelcey, B.

    This study investigated the relationship of teachers’ reading knowledge with students’ reading achievement using a direct teacher knowledge assessment rather than indirect proxies (e.g., certification). To address the inequitable distribution of teachers’ knowledge resulting from differences in teachers’ backgrounds and the disparities in how schools attract and cultivate knowledge, the study developed multilevel propensity score methods to identify comparable teachers on the basis of both teacher and school backgrounds. Results suggest that schools are complexly associated with differences in teachers’ knowledge and that comparisons which ignore the relevance of schools may be misleading. By comparing teachers with similar personal and school backgrounds, results show measured knowledge is significantly associated with students’ achievement in reading comprehension but not word analysis. The findings support policies which leverage school capacities to develop the specialized knowledge needed for teaching reading.



  • "Staffing to the Test": Are Today's School Personnel Practices Evidence Based?
    by Cohen-Vogel, L.

    Faced with mounting policy pressures from federal and state accountability programs, school leaders are reallocating curricula, time, even diet in an attempt to boost student achievement. To explore whether they are using test score data to reallocate their teacher resources as well, I designed a cross-case, cross-sectional study and explored principals’ reported staffing practices in one higher performing and one lower performing elementary school in each of five Florida school districts. Findings show that school leaders are "staffing to the test" by hiring, moving, and developing teachers in an effort to increase their schools’ overall performance. The paper discusses the implications of evidence-based staffing for policy, practice and future research.



  • The Causal Effect of Federal Work-Study Participation: Quasi-Experimental Evidence From West Virginia
    by Scott-Clayton, J.

    Since 1964, the Federal Work-Study (FWS) program has provided funds to subsidize the wages of student employees, but it has never been studied directly. I use an instrumental variables difference-in-difference framework with administrative data from West Virginia to identify causal effects, comparing eligible and ineligible students across institutions with higher and lower FWS availability and using differences in FWS availability to instrument for actual FWS participation. I find no evidence that FWS participation improves academic outcomes for the full sample, but this masks significant negative effects for women and some significant positive effects for men. Although results should be interpreted cautiously given limitations of the sample, they represent the first direct, quasi-experimental evidence regarding the effect of the program.



  • Acknowledgments

  • Evaluating Criteria for English Learner Reclassification: A Causal-Effects Approach Using a Binding-Score Regression Discontinuity Design With Instrumental Variables
    by Robinson, J. P.

    When English learners are "reclassified" as fluent English proficient, often their instructional setting changes, including a significant reduction in or elimination of English language development services. Depending on a child’s language skills, this instructional change could hinder future development or provide needed opportunities for learning advanced material. By establishing assessment-based guidelines for reclassification, policy makers have tremendous influence on when these settings change. The author highlights this policy lever for guiding reclassification decisions and identifies a method for rigorously evaluating whether the threshold for transitioning between settings is appropriate. This method—binding-score regression discontinuity with an instrumental variable—was then implemented to obtain unbiased effects of reclassification on academic outcomes for students on the cusp of meeting reclassification criteria to provide credible policy recommendations for maintaining or shifting assessment-based reclassification thresholds. The method detailed here can be used by policy makers to evaluate their own assessment-based guidelines.



  • The Path Not Taken: How Does School Organization Affect Eighth-Grade Achievement?
    by Schwartz, A. E., Stiefel, L., Rubenstein, R., Zabel, J.

    Although rearranging school organizational features is a popular school reform, little research exists to inform policymakers about how grade spans affect achievement. This article examines how grade spans and the school transitions that students make between fourth and eighth grade shape student performance in eighth grade. The authors estimate the impact of grade span paths on eighth-grade performance, controlling for school and student characteristics and correcting for attrition bias and quality of original school. They find that students moving from K–4 to 5–8 schools or in K–8 schools outperform students on other paths. Results suggest four possible explanations for the findings—the number and timing of school changes, the size of within-school cohorts, and the stability of peer cohorts.



  • Running in Place: Low-Income Students and the Dynamics of Higher Education Stratification
    by Bastedo, M. N., Jaquette, O.

    The increasing concentration of wealthy students at highly selective colleges is widely perceived, but few analyses examine the underlying dynamics of higher education stratification over time. To examine these dynamics, the authors build an analysis data set of four cohorts from 1972 to 2004. They find that low-income students have made substantial gains in their academic course achievements since the 1970s. Nonetheless, wealthier students have made even stronger gains in achievement over the same period, in both courses and test scores, ensuring a competitive advantage in the market for selective college admissions. Thus, even if low-income students were "perfectly matched" to institutions consistent with their academic achievements, the stratification order would remain largely unchanged. The authors consider organizational and policy interventions that may reverse these trends.



  • Determinants of High Schools' Advanced Course Offerings
    by Iatarola, P., Conger, D., Long, M. C.

    This article examines the factors that determine a high school’s probability of offering Advanced Placement (AP) and International Baccalaureate (IB) courses. The likelihood that a school offers advanced courses, and the number of sections that it offers, is largely driven by having a critical mass of students who enter high school with eighth-grade test scores that are far above average. The number and qualifications of the instructional staff, in contrast, play a very small role. The results suggest that the willingness of schools to offer advanced courses is driven by real, perceived, or created student demand and that there may be few resource constraints that prevent schools from supplying advanced courses.



  • The Effectiveness of Distance Education Across Virginia's Community Colleges: Evidence From Introductory College-Level Math and English Courses
    by Xu, D., Jaggars, S. S.

    Although online learning is rapidly expanding in the community college setting, there is little evidence regarding its effectiveness among community college students. In the current study, the authors used a statewide administrative data set to estimate the effects of taking one’s first college-level math or English course online rather than face to face, in terms of both course retention and course performance. Several empirical strategies were used to minimize the effects of student self-selection, including multilevel propensity score. The findings indicate a robust negative impact of online course taking for both subjects. Furthermore, by comparing the results of two matching methods, the authors conclude that within-school matching on the basis of a multilevel model addresses concerns regarding selection issues more effectively than does traditional propensity score matching across schools.



  • A Multistate District-Level Cluster Randomized Trial of the Impact of Data-Driven Reform on Reading and Mathematics Achievement
    by Carlson, D., Borman, G. D., Robinson, M.

    Analyzing mathematics and reading achievement outcomes from a district-level random assignment study fielded in over 500 schools within 59 school districts and seven states, the authors estimate the 1-year impacts of a data-driven reform initiative implemented by the Johns Hopkins Center for Data-Driven Reform in Education (CDDRE). CDDRE consultants work with districts to implement quarterly student benchmark assessments and provide district and school leaders with extensive training on interpreting and using the data to guide reform. Relative to a control condition, in which districts operated as usual without CDDRE services, the data-driven reform initiative caused statistically significant districtwide improvements in student mathematics achievement. The CDDRE intervention also had a positive effect on reading achievement, but the estimates fell short of conventional levels of statistical significance.



  • The Effectiveness of Private Voucher Education: Evidence From Structural School Switches
    by Lara, B., Mizala, A., Repetto, A.

    In this article the authors analyze the effect of private voucher education on student academic performance using new data on Chilean students and a novel identification strategy. Most schools in Chile provide either primary or secondary education. The authors analyze the effect of private voucher education on students who are forced to enroll at a different school to attend secondary education once graduated from primary schooling—structural switches. Moreover, the data set the authors use in this article contains information on previous academic achievement and thus allows them to identify differences in students’ unobservable characteristics. Using a number of propensity-score-based econometric techniques and the changes-in-changes estimation method, the authors find that private voucher education leads to small, sometimes not statistically significant differences in academic performance. The estimated effect of private voucher education amounts to about 4% to 6% of one standard deviation in test scores. In contrast, the literature on Chile based on cross-sectional data had previously found positive effects of about 15% to 20% of one standard deviation.



  • Public School Response to Special Education Vouchers: The Impact of Florida's McKay Scholarship Program on Disability Diagnosis and Student Achievement in Public Schools
    by Winters, M. A., Greene, J. P.

    The authors expand on research evaluating public school response to school choice policies by considering the particular influence of voucher programs for disabled students—a growing type of choice program that may have different implications for public school systems from those of more conventional choice programs. The authors provide a theoretical framework to show that special education vouchers could influence both school quality and the likelihood that a school will choose to identify the marginal child as disabled. Using a rich panel data set from Florida, the authors find some evidence that competition from a voucher program for disabled students decreased the likelihood that a student was diagnosed as having a mild disability and was positively related to academic achievement in the public schools.



  • Educational Choice and Student Participation: The Case of the Supplemental Educational Services Provision in Chicago Public Schools
    by Steinberg, M. P.

    The Supplemental Educational Services (SES) provision of the No Child Left Behind Act offers free tutoring services to students attending perennially underperforming schools. The author assesses the extent to which Chicago Public Schools students most in need of and who could potentially benefit the most from additional academic instruction participate in SES. The author uses multilevel cohort and longitudinal analyses to explore the characteristics of students participating in SES in Chicago Public Schools from 2004–2005 to 2007–2008. The results of this study, among the first empirical evaluations of SES participation, suggest that students with higher observed cognitive achievement are less likely to engage the SES provision, whereas students with better noncognitive performance are more likely to participate in SES.