Document Type

Conference Proceeding

Abstract

Propensity score matching (PSM) is a popular technique for selecting a sample in observational research that mimics the desirable qualities of a randomized controlled trial. This paper introduces a new algorithm for propensity score matching that iteratively selects only the mutual best matching treatment-control pairs. The new approach, referred to here as iterative matching, is compared to the popular nearest neighbor with caliper method. The utility and importance of the new algorithm is demonstrated in an applied example through an ANCOVA examining the efficacy of i-Ready Diagnostic and Instruction in improving scores on the Florida Standards Assessment (FSA). Results show that the iterative matching algorithm results in fewer matched pairs than nearest neighbor with caliper; however, when the treatment-to-control ratio is balanced in the sampling pool, iterative matching tends to result in slightly higher quality matches. In the applied example, the effect of i-Ready on FSA scores, controlling for prior year FSA scores, is statistically significant for a sample constructed using iterative matching, but not for the nearest neighbor with caliper-matched sample or the unmatched sample. Overall, this study demonstrates the importance of PSM and the choice of PSM method while also providing efficacy evidence for i-Ready Diagnostic and Instruction.

Download
COinS