Pitfalls of analyzing perinatal outcomes by health care providers

The recent study by Stoll and colleagues1 showed a substantial reduction in perinatal death (relative risk [RR] 0.44), low birth weight (RR 0.24), preterm birth (RR 0.39), an Apgar score of less than 7 at 5 minutes (RR 0.63) and cesarean delivery (RR 0.17) when care was provided by midwives. The analyzes were based on the most responsible provider (MRP), and risk stratification was carried out with a risk scoring system. Several potential methodological and analytical errors need consideration before these inferences can be made.

This risk scoring system could not have substantially reduced the selection bias. Many individual risk factors in this scoring system not only directly associate with specific outcomes, but also affect MRP assignments; for example, diagnosis of small-for-gestational-age babies (score 3) predetermines one of the primary outcomes, low birth weight, and strongly associates with the other 4 primary outcomes. In addition, such pregnancies are more likely to be cared for by an obstetrician. On the other hand, malpresentation (score 3), increases the risk of cesarean delivery with no substantial impact on the other 4 outcomes. Therefore, pregnancies with the exact same antepartum risk score could have a completely different a priori risk of most, if not all, of the primary outcomes. This creates a very substantial selection bias.

The second major potential problem relates to the MRP assignment for in-hospital services. Agreement was 93% for obstetricians and 94% for midwives; this degree of imprecision can easily account for the observed associations. This proportion of incorrect assignments (7% for obstetricians and 6% for midwives) is, at least in part, because of patients being transferred from midwives to obstetricians. These are relatively large and systematic, rather than random, misclassifications are important when considering perinatal death, low birth weight, preterm birth and Apgar scores of less than 7 at 5 minutes. It is quite likely that a substantial proportion of these events occurred among pregnancies that should have been assigned to the midwife group but instead were incorrectly assigned to the obstetrician group. Indeed, Thiessen and colleagues2 indicated that their results, which also relied on MRP, “provide a heightened awareness regarding transfer of care and the mislocation of provider type.” Frost and colleagues3 evaluated data elements of the British Columbia Perinatal Data Registry and reported less than perfect validity for the delivery provider.

Stoll and colleagues’1 study also did not consider etiology. For example, babies with known genetic or structural anomalies are at a higher risk of perinatal death, and most of these pregnancies are under the care of obstetricians. Biological plausibility of many of the associations eliminated is also lacking. The higher incidence of perinatal death, preterm birth, low birth weight or low Apgar score under obstetric care has no mechanistic basis.

Therefore, conclusions regarding quality of care by midwives versus obstetricians simply based on associations cannot be considered credible unless these concerns are resolved by a higher-stringency study design.