Article summary
Many scholars classify documents into categories, but social scientists often need accurate estimates of the proportion of documents in each category. Parametric “classify-and-count” approaches can be highly model-dependent and may produce greater bias in proportion estimates even when the percent of documents correctly classified rises. Direct, nonparametric estimation of proportions avoids some of those model-dependence problems but can fail when language meaning shifts between training and test sets or when categories use very similar language.