Article summary
This article explains the pretrain–finetune paradigm for transformer language models and shows how applied political scientists can use it to classify political text with relatively little labeled data and modest computation. The author frames mechanisms behind pretraining and finetuning, addresses common misconceptions about costs and complexity, and demonstrates the approach by identifying toxic language in conversation threads that follow U.S. Senators’ tweets. An online tutorial accompanies the article to help nontechnical scholars adopt the workflow.