New preprint! Recovering Diversity Without Losing Alignment: A DPO Recipe for Post-Trained LLMs is now on arXiv.