Nectar (7-wise Ranked Preference Data)

Nectar is a preference/ranking dataset: 182,954 prompts drawn from lmsys-chat-1M, ShareGPT, Anthropic HH-RLHF, UltraFeedback, Evol-Instruct, and Flan, each paired with 7 responses (from GPT-4, GPT-3.5-turbo, Llama-2-7B-chat, Mistral-7B-Instruct and others) that are ranked 7-wise using GPT-4-based judgments. It underpins the Starling-RM reward model and Starling-7B. Classified as preference data rather than plain SFT because the value is in the ranked comparisons.