LongCounsel-8: A Benchmark Suite for Longitudinal Depression Tracking from Multi-Session Counseling Dialogues

arXiv:2609.03507v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used to assess depression in counseling conversations, where psychology evidence suggests that tracking change across sessions matters more than any single score. However, existing dialogue benchmarks score each session on its own, and an accurate score for every session does not guarantee that a method follows the changes between sessions. We introduce LongCounsel-8, a benchmark for tracking depression across counseling sessions. Its systematic pipeline takes client profiles from real counseling cases, depression scores from real longitudinal data, and turn length and form from real counseling dialogues, yielding 7,749 five-session trajectories from three LLMs. Every session is generated to express a preset score on the PHQ-8, a standard eight-symptom depression questionnaire. Validation shows that the labels are self-consistent and the dialogues close to real counseling. Following clinical research on treatment monitoring, we propose a trend-based evaluation protocol reflecting real-world tracking needs: changes between sessions, crossings of the screening cutoff, trajectory types, and symptom-level changes. Extensive experiments show all five existing methods tested fall short in tracking trends: none estimates changes better than a no-change baseline, at most 40% of cutoff crossings are detected, and adding earlier sessions lowers change error but causes more missed improvements.