Bilkent University
Department of Computer Engineering

---

M.S.THESIS PRESENTATION

 

Audio-Language Multiple Instance Learning for Depression Severity Assessment

 

Uğur Can Altun
Master Student
(Supervisor: Assoc.Prof.Hamdi Dibeklioğlu )

Computer Engineering Department
Bilkent University

Abstract: Automatic assessment of depression severity has received substantial scientific attention due to its potential to support early diagnosis and intervention. This thesis presents a novel audio-language approach for estimating depression severity. The proposed method builds on prompt-conditioned audio-language representations, enriches them with emotion and semantic features through a novel cross-attention mechanism, and facilitates learning from small clinical datasets using a framework that combines multiple instance regression with ordinal contrastive learning to capture nuanced differences across PHQ-8 ranges. The main contributions of this thesis are as follows: (1) employing an audio-language model as a joint auditory-textual utterance embedding model; (2) introducing Multimodal Joint Gated Cross-Attention to integrate concepts that characterize complementary aspects of the same speech content; and (3) proposing Collectively Assumed Multiple Instance Regression (CA-MIR), a learning framework designed for limited clinical data, together with the Segment-Level Cross-Patient Ranking Contrastive (SCPR-Con) loss. Comprehensive experiments systematically evaluate the individual architectural and learning components. In addition, latent instance scores associated with sentences in the speech content are analyzed to provide insight into how acoustic and semantic cues contribute to the model’s predictions. Finally, the proposed method is compared with existing approaches and is shown to outperform prior work.

 

DATE: September 7, Monday @ 13:30

Place: EA 516