Bilkent University
Department of Computer Engineering

---

CS 590/690 SEMINAR

 

Refedit: Zero Shot Reference-Based Editing with Text-to-Image Diffusion Models

 

Arda Göktoğan
Master Student
(Supervisor: Asst.Prof. Ayşegül Dündar Boral)

Computer Engineering Department
Bilkent University

Abstract: Image editing, especially the task of adding objects to existing scenes, requires a delicate balance between preserving the integrity of the original image and ensuring the new elements blend seamlessly. While advancements in diffusion models have made significant strides in improving image quality and stability, challenges remain, particularly when it comes to integrating objects in specific contexts, such as placing eyeglasses on a person's face. Pre-trained diffusion models often face difficulties in maintaining coherence across the entire scene and preventing unintended alterations to irrelevant areas. Furthermore, tasks that require reference-based editing—where a specific object must be incorporated—pose additional complexities. In this work, we introduce a novel method that combines reference images with textual prompts, enabling more precise and contextually aware image edits. This approach allows for highly flexible, realistic edits using pre-trained models, all without the need for additional training or task-specific datasets. By leveraging both visual and descriptive inputs, we address key limitations of existing methods and offer an efficient solution for realistic, reference-based image editing.

 

DATE: November 10, Monday @ 15:50 Place: EA 502