Multi-Object Reward-Guided Optimization for Empty-Room-to-Furnished-Scene Image Editing

TY - JOUR

T1 - Multi-Object Reward-Guided Optimization for Empty-Room-to-Furnished-Scene Image Editing

AU - Zhang, Naidan

AU - Aliabadi, Mehri R.

AU - Shih, Frank Y.

N1 - Publisher Copyright: © 2026, World Scientific Publishing Company.

PY - 2026

Y1 - 2026

N2 - Recent instruction-guided image editing models support flexible single-object manipulation, but multi-object editing in one scene remains challenging. In empty-room-to-furnished-scene editing, we aim to insert all requested furniture items, preserve source identity, control spatial placement, and maintain plausible room layout. Existing multi-reference editors often treat furniture references as a holistic cue, causing missing objects, identity drift, counting errors, unrealistic placement, and background damage. In this paper, we propose a novel method of Multi-Object Reward-Guided Optimization (MORGO), which formulates the task as multi-object furniture fusion. It builds a reference-guided synthetic dataset with empty rooms, moodboards, instructions, pseudo-furnished targets, and object-level metadata for source tracing. Furthermore, we develop a source-aware Visual Question Answering (VQA) evaluator for furniture-level rewards, supported by classical image-similarity metrics as an auxiliary audit. This evaluation is further converted into a reward signal for Direct Preference Optimization (DPO) fine-tuning. We have built a dataset, which contains 5000 images of furniture, varying randomly from 1 to 9 objects in an image. Experimental results show that our method achieves the evaluation metrics (Formula presented), that outperforms the existing methods of Qwen base and OmniGen2.

AB - Recent instruction-guided image editing models support flexible single-object manipulation, but multi-object editing in one scene remains challenging. In empty-room-to-furnished-scene editing, we aim to insert all requested furniture items, preserve source identity, control spatial placement, and maintain plausible room layout. Existing multi-reference editors often treat furniture references as a holistic cue, causing missing objects, identity drift, counting errors, unrealistic placement, and background damage. In this paper, we propose a novel method of Multi-Object Reward-Guided Optimization (MORGO), which formulates the task as multi-object furniture fusion. It builds a reference-guided synthetic dataset with empty rooms, moodboards, instructions, pseudo-furnished targets, and object-level metadata for source tracing. Furthermore, we develop a source-aware Visual Question Answering (VQA) evaluator for furniture-level rewards, supported by classical image-similarity metrics as an auxiliary audit. This evaluation is further converted into a reward signal for Direct Preference Optimization (DPO) fine-tuning. We have built a dataset, which contains 5000 images of furniture, varying randomly from 1 to 9 objects in an image. Experimental results show that our method achieves the evaluation metrics (Formula presented), that outperforms the existing methods of Qwen base and OmniGen2.

KW - controllable generation

KW - instruction-guided image editing

KW - Multi-image fusion

KW - preference optimization

KW - scene synthesis

KW - visual grounding

UR - https://www.scopus.com/pages/publications/105050282374

UR - https://www.scopus.com/pages/publications/105050282374#tab=citedBy

U2 - 10.1142/S021800142659041X

DO - 10.1142/S021800142659041X

M3 - Article

SN - 0218-0014

JO - International Journal of Pattern Recognition and Artificial Intelligence

JF - International Journal of Pattern Recognition and Artificial Intelligence

M1 - 2659041

ER -