NeurIPS 2026 Evaluations & Datasets Track

OverLay++Dense-Overlap Layout-to-Image Generation Dataset

* Equal contribution

500KImages
6.6Objects per image
1.67×Annotation density
> 6×Object caption length
OverLay++ teaser: dense annotations, dataset statistics, and generated examples View full size ↗
Figure 1: OverLay++: A dense and overlap-aware dataset for Layout-to-Image generation. Left: OverLay++ provides dense object annotations with rich local captions for fine-grained spatial and attribute control. Center: Our dataset contains 500K images with 6.6 objects per image, yielding denser annotations and richer captions. Right: Training on OverLay++ improves generation quality in dense and overlapping scenes, producing better object fidelity and spatial alignment.

Video

Abstract

Layout-to-Image generation has made substantial progress in spatial and object-level control. However, existing methods still struggle with complex scenes containing many overlapping and interacting objects. We argue that training data is a particular bottleneck: existing datasets lack examples with dense, complex object interactions. To address this gap, we introduce OverLay++, a large-scale Layout-to-Image dataset with structurally complex scenes. OverLay++ contains approximately 500K images with an average of 6.6 objects per image, exceeding existing datasets by 1.67× in annotation density. Beyond annotation density, OverLay++ provides rich semantic detail with object captions over six times longer than in current datasets. Our dataset generation pipeline is simple and produces dense, overlapping object annotations with rich per-object captions. Across multiple benchmarks, state-of-the-art Layout-to-Image methods trained on the OverLay++ dataset show consistent improvement and faster convergence, demonstrating the importance of dense, overlap-aware, and caption-rich supervision for controllable image generation.

Method

We construct OverLay++ from the LAION-Aesthetics V2 4.75 subset using a four-stage pipeline: image filtering, object detection, scene filtering, and hierarchical captioning. The pipeline prompts a vision-language model to produce overlapping object proposals, then applies per-object grounded validation to keep only accurate detections. Each retained image receives short and long captions at both the scene and object levels.

Four-stage OverLay++ dataset generation pipeline View full size ↗
Figure 2: Dataset generation pipeline for the OverLay++ dataset. Four stages: (I) image filtering via aesthetics, resolution, and center cropping; (II) object detection for bounding boxes and labels; (III) scene filtering to remove low-quality or inconsistent samples, and (IV) hierarchical captioning producing scene-level and object-level captions (long and short).

Dataset Statistics

Dataset Images Objects ↑ IoU ↑ Dist ↓ Local short ↑ Local long ↑ Global short ↑ Global long ↑
LayoutSAM 2.66M 3.93 0.22 0.22 1.99 11.93 — 81.81
EliGen 496K 2.56 0.26 0.24 2.12 — 17.76 —
GRIT 20.5M 1.78 0.16 0.25 2.40 — 17.47 —
OverLay++ dataset 500K 6.59 0.31 0.17 15.89 89.70 19.98 52.24

Table 1: Dataset statistics for Layout-to-Image datasets. Our dataset has higher object density, stronger spatial interactions, and richer caption supervision than prior datasets. Caption lengths are measured in number of words. Bold and underline mark the best and second-best values in each column.

Key Results

Result 1 — Improved Visual Coherence and Object Fidelity

Models trained on the OverLay++ dataset produce more coherent and detailed images than those trained on prior datasets.

Qualitative comparison of models trained on EliGen and OverLay++ View full size ↗
Figure 3: Qualitative comparison of L2I generation. Models trained on OverLay++ produce more coherent and detailed images than those trained on prior datasets. Common failure cases of prior datasets include missing objects, missing attributes, and poor handling of dense object layouts.

Result 2 — Improved Attribute Alignment and Image Quality

On LayoutSAM-Eval, EliGen-SD3 trained on the OverLay++ dataset achieves state-of-the-art region-wise quality.

Model Spatial (%) ↑ Color (%) ↑ Texture (%) ↑ Shape (%) ↑ FID ↓ IS ↑
GLIGEN77.5349.4155.2952.7221.9220.57
Ranni41.3824.1025.5723.3527.2419.81
MIGC85.6666.9771.2469.0621.1919.65
InstanceDiff87.9969.1672.7871.0819.6720.02
Be Yourself53.9931.7335.2632.7528.1017.98
HiCo87.0469.1972.3671.1022.6120.15
SiamLayout-SD392.6774.4577.2175.9319.1022.04
SiamLayout-SD3 (OverLay++ dataset) 93.30 (+0.68%) 75.99 (+2.07%) 78.77 (+2.02%) 78.41 (+3.27%) 17.96 (-5.97%) 21.60
EliGen-SD393.0482.9686.7386.3525.1719.21
EliGen-SD3 (OverLay++ dataset) 95.45 (+2.59%) 90.03 (+8.52%) 93.14 (+7.39%) 92.81 (+7.48%) 19.61 (-22.09%) 19.54 (+1.72%)

Table 2: Results on LayoutSAM-Eval. Comparison of region-wise and global quality across different methods. Our methods consistently outperform their corresponding baselines, achieving substantial gains in region-wise quality while maintaining competitive global image quality. Improvements over baseline are shown in green, and bold and underline denote the best and second-best results among all compared methods.

Result 3 — Improved Spatial Alignment in Overlapping Layouts

Training on the OverLay++ dataset improves both EliGen-SD3 and SiamLayout-SD3 on OverLayBench benchmark across the Simple, Regular, and Complex splits.

Simple

Method mIoU ↑ O-mIoU ↑ SRE ↑ SRR ↑ CLIPG ↑ CLIPL ↑ FID ↓
GLIGEN60.5436.2249.9978.7234.1724.7531.27
InstanceDiff71.2149.9977.7187.4934.2527.6936.17
MIGC58.6432.1563.4181.6033.0726.4931.64
HiCo69.4747.2367.7586.0835.2527.0429.21
3DIS65.7538.3886.2486.9835.8529.6729.18
SiamLayout-SD358.7832.5272.3484.4537.2927.4927.51
SiamLayout-SD3 (OverLay++ dataset) 61.01 (+3.79%) 37.23 (+14.48%) 75.19 (+3.94%) 85.60 (+1.36%) 37.03 27.59 (+0.36%) 26.73 (-2.84%)
EliGen-SD366.6741.3485.7088.6737.0128.5730.29
EliGen-SD3 (OverLay++ dataset) 70.39 (+5.58%) 47.09 (+13.91%) 91.83 (+7.15%) 91.06 (+2.70%) 36.27 29.15 (+2.03%) 30.28 (-0.03%)

Regular

Method mIoU ↑ O-mIoU ↑ SRE ↑ SRR ↑ CLIPG ↑ CLIPL ↑ FID ↓
GLIGEN52.4626.5344.8877.4633.9323.4252.22
InstanceDiff60.0834.1572.5183.3633.0926.1959.73
MIGC47.4220.0656.6777.8532.7224.9954.24
HiCo55.0229.6058.2479.8933.9125.3449.07
3DIS55.6627.2980.8083.6935.4228.1248.56
SiamLayout-SD347.0420.6762.6078.3136.6725.5545.57
SiamLayout-SD3 (OverLay++ dataset) 48.41 (+2.91%) 23.32 (+12.82%) 65.53 (+4.68%) 80.39 (+2.66%) 36.49 25.66 (+0.43%) 45.37 (-0.44%)
EliGen-SD356.7929.2481.3483.5736.5427.2049.15
EliGen-SD3 (OverLay++ dataset) 62.40 (+9.88%) 36.66 (+25.38%) 89.45 (+9.97%) 86.75 (+3.81%) 35.64 28.17 (+3.57%) 49.09 (-0.12%)

Complex

Method mIoU ↑ O-mIoU ↑ SRE ↑ SRR ↑ CLIPG ↑ CLIPL ↑ FID ↓
GLIGEN50.7923.8541.7079.9333.9222.7557.32
InstanceDiff53.6825.6366.0280.3432.3325.5366.32
MIGC40.0413.2647.8074.4831.9324.2066.52
HiCo46.5620.3548.8875.1933.1524.4155.78
3DIS50.6521.7574.3181.5735.1127.3554.90
SiamLayout-SD344.2418.0552.1079.9836.5524.7653.29
SiamLayout-SD3 (OverLay++ dataset) 45.81 (+3.55%) 20.42 (+13.13%) 57.54 (+10.44%) 81.60 (+2.03%) 36.15 24.97 (+0.85%) 52.21 (-2.03%)
EliGen-SD350.5322.9773.7582.2236.4926.3253.58
EliGen-SD3 (OverLay++ dataset) 57.90 (+14.59%) 30.99 (+34.92%) 83.30 (+12.95%) 85.59 (+4.10%) 35.46 26.97 (+2.47%) 53.63

Table 3: Results on OverLayBench. Comparison of generation quality across different methods and complexity settings. Our methods consistently outperform their corresponding baselines, achieving substantial gains in region-aware metrics while maintaining competitive overall image fidelity. Improvements over the baseline are shown in green, and bold and underline indicate the best and second-best results among all compared methods.

Result 4 — Improved Generation Quality in Dense Scenes

Training on the OverLay++ dataset improves both EliGen-SD3 and SiamLayout-SD3 on DenseLayout benchmark.

Model mIoU (%) ↑ Color (%) ↑ Texture (%) ↑ Shape (%) ↑
EliGen-SD345.7937.6539.9839.76
EliGen-SD3 (OverLay++ dataset) 46.92 (+2.47%) 40.61 (+7.86%) 42.91 (+7.33%) 42.82 (+7.70%)
SiamLayout-SD315.5411.6912.3412.17
SiamLayout-SD3 (OverLay++ dataset) 17.10 (+10.04%) 13.15 (+12.49%) 13.77 (+11.59%) 13.61 (+11.83%)

Table 4: Results on DenseLayout. EliGen-SD3 and SiamLayout-SD3 trained on OverLay++ outperform their corresponding baselines on DenseLayout benchmark. Improvements over baselines are highlighted in green, and bold indicates the best results among all compared methods.

Result 5 — Accelerated Training Convergence

Training on the OverLay++ dataset accelerates convergence across all OverLayBench splits.

Convergence on the OverLayBench Simple split
(a) Simple
Convergence on the OverLayBench Regular split
(b) Regular
Convergence on the OverLayBench Complex split
(c) Complex

Figure 4: Convergence analysis. Comparison of mIoU and O-mIoU versus training steps for Simple, Regular, and Complex settings. The model trained on OverLay++ achieves higher scores across all difficulty levels and demonstrates stable convergence behavior.

Citation

If you find OverLay++ useful, please consider citing:

@misc{aggarwal2026overlaydenseoverlaplayouttoimagegeneration,
  title={OverLay++: Dense-Overlap Layout-to-Image Generation Dataset},
  author={Shivansh Aggarwal and Shresth Grover and Divyansh Srivastava and Haiyang Xu and Bingnan Li and Xiang Zhang and Ethan J. Armand and Chuan Li and Jianwen Xie and Zhuowen Tu},
  year={2026},
  eprint={2610.09071},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2610.09071},
}

Acknowledgment

This work is supported by U.S. National Science Foundation Award IIS-2433768 and IIS-2127544. This work also used Delta and DeltaAI at National Center for Supercomputing Applications (NCSA) through allocation CIS260179 from the Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support (ACCESS) program, which is supported by U.S. National Science Foundation grants #2138259, #2138286, #2138307, #2137603, and #2138296. We also thank Lambda, Inc. for their valuable compute resource help.