Integrating aerial imagery-based scene generation into applications like autonomous driving and gaming enhances realism in 3D environments, but challenges remain in creating detailed content for occluded areas and ensuring realtime, consistent rendering. In this paper, we introduce Skyeyes, a novel framework that can generate photorealistic sequences of ground view images using only aerial view inputs, thereby creating a ground roaming experience. More specifically, we combine a 3D representation with a view consistent generation model, which ensures coherence between generated images. This method allows for the creation of geometrically consistent ground view images, even with large view gaps. The images maintain improved spatial-temporal coherence and realism, enhancing scene comprehension and visualization from aerial perspectives. To the best of our knowledge, there are no publicly available datasets that contain pairwise geo-aligned aerial and ground view imagery. Therefore, we build a large, synthetic, and geo-aligned dataset using Unreal Engine.

Creating large-scale, high-quality 3D simulation environments is crucial for applications like autonomous driving, gaming, and robotics. However, traditional methods in the gaming industry often rely on labor-intensive handcrafting, which is both time-consuming and costly, limiting their scalability and realism in depicting real-world landscapes. Aerial imagery plays a significant role in addressing this challenge due to its wide coverage and ease of acquisition. It provides a practical resource for generating largescale 3D terrains and environments. However, transforming aerial views into accurate ground-level views remains a complex problem due to the significant differences between aerial and ground perspectives.

We utilized two distinct scene simulation platforms, CARLA Simulator [5] and CitySample [6] from Unreal CARLA Simulator is an open-source platform designed for the development, training, and validation of autonomous driving systems. The sequences are extracted from varies maps including Town01, Town02, Town03, Town04 and Town05 where we manually locate the start and end point of each lane (See Fig. 3a for an example of lane selection of Town 01). CitySample project is created by Ubisoft in Unreal Engine 5. We selected the smaller city level within this project for data extraction. Given the expansive scale of the map, we segmented it into multiple regions (refer to 3b for details) and appoint multiple lanes inside a region (refer to 3c for details). Please refer to our supplementary materials for a more detailed description of data collection process.

Qualitative Comparison We present the visual performance comparison of the baselines and our method in Figure 5, showcasing the effectiveness of each approach in rendering visually realistic terrain. As observed, SuGaR, while rendering geometry and color accurately, results in a somewhat blurred image, primarily due to splatting effects when viewed from this extreme perspective, especially in comparison to aerial views. ControlNet appears to produce photorealistic images with less artifacts, their textures are significantly different from the ground truth images. Compared with GVG, our method produces less artifacts and largely maintain intra-frame consistency.