부산대학교 도서관

국외단행본

Disentangled Visual Generative Models [electronic resource].

University of California, Berkeley. Computer Science.

[S.l.] : University of California, Berkeley. 2024 Ann Arbor : ProQuest Dissertations & Theses , 2024

1 online resource(116 p.)

Source: Dissertations Abstracts International, Volume: 86-04, Section: B.
Advisor: Efros, Alexei A.

Thesis (Ph.D.)--University of California, Berkeley, 2024.

Generative modeling promises an elegant solution to learning about high-dimensional data distributions such as images and videos - but how can we expose and utilize the rich structure these models discover? Rather than just drawing new samples, how can an agent actually harness p(x) as a source of knowledge about how our world works? This thesis explores scalable inductive biases that unlock a generative model's disentangled understanding of visual data, enabling much richer interaction and control as a result.First, I propose a representation of scenes as collections of feature "blobs", where a generative adversarial network (GAN) learns - without any labels - to bind each blob to a different object in the images it creates. This allows GANs to more gracefully model compositional scenes, in contrast to typical unconditional models which are constrained to highly-aligned single-object data. The trained model's representation can easily be modified to counterfactually manipulate objects in both generated and real images. Next, I consider methods that do not impose bottlenecks on architectures during training, facilitating their application to more diverse, uncurated data. I show that the internals of diffusion models can be used to meaningfully guide generation of new samples, without any further fine-tuning or supervision. Energy functions derived from a small set of primitive properties of denoiser activations can be combined to impose arbitrarily complex conditions on the iterative diffusion sampling procedure. This allows for control over attributes such as the position, shape, size, and appearance of any concept that can be described in text.I also demonstrate that the distribution learned by a text-to-image model can be distilled to generate compositional 3D scenes. Predominant approaches focus on creating 3D objects in isolation rather than scenes with several entities interacting. I propose an architecture that, when optimized so its outputs are on-manifold for the image generator, creates 3D scenes decomposed into the objects they contain. This provides evidence that scale alone suffices for a model to infer the actual 3D structure latent to a world it observes only through 2D images.Finally, I conclude with a perspective on the interplay between emergence, control, interpretability, and scale, and humbly attempt to relate these themes to the pursuit of intelligence.

9798384447214