publication . Preprint . 2020

Layout Generation and Completion with Self-attention

Gupta, Kamal; Achille, Alessandro; Lazarow, Justin; Davis, Larry; Mahadevan, Vijay; Shrivastava, Abhinav;
Open Access English
  • Published: 25 Jun 2020
Abstract
We address the problem of layout generation for diverse domains such as images, documents, and mobile applications. A layout is a set of graphical elements, belonging to one or more categories, placed together in a meaningful way. Generating a new layout or extending an existing layout requires understanding the relationships between these graphical elements. To do this, we propose a novel framework, LayoutTransformer, that leverages a self-attention based approach to learn contextual relationships between layout elements and generate layouts in a given domain. The proposed model improves upon the state-of-the-art approaches in layout generation in four ways. Fi...
Subjects
free text keywords: Computer Science - Computer Vision and Pattern Recognition, Computer Science - Machine Learning
Related Organizations
Download from
49 references, page 1 of 4

1. Ashual, O., Wolf, L.: Specifying object attributes and relations in interactive scene generation. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 4561{4569 (2019) [OpenAIRE]

2. Biederman, I.: On the semantics of a glance at a scene. In: Perceptual organization, pp. 213{253. Routledge (2017) [OpenAIRE]

3. Brock, A., Donahue, J., Simonyan, K.: Large scale gan training for high delity natural image synthesis. arXiv preprint arXiv:1809.11096 (2018)

4. Capobianco, S., Marinai, S.: Docemul: a toolkit to generate structured historical documents. CoRR abs/1710.03474 (2017), http://arxiv.org/abs/1710.03474 [OpenAIRE]

5. Chang, A.X., Monroe, W., Savva, M., Potts, C., Manning, C.D.: Text to 3d scene generation with rich lexical grounding. CoRR abs/1505.06289 (2015), http:// arxiv.org/abs/1505.06289

6. Chen, Q., Koltun, V.: Photographic image synthesis with cascaded re nement networks. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 1511{1520 (2017)

7. Chen, X., Shrivastava, A., Gupta, A.: Neil: Extracting visual knowledge from web data. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 1409{1416 (2013)

8. Deka, B., Huang, Z., Franzen, C., Hibschman, J., Afergan, D., Li, Y., Nichols, J., Kumar, R.: Rico: A mobile app dataset for building data-driven design applications. In: Proceedings of the 30th Annual Symposium on User Interface Software and Technology. UIST '17 (2017)

9. Dinh, L., Sohl-Dickstein, J., Bengio, S.: Density estimation using real nvp. arXiv preprint arXiv:1605.08803 (2016) [OpenAIRE]

10. Dong, H., Yu, S., Wu, C., Guo, Y.: Semantic image synthesis via adversarial learning. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 5706{5714 (2017)

11. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in neural information processing systems. pp. 2672{2680 (2014)

12. Gupta, T., Schwing, A., Hoiem, D.: Vico: Word embeddings from visual cooccurrences. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 7425{7434 (2019) [OpenAIRE]

13. Hinz, T., Heinrich, S., Wermter, S.: Generating multiple objects at spatially distinct locations. CoRR abs/1901.00686 (2019), http://arxiv.org/abs/1901.00686

14. Holtzman, A., Buys, J., Forbes, M., Choi, Y.: The curious case of neural text degeneration. arXiv preprint arXiv:1904.09751 (2019) [OpenAIRE]

15. Hong, S., Yang, D., Choi, J., Lee, H.: Inferring semantic layout for hierarchical textto-image synthesis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 7986{7994 (2018)

49 references, page 1 of 4
Abstract
We address the problem of layout generation for diverse domains such as images, documents, and mobile applications. A layout is a set of graphical elements, belonging to one or more categories, placed together in a meaningful way. Generating a new layout or extending an existing layout requires understanding the relationships between these graphical elements. To do this, we propose a novel framework, LayoutTransformer, that leverages a self-attention based approach to learn contextual relationships between layout elements and generate layouts in a given domain. The proposed model improves upon the state-of-the-art approaches in layout generation in four ways. Fi...
Subjects
free text keywords: Computer Science - Computer Vision and Pattern Recognition, Computer Science - Machine Learning
Related Organizations
Download from
49 references, page 1 of 4

1. Ashual, O., Wolf, L.: Specifying object attributes and relations in interactive scene generation. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 4561{4569 (2019) [OpenAIRE]

2. Biederman, I.: On the semantics of a glance at a scene. In: Perceptual organization, pp. 213{253. Routledge (2017) [OpenAIRE]

3. Brock, A., Donahue, J., Simonyan, K.: Large scale gan training for high delity natural image synthesis. arXiv preprint arXiv:1809.11096 (2018)

4. Capobianco, S., Marinai, S.: Docemul: a toolkit to generate structured historical documents. CoRR abs/1710.03474 (2017), http://arxiv.org/abs/1710.03474 [OpenAIRE]

5. Chang, A.X., Monroe, W., Savva, M., Potts, C., Manning, C.D.: Text to 3d scene generation with rich lexical grounding. CoRR abs/1505.06289 (2015), http:// arxiv.org/abs/1505.06289

6. Chen, Q., Koltun, V.: Photographic image synthesis with cascaded re nement networks. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 1511{1520 (2017)

7. Chen, X., Shrivastava, A., Gupta, A.: Neil: Extracting visual knowledge from web data. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 1409{1416 (2013)

8. Deka, B., Huang, Z., Franzen, C., Hibschman, J., Afergan, D., Li, Y., Nichols, J., Kumar, R.: Rico: A mobile app dataset for building data-driven design applications. In: Proceedings of the 30th Annual Symposium on User Interface Software and Technology. UIST '17 (2017)

9. Dinh, L., Sohl-Dickstein, J., Bengio, S.: Density estimation using real nvp. arXiv preprint arXiv:1605.08803 (2016) [OpenAIRE]

10. Dong, H., Yu, S., Wu, C., Guo, Y.: Semantic image synthesis via adversarial learning. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 5706{5714 (2017)

11. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in neural information processing systems. pp. 2672{2680 (2014)

12. Gupta, T., Schwing, A., Hoiem, D.: Vico: Word embeddings from visual cooccurrences. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 7425{7434 (2019) [OpenAIRE]

13. Hinz, T., Heinrich, S., Wermter, S.: Generating multiple objects at spatially distinct locations. CoRR abs/1901.00686 (2019), http://arxiv.org/abs/1901.00686

14. Holtzman, A., Buys, J., Forbes, M., Choi, Y.: The curious case of neural text degeneration. arXiv preprint arXiv:1904.09751 (2019) [OpenAIRE]

15. Hong, S., Yang, D., Choi, J., Lee, H.: Inferring semantic layout for hierarchical textto-image synthesis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 7986{7994 (2018)

49 references, page 1 of 4
Powered by OpenAIRE Research Graph
Any information missing or wrong?Report an Issue