Multi-Layer Lighting-Aware Style Transfer with Guidance Map Transformation

Muhammad Arief; Hideki Todo; Koji Mikami; Mochamad Hariadi

doi:10.48084/etasr.13520

Authors

Muhammad Arief Department of Media Science, Tokyo University of Technology, Hachioji, Tokyo, Japan
Hideki Todo Department of Computer Science, Faculty of Engineering, Takushoku University, Tokyo, Japan
Koji Mikami Department of Media Science, Tokyo University of Technology, Hachioji, Tokyo, Japan
Mochamad Hariadi Department of Computer Engineering, Institut Teknologi Sepuluh Nopember Surabaya, Indonesia

Volume: 15 | Issue: 6 | Pages: 30144-30153 | December 2025 | https://doi.org/10.48084/etasr.13520

Received: 19 July 2025 | Revised: 26 August 2025 | Accepted: 6 September 2025 | Online: 8 December 2025

Corresponding author: Muhammad Arief

Abstract

The present study addresses the challenge of animating high-frequency stylized details without introducing deforming artifacts or losing lighting impressions. While Lit-Sphere reliably produces coherent low-frequency shading, it often distorts the fine patterns in 3D animations. Conversely, StyleBlit can transfer high-frequency style elements but fails to animate the dynamic lighting effects. The proposed hybrid pipeline combines Lit-Sphere for low-frequency shading with a patch-based transfer inspired by StyleBlit. To achieve the lighting control, common normal maps are employed as guidance, enabling practical lighting control through normal map transformations. Additionally, local editing, posterization, and normal map smoothing are investigated to expand the range of achievable animation effects.

Keywords:

rendering, non-photorealistic rendering, style transfer, lit-sphere, patch-based transfer

Downloads

Download data is not yet available.

References

D. Prasetyo, N. Ramadhani, M. Hariadi, and I. R. Mutiaz, "Lighting Dynamics for Emotional Perception: A Technology-Driven Virtual Simulation Approach," Engineering, Technology & Applied Science Research, vol. 15, no. 2, pp. 22196–22202, Apr. 2025. DOI: https://doi.org/10.48084/etasr.10434

P.-P. J. Sloan, W. Martin, A. Gooch, and B. Gooch, "The lit sphere: a model for capturing NPR shading from art," in Proceedings of Graphics Interface 2001, CAN, June 2001, pp. 143–150.

D. Sýkora et al., "StyleBlit: Fast Example-Based Stylization with Local Guidance," Computer Graphics Forum, vol. 38, no. 2, pp. 83–91, 2019. DOI: https://doi.org/10.1111/cgf.13621

J. Mitchell, M. Francke, and D. Eng, "Illustrative rendering in Team Fortress 2," in Proceedings of the 5th international symposium on Non-photorealistic animation and rendering, New York, NY, USA, Aug. 2007, pp. 71–76. DOI: https://doi.org/10.1145/1274871.1274883

P. Barla, J. Thollot, and L. Markosian, "X-toon: an extended toon shader," in Proceedings of the 4th international symposium on Non-photorealistic animation and rendering, New York, NY, USA, June 2006, pp. 127–132. DOI: https://doi.org/10.1145/1124728.1124749

C.-R. Yan, M.-T. Chi, T.-Y. Lee, and W.-C. Lin, "Stylized Rendering Using Samples of a Painted Image," IEEE Transactions on Visualization and Computer Graphics, vol. 14, no. 2, pp. 468–480, Mar. 2008. DOI: https://doi.org/10.1109/TVCG.2007.70440

C. D. Kulla, J. D. Tucek, R. J. Bailey, and C. M. Grimm, "Using texture synthesis for non-photorealistic shading from paint samples," in 11th Pacific Conference onComputer Graphics and Applications, 2003. Proceedings., Oct. 2003, pp. 477–481. DOI: https://doi.org/10.1109/PCCGA.2003.1238298

L. A. Gatys, A. S. Ecker, and M. Bethge, "Image Style Transfer Using Convolutional Neural Networks," in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016, pp. 2414–2423. DOI: https://doi.org/10.1109/CVPR.2016.265

J. Liao, Y. Yao, L. Yuan, G. Hua, and S. B. Kang, "Visual Attribute Transfer through Deep Image Analogy." arXiv, June 06, 2017. DOI: https://doi.org/10.1145/3072959.3073683

H. Huang et al., "Real-Time Neural Style Transfer for Videos," in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017, pp. 7044–7052. DOI: https://doi.org/10.1109/CVPR.2017.745

A. Šubrtová, M. Lukáč, J. Čech, D. Futschik, E. Shechtman, and D. Sýkora, "Diffusion Image Analogies," in ACM SIGGRAPH 2023 Conference Proceedings, New York, NY, USA, July 2023, pp. 1–10. DOI: https://doi.org/10.1145/3588432.3591558

J. Fišer et al., "StyLit: illumination-guided example-based stylization of 3D renderings," ACM Trans. Graph., vol. 35, no. 4, July 2016, Art. no. 92:1-92:11. DOI: https://doi.org/10.1145/2897824.2925948

A. Hertzmann, C. E. Jacobs, N. Oliver, B. Curless, and D. H. Salesin, "Image analogies," in Proceedings of the 28th annual conference on Computer graphics and interactive techniques, New York, NY, USA, Aug. 2001, pp. 327–340. DOI: https://doi.org/10.1145/383259.383295

P. Bénard et al., "Stylizing Animation By Example," ACM Transactions on Graphics, vol. 32, no. 4, 2013, Art. no. 119:1. DOI: https://doi.org/10.1145/2461912.2461929

Q. Zheng, Z. Li, and A. Bargteil, "Learning to Shadow Hand-drawn Sketches." arXiv, Apr. 02, 2020. DOI: https://doi.org/10.1109/CVPR42600.2020.00746

L. Zhang, E. Simo-Serra, Y. Ji, and C. Liu, "Generating Digital Painting Lighting Effects via RGB-space Geometry," ACM Trans. Graph., vol. 39, no. 2, Feb. 2020, Art. no. 13:1-13:13. DOI: https://doi.org/10.1145/3372176

H. Todo, K. Anjyo, and S. Yokoyama, "Lit-Sphere extension for artistic rendering," The Visual Computer, vol. 29, no. 6, pp. 473–480, June 2013. DOI: https://doi.org/10.1007/s00371-013-0811-7

M. Arief, H. Todo, K. Mikami, and K. Kondo, "Region Control in Stylized Shading Using Radial Transformation within Texture Projection," IIEEJ Transactions on Image Electronics and Visual Computing, vol. 7, no. 1, pp. 36–45, 2019.

M. Ashikhmin, "Synthesizing natural textures," in Proceedings of the 2001 symposium on Interactive 3D graphics, New York, NY, USA, Mar. 2001, pp. 217–226. DOI: https://doi.org/10.1145/364338.364405