Automated Cleanup of Unity's Generated Shaders

TL;DR

For my work at Thunderful Development I made a neat little tool which uses regular expressions to simplify shader files generated from Unity's Shader Graph for the sake of readability. Unity asked me to publish it, I got permission from work to do so, and now it is available on GitHub.

If you want to know the technical details of how it works you should check out the examples listed on the GitHub page or perhaps even read the source code. If you want to know why I felt a need to write it and how I went about it, keep reading this article.

Introduction

Sometimes in my work I've had to optimize shaders which were originally created using Unity's Shader Graph. Shader Graph is a great tool which allows artists to create shaders without any programming knowledge. Unfortunately, these graphs are not the easiest format to work with when optimizing shaders. This then leads to cases where shaders are generated using Shader Graph, converted to a pure shader file, and then optimized.

Every time I've found myself optimizing such a shader I've made the same observations. Primarily that the code generated by Shader Graph is... not intended to be human readable. Thousands of lines consisting mainly of declarations of variables with automated names such as _Property_37502285_Out_0 followed by generated function calls obscuring even the simplest operation.

On top of this, it becomes hard to analyze the data flow when most variables are only used once. Even worse, every time an existing variable is passed to a subgraph it is done via a new alias only used for that one call. So even the values which are constantly reused get hard to follow.

As an example, here is a screenshot of a section from a randomly selected Shader Graph in the project I'm currently working on.

Example segment of a Shader Graph

Here is the shader code generated by Unity for this portion. 

 #if defined(KEYWORD_PERMUTATION_0) || defined(KEYWORD_PERMUTATION_1) || defined(KEYWORD_PERMUTATION_2) || defined(KEYWORD_PERMUTATION_3)
float _Property_89116ACD_Out_0 = _VertexOffsetAmount;
#endif
#if defined(KEYWORD_PERMUTATION_0) || defined(KEYWORD_PERMUTATION_1) || defined(KEYWORD_PERMUTATION_2) || defined(KEYWORD_PERMUTATION_3)
float _Multiply_602F0AD4_Out_2;
Unity_Multiply_float(_Remap_A65749B9_Out_3, _Property_89116ACD_Out_0, _Multiply_602F0AD4_Out_2);
#endif
#if defined(KEYWORD_PERMUTATION_0) || defined(KEYWORD_PERMUTATION_1) || defined(KEYWORD_PERMUTATION_2) || defined(KEYWORD_PERMUTATION_3)
float3 _Multiply_30325343_Out_2;
Unity_Multiply_float(IN.ObjectSpaceNormal, (_Multiply_602F0AD4_Out_2.xxx), _Multiply_30325343_Out_2);
#endif
#if defined(KEYWORD_PERMUTATION_0) || defined(KEYWORD_PERMUTATION_1) || defined(KEYWORD_PERMUTATION_2) || defined(KEYWORD_PERMUTATION_3)
float3 _Add_3A54B679_Out_2;
Unity_Add_float3(IN.ObjectSpacePosition, _Multiply_30325343_Out_2, _Add_3A54B679_Out_2);
#endif
#if defined(KEYWORD_PERMUTATION_0) || defined(KEYWORD_PERMUTATION_1) || defined(KEYWORD_PERMUTATION_2) || defined(KEYWORD_PERMUTATION_3)
float _Split_3764AD50_R_1 = IN.VertexColor[0];
float _Split_3764AD50_G_2 = IN.VertexColor[1];
float _Split_3764AD50_B_3 = IN.VertexColor[2];
float _Split_3764AD50_A_4 = IN.VertexColor[3];
#endif
#if defined(KEYWORD_PERMUTATION_0) || defined(KEYWORD_PERMUTATION_1) || defined(KEYWORD_PERMUTATION_2) || defined(KEYWORD_PERMUTATION_3)
float3 _Lerp_5F9C0AF5_Out_3;
Unity_Lerp_float3(IN.ObjectSpacePosition, _Add_3A54B679_Out_2, (_Split_3764AD50_R_1.xxx), _Lerp_5F9C0AF5_Out_3);
#endif

And here is what this section becomes after running my shader cleanup tool: 

#if defined(KEYWORD_PERMUTATION_0) || defined(KEYWORD_PERMUTATION_1) || defined(KEYWORD_PERMUTATION_2) || defined(KEYWORD_PERMUTATION_3)
float _Multiply_602F0AD4_Out_2 = _Remap_A65749B9_Out_3 * _VertexOffsetAmount;
float3 _Multiply_30325343_Out_2 = IN.ObjectSpaceNormal * (_Multiply_602F0AD4_Out_2.xxx);
float3 _Add_3A54B679_Out_2 = IN.ObjectSpacePosition + _Multiply_30325343_Out_2;
float3 _Lerp_5F9C0AF5_Out_3 = lerp(IN.ObjectSpacePosition, _Add_3A54B679_Out_2, (IN.VertexColor[0].xxx));
#endif

Still fairly dense with useless variable names, but a lot shorter and easier to read. Functionally, these are identical and will compile to the same instructions.

Background

The way Unity generates code from a Shader Graph is very straightforward and follows a few simple patterns. This means that if you can come up with general rules for how to rewrite them for legibility you could automate conversion of these generated files to some equivalent but slightly more readable format. I've had this thought now and again for the past few years, but whenever I was optimizing a specific shader it never quite seemed worthwhile. Until now.

While working on the porting team for Wavetale a large portion of my time was spent optimizing a few of the most expensive shaders in the game. Unfortunately, these were originally designed using Unity's Shader Graph and later converted to regular shaders.

One of the first things I tried to do was manually cleaning things up for readability. I analyzed the recurring patterns as I started from the top, and soon realized it wasn't feasible to go through and rewrite thousands of lines of generated code. Instead, I scrapped the idea and decided to only rewrite the most performance sensitive portions of the shader, using the slightly outdated original Shader Graph as a reference. This works well, given the straightforward translation from graph to code, but I spent a fair amount of time and effort trying to look past the dense yet lengthy format of the code.

A few months later, I returned to this shader after exhausting my list of obvious graphics-related optimizations in the game. I decided to take another look to see if I could spot some non-obvious improvement. It was at that point that I decided to see if I could clean up the entire file without having to do so manually. I revisited the list I'd made of all the patterns and potential replacements. Then, I made this happen using a combination of Vim macros and Vim's built-in regex search and replace command.

An hour and fifteen minutes later I had reduced the code length by 25% and the remaining code was much simpler to follow. With everything on a cleaner and more familiar format I immediately started to notice less obvious optimizations.

Introducing the tool

When I showed the resulting simplification to some coworkers, they asked how I had done it. I explained that it was a fairly mechanical process, mainly based on regex, and blurted out that it wouldn't be hard to make this into an automated tool. That thought stuck with me, and a few days later I sat down and wrote a simple editor script which takes a shader file as input and essentially runs an extended version of the regex replacements I'd used manually.

Shortly after this, someone suggested I contact Unity to ask if they could simply generate better code to begin with. I ended up doing so, including my tool in the ticket as an example of the type of improvements I desired. The main takeaway from their response was that they liked the tool and wanted me to publish it on GitHub.

After a few rounds of internal discussion I was given the green light to publish the tool and write this article introducing it. The code is available here.

The results

First and foremost, the resulting code is a lot easier to read, which makes it better to work with and optimize. While it does shorten the code, the main improvement is that it becomes much easier to grok.

The shaders I was working with when developing the tool contain a few thousand lines spread across a large number of subgraphs. For these, the results are a ~35% reduction in length. For other shaders I've typically seen a reduction of 20-25%.

Combining this with some manual steps such as removing unused passes from the generated file can reduce the length even further. The shader I spent the most time with was originally 6256 lines long. Removing unused passes brought it down to 3226. Running the current version of my script on this brings it down further to 2076 lines (a 36% reduction). After some manual optimizations the final version ended up being 1848 lines.

About the author

Hello,

My name is Daniel "Agentlien" Kvick and I'm a Software Engineer with a passion for games.
I currently work as a Graphics Programmer at Thunderful Development.

Here you'll find a selection of things I have worked on.