Unity Asset Management On Steroids

“Show me all car models I own”

19 min readMar 22, 2024

If you are working with Unity, you will in many cases utilize assets you did not create yourself to speed up your development process. It can quickly become a challenge to find exactly what you need. Especially if you own many assets it is nearly impossible to answer seemingly simple questions like the one above.

I spent the last two years working on this very topic while creating my own asset management solution called Asset Inventory. This post will answer the above and many more related questions and illustrate how to get there, and I will take you along on the journey. I will show specific problems, the technical foundation and also whenever I succeeded, how I solved them.

Formats

Assets come in various forms. They can be models, shaders or textures for your graphics, scripts for additional logic, tooling for a better editor experience, sound files for effects and music, even tutorials or documents and basically anything that can be helpful.

Since single files alone often do not make sense but have dependencies or are provided in bundles, Unity makes use of two different packaging formats:

Files ending with the .unitypackage extension, which are basically tar archives that contain a very specific and rather cryptic folder structure.

Asset packages as stored in the file system

Packages that are referenced from a remote server (a registry) via name and version. Registry packages usually only contain logic and no reusable art or music assets since they are not immediately visible in the editor, but there are hybrid forms like the newer Meta XR packages as well. Their big advantage is that they are strictly versioned and switching between versions is just a click. They are not materialized in the project folder but only linked to, not cluttering the project.

Sources

Unity asset packages are most commonly downloaded from the Unity Asset Store (free or after purchase) but there are also third party stores like Synty, and users can also export their own packages for easier reuse between projects.

Registry packages most commonly come from the central Unity registry but everybody can setup a remote registry to use which is then included as a scoped registry. Also pointing to public package.json, e.g. Github repositories, or even a local one is easily possible.

Example showing Unity, third party scoped registry, web and local references in action

In addition Unity developers often have media libraries on their hard drives with textures and sound files from the Internet or various sources, which can directly be drag and dropped into the editor. Sometimes these are also zipped to preserve storage space.

Some of my rather random folders with assets

Dilemma

The result of this is that you end up with a list of packages. The metadata for both types differs but commonly they have a name, description, release notes, publisher info and tags. The Asset Store also supports media like screenshots and video links the authors can provide while registry packages expose dependencies in addition.

The package list as shown in the Unity Package Manager (2023.2)

This simple list is typically not enough to know what exactly is contained in each. Also assets you downloaded from somewhere else don’t show up. This typically leads to the situation that one imports many packages and then painstakingly cherry-picks assets to use. The unused assets either remain in the project, progressively clutter it, slow it down with additional recompilation times and increase the size often dramatically. One can then use specialized asset clean-up utilities wich try to identify unused assets and remove them, hopefully without affecting the project negatively.

→ Doesn’t that process sound utterly wrong though?

Do we actually care about packages? Or do we rather care about assets? And it is not really important in which packages these assets are contained. So here is the big question:

→ Why are packages our entry point at all?

It just doesn’t make sense from an artistic or time-efficiency perspective. Why spend time sifting through package after package to find something? And also financially it can be disadvantageous because you might buy something although you already own perfectly fitting assets hidden in another package.

A New Hope

Being quite an asset hoarder myself, owning more than 2000 packages by now, this existing process has always frustrated me. And one day, two years ago, I decided to finally do something about it. I started with:

→ How would my ideal asset workflow look like?

One alternative could be to buy only individual assets like single textures. This would lead to many micro-packages existing that contain exactly one thing so nothing is hidden anywhere. For all kinds of logistical and financial reasons this is a impractical, so I actually don’t mind the current packaged way, being able to buy many sounds and icons for example in bulk for a reduced price. That is nice.

So if the purchase experience stays the same, the search experience needs to change. And here comes the big thought:

→ What, if we could search by asset, instead of by package. What would that entail?

Step 1 — An Index

From a search perspective, we would need an index containing all assets, with appropriate metadata to filter for. So I created one. My approach:

Unity already stores all downloaded packages (both assets and registry) in special cache folders

.unitypackage files are just tar archives which can easily be extracted

Root folder of extracted package contents, one directory per file

All files are stored in individual folders directly under the root folder, containing the asset itself, a file containing the actual target file path and name, the meta file and in most cases a preview image

I decided to use SQLite which is a great lightweight database format that can easily handle millions of entries. I created two initial tables: Assets and AssetFiles. Assets contains the package information and the other the list of linked asset files. Through a Unity editor script I iterate over all existing packages in the cache, extract them and store the information in the database.

Using a new dedicated search UI the magic can now begin. Type in: “car”. And voila. (Quick side-note: screenshots shown here are not historically accurate but taken from the current version)

Asset-centric view across six different packages

The above result I basically had after a week of work, catering for some first special cases (of which there are MANY) and I was really happy. So I polished the state up, added a first documentation and created my first ever Asset Store package. I’ll write a separate post about this experience which I believe can be really insightful for aspiring Asset Store publishers. Basically after a month it was live and reception was tremendous.

“This should be already integrated into the editor”, “Transformative Efficiency”, “Phenomenal!”, “Witchcraft”, “Better Asset Manager!”

It seemed to have hit a nerve and provide something really useful that didn’t exist before. Besides that emotional uplift, also many users and tons of feature requests started coming in. So this is what I did the last two years. Eleven major updates since the initial release and now a version 2 which takes everything to the next level. I have the best community ever and loved every step along the way. But step by step:

Step 2 — More Sources

In order to index something, it needs to be available in the local Unity cache. Especially if you own many assets or purchase new ones there will be gaps in the index until you download these. Packages on the Asset Store are encrypted so downloading them manually is not the solution. Lucky enough there is a way to trigger a download through Unity which will cause it to download and decrypt. I added a way to do so and delete the downloaded file afterwards again to save space so that indeed every asset is correctly indexed. This ensures 100% of assets from the Asset Store are indexed.

Dealing with free-floating assets required a different approach. I created three different special-purpose indexers:

Custom Packages: Scans for any .unitypackage files and indexes the content. The initial drawback was that the nice metadata from the Asset Store (which I download via API) like publisher info, media files etc. were missing. Luckily there are two solutions for this. First, many packages actually contain a header stating their Id and some basic metadata. Using this Id and the Asset Store API one can then download the additional metadata. That left the question what to do with packages that don’t have a header but are also available on the Asset Store, just purchased from somewhere else? I decided the easiest would be to manually enter an Id (as contained in the Asset Store URL like https://assetstore.unity.com/packages/tools/utilities/asset-inventory-226927) to connect them this way.

Media Files: Scans for images, sound files and 3d models
Archives: Scans for .zip files and indexes their content, using the media files indexer

Configuring additional folders to scan and their importers

This dealt nicely with all non-Unity managed files so at this point we do have an Asset Manager that can do a 100% index of everything purchased and everything stored locally.

An unexpected issue comes into play with Unity 2022+ where it introduced the option to specify custom cache locations, without exposing the location actually through an API (to this day). Luckily they do have some private functions to gather this info and the power of reflection is strong with us.

Step 3 — A Better Index

Having everything in a database now gives us the chance to store more information than just the filenames. Wouldn’t it be great to know the dimensions of an image and even be able to filter for these? Or the length of sound clips to easily find short foot steps or longer music pieces? So I extended the indexers to also extract this information. A nice community suggestion was to filter for color as well, making it easy to find specific styles.

Advanced filters and color search in action

Speaking of audio, effectively going through a big audio library can be quite a challenge. I added auto-play and optional looping features upon selection making it really fast to identify audio. Scrubbing helps navigating inside big files.

The next item to tackle was how to actually search. Typing in “car” will also return items like “cards”. So now there are two modes to search even better:

“+car -cards”: use plus and minus to state what must and must not occur, or ~ to force a literal search, otherwise the words can appear in any order
“=” at the beginning to initiate an expert search which allows to use SQL syntax directly, like “=AssetFile.Width > 3000 and AssetFile.FileName not like “%Normal%” and Asset.DisplayName like “%icon%””

Step 4 — A Better Import

At this state we can magically find anything we ever purchased. But what now? Once we find it, importing the whole package again just for this item seems really wrong, having come so far.

Thinking of the problem from a first principles perspective, it is like that: files like images and sound files can be imported on their own. But other files in Unity will have dependencies, like prefabs pointing to models and materials and animations and such. And materials will point to textures again…

I was thinking about a complicated technique to determine such dependencies but then realized an elegant short-cut: each asset in Unity has a corresponding .meta file next to it, typically hidden. This file contains all kinds of information and also a GUID to uniquely identify it. That is puzzle piece one.

The other is, that materials, prefabs etc. are just text files in yaml format (if serialized as text which is the default). And if a file has dependencies, the GUIDs of those other files will appear inside that file.

So all that actually needs to be done is traverse this file tree gathering all dependencies starting from the initial file. And then materialize this subset of files and copy it to the Unity project. And voila, we have a mechanism to get exactly what one needs, no clutter, no wasted space, no additional computation cycles. Determining script dependencies would require a different approach so I typically do not import scripts if not explicitly wanted by the user.

Let’s not rejoice just yet though. Unity throws a wrench into these nicely turning gears. Ever since the introduction of render pipelines imported assets will show up purple in case you are in a URP project and the materials are not immediately compatible. Either the packages provide a URP compatibility shader or you run the render pipeline converter which does a nice job of assigning new shaders and shuffling the textures around.

So after struggling with this issue for a long time I thought of a really simple experiment: what, if we just call the render pipeline converter automatically if in a URP project? And even if that goes a bit overboard because it will scan all project materials (there is no API to supply just one material) it is surprisingly effective and does the job.

Time for a new status check: we now have elegantly found individual files, selectively imported these into the project and made them compatible with the project for instant usage. Beau.ti.ful.

Step 5 — A Better Package Manager

Now that assets are mostly sorted out, let’s turn our eyes to managing packages. That is supposed to be the strength of the Unity Package Manager and admittedly with 2023+ it has become a lot better but it still leaves very much to be desired. Also, not many users are on 2023+.

Back to the drawing board, what features would I personally want for packages (and hint: consequently also implemented)?

Overview: the minimum should be a list, showing what is available. This list should be searchable and sortable, best by multiple criteria. And grouping would be great once we start thinking about bulk actions affecting more than one package. Also there should be an icon.
Bulk actions: When starting a new project, I often need a specific set of packages that I always import. That should be a one-click action. Also, when updates are available I want to download these to the cache all at once.

Bulk info & actions when multiple packages are selected

Tagging: I want to specify tags and apply them to my packages for easier grouping. And I want to bulk import all items in a tag group, e.g. the “essentials” group, at once.
Information: I’d love to see all available information for a package, like ratings, last update, description, media files, maybe even my purchase date and the price.

Package media files instantly visible and quickly browsable

All-in-one: The package list should contain all my purchases, all registry packages, all my custom packages, all my archives, media libraries etc. to give me a full overview of everything I have access to.
Importing: I would like to keep my Assets root folder clean and put all asset packages into a dedicated sub folder. When importing registry packages from a scoped registry I want that registry to be automatically added.
Intelligent updates: Registry packages receive a lot of updates. I’d love to have a single icon indicating that there is something new available which requires my attention. But I am for example not interested in preview versions. Or I am, but only for a specific package. I’d love to be able to specify an update strategy for each package and then I can trust the update indicator, that there indeed is something for me to check and update without wasting my time.

As a gimmick and asset hoarder myself it would be nice to know the net-worth of my asset library. There is no API I could find that will return the actual purchase price. Only the current price is available. Better than nothing though. This allows us to calculate the current value of all assets without discounts and bundles (for any selection).

Example breakdown of current asset value

Step 6— Better Preview Images

When I showed the package structure earlier we saw there was a preview image stored as well. Unity creates these very conveniently automatically for assets. A technical limit that Unity imposes is that these are limited to 128x128 pixels. This is ok’ish when browsing in the small project window but it does not allow to zoom in. Dimensions of around 256x256 would be preferable. This is one often requested enhancement by the community.

What alternatives are there? Simple upscaling looks very blocky. For image files that have higher resolutions than 128 we can actually render previews ourselves at the target size and replace the original ones leading to nice and crisp images. The drawback is that they consume more disk space. For my 1 million indexed files this amounts to 12Gb by now (instead of around 6Gb).

Other assets like prefabs are more complicated. These typically cannot be handled outside the Unity editor. Luckily enough there is the PreviewSceneStage API for creating preview images and using this way we could store also these images into a higher resolution render texture. This is utilized by a couple of other assets for different purposes and something I plan to adopt as well in the future.

Step 7 — Backup

Having most of the building stones for good asset management in place we can now look a bit broader. You might already have caught a distinction above when I talked about registry package versions and how easy it is to go back and forth between them as they are not materialized into the project itself.

This does not apply to the much bigger category of Asset Store packages though, which come as .unitypackage files and are basically copied 1:1 into the project. There is no concept in the Asset Store to go back to earlier versions. There is just latest (with the small distinction of supported Unity version that acts as an upper boundary) and whatever you have in your local cache. As soon as you download a new version, that old one is lost.

Packages also follow a lifecycle. There are 3 states (if you don’t count draft to which only the author has access). To illustrate the typical distribution a bit here the states and how many of my assets fall into each category:

published: 1598
deprecated: 313
disabled: 29

As soon as a package reaches the disabled state, a download will not be possible anymore and there will be no more way to get the package at all. New versions can also introduce bugs that can jeopardize a projects health or releasability.

The strategy I came up with to also support multi-versioning for asset packages is to copy them out of the cache and into another directory. This will require a significant amount of space, so it can be undesirable to backup every asset and also not every single version of an asset. One is probably only interested in the latest patch version and most likely only back a couple of minor versions.

In the folder screenshot above you can see that there is no version 11.3.0 for the Asset Store Publishing Tools since there is a higher patch version and the old one was cleaned up. Once this structure is in place it is easy to go back to specific versions. Since the backup folder is not used a lot it can be offloaded to a cheap, large and slow external drive.

Step 8 — Reverse Lookup

A question that often comes up for various reasons is to determine what assets are used in a given project. One might want to know which package a file originated from. Or what license that package has. Or compile a credits file for the current project in an enterprise environment requiring specific compliance rules (e.g. only using approved licenses).

There are a couple of approaches one can take here. A quick win is to utilize the GUIDs again. Since everything is indexed, we can do a GUID lookup in the database and find out which package it belongs to. Now, there are two issues:

It can belong to multiple packages, in case it is a common or shared file downloaded from the net or something a publisher reuses between packages.
We cannot derive the version from the GUID since GUIDs are typically kept identical between package versions.

To solve this we would somehow need to remember from which package the original file was installed from. We could for example store such information in the database or in a file next to the asset. But that would become messy. Interestingly enough also Unity realized this is a bad situation and with 2023+ nicely adds additional attributes into the meta files themselves to pinpoint the asset origin:

In an upcoming update I will incorporate these fields in case they exist.

Step 9 — Export

Exporting data allows you to do your own analysis and build additional tooling on top. There are three types of export an asset management tool should provide:

Asset Export: since packages come in a proprietary format, it should be possible to (also bulk) export textures and other common types so that these can be reused in other scenarios and also engines.

Asset exporter for selectable file type groups

Metadata Export: package information should be available as CSV to quickly import into Excel and other spreadsheets for analysis.

Raw Data Access: this is the most versatile access and can be utilized to create custom web frontends and the like (actually done by one University). SQLite is a broadly used database and interfacing with it is super easy, barely an inconvenience.

Step 10 — One Index, Multiple Devices

Having the index finally done after some hours of intense work for the computer, multiple users raised the question what to do if they utilize multiple devices or have other team members. Having everyone create their own index of the same data again does not really make sense and should be avoided.

A possible solution for this is using network shares or mounted drives and while that initially works, the biggest issue will be hardcoded paths inside the database. These are very convenient for processing but drive letters will probably be different on each computer so indexed files will not be found.

After a lot of tinkering, the best solution seemed to be to introduce the concept of keys, that encode a physical drive location. So instead of “c:/temp” it would just be “[temp]”. Then on each computer that key would need to be mapped one time to its concrete path on that specific device. I wrote a tool to convert from hard coded paths to keys and back conveniently and also track how many computers access the database for easy reference.

Conversion wizard for cross-device usage, [Pictures] already being relative

Step 11 — Third Party & API

The final piece of the asset manager of my dreams is reusability from other assets. Imagine you are providing a tool for the editor to create terrains. Your users will need to pick their textures, prefabs to spawn etc. This is a prime candidate to interface with such an asset management solution to quickly find and materialize anything you need without any clutter.

The best way to do so is to rely on an automatic define symbol for C#. This way you can write code which is only compiled if the asset is available, otherwise there will be no compile errors.

Example of API calls to the Asset Inventory

One very interesting capability that is possible when having a complete index of an asset is to derive complimentary assets. Imagine you want your users to select a texture to use in a terrain. You most likely have not only one but multiple slots, e.g. for albedo, normals and more. Wouldn’t it be great if users only needed to select one texture and the other slots are automagically filled? Since texture files follow some rough naming conventions I came up with a heuristic to do a parallel search and materialize and return the full set of complimentary textures in case they exist.

Example of API calls using texture heuristics

Closing & Outlook

This brings me to the end of my post. I hope it was interesting and you learned a thing or two about Unity and how it handles assets and packages. And I hope I could inspire you with a radical new asset workflow that does not use packages as the entry point anymore, but the actual content. It took a lot of hard work to get there, but it is truly transformative.

I have been using Asset Inventory nearly every day for the last two years now, either during development or when working on projects small and large. It’s the first thing I usually install and it saves me so much time.

With version 2 I have finally reached a state where I am confident to call it a Package Manager 2.0, something that surpasses the built in features of Unity on nearly every level and in many ways goes much deeper, while also working backwards down to Unity 2019.4.

It is one of my most ambitious projects I ever worked on. Is it finished? It has come a very long way, for sure. And it is battle-tested by a big community and production ready. The roadmap on Trello is getting smaller finally, but is still long and I am eager to tackle the next challenges.

If you want to engage or influence the journey please leave a comment or come to the great Discord community.

→ Visit Asset Inventory on the Asset Store