I had the opportunity in October to attend Game Sound Con 2022 as a speaker for Mix Universe! The talk goes over how it all started and what I plan to do in the future. I highly recommend giving it a watch as it shows a lot of the thought processes and design challenges along the way!
The MetaSound that I will be talking about is in this video.
In short, my goal for this was to try out some newer 5.1 quality of life improvements for MetaSound and the grain delay node.
This link says it best.
“A DAW and a game audio engine serve totally different needs and have opposed technical constraints and use cases. A game audio engine is about building audio experiences with interactivity and runtime procedural generation as the core focus. “
MetaSound is Unreal’s new audio plugin in UE5 aiming to elevate functionality usually required by sound designers for games.
Here are 2 videos that will be really helpful in understanding what is going on here.
You can try this MetaSound out for yourself by downloading from gumroad!
https://chriszuko.gumroad.com/l/awgbr
The grain delay is a bit of a wild node. The biggest hurdle is giving it some audio and a trigger.
I ended up taking this pulse and delaying it further to offset when it decides to spawn grains.
This gave me a wild ethereal type reverb effect. For the audio going in, I needed to tune that a bit more.
I am filtering and compressing the audio in order to reduce artifacts that can happen when a sound gets a bit too loud inside of the grain delay. Some whacky things can happen.
I also realized quickly that transients from a kick, hi-hat, or snare weren’t going to work well for the kind of effect I was going for. It sounded jarring, so I made sure that the only audio going into the grain delay was the pad and the lead instruments.
The entire Grain delay setup looks like this (Again you can download this metasound above if you wanna poke around in it more)
I know this might seem overwhelming, but a lot of what I was doing was straight up experimenting, so I encourage you to try something out yourself and tinker with the MetaSound on your own!
Previously, I talked through the entire process here to get to a demo video that was released about a year ago showing what I was calling at the time “Project Mix”. This has now turned into a full musical sandbox game that I am calling Mix Universe!
Link to that article. https://chriszuko.com/project-mix-current-progress/
A musical sandbox like no other where you connect nodes together to make a truly spectacular audio visual experience.
I started this thing with very simple beginnings and I am excited to talk about that more in-depth in the future. This post is mostly just to get things up to date here on my main site.
I want to get this thing out there in early access so folks can enjoy it without worrying about features completely destroying everything they’ve made. Early access launch will allow this.
Some fun screenshots
Thanks to everyone who has supported me along the way! I am excited for what the future may hold here and am glad to have made it this far in development!
If you’ve started to dive into making systems for your game using MetaSounds, you may have found yourself with some minor hitches or latency when playing the sounds.
These tips might help!
When this is on, it reduces the CPU cost during play, however, it can lead to latency with larger MetaSounds. It might be worth turning it off (Mix Universe has it off to ensure that a sound will always play at the right time)
au.MetaSound.EnableAsyncGeneratorBuilder 0
I’ve found 28 is a sweet spot for latency and performance for lower end machines. Lower numbers will increase latency, but decrease CPU usage. Higher numbers will decrease latency, but increase CPU usage. Default is 100, and this is described in code as “blocks per second” when processing the audio for a MetaSound.
au.MetaSound.BlockRate 28
PCM is uncompressed and has a high memory footprint, but will result in the fastest audio playback possible.
ADPCM is 4x compression which may cause artifacts, but typically is ok for sounds that don’t have a ton of high frequency detail.
What these formats avoid is any types of decoders which will cause CPU performance overhead and latency. (Especially on something like the Quest)
You can monitor this using stat audio
Here is a pretty complex MetaSound I am using for example. I have found around 30 is the sweet spot right before it starts to cause real issues. It may vary depending on the input type, but this was my findings with ints and floats.
Right now, the only real way to improve this is to just simply reduce the number of inputs you are sending to a MetaSound. It doesn’t really matter which ones are sent during play.
Similarly, any nodes that use a large buffer size or a copy of a previous audio buffer can add up really quickly. Reducing lookahead time helps for compressors and also just in general, not using too many of these in your graph. I’ve found 1 of each is fine, but if you start pushing it too far it will start to show when playing your sound!
As MetaSound evolves, you will notice diamond shape connection points. These are constructor pins and are very important for performance. They are only ever evaluated once and cannot change with inputs, so that means they can be optimized to be super cheap. In the case of a delay, this allows you to set the max allowed delay time to limit the audio buffer allocated for the delay form ever going over a certain length. Smaller numbers in this make the node much cheaper.
Then it is time to dive in deeper. There are many ways to do this, but I personally get the most info initially from doing a simple CPU profile using stat startfile and stat stopfile
stat startfile stat stopfile
Then using the unreal frontend to see what is happening when a hitch occurs!
If you are new to performance profiling, I would highly recommend this talk as it goes over the basics on the thought process when it comes to solving a performance problem!
Then finally, if that isn’t enough, you can look into Unreal Insights a bit more in-depth which will give you the most detail about what is happening to your frames!
I hope this helps and you can continue making awesome MetaSounds in UE5!