I had the opportunity in October to attend Game Sound Con 2022 as a speaker for Mix Universe! The talk goes over how it all started and what I plan to do in the future. I highly recommend giving it a watch as it shows a lot of the thought processes and design challenges along the way!
Mix Universe is still trucking along! You can check out the Mix Universe website for more details on the game!
What’s new and what is fixed?
- Patterns are upgraded now to allow them to chain when they aren’t set to looping.
- Patterns more clearly have start and end points indicated by spherical nodes that you can connect to for activation.
- Patterns now have randomization controls to allow for mixes to have much more variety
- All Nodes can adjust their height using Q and E which allows for a lot more visual flexibility when laying out mixes.
- New Rock Visualizer Node – Similar to the planet node, instead its a rock.
- New End Point Node – Stops the mix and shows stats on it.
- Every Mix now has an “Origin” layer which you can access in the lower left section of the edit panel. This layer controls which layers 1-6 are activated by default at the start of the mix.
- Layer Switchers now show visually which layer they are set to switch to.
- Headbob’s character model has been updated.
- The default space theme’s lighting and fog has been updated and particles for the main atmosphere have been fixed.
- Multi-Edit double click functionality has been added for Samples, Modulators, Link Duration, Note Duration, Note Octave, and Note Value.
- Onscreen UI limit has been increased from 10 to 32 to allow for up to 32 nodes to edit at once.
- Simple edit mode has been added which displays a much simpler “Selected” UI.
- Added mix history when mixes are uploaded to the store, now if someone is to save a mix off of someone elses mix, we can see where the original mix came from and who made it.
- Added a profile name you can change in the options menu if you don’t want to use your steam name for artist info on mixes.
- Added a mix artist overlay which shows title and artist names.
- Improved CPU performance when it comes to elements in the scene, still have a ways to go though for much mega mixes.
- Fixed issues with UI’s getting stuck during complex edits or longer mix sessions.
Not too much longer now!
I will be gearing up for an early access launch February 2023. This means that the playtest will be closed to any new players automatically in 2 weeks! Please jump in now if you wanna give feedback. Otherwise, I’ll see you on early access launch day (To be announced)
Shoutout to the community so far! Here are some cool mixes that players have made so far!
The MetaSound that I will be talking about is in this video.
In short, my goal for this was to try out some newer 5.1 quality of life improvements for MetaSound and the grain delay node.
What is MetaSound? A DAW?
This link says it best.
“A DAW and a game audio engine serve totally different needs and have opposed technical constraints and use cases. A game audio engine is about building audio experiences with interactivity and runtime procedural generation as the core focus. “
MetaSound is Unreal’s new audio plugin in UE5 aiming to elevate functionality usually required by sound designers for games.
Important Resources I used to get things going.
Here are 2 videos that will be really helpful in understanding what is going on here.
This Study’s MetaSound is free to download and tinker with below!
Key things this MetaSound above can do.
- Uses an alterable scale to determine the chord choices
- Uses bool arrays to determine the beat
- Generates a Lead, Kick, Snare, Hi Hat, and Pad Chords
- Routes everything through multiple FX such as filtering and Grain delay.
You can try this MetaSound out for yourself by downloading from gumroad!
Quick Grain Delay Notes
The grain delay is a bit of a wild node. The biggest hurdle is giving it some audio and a trigger.
I ended up taking this pulse and delaying it further to offset when it decides to spawn grains.
This gave me a wild ethereal type reverb effect. For the audio going in, I needed to tune that a bit more.
I am filtering and compressing the audio in order to reduce artifacts that can happen when a sound gets a bit too loud inside of the grain delay. Some whacky things can happen.
I also realized quickly that transients from a kick, hi-hat, or snare weren’t going to work well for the kind of effect I was going for. It sounded jarring, so I made sure that the only audio going into the grain delay was the pad and the lead instruments.
The entire Grain delay setup looks like this (Again you can download this metasound above if you wanna poke around in it more)
I know this might seem overwhelming, but a lot of what I was doing was straight up experimenting, so I encourage you to try something out yourself and tinker with the MetaSound on your own!
Previously, I talked through the entire process here to get to a demo video that was released about a year ago showing what I was calling at the time “Project Mix”. This has now turned into a full musical sandbox game that I am calling Mix Universe!
Link to that article. https://chriszuko.com/project-mix-current-progress/
What is Mix Universe?
A musical sandbox like no other where you connect nodes together to make a truly spectacular audio visual experience.
I started this thing with very simple beginnings and I am excited to talk about that more in-depth in the future. This post is mostly just to get things up to date here on my main site.
Heading towards an early access launch!
I want to get this thing out there in early access so folks can enjoy it without worrying about features completely destroying everything they’ve made. Early access launch will allow this.
Some fun screenshots
Thanks to everyone who has supported me along the way! I am excited for what the future may hold here and am glad to have made it this far in development!
If you’ve started to dive into making systems for your game using MetaSounds, you may have found yourself with some minor hitches or latency when playing the sounds.
These tips might help!
Try toggling the Async MetaSound Generator.
When this is on, it reduces the CPU cost during play, however, it can lead to latency with larger MetaSounds. It might be worth turning it off (Mix Universe has it off to ensure that a sound will always play at the right time)
Set the MetaSound BlockRate to a lower value
I’ve found 28 is a sweet spot for latency and performance for lower end machines. Lower numbers will increase latency, but decrease CPU usage. Higher numbers will decrease latency, but increase CPU usage. Default is 100, and this is described in code as “blocks per second” when processing the audio for a MetaSound.
Make sure to enable stream caching and force streaming on your audio files.
Try choosing ADPCM or PCM for your audio compression.
PCM is uncompressed and has a high memory footprint, but will result in the fastest audio playback possible.
ADPCM is 4x compression which may cause artifacts, but typically is ok for sounds that don’t have a ton of high frequency detail.
What these formats avoid is any types of decoders which will cause CPU performance overhead and latency. (Especially on something like the Quest)
You can monitor this using stat audio
Consolidate or reduce inputs
Here is a pretty complex MetaSound I am using for example. I have found around 30 is the sweet spot right before it starts to cause real issues. It may vary depending on the input type, but this was my findings with ints and floats.
Right now, the only real way to improve this is to just simply reduce the number of inputs you are sending to a MetaSound. It doesn’t really matter which ones are sent during play.
Reduce usage of Compression or Delay Nodes
Similarly, any nodes that use a large buffer size or a copy of a previous audio buffer can add up really quickly. Reducing lookahead time helps for compressors and also just in general, not using too many of these in your graph. I’ve found 1 of each is fine, but if you start pushing it too far it will start to show when playing your sound!
If you see constructor pins, they can save you!
As MetaSound evolves, you will notice diamond shape connection points. These are constructor pins and are very important for performance. They are only ever evaluated once and cannot change with inputs, so that means they can be optimized to be super cheap. In the case of a delay, this allows you to set the max allowed delay time to limit the audio buffer allocated for the delay form ever going over a certain length. Smaller numbers in this make the node much cheaper.
What if none of this works?
Then it is time to dive in deeper. There are many ways to do this, but I personally get the most info initially from doing a simple CPU profile using stat startfile and stat stopfile
stat startfile stat stopfile
Then using the unreal frontend to see what is happening when a hitch occurs!
If you are new to performance profiling, I would highly recommend this talk as it goes over the basics on the thought process when it comes to solving a performance problem!
Then finally, if that isn’t enough, you can look into Unreal Insights a bit more in-depth which will give you the most detail about what is happening to your frames!
I hope this helps and you can continue making awesome MetaSounds in UE5!