3D Audio and audio effects

This is a READ-ONLY snapshot of the Godot Community Forums when TwistedTwigleg was the owner, which is hosted for archival reasons only. The current Godot Community Forums can be found here.

Some links may point to external websites, including the current Godot Community Forums. These websites are not associated with TwistedTwigleg.

Leakbang

I've been experimenting with the audio system implemented in Godot, mainly with the AudioStreamPlayer3D and the Audio effects. How can I make objects like walls affect how the sound propagates. For example when there is a wall between the player and the audioplayer, the audio gets muffled muffled. I want to achieve something like this: (P) is the player and their looking angle. (S) is the audio source. Black lines are walls. In reality, the player should hear the audio to their right, where the red X is. But in the current system, the audio is heared right in the front of the player. I believe the theory behind it is that, because the wall muffles the audiosource, the opening in the wall allows the audio to propagate easier and sound as if it is coming from the right (the red X) So how can I achieve something like this?

Erich_L

I wonder if whoever can come up with a good solution to this can tie it in with the new room, room group, and portal nodes. Those were the first thing I thought of when I saw @Leakbang's picture and feel like a lot of the needed information overlaps.

dotted

You could find a first order simulation by using a bit of pathfinding (a-star) and perhaps a special nav mesh (if needed). It would give you a first order approximation of reflection off of surfaces because navigating obstacles would give you the actual distance traveled. The last way point changing direction before reaching a player would give you the apparent direction of the sound, so the sound would need to be attenuated by the distance it traveled to there, then played from that point. Other than that simulating the wave propagation to count the surfaces struck. It's a kind of ray tracing with massively different details v light propagation and a vastly different kind of 'viewport'

There's a (mac os) game called dark echo that visualizes sound in a 2d space. You might want to watch it to gain some inspiration, or the dev might actually speak on the topic of how he visualized sound propagation.

I expect you'd need a greatly simplified map to handle any simulation of sound propagation.

https://www.darkechogame.com

cybereality

Yes, you can do it with navigation meshes, this is a cool trick I hadn't considered. To do it properly, you'd need to use ray tracing (to accurately account for bounces off the wall) but this is overkill and not needed. Actually, I think if you just used a single ray cast from the player to the source of the sound, and if it collided with anything (or let's say wall objects rather than other small items) then just muffle the sound by changing the decibels. This is not realistic, and won't move the position of the sound, but it would be computationally cheap and may be good enough.

Leakbang

I did check out the Dark echo game you mentioned, but to be honest, I don't think that game has much going in terms of sound for it. It's visual effects were neat but I'm almost certain they were independent of the game sound calculations. Perhaps one of the best games with the best sound engine is Thief 2. I know the modern sound engines are more complex but I still think that sounded much better. You could close your eyes and feel the entire room. But that's a separate can of worms. I don't think its good/feasible to actually make such a system with Raycasts and or pathfinding. I think it'll be slow and weak. I wonder if any good open-source sound engines exist to begin with.

dotted

@Leakbang said: I did check out the Dark echo game you mentioned, but to be honest, I don't think that game has much going in terms of sound for it. It's visual effects were neat but I'm almost certain they were independent of the game sound calculations. Perhaps one of the best games with the best sound engine is Thief 2. I know the modern sound engines are more complex but I still think that sounded much better. You could close your eyes and feel the entire room. But that's a separate can of worms. I don't think its good/feasible to actually make such a system with Raycasts and or pathfinding. I think it'll be slow and weak. I wonder if any good open-source sound engines exist to begin with.

I don't talk about DarkEcho in terms of sound but how they visualized its propagation. How else do you think the enemy AI 'heard' the player? Ray Cast collision.

The problem is direction. It will be unconvincing without directional change. Apparent direction will change according to walls. Navigation will give the first order 'shortest' and therefore loudest path from source to player ear. If your sound source is an enemy (or the player) you can leverage pathing you'd have anyway.

You are talking to an ex-Thief junkie. Thief2 was a sad reboot of a game where the original did sound much better. Compared to TTLG Thief, Thief2 was soundscaped but not 'alive'. An immense disappointment for some.

Eidos Montreal had auditory landscaping (like TTLG Thief, and all this is easily done with Godot) and some selective play back but it did not match the DarkEngine. By DarkEngine standards it made the game far more Call of Poopy visually oriented.

cybereality

Yeah, there are a bunch of libraries for 3D sound (but some may be Windows only or have proprietary licenses, that may make it difficult to use).

https://valvesoftware.github.io/steam-audio/ https://developer.nvidia.com/vrworks-audio-sdk-depth https://gpuopen.com/true-audio-next/ https://hub.vive.com/storage/3dsp/

dotted

@cybereality

unfortunately almost all of them use hardware specific stuff (NVIDIA)

Open CL might require a custom build. That's a native library.

Leakbang

@dotted said: You are talking to an ex-Thief junkie. Thief2 was a sad reboot of a game where the original did sound much better. Compared to TTLG Thief, Thief2 was soundscaped but not 'alive'. An immense disappointment for some.

Wow, that's the first time I'm hearing that. I liked Thief 1's soundtrack a bit more than 2, but in my opinion, 2 eclipsed the first one on every front. I'm very shocked by what you said. Why do you think Thief 1's sound better than Thief 2? I also think that Thief 2 provided more opportunities to utilize sound to the player's advantage so that could also be a factor in why I think it sounded better. Just so we're clear I'm talking about Thief 1 (The dark project) and Thief 2 (The metal age) and not the Thief 2014 reboot.

dotted

Whoops I had thought you were talking about Eidos Montreal's effort not TTLG's series.

I can only say, TTLG > Eidos Montreal.

I think I liked Thief1 story a bit better, because it was a bit more pure dark fantasy and I really liked the music there.

cybereality

Honestly, I think ray casts would be fast enough in GDScript. You cast a ray from the player to the sound, which is cheap (and you don't have to do this every frame, maybe just when the sound starts playing or every second or two). If it has hit the wall of the room, you know it's being occluded. Then you would need to set "open areas" with the editor (just Position3D objects) to denote a door frame or window. Then find the closest "open area" to both the sound emitter and the player and move the sound to that position. Also, you would need to attenuate the sound, basically by lowering the volume based on the distance from the original sound position. This is a hack, and requires manual setup for the "open areas" but it should work in theory and it would be very fast.

dotted

@cybereality said: Honestly, I think ray casts would be fast enough in GDScript. You cast a ray from the player to the sound, which is cheap (and you don't have to do this every frame, maybe just when the sound starts playing or every second or two). If it has hit the wall of the room, you know it's being occluded. Then you would need to set "open areas" with the editor (just Position3D objects) to denote a door frame or window. Then find the closest "open area" to both the sound emitter and the player and move the sound to that position. Also, you would need to attenuate the sound, basically by lowering the volume based on the distance from the original sound position. This is a hack, and requires manual setup for the "open areas" but it should work in theory and it would be very fast.

The down side is it adds the need for maps to indicate 'portals' and imagine a map containing a set of pillars of various widths. With that geometry they are not applicable.

Again, if there is any pathing between source and listener (as with a player making foot falls detected by AI), the AI would likely need pathing to follow or attack a player.

Since Leakbang is referring to Thief, this would be a common case for AI. I don't think this is much overhead if sound sources amounted to about 5 or 6.

cybereality

Correct. I hadn't considered that. I think the path finding is pretty optimized (it's in C++) so I don't think there is huge overhead as I've seen games with dozens of enemies in Godot that run fine. So if you used that, for let's say only 6 sounds (the closest to the player), that should be fast enough.

Leakbang

I think the pathfinding setup would have a lot of problems. For example, this situation below: There are two equally long paths to the player. The sound should be heard from both sides, but with pathfinding, only one of these paths will match. Also the material and thickness of the wall is ignored. There are too many difficult to solve problems with this system that I think make it impossible to create a robust diy sound engine. However, as I was writing this post... something stuck me. I don't know how to do this, but I think this may be possible using Areas to reverb sound. Here's what I have in mind: The black lines are the room walls and a giant Area with a Normal Audio Bus covers the room. The red lines represent a sound vacuum. The Audio Bus will be set so whatever sound enters it, gets muffled. I think this automatically makes it so that the sound comes from the sides rather than the front. But the problem is that... the Audio Bus only affects sounds that are inside the (Red) Area. Not other sounds leaking into it. But, there may be a way for it to affect external sounds as well. I think this is worth a try. So what do you think? Any ideas on how to do this?

cybereality

Yeah, that would work for muffling the sound, but it wouldn't make it appear to come from the side. But you could also muffle the sound with the ray cast, as I mentioned, and that wouldn't require extra setup.

Leakbang

@cybereality said: Yeah, that would work for muffling the sound, but it wouldn't make it appear to come from the side. But you could also muffle the sound with the ray cast, as I mentioned, and that wouldn't require extra setup.

I mean that there would be two areas. One that covers the entire room, and one that covers the sound vaccum (the red rectangle)

The raycast method would allow me to muffle the audio if the player doesn't have line of sight with the object, but it is very limited and I can't achieve the directional audio.