Add terrain-based footstep sound system - #2941
Leonardo-UTP wants to merge 5 commits into
Conversation
|
Play this branch at https://play.threadbare.game/branches/Leonardo-UTP/issue-1095-footstep-sounds/. (This launches the game from the start, not directly at the change(s) in this pull request.) |
manuq
left a comment
There was a problem hiding this comment.
This needs a demo for testing. At least 2 materials added in a TileSet and 2 different sounds, to test how it works. It is worrying that the play_footstep() method will be called in every footstep, iterating all the tilemap layers.
| for layer in _footstep_layers: | ||
| var coord := layer.local_to_map(layer.to_local(global_position)) | ||
| var tile_data := layer.get_cell_tile_data(coord) |
There was a problem hiding this comment.
All TileMapLayers will be iterated every time a footstep is played from AnimationPlayer? This doesn't sound performant at all.
Also we use several tilemap layers. I wonder what will happen if there is a "stone material" layer below a "grass material" layer. Will this algorithm play the grass material sound effect?
There was a problem hiding this comment.
On my side, I also considered the performance aspect while developing the system, mainly because of the complexity of integrating the sounds with the different tiles and TileMapLayers.
To develop the system, I first reviewed how similar situations had been handled in other parts of the project. From that review, I found some options that I could use, such as identifying specific TileMapLayers through groups or using scripts associated with the layers so that FootstepSound could recognize which ones it should check. However, these alternatives would also require manually configuring the TileMapLayers that should participate in the system.
I also took into account that scenes can have more complex TileMapLayer structures, with the terrain distributed across different layers. Because of this, the system was designed to adapt to this structure without depending on a specific TileMapLayer.
I ultimately chose the current approach as a more centralized solution. When the scene starts, the system finds the TileMapLayers whose TileSets contain a Custom Data Layer named "material" and stores them. Then, only when a footstep is triggered by the animation, those TileMapLayers are checked to identify the material at the player's position.
When multiple materials overlap, the last valid material found during that check is currently used. This was also one of the reasons why I used Fray's End for testing, due to its map structure and the more complex distribution of its TileMapLayers. This allowed me to test the system in situations involving different materials and overlapping layers.
There was a problem hiding this comment.
When I briefly researched how to do terrain-specific footstep sounds, this is one of the ways that I came across. I also share the concern about performance but that is something that the profiler can tell us: what's the cost of iterating a (small) list of TileMapLayer nodes? Is it significant compared to the 16.6ms budget we get per 60 FPS frame? I think we should check that.
Another way you could imagine doing it is:
- Have a new physics layer for "footstep-affecting surface"
- Paint it over every tile that has a footstep material
- Put an Area2D on the player's feet that detects entering/exiting
- Only when entering/exiting, list the layers this detector is colliding with and pick the "top" one, update the stream on the player
- Now each footstep doesn't involve a for loop, just
play()on the audiostreamplayer
I actually suspect that this might be more costly for the engine because it has to deal with more absolutely enormous physics shapes for the floor layers, but this is guesswork!
|
In this case, I integrated three sounds created by me: sand, stone, and wood. I added them to the Player's sound folder. Additionally, I added a test map to the dev scenes. For this scene, I used the TileMapLayer structure from Fray's End as a reference, placing the corresponding tiles to test the different sounds and different situations. Regarding the audio files, it was necessary to add the corresponding license information. In this case, I used my contributor name, Leonardo-UTP, as the author, since the three sounds were created by me. This information was also necessary for the Licensing/REUSE check to pass correctly. |
manuq
left a comment
There was a problem hiding this comment.
@Leonardo-UTP impressive! Thanks for the demo level, it's very useful, and you also used the same explanations as other demos in Dev Archipelago. I'm on the fence about the performance of this, iterating over all TileMapLayers on each footstep seems bad to me. But I have to say it works very well and I can't think of a different solution. @wjt can you take a quick look and enlighten us here? Here is a recording with audio:
footsteps.webm
There was a problem hiding this comment.
If the assets are created by you, you shouldn't add them to "third-party". Instead add them to assets/first_party/sounds/. And for files in that location, the license file is not needed, because it's defined in REUSE.toml:
Line 74 in 678e080
There was a problem hiding this comment.
Regarding the sound files and their licenses, I have already made the requested changes. I moved the three sounds I created to assets/first_party/sounds/ and removed their individual .license files, since REUSE.toml already covers .wav files located under first_party. After making these changes, I tested the demo again and the sounds continue to work correctly. The Build, Licensing, and Linting/Formatting checks also passed successfully.
Finally, I also wanted to ask if it would be possible to document these sounds in their corresponding issues (#1046, #1049, and #1050). I have documented the process I followed to create each of them, and I would like to add that information to their respective issues, along with the sound creation process.
There was a problem hiding this comment.
Please go ahead and comment on those issues
| for layer in _footstep_layers: | ||
| var coord := layer.local_to_map(layer.to_local(global_position)) | ||
| var tile_data := layer.get_cell_tile_data(coord) |
There was a problem hiding this comment.
When I briefly researched how to do terrain-specific footstep sounds, this is one of the ways that I came across. I also share the concern about performance but that is something that the profiler can tell us: what's the cost of iterating a (small) list of TileMapLayer nodes? Is it significant compared to the 16.6ms budget we get per 60 FPS frame? I think we should check that.
Another way you could imagine doing it is:
- Have a new physics layer for "footstep-affecting surface"
- Paint it over every tile that has a footstep material
- Put an Area2D on the player's feet that detects entering/exiting
- Only when entering/exiting, list the layers this detector is colliding with and pick the "top" one, update the stream on the player
- Now each footstep doesn't involve a for loop, just
play()on the audiostreamplayer
I actually suspect that this might be more costly for the engine because it has to deal with more absolutely enormous physics shapes for the floor layers, but this is guesswork!
| if not tile_data: | ||
| continue | ||
|
|
||
| var footstep_material: Variant = tile_data.get_custom_data("material") |
There was a problem hiding this comment.
I don't have a better suggestion but the problem with calling this "material" is that Godot already has a concept of a "material" (e.g. physics material) and it's not this...
"Fabric"? https://en.wiktionary.org/wiki/Thesaurus:fabric#English and https://en.wiktionary.org/wiki/Thesaurus:material#English
Or be explicit and call the custom data "footstep_material" so it's clear it's not the other kinds of material that Godot has.
|
I’ll be a bit busy today with the program wrapping up, but I’ll be happy to continue with the remaining tests for the sound system as soon as I can. |
Description
Adds support for terrain-based footstep sounds.
The player can now detect the material of the tile beneath them using a TileSet Custom Data Layer named
material. When a footstep is triggered by the walk animation, the corresponding sound configured for that material is played.If no sound is configured for the detected material, the existing default footstep sound is used as a fallback.
Changes
FootstepSoundcomponent to the player.Fixes #1095