Jeff Charles (Aug 01 2024 at 19:43): Jeff Charles (Aug 01 2024 at 19:43)::wave: I'm looking at how to track the size of memory allocations for the MemoryCreator's `new_memory function from a Wasmtime embedding. It seems like a decorator or proxy similar to MemoryCreatorProxy would work the most conveniently. But I'm not sure how to set up a proxy around the DefaultMemoryCreator given it's not public AFAICT. I'd prefer not to have my own implementation of MmapMemory if possible.
Any tips :smile:?
Alex Crichton (Aug 01 2024 at 19:45):Would ResourceLimiter work for your use case?
Lann Martin (Aug 01 2024 at 20:30):You can see how Spin tracks this here: https://github.com/fermyon/spin/blob/main/crates/core/src/limits.rs#L28-L29 (and here)
Note its a bit awkward in order to handle multi-memory usage...
Lann Martin (Aug 01 2024 at 20:36):Also note that this isn't a very comprehensive accounting of memory usage; you'd need to track usage for tables and instances at a minimum to get a more precise idea of how much memory a guest is using.
Jeff Charles (Aug 01 2024 at 21:33):ResourceLimiter could maybe work but I'd have to significantly refactor things, which is probably the appropriate thing to do. For context, I'm looking at how to get the functionality in wasmtime-rb's TrackedMemoryCreator working with Wasmtime versions later than 20. Basically the point of the TrackedMemoryCreator is to globally report mmap allocations to the Ruby VM. The default config for the gem configures the memory creation to use that memory creator. With the wasmtime-runtime crate removed, that approach no longer works.
I'm thinking I could hypothetically have any Wasmtime APIs that wasmtime-rb calls that would return a store add one of my ResourceLimiters by default and decorate any new resource limiters the user adds with my tracking resource limiter. But I'm a little concerned it could be easy to miss somewhere where it should be added (and remembering to ensure it's added appropriately for any new Wasmtime APIs we call in updates). I can give it a try anyway and see how it goes.
Alex Crichton (Aug 01 2024 at 22:12):Oh ResourceLimiter only knows about sizes, not pointers, does ruby need to know the regions of linear memory?
Jeff Charles (Aug 02 2024 at 13:35):It doesn't need to know the regions, mmapping memory or freeing it just needs to have _something_ that calls void rb_gc_adjust_memory_usage(ssize_t diff) with the size of the allocation or amount that was freed just after those operations happen.
Alex Crichton (Aug 02 2024 at 13:54):Ah in that case I think ResourceLimiter is probably what you want as that's exactly the callback you get to customize
Notification Bot (Aug 02 2024 at 20:49):Jeff Charles has marked this topic as resolved.
Last updated: Nov 22 2024 at 17:03 UTC