Kirp (Aug 05 2024 at 23:04): Kirp (Aug 05 2024 at 23:04):Was reading this re rustc stackoverflow and the culprit was the size of isle_opts.rs https://sunshowers.io/posts/rustc-segfault-illumos/ - coincidentally was also talking earlier about the size of the rust code being generated by islec resulting in slow compiles - wonder if there is any feasible solution to split or shrink the code?
Chris Fallin (Aug 05 2024 at 23:38):Well, the ultimate culprit is the uncontrolled recursion in rustc's parser -- we are generating valid Rust, albeit one large function but the language does not specify limits on function size in the semantics so I believe the proper fix is where that post landed, i.e. fixing rustc
Chris Fallin (Aug 05 2024 at 23:39):even moreso given that rustc's parser is making efforts to not overflow the stack, but failed on illumos due to some missing plumbing; that shows the implementation intent
Chris Fallin (Aug 05 2024 at 23:41):The code is generated from our set of rewrite rules; fundamentally it all has to be there as long as we have those rules; it's always possible that there are constant-factor improvements in the code we generate, but it's pretty lean already (there are just a lot of cases). "Slow builds" is a little relative IMHO -- we're embodying a large collection of domain-specific logic. For comparison, how long does an LLVM backend take to build, etc... (the problem would be more urgent if we were seeing slowdowns relative to another compiler of the same complexity IMHO)
bjorn3 (Aug 06 2024 at 07:47):Chris Fallin said:
even moreso given that rustc's parser is making efforts to not overflow the stack, but failed on illumos due to some missing plumbing; that shows the implementation intent
It is actually missing a place where stacker is used to extend the stack. The reason it crashed on Illumos is not because it didn't extend the stack, but because stacker extended the stack with a small segment when not strictly needed. Rustc runs on a 4MiB(?) stack by default, which masked the missing stacker call, but then on Illumos the first stacker call that would consider enough stack remaining on other OSes would switch to a new stack segment which is much smaller. (128KiB?) If enough stacker calls existed, this would be fine, but as a stacker call somewhere in the parser was missed, it would run out of this 128KiB stack segment and crash.
bjorn3 (Aug 06 2024 at 07:49):Is it possible for ISLE to subdivide large functions into multiple functions. LLVM is much slower at compiling a single huge function than many smaller ones anyway, so it should help with making Cranelift itself be built faster too.
Chris Fallin (Aug 06 2024 at 15:33):Is it possible for ISLE to subdivide large functions into multiple functions
WIth more analysis, yeah, that's theoretically possible; we'd need to analyze for captures to create the arguments. The trickier part is that we'd need to ensure the control flow remained identical: we generate tree-shaped matching code but with fallthroughs so we'd need to get the backtracking right
Last updated: Nov 22 2024 at 16:03 UTC