froydnj opened Issue #1386:
Feature
While looking at rewriting
lucetc
to useobject
instead offaerie
(primarily to get Windows support "for free" inlucetc
), I think the only reason that "raw"faerie
interfaces need to be used instead of going throughcranelift-module
is that the definition of the stack probe cannot be done as clif:as the stack probe must be defined as a certain sequence of instructions that other code needs to know the layout of. And various later things after the above point need to know about the existence of the stack probe, so they can't be written in terms of the
cranelift-module
interface and need to be written with "raw"faerie
.The straightforward way to rewrite this code (and other code using
faerie
) to useobject
is to make the appropriate translations between the two APIs. But there's nothing about the non-stack probe code thatcranelift-module
can't already handle -- basically defining a bunch of tables with assorted relocations. So I propose adding an interface tocranelift-module::Backend
that enables you to define a function in terms of the raw bytes for that function, rather than going through cranelift's code generation.Benefit
If you squint right, this interface is a primitive form of inline assembly, which has proven useful in other compilers for similar reasons (tightly controlled instruction sequences, as well as access to functionality not directly supported by the compiler or doing things that are Hard To Do in the original language).
For the above example, this interface enables the stack probe to be defined in terms of
cranelift-module
interfaces, and subsequently all the remaining pieces to be defined in terms ofcranelift-module
interfaces as well, thus eliminating the need to drop down to a lower-level interface. Switching betweenfaerie
andobject
would get a lot easier as well -- a handful of lines changed, rather than rewriting a bunch of object definitions.Implementation
This functionality should be straightforward to add: all of the
cranelift-module
implementations already have to have a way of taking bytes and associating a function name with those bytes, insideBackend::define_function
. This interface would just sidestep the code generation parts.I don't know what the exact signature should look like. Something like
define_function_bytes(&mut self, id: FuncId, name: &str, bytes: &[u8], namespace: &ModuleNamespace<Self>)
would work for the above use case. Depending on how Right people think a first cut should be, maybe you'd want to include an argument specifying relocations so external functions could be called. I'm inclined to do the simplest thing possible for now.Alternatives
The alternative that I see is defining some sort of inline assembly bits for the cranelift code generator, which seems like an overwhelmingly large project (although maybe that would have to happen at some point, given potential inline assembly support in
rustc
?). Maybe there are other alternatives that I haven't thought of.
philipc commented on Issue #1386:
lucetc
needs to also define the trap sites in the stack probe function, so the proposed API doesn't seem to be enough to avoid the raw interface.But there's nothing about the non-stack probe code that cranelift-module can't already handle -- basically defining a bunch of tables with assorted relocations
The problem is that the function manifest and trap table can't be generated until
Module::finish
is called because the information they require is in the product.You're probably aware of this, but I did convert
lucetc
to usecranelift-object
9 months ago (but it wasn't merged). Is the problem thatlucet
wants to be able to keepfaerie
and switch between backends, rather than removefaerie
support completely?
froydnj commented on Issue #1386:
lucetc
needs to also define the trap sites in the stack probe function, so the proposed API doesn't seem to be enough to avoid the raw interface.That's a good point; the interface would also need to specify trap information, which would require extending backends to expose some kind of portable way of expressing trap sites. But that seems pretty straightforward, since
cranelift-object
andcranelift-faerie
do trap collection in very similar ways already. (cranelift-simplejit
doesn't deal with traps, so maybe that complicates things a little bit.)But there's nothing about the non-stack probe code that cranelift-module can't already handle -- basically defining a bunch of tables with assorted relocations
The problem is that the function manifest and trap table can't be generated until
Module::finish
is called because the information they require is in the product.The function manifest is defined prior to the conversion out of
cranelift-module
:It does have to be fixed up to reflect the actual length of the stack probe:
but that bit would be unnecessary if
cranelift-module
were aware there was actual code associated with the stack probe.The actual trap tables lucet needs to define:
can be defined with information retrieved from
cranelift-module
, assuming that trap site information is exposed in some way.You're probably aware of this, but I did convert
lucetc
to usecranelift-object
9 months ago (but it wasn't merged). Is the problem thatlucet
wants to be able to keepfaerie
and switch between backends, rather than removefaerie
support completely?My understanding is that it is desirable to keep
faerie
support since that is a known quantity, and in case problems show up withobject
. The above proposal is intended to move enough functionality intocranelift-module
such that switching betweenfaerie
andobject
is easier than rewriting lucetc/lucetc/src/output.rs and the associated other places that touchfaerie
internals to supportobject
in parallel.
philipc commented on Issue #1386:
That's all fine, I'm not opposed to the idea, but it sounded incomplete to me. I did something similar initially but didn't follow through because it wasn't enough benefit, but I didn't go the whole way to try to do the function manifest and trap tables via
cranelift-module
. For defining the stack probe, my approach was to add an API that accepted a function parameter with a signature similar toContext::emit_to_memory
, which lets the caller define relocs/traps/stackmap. The existingModule::define_function
could then be implemented using the new API too.
froydnj commented on Issue #1386:
I realized as I was writing #1400 and experimenting with how it would work with
lucet
that some sort of API to expose trap information fromcranelift-module
will also be required...orlucet
can have some trait thatcranelift-object
andcranelift-faerie
could implement. Not sure yet which is better.
froydnj commented on Issue #1386:
I think this was taken care of by #1400. I opened #1404 for exposing trap information in some way.
froydnj closed Issue #1386:
Feature
While looking at rewriting
lucetc
to useobject
instead offaerie
(primarily to get Windows support "for free" inlucetc
), I think the only reason that "raw"faerie
interfaces need to be used instead of going throughcranelift-module
is that the definition of the stack probe cannot be done as clif:as the stack probe must be defined as a certain sequence of instructions that other code needs to know the layout of. And various later things after the above point need to know about the existence of the stack probe, so they can't be written in terms of the
cranelift-module
interface and need to be written with "raw"faerie
.The straightforward way to rewrite this code (and other code using
faerie
) to useobject
is to make the appropriate translations between the two APIs. But there's nothing about the non-stack probe code thatcranelift-module
can't already handle -- basically defining a bunch of tables with assorted relocations. So I propose adding an interface tocranelift-module::Backend
that enables you to define a function in terms of the raw bytes for that function, rather than going through cranelift's code generation.Benefit
If you squint right, this interface is a primitive form of inline assembly, which has proven useful in other compilers for similar reasons (tightly controlled instruction sequences, as well as access to functionality not directly supported by the compiler or doing things that are Hard To Do in the original language).
For the above example, this interface enables the stack probe to be defined in terms of
cranelift-module
interfaces, and subsequently all the remaining pieces to be defined in terms ofcranelift-module
interfaces as well, thus eliminating the need to drop down to a lower-level interface. Switching betweenfaerie
andobject
would get a lot easier as well -- a handful of lines changed, rather than rewriting a bunch of object definitions.Implementation
This functionality should be straightforward to add: all of the
cranelift-module
implementations already have to have a way of taking bytes and associating a function name with those bytes, insideBackend::define_function
. This interface would just sidestep the code generation parts.I don't know what the exact signature should look like. Something like
define_function_bytes(&mut self, id: FuncId, name: &str, bytes: &[u8], namespace: &ModuleNamespace<Self>)
would work for the above use case. Depending on how Right people think a first cut should be, maybe you'd want to include an argument specifying relocations so external functions could be called. I'm inclined to do the simplest thing possible for now.Alternatives
The alternative that I see is defining some sort of inline assembly bits for the cranelift code generator, which seems like an overwhelmingly large project (although maybe that would have to happen at some point, given potential inline assembly support in
rustc
?). Maybe there are other alternatives that I haven't thought of.
Last updated: Dec 23 2024 at 14:03 UTC