For six weeks, the native Protos executable ran in interpreter mode. The investigation started with what looked like an unboxing error, went through a GraalVM issue, and ended with a change in how I debugged it. I stopped chasing the latest compiler error and started reading the full paths that led to it.
On 17 November, I saw opt done and Tier 2 in the native Protos executable’s log again. Four successful compilations, zero failures, and a 1 on standard output. That 1 was exactly what I expected. It was also what the interpreter could return when compilation failed. The difference was in the four lines above it.
October. The error that looked like unboxing
Protos is built on Truffle. With GraalVM Native Image, I can compile the runtime and interpreter ahead of time and ship them as an executable. Once that executable is running, it can also use the Truffle compiler to optimise programs written in Protos. These happen at two different stages. Native Image builds the executable, and the JIT compiles the program it runs.
The problem showed up at that second stage. Truffle tried to compile a Bytecode DSL root at Tier 2, and Graal bailed out with this message:
Object of type Lcom/oracle/truffle/api/impl/FrameWithoutBoxing;
should not be materialized
My first suspicion was unboxing. The name FrameWithoutBoxing pointed to the area I was working on, where Protos tries to keep primitive values such as long without wrapping them in Java objects. I wanted to know whether one of our specialisations was forcing Truffle’s virtual frame to become an actual object during compilation. I kept removing pieces until I was left with a Bytecode DSL root independent of Protos. Its body just returned the constant 1L.
b.beginRoot();
b.beginReturn();
b.emitLoadConstant(1L);
b.endReturn();
b.endRoot();
With no locals, no continuations, no Protos operations, and no boxing elimination enabled in the reproducer, the error still occurred. The same root compiled at Tier 2 on the JVM. On 1 October, I opened oracle/graal#14579 with the minimal project, the traces, and instructions to reproduce it.
Meanwhile, I needed to keep a usable native executable. On 2 October, I switched the Native Image build to Truffle’s fallback runtime:
-Dtruffle.UseFallbackRuntime=true
The executable was still built with Native Image and could run Protos in the interpreter. The JVM edition kept its JIT. The cost was explicit. The native edition lost the ability to optimise the programs it ran. PLAT045 platform decision
November. GraalVM closes the case and the investigation returns to Protos
GraalVM replied on 12 November. The generated class MiniPlainRootGen was configured to initialise at run time. That left its execute method outside runtime compilation support. During partial evaluation, Graal saw an opaque call it could not look inside. The frame escaped through that call, so the compiler ended up trying to materialise it. For the reproducer, the fix was to initialise the repro package at image build time. GraalVM closed the case with that explanation.
--initialize-at-build-time=repro
That changed how I read the original message. FrameWithoutBoxing was the object that escaped. The explanation lay in the call it escaped through. Fixing that call in a minimal project left the question I cared about open. Which paths was Protos itself making reachable when I enabled its compiler?
The first attempt with Protos returned Blocklisted methods are reachable for runtime compilation and pointed to BigInteger.valueOf(long) through generic interop dispatch. I also tried initialising an entire package at image build time. The error moved into the image heap, first with ProtosRegexSemanticTransferFamily and then with ProtosClosureValue. The option that fixed the minimal project had a much wider effect on a real interpreter package. Investigation record for 13 November
Chasing the carrot. One more method and surely it will build
For a while, I chased each method as it appeared. I changed a boundary, adjusted when a class initialised, or reviewed a dispatch path. I rebuilt the image and the next failure gave me another name. Every iteration felt like progress towards the end. The compiler was showing me different paths through the same graph.
I also had to reconsider the liberties AI had taken while implementing parts of the runtime. I had accepted Java solutions because they matched Protos semantics and passed the tests. Some connected paths that Truffle tries to keep separate. An operation that should have been easy to specialise ended up pulling in host lifecycle logic such as terminating an actor and cleaning up its I/O resources.
One example was the path through ProtosActor.markTerminated(). The method at the end of a trace could belong to a Java library. The decision I needed to revisit was several steps earlier, in Protos, where the guest execution path entered actor cleanup. I put a @TruffleBoundary on terminalActorIoLifecycleCleanupForRuntime so partial evaluation would leave that cleanup as a call. The actor still ran its cleanup when required. What changed was how far the compiler went when specialising the guest program.
From there, I started checking those decisions against GraalPy and other Truffle languages. In Protos, operations such as while and + have message syntax and semantics. On the usual execution path, I want Truffle to run them as specialised internal operations. A Java implementation can return the right value and still lead the compiler down paths it should never have had to analyse.
The tools were already leaving clues
The pace changed when I stopped reading the final method in the error and started processing the full Native Image svm_err*.md reports. The reachability reports included paths I could extract and group. I also used javap to check which class actually declared certain methods in the generated bytecode. That let me compare the sets of findings between builds and see what had cleared and what remained.
The clearest example came from an analysis that flagged 227 methods. Of those, 224 went through ProtosActor.markTerminated(), and two more went through ProtosBufferedByteIo.pump() and doFlush(). Read one by one, they looked like hundreds of repairs. Grouped by the paths that made them reachable, they showed where to focus.
I also learned to be wary of a zero. In one intermediate build, all 137 findings in one category disappeared, but Native Image still failed for a different reason. The zero described what I had extracted from that report. To call the JIT restored, I still had to build the binary, run it, and see Truffle complete a real compilation.
The final test. A 1 with four compilations behind it
The probe I ran against the native executable began with this Protos code:
x: 1
f: () => { x }
g: () => { (() => false).whileTrue(() => { null }) }
The generator added 500 pairs of calls to f() and g(), followed by print(f()). That exercised a closure capturing a variable and a path through loop control. I ran the probe with compilation tracing and immediate compilation enabled, and background compilation disabled. Here is the abridged log:
[engine] opt done ... ProtosBytecodeRootNodeGen ... |Tier 2|
[engine] opt done ... ProtosBytecodeRootNodeGen ... |Tier 2|
[engine] opt done ... ProtosBuiltinCallRootNode ... |Tier 2|
[engine] opt done ... ProtosBuiltinCallRootNode ... |Tier 2|
1
The run recorded four Tier-2 compilations, two for generated Bytecode DSL roots and two for built-in call roots. It recorded zero compilation failures, zero FrameWithoutBoxing failures, and zero fallback runtime markers. The native regression suite passed too. In the separate JUnit image, the shared Polyglot test context now initialises when the image runs, and all 38 TLS tests passed. I115 acceptance evidence
The change landed in the Protos repository as 0.4.155-SNAPSHOT, with a test that requires positive Tier-2 evidence from the native executable. The full transcript is from the preceding local run on 0.4.154-SNAPSHOT. After pushing the change, I ran the tests locally again and closed the Protos tracking issue. Recovery commit · UPSTREAM004 closure
What would have saved me the most time in October was knowing how to read a reachability trace as a record of runtime decisions. The final method told me where Graal stopped. The full path told me which decision in Protos had brought it there. And the 1 printed on the screen all along only told me the work was done once I could see, just above it, what had been compiled.