When decoding jxl-rs first gives us the basic info from the file header, this is enough to allocate the buffers, get the size, and pixel format. And then jxl-rs will later make available the frame header; it seems as though jxl-rs will only make it available relatively late, after parsing a decent chunk of the frame data. Animated images do need to wait for the frame header to get the frame duration before they can decode. But non-animated images can get useful pixel data as soon as we have basic info. We were waiting until we had a frame header (jxl_decoder_is_frame_ready which returns true once image/rust/jxl/src/decoder.rs observes the frame_header) for all jxl images, but we can get useful pixel data via flush_pixels much sooner than that.
The page https://random-stuff.jakearchibald.com/apps/partial-img-decode/?demo=fox-progressive.jxl&density=2 probably shows the biggest improvement this change can make. Before this change we would show the first image at around 55k. After this change we show an image around 2.5k.
We split BeginFrame into AllocateFrameBuffers and EnsureSurfacePipe so that we can call AllocateFrameBuffers as soon as we observe basic info, but not pass that data to the surface pipe until a flush_pixels call returns true, at which point we can create the surface pipe and start making that pixel data user visible. These functions now return nsresult instead of FrameOutputResult (which seemed more confusing to me).
ProcessAvailableData gets a refactor that I think makes it a bit easier to follow. (I'd like to do more refactoring: these processing loops seem harder to follow than necessary.)
This also includes a refactoring of the main loop in process_data in decoder.rs. Most of the code we were executing for ProcessingResult::Complete should also happen for ProcessingResult::NeedsMoreInput: just because we need more data does not mean that we can't look for the basic info.
As part of this change I re-did all of our progressive image tests (one mochitest, one wpt reftest, some gtests) to better test this feature. The gtest changes are a significant improvement and I'm still working them into landable state, but the wpt and mochitest should be good coverage to land this.
Differential Revision: https://phabricator.services.mozilla.com/D298797
Timothy Nikkel
d475dafdb7