Repository navigation
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…layer - IfcByteSource: sync random access over bytes or a Blob (FileReaderSync with an LRU page cache); feeds web-ifc's load callback and the resolver. - IfcEntityResolver reads through any IfcByteSource, captures the header, and gains an uncached parse() for sweeps. - IfcResolverLineApi: the tape-reading subset of IfcAPI over the resolver, pinned against IfcAPI line by line on every fixture. - ProcessData.file/source: web-ifc opens through the reader; the property pass reads through the resolver instead of a second web-ifc instance. - Geometry is written to the builder as it arrives instead of held as JS arrays until the end. - Fix embind handle leaks (mesh.geometries, id vectors, swept-disk probe), an unused web-ifc re-Init after Dispose, quadratic unshift loops and an indexOf-per-child spatial walk. - Finer progress (within a category) and explicit phases. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Extraction (web-ifc meshes, hashing, shell building) depends on one element alone; assembly (dedup of geometry and local transforms, id allocation) replays the single-pass decisions in order. Shells travel as typed arrays. The grid reader reads through IfcLineApi with the coordination matrix passed in, so it no longer needs an open web-ifc model. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
IfcProjector writes, for a batch of elements, a standalone IFC holding only what their geometry reads: down-references plus the four reverse edges web-ifc's IfcCache follows (voids, inherited through aggregation; styled items; material associations, cut down to the batch; material definitions) and the project's units. Each batch opens in a web-ifc of its own, in this thread or in workers (serveIfcGeometryWorker), and is assembled in order. The origin is reproduced exactly: a probe batch reports which element set it, and every later batch streams that element first. Pinning the matrix with SetGeometryTransformation instead rounds a few transforms differently. The extractor skips building geometry the assembler already holds, which the split had lost. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
IfcStatementScanner's byte loop becomes IfcStatementCursor, which reports statements positionally, interns type names by a hash taken while scanning, skips through statement bodies, and keeps its state in locals. IfcEntityResolver.fromSource indexes through it directly: ~50 MB/s through the stream becomes ~250 MB/s. Also: parallel mode in the StreamingImport demo, and batches sized so every executor gets several. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Moves the .frag dump/compare out of the parity CLI into src/testing so tests can use it, and adds replicate.ts, which builds large synthetic IFC files from a real one for testing past the sizes real fixtures reach. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The property pass splits into prepare (every entity's attributes and every relation, keyed by id) and finish (the layout, items with geometry first). With geometry in workers, prepare runs on the coordinating thread meanwhile, yielding every few milliseconds so batch results keep flowing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- IfcChunkedBytesSource holds a file as several buffers, so files past the 2 GB a single Blob read allows can still be read into memory. - IfcBlobSource pages default to 4 MB (each FileReaderSync call is a round trip to the browser process) with a 128 MB cache. - IfcLineIndexBuilder takes a size hint, grows by 1.5x, and finalizes without copying when its arrays are mostly full. - Demo: largest web-ifc heap across workers, elapsed time while converting, page and resident-budget parameters, bench screenshots. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Relations are an edge list in typed columns, grouped by a stable radix sort when read, instead of an object and arrays per entity. - Items with geometry are typed columns filled on arrival, with material indices assigned then, instead of an object graph kept to the end. - The shell dedup key is digested to 64 bits: it crosses from workers and is held once per geometry. - The projected reader's processing order is two typed arrays. - Bench: sampled live heap of the import worker (--heap). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- README: how the pipeline works, what changed in the library, parity evidence, measurements against the previous importer on the test models and a 2.4 GB synthetic file, the workers/memory trade-off, and limits. - IfcGeometryExtractor and its per-geometry methods take destructured objects. - Bench reports the geometry workers' largest web-ifc heap. - Examples index lists the StreamingImport demo. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…progress Batches of consecutive elements were cut by count, and exporters write costly elements next to each other: one batch then held a run of CSG-heavy members, or 150 instances of a 22k-face mapped brep, and bounded the whole conversion. The projector now estimates each element's meshing cost as it walks its closure (booleans, openings, geometry bytes, and a mapped representation again per instance, since web-ifc re-meshes every instance), batches are cut at a cost worth targetBatchMs, and finished batches calibrate how many ms a unit of cost is. Slowest models: 18.7 -> 11.7 s, 6.7 -> 4.6 s, 10.9 -> 7.9 s; no change on the rest. Also: batch durations in IfcImporter.stats, and the demo shows progress while loading into the viewer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Estimating a representation map's cost with a walk of its own re-read much of an instanced file, and planning a paged 2.4 GB file went from 31 s to 75 s. The map is now measured as part of the batch's walk, and the walk keeps index positions on its stack, looking each statement up once. README: how batches are planned and why not cluster, the corrected account of what batching duplicates (web-ifc meshes shared geometry per element in any mode), the output cap, and results at this commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…heap claim Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One model in one file gives the viewer nothing to draw until all of it is loaded, in one worker, properties included. `splits` writes a conversion as several models instead, as cutting the IFC up with the splitter does, but from one conversion: one origin, one set of local ids. - Geometry splits hold the meshes and the local id and category of the item each belongs to, which is all drawing, picking and hiding read. - The reference split holds the grids and alignments, which the viewer draws from their attributes. - The data split holds everything else, for every item, elements included: attributes, relations, GUIDs, the spatial structure and the metadata. Property sets alone are 2-7% of a model; most of what rendering never reads belongs to the elements themselves. - `process` resolves to a manifest that lists the splits and says, for each local id, which split has its geometry and which its attributes (`findItemSplits`). Splits are planned with the batches: elements are counted into the current split and the batch that would overfill it is cut short, so a split is a run of consecutive batches. A split is also ended when its model passes `maxBytes`. Each is assembled on its own, into a builder of its own, and handed to `onSplit` as soon as its last batch is assembled; `onSplit` is awaited, and assembly with it. The config follows the one discussed in ThatOpen#180, without a `splitBy` for now. Writing a model's meshes table moves out of IfcGeometryProcessor into MeshesWriter, one per model written. Without `splits` the output is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Splits are put back together (`dumpSplits`) and compared with the single model: on every fixture, cut by count and by size, with the manifest checked against what each split holds and every split loaded in the viewer's model. Shells are compared as shapes rather than as lists of numbers. Geometry is deduplicated by a key made of a mesh's rounded vertices and its size, not its triangles, so one model stores two meshes with the same key once; when a split boundary falls between them, each is stored as itself: points that differ by what the rounding hides, sometimes in another order, or with a flat face cut into loops another way. `compare` counts those when given a tolerance and still fails on anything else. Grids and alignments, whose local ids the importer makes up, can be compared by what they hold. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The demo writes the model as splits when "Several models" is ticked, and loads each as it arrives: geometry into the scene, grids and alignments drawn from the reference split, the data split into a thread group of its own. Clicking an element reads its data from the data split, found through the manifest. It reports when the first geometry is on screen and when everything is loaded, and bench.mjs passes both on, with an item read back from the data split. convert.ts writes splits and a manifest with SPLIT_ITEMS, and parity.ts takes a manifest in place of a .frag. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
What a converted model is made of, what splits buy when loading (1.3-4.6x faster viewer load, first geometry on screen in a sixth to a half of the time), what they cost, how they compare with the single model on four real models, and what is left: data is one split, and nothing in the viewer loads a manifest yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ShaMan123
force-pushed
the
poc/ifc-split-output
branch
from
October 12, 2026 06:45
519f262 to
570e24d
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
/private/tmp/claude-501/-Users-shaman123-DEV-engine-fragment/f6e4263e-da2d-4e66-ae01-e48ccaf947b7/scratchpad/pr-splits.md