The aligner is dead. Long live the aligner.

AGENTS.md

@@ -64,10 +64,18 @@ Two orthogonal mixin axes: **braindata** (the array plus subject/transform ident
 Flow: dataviews → JSON + PNG mosaics (`data.py:Package`) + compressed CTM surface packs (`cortex/brainctm.py`, cached per subject) → Tornado template (`template.html`, extended by `static.html` etc., resolved via `FallbackLoader` so user template dirs can override) → Three.js app in `resources/js/`.
 
 - `view.py`: `show` (live Tornado server from `serve.py`, returns a `JSProxy` websocket RPC handle for driving JS from Python) and `make_static` (self-contained directory; `htmlembed.py` can inline everything into a single file).
+- `serve.WebApp` binds its listening sockets in `__init__` through `bind_local_sockets`, one per address the names in `serve.LOCAL` resolve to (the loopback addresses and the machine's hostname), all on one port, unless a caller passes another `address`; `view.show` opens up to every interface only when the `domain_name` config option names a domain to reach the viewer under.
+- Every handler is wrapped in `serve.TokenGuard` (by `serve.guarded`, in `WebApp.run`), which demands the server's `token`: `serve.WebApp.url` puts it in the address as `?token=`, the guard hands it back as a per-port cookie, and requests without either get a 403. Build URLs with `server.url(page)` rather than by hand, and pass `token=""` to turn the check off.
+- `jsplot.Axes3D.draw` renders every entry of `this.views` as a viewport; a view may carry a `camera` of its own, a `prepare(width, height)` callback, an `overlay(camera)` one that draws over the view once the scene is in, and a `surf` index, which is how `mriview.Viewer.setSliceViews` puts the three slice planes beside the 3D view (each drawn down its own axis by an orthographic camera, the surface left to the 3D view, the crosshair laid over all four from a scene of its own). `setGrid` rebuilds the views on every `setData`, so the split is applied again after it. The scissor test is turned off once the views are drawn, because the passes that follow — the picker's among them — cover the whole canvas. `setOrthographic` swaps the camera the controls move for an orthographic one, where the radius sets the frustum rather than the distance; the lights are children of the camera and the renderer only finds the lights that are in the scene, so the camera is put into the scene with them.
+- `surface.<subject>.low poly` draws a surface from faces cut over groups of that many mm: `utils.get_lod` cuts them with `polyutils.decimate_faces` from the pack's own mesh, keyed on the pack's name in the subject cache dir, and `CTMHandler` cuts one the first time it is asked for, while `make_static` writes the widths in `utils.LOD_CELLS` and names them in the pack json as `lod_cells`. The coarse vertices are vertices of the surface, so only the index buffer and the normals change and everything held per vertex stays put; `mriview.Surface._applyLod` swaps them, keeping what the surface was loaded with to go back to.
+- `surface.<subject>.shading` picks how a face is colored and lit: the default interpolates a color across it and lights every pixel, while flat, gouraud and phong give it one color, read where its first corner is. One color to a face needs a vertex per corner, so `Surface._setFacets` writes the drawn faces out that way (`facepos`/`facewm` carry the read position, `FACECOLOR` in `shaderlib.surface_pixel`), which it does below `facet_limit` faces (a surface of its own detail would take hundreds of megabytes, and its faces are smaller than a pixel in any case). Data held a vertex at a time arrives for the surface as it is held, so `Surface.setDataAttribute` writes each frame out for the faces as it comes. `surface.<subject>.outline` draws the surface again through `Shaders.outline`, inside out and grown by `outline width` of the screen (as many pixels up as across, from the `viewport` uniform `Surface.prerender` keeps current), which leaves a line around it; `outline corners` set to round adds a `THREE.PointCloud` of dots of the line's width at the vertices of the faces being drawn (`Surface._dotGeometry`) that face away from the eye, as the line is drawn from the faces that do, so the surface covers them except where they round off a corner of the silhouette; `Surface.prerender` keeps them out of the depth image the labels are hidden by.
+- `surface.<subject>.lighting.ambient occlusion` takes the sky a vertex misses out of the light on it. `polyutils.vertex_occlusion` bakes it a vertex at a time by following rays over the hemisphere of the normal through a voxel grid of the surface, and `utils.get_occlusion` works it out on each surface the viewer mixes between (the folded surface at depth 0 and 1, the pial and white matter surfaces, then the pack's `names`, zeros for the flat one), in the pack's numbering, writing a byte per vertex to `<pack>_ao.bin` beside it; `CTMHandler` makes the file the first time it is asked for, on a thread, and `make_static` writes it with the pack. The viewer fetches it on the first toggle and writes the values into spare components of attributes that are already there (the two depths as two bytes in the w of `wm`, mixed by `use_thickmix` like the position; each `mixSurfs`'s w; `auxdat.z` without a white matter surface), since the shaders are at WebGL's 16-slot limit; `mixfunc_occlusion` in the shader mixes them with the same weights as the positions, so what is dark follows the view and the depth, and `occlusion strength` (the `occlusionStrength` uniform) scales how much is taken out of the light.
+- `Axes3D.getImage` draws into a `WebGLRenderTarget`, which carries one sample a pixel however the canvas was made (`antialias` is asked for only off a retina display, and a render target cannot be multisampled in webgl 1), so an image saved from the viewer has hard edges. `Save image.Antialias` renders it that many samples across a pixel and `mriview.getTexture` hands the buffer to `downsampleTexture`, which averages each block back down to a pixel with each sample's color weighted by its alpha, so the edge of the surface keeps the color of the surface and fades into the transparent background rather than into black.
 - The bundled Three.js is **r69** — very old; the shader pipeline depends on its conventions, do not casually upgrade.
 - All GLSL lives in `resources/js/shaderlib.js` as arrays of string lines assembled per-configuration — there are no `.glsl` files.
 - CTM packs reorder vertices: `cortex.utils.get_ctmmap` / `get_ctm2webgl_map` translate between CTM/WebGL ordering and the original surface ordering. Indexing viewer data with original vertex indices without remapping is a classic bug.
 - `cortex/export/headless.py` (`headless_viewer`) runs the viewer in headless Chromium via Playwright for screenshots/tests.
+- `cortex/webgl/aligner.py` + `resources/js/aligner.js` (`aligner.html`, `resources/css/aligner.css`) is the browser-based manual aligner behind `cortex.align.webgl_manual`. It reuses the CTM pack, the mosaic PNG loader (`dataset.VolumeData`) and the `Shaders.aligner_*` builders in `shaderlib.js`, and works in a world frame that is the reference volume's voxel grid in mm permuted to RAS (`reference_frame`); the edited transform is the pycortex `coord` transform. Its `display` control picks one of three displays; the data view draws the surfaces in the anatomy's own frame with the morph targets of the CTM pack (`Shaders.aligner_volume({morphs: n})`), so they inflate and flatten there while the alignment only moves where the volume is sampled. `/save` writes to the filestore, so it takes only posts carrying the per-page `save_token` from the page config, and `JSAligner.save` polls the page's save state rather than returning while the request is still out.
 - Some JS resources use CRLF line endings (e.g. `dataset.js`); keep the existing endings when editing.
 - `setup.py` explicitly enumerates `cortex.webgl` `package_data` patterns — new resource subdirectories must be added there or they won't ship in wheels.
 

cortex/align.py

@@ -14,90 +14,143 @@
 from .xfm import Transform
 
 
-def mayavi_manual(subject, xfmname, reference=None, **kwargs):
-    """Open GUI for manually aligning a functional volume to the cortical surface for `subject`. This
-    creates a new transform called `xfm`. The name of a nibabel-readable file (e.g. nii) should be
-    supplied as `reference`. This image will be copied into the database.
-
-    To modify an existing functional-anatomical transform, `reference` can be left blank, and the
-    previously used reference will be loaded.
-
-    <<ADD DETAILS ABOUT TRANSFORMATION MATRIX FORMAT HERE>>
-
-    When the GUI is closed, the transform will be saved into the pycortex database. The GUI requires
-    Mayavi support.
+def webgl_manual(
+    subject: str,
+    xfmname: str,
+    reference: Optional[str] = None,
+    view_only: bool = False,
+    cmap: Optional[str] = None,
+    mesh_color: Optional[str] = None,
+    mesh_opacity: Optional[float] = None,
+    open_browser: Optional[bool] = None,
+    autoclose: Optional[bool] = None,
+    port: Optional[int] = None,
+    recache: bool = False,
+    types: tuple[str, ...] = ("inflated",),
+    title: Optional[str] = None,
+    display_url: bool = True,
+    token: Optional[str] = None,
+    template: str = "aligner.html",
+):
+    """Open the browser-based aligner for manually aligning a functional volume
+    to the cortical surface of `subject`.
+
+    The functional reference volume stays on its own voxel grid, so its slices
+    are shown without resampling, and the pial and white matter surfaces are
+    moved into its space; in each slice view the surfaces are cut off at the
+    displayed slice, so that their outline can be compared with the anatomy
+    in the image. Only rotations and translations are possible.
+
+    The page shows the coronal, axial and sagittal slices and a 3D view of
+    the three slices. In a slice view, a left drag moves the cursor, which
+    sets the slices of the other views and is the pivot of rotations; the
+    wheel (or ``[`` and ``]``) changes the slice, ctrl + wheel zooms, and a
+    middle (or shift + left) drag pans. A right drag, the WASD keys or the
+    arrow keys translate the surfaces in the plane of the view under the
+    mouse, ctrl + right drag or ``q`` / ``e`` rotate them about the cursor;
+    shift makes the keyboard steps ten times smaller and ctrl + z undoes. The panel on
+    the right holds the Save button, the view mode (surface outlines on the
+    slices, or the volume painted on the surface, also toggled with ``m``),
+    the colormap with its range, brightness, contrast, gamma and flip, the
+    color and opacity of the surfaces, the slices and the keyboard steps.
+
+    The ``transform`` field above the Save button holds the name the
+    alignment is saved under. It starts as `xfmname`; editing it saves the
+    alignment as a new transform and leaves the one it was loaded from
+    alone. An asterisk on the Save button and in the window title marks an
+    alignment that differs from the one last saved.
+
+    When Save is pressed the transform is stored into the pycortex database,
+    as a 'coord' transform. A new transform requires `reference`, which is
+    copied into the database; an existing transform is loaded together with
+    its stored reference.
+
+    Saving also deletes the masks cached for the transform it writes, since
+    they were cut out of the reference volume through the alignment being
+    replaced. The page warns about this when it opens a transform that has
+    masks, and the save message names the ones it deleted. Data already
+    masked with them has to be masked again from the volumes. Pass
+    ``view_only=True`` to inspect an alignment without saving.
 
     Parameters
     ----------
     subject : str
         Subject identifier.
     xfmname : str
-        String identifying the transform to be created or loaded.
+        Name of the transform to create or modify.
     reference : str, optional
-        Path to a nibabel-readable image that will be used as the reference for this transform.
-        If given the default value of None, this function will attempt to load an existing reference
-        image from the database.
-    kwargs : dict
-        Passed to mayavi_aligner.get_aligner.
+        Path to a nibabel-readable functional volume, required for a new
+        transform. Must be None for an existing transform.
+    view_only : bool, optional
+        Open the aligner without the possibility to save, to inspect an
+        alignment. Default False.
+    cmap : str, optional
+        Initial colormap for the reference volume, one of the 1D pycortex
+        colormaps. Defaults to the `colormap` option of the `webgl_aligner`
+        section of the config file.
+    mesh_color : str, optional
+        Initial color of the surface outlines, as a matplotlib color.
+        Defaults to the `mesh_color` config option.
+    mesh_opacity : float, optional
+        Initial opacity of the whole surfaces in the 3D view (0 shows only
+        their outlines on the slices). Defaults to the `mesh_opacity` config
+        option.
+    open_browser : bool, optional
+        Open the aligner in the default browser. Defaults to the
+        `open_browser` option of the `webshow` config section.
+    autoclose : bool, optional
+        Stop the server when the last browser window disconnects. Defaults
+        to the `autoclose` option of the `webshow` config section.
+    port : int, optional
+        Port of the server; a free port is picked when None.
+    recache : bool, optional
+        Regenerate the cached surface (CTM) files. Default False.
+    types : tuple of str, optional
+        Surface types included in the CTM pack, to share the cache with the
+        viewer. Default ("inflated",).
+    title : str, optional
+        Title of the browser window. Defaults to the subject and the
+        transform.
+    display_url : bool, optional
+        When `open_browser` is False, display an IPython link to the
+        aligner. Default True.
+    token : str, optional
+        The session token the server demands, which the address it prints
+        carries and the page then keeps in a cookie. A new one is made for
+        each aligner; pass '' to take requests from anything that reaches
+        the port, which a script talking to the server itself may want.
+    template : str, optional
+        Name of the tornado template of the page. Default 'aligner.html'.
 
     Returns
     -------
-    m : 2D ndarray, shape (4, 4)
-        Transformation matrix.
+    handle : cortex.webgl.aligner.JSAligner or cortex.webgl.serve.WebApp
+        A handle to the running aligner: ``handle.get_xfm()`` returns the
+        current transform, ``handle.save()`` saves it. When the aligner is
+        started with ``open_browser=False`` the tornado server is returned
+        instead; its ``get_client()`` returns the handle once a browser has
+        connected.
     """
-
-    warnings.warn("This is the old cortex.align.manual(), and has been "
-                  "deprecated. Please use the new cortex.align.manual() "
-                  "(previously cortex.align.fs_manual()), which uses "
-                  "the `freeview` program in the freesurfer suite, to "
-                  "perform manual alignment.", DeprecationWarning
-                  )
-    from .database import db
-    from .mayavi_aligner import get_aligner
-    def save_callback(aligner):
-        db.save_xfm(subject, xfmname, aligner.get_xfm("magnet"), xfmtype='magnet', reference=reference)
-        print("saved xfm")
-
-    def view_callback(aligner):
-        print('view-only mode! ignoring changes')
-
-    # Check whether transform w/ this xfmname already exists
-    view_only_mode = False
-    try:
-        db.get_xfm(subject, xfmname)
-        # Transform exists, make sure that reference is None
-        if reference is not None:
-            raise ValueError('Refusing to overwrite reference for existing transform %s, use reference=None to load stored reference' % xfmname)
-
-        # if masks have been cached, quit! user must remove them by hand
-        from glob import glob
-        if len(glob(db.get_paths(subject)['masks'].format(xfmname=xfmname, type='*'))):
-            print('Refusing to overwrite existing transform %s because there are cached masks. Delete the masks manually if you want to modify the transform.' % xfmname)
-            checked = False
-            while not checked:
-                resp = input("Do you want to continue in view-only mode? (Y/N) ").lower().strip()
-                if resp in ["y", "yes", "n", "no"]:
-                    checked = True
-                    if resp in ["y", "yes"]:
-                        view_only_mode = True
-                        print("Continuing in view-only mode...")
-                    else:
-                        raise ValueError("Exiting...")
-                else:
-                    print("Didn't get that, please try again..")
-    except IOError:
-        # Transform does not exist, make sure that reference exists
-        if reference is None or not os.path.exists(reference):
-            raise ValueError('Reference image file (%s) does not exist' % reference)
-
-
-
-
-    m = get_aligner(subject, xfmname, epifile=reference, **kwargs)
-    m.save_callback = view_callback if view_only_mode else save_callback
-    m.configure_traits()
-
-    return m
+    from .webgl import aligner
+
+    return aligner.show(
+        subject,
+        xfmname,
+        reference=reference,
+        view_only=view_only,
+        cmap=cmap,
+        mesh_color=mesh_color,
+        mesh_opacity=mesh_opacity,
+        open_browser=open_browser,
+        autoclose=autoclose,
+        port=port,
+        recache=recache,
+        types=types,
+        title=title,
+        display_url=display_url,
+        token=token,
+        template=template,
+    )
 
 
 def fs_manual(subject, xfmname, **kwargs):
@@ -136,8 +189,8 @@ def manual(
     ALSO: all the freesurfer environment stuff shouldn't be necessary, except that
     I don't know what vox2ras-tkr is doing.
 
-    Renamed from fs_manual() to manual(), since old manual() function was no longer 
-    supported (or functional) for a while due to changes in mayavi.
+    Renamed from fs_manual() to manual(), since the old manual() function had
+    been unsupported for a while.
 
 
     Parameters

cortex/dataset/view2D.py

@@ -7,7 +7,7 @@
 import numpy.typing as npt
 
 from .. import options
-from .views import Dataview, Volume, Vertex, VolumeRGB, VertexRGB
+from .views import Dataview, Volume, Vertex, VolumeRGB, VertexRGB, json_attrs
 from .viewRGB import warn_alpha_range
 from .braindata import BrainData, VolumeData, VertexData
 
@@ -146,7 +146,7 @@ def to_json(self, simple=False):
         """
         sdict = dict(data=[[self.dim1.name, self.dim2.name]],
             state=self.state, 
-            attrs=self.attrs, 
+            attrs=json_attrs(self.attrs), 
             desc=self.description,
             cmap=[self.cmap] )