Scene encoders: sparse-voxel networks¶

Seed-based · 5 methods--set backbone=8 options

These encoders quantise the cloud to a voxel grid they own and run sparse convolutions on the occupied voxels, then hand every point the feature of its voxel. The grid edge, 5 mm in every preset, is their unit of length. What all scene encoders share is on the overview.

Use it¶

GraspNet-baselineGSNetScale-Balanced-GraspEconomicGraspFineGrasp

grasppanda compose graspnet_baseline/graspnet-baseline_pointnet2_g1b-realsense_18e \
    --set backbone=minkunet34c -o graspnet-baseline_minkunet34c.yaml
grasppanda compose graspnet_baseline/graspnet-baseline_pointnet2_g1b-realsense_18e --list backbone

grasppanda compose gsnet/gsnet_sunet14_g1b-realsense_10e \
    --set backbone=minkunet34c -o gsnet_minkunet34c.yaml
grasppanda compose gsnet/gsnet_sunet14_g1b-realsense_10e --list backbone

grasppanda compose scale_balanced_grasp/sbg_pointnet2-transformer_g1b-realsense_18e \
    --set backbone=minkunet34c -o scale-balanced-grasp_minkunet34c.yaml
grasppanda compose scale_balanced_grasp/sbg_pointnet2-transformer_g1b-realsense_18e --list backbone

grasppanda compose economicgrasp/economicgrasp_sunet14_g1b-realsense_10e \
    --set backbone=minkunet34c -o economicgrasp_minkunet34c.yaml
grasppanda compose economicgrasp/economicgrasp_sunet14_g1b-realsense_10e --list backbone

grasppanda compose finegrasp/finegrasp_sunet14_g1b-realsense_10e \
    --set backbone=minkunet34c -o finegrasp_minkunet34c.yaml
grasppanda compose finegrasp/finegrasp_sunet14_g1b-realsense_10e --list backbone

The first command writes a checked config; the second lists every option of the slot under that method, with what was measured. Then grasppanda probe <file> tries the config on real frames and grasppanda train <file> trains it (Swap a component). Each method page lists the same: GraspNet-baseline · GSNet · Scale-Balanced-Grasp · EconomicGrasp · FineGrasp.

Options¶

A probe shows that a combination trains on real frames of GraspNet-1Billion, not how good it is. Accuracy is claimed only for a method trained and evaluated here.

Option	What it is	Runs	Trains like shipped
`minkunet18a`	MinkUNet18A of MinkowskiEngine: two residual blocks per stage, on a 5 mm grid.	5/5	4/5 on a par · 1 worse worse under EconomicGrasp
`minkunet18a-openscene`	MinkUNet18A started from the OpenScene weights (distilled from CLIP features, ScanNet).	5/5	4/5 on a par · 1 worse worse under EconomicGrasp
`minkunet34c`	MinkUNet34C (Res16UNet34C) of MinkowskiEngine, trained from scratch on a 5 mm grid.	5/5	4/5 on a par · 1 worse worse under EconomicGrasp
`minkunet34c-pointcontrast`	MinkUNet34C started from the PointContrast weights (pre-trained on partial RGB-D frames).	5/5	5/5 on a par
`oacnns`	OA-CNNs (Pointcept oacnns-v1m1) on a 5 mm grid: a sparse CNN with adaptive pooling.	2/5	2/2 on a par
`sparse-unet14`	SparseUNet14: a 14-layer sparse residual U-Net on a 5 mm grid, one feature per point.	2/2 own in GSNet, EconomicGrasp, FineGrasp	2/2 on a par
`spunet`	SpUNet, the deep sparse U-Net of Pointcept (MinkUNet-34C block counts), on a 5 mm grid.	4/5 not FineGrasp (unstable twice in the short probe)	4/4 on a par
`spunet-gaussiancross`	SpUNet started from the GaussianCross weights (self-supervised on ScanNet, MIT), 5 mm grid.	5/5	3/5 on a par · 1 diverging · 1 worse worse under GSNet; diverging under Scale-Balanced-Grasp

What the columns say, and who is left out

Runs: out of the methods the option is a swap under, those under which its probe trained on real frames without error. A method that ships the option is named own and not counted; a method still undecided is counted and named. Measured on 20 training iterations on 8 RealSense frames of GraspNet-1Billion (one scene), seed 0, one run per cell; nothing was measured on GraspClutter6D or on Kinect frames.
Trains like shipped: out of the methods under which the swap ran, those under which every comparable loss term ended within the method's tolerance of the component it ships, with nothing rising; a method left out is named with why. Measured on 250 iterations on 8 RealSense frames of GraspNet-1Billion (one scene), seed 0, at a constant learning rate (FineGrasp: its own warm-up), beside the method as shipped trained the same way twice.
How we measure defines every word of these columns.

Who is left out, and why:

oacnns, runs: not GSNet, EconomicGrasp, FineGrasp (unstable twice in the short probe).
oacnns, trains like shipped: not compared under GSNet, EconomicGrasp, FineGrasp: the swap did not run at level 1.
spunet, trains like shipped: not compared under FineGrasp: the swap did not run at level 1.

Sources and licences

Code base: the upstream repository, commit and licence that THIRD_PARTY_NOTICES.md lists for the class's file (code adapted from it, or a module tree that follows it so that its weights load); written here: no third-party code. Extra: the optional install the class needs (every extra). Output: where the features live and how many channels the preset returns; compose sets a free out_channels to the width the method's other stages need. Released weights carry licences of their own.

Class	Options	Work	Code base	Extra	Output
`MinkUNet`	`minkunet18a`, `minkunet18a-openscene`, `minkunet34c`, `minkunet34c-pointcontrast`	MinkUNet (Choy et al., CVPR 2019)	NVIDIA/MinkowskiEngine @ `02fc608`, MIT	`spconv`	dense · width set by compose
`OACNNs`	`oacnns`	OA-CNNs (Peng et al., CVPR 2024), Pointcept oacnns-v1m1	Pointcept/Pointcept @ `1342eda`, MIT	`spconv`	dense · width set by compose
`SparseUNet14`	`sparse-unet14`	MinkUNet14D (Choy et al., CVPR 2019), as GSNet, EconomicGrasp and FineGrasp use it	written here	`spconv`	dense · width set by compose
`SpUNet`	`spunet`, `spunet-gaussiancross`	SpUNet of Pointcept, MinkUNet-34C layout (Choy et al., CVPR 2019)	Pointcept/Pointcept @ `1342eda`, MIT	`spconv`	dense · width set by compose

Released weights that load by name

Each file was downloaded and loaded strictly into its class (every tensor of the class from the file, with its shape). Write the name as pretrained: of the encoder; building the model without the file prints the command that fetches it. A file on a hub is fetched at the revision it was loaded from (the last column). No file was evaluated for grasping.

`pretrained:`	Class	Work	Trained on	Licence	MB	Strict load with	File, revision
`gaussiancross-scannet-lin`	`SpUNet`	GaussianCross (Yao et al., ACM MM 2025)	GaussianCross self-supervised pre-training on ScanNet: the linear-probing checkpoint of the release, whose class layer lies outside the encoder	MIT	157	`out_channels: null`	hub @ `3cbceee`
`gaussiancross-scannet`	`SpUNet`	GaussianCross (Yao et al., ACM MM 2025)	GaussianCross pre-training, fine-tuned for ScanNet semantic segmentation	MIT	470	defaults	hub @ `3cbceee`
`volt-teacher-scannet`	`SpUNet`	Volt (Yilmaz et al., ECCV 2026)	ScanNet semantic segmentation; a teacher network of Volt	MIT	470	defaults	hub @ `9dcb91d`
`pointcontrast-nce`	`MinkUNet`	PointContrast (Xie et al., ECCV 2020)	PointContrast self-supervised pre-training (PointInfoNCE) on pairs of partial ScanNet RGB-D frames; Res16UNet34C, 3 input channels	unstated (code MIT; ScanNet terms apply)	303	`out_channels: 32`	dropbox.com
`mask3d-scannet`	`MinkUNet`	Mask3D (Schult et al., ICRA 2023)	the Res16UNet34C backbone inside Mask3D, trained for instance segmentation of ScanNet (the model its S3DIS models are fine-tuned from); 3 input channels	unstated (code MIT; ScanNet terms apply)	159	`out_channels: 20`, `stem_kernel: 5`	omnomnom.vision.rwth-aachen.de
`openscene-scannet-openseg`	`MinkUNet`	OpenScene (Peng et al., CVPR 2023)	OpenScene: MinkUNet18A distilled on ScanNet (2 cm voxels) to predict per-point OpenSeg / CLIP features (768 channels); its input is geometry alone, three constant ones per point, exactly what GSNet, EconomicGrasp and GraspNet-baseline feed (FineGrasp feeds six channels)	unstated (code Apache-2.0; ScanNet terms apply)	187	`layers: [2, 2, 2, 2, 2, 2, 2, 2]`, `planes: [32, 64, 128, 256, 128, 128, 96, 96]`, `stem_kernel: 5`, `out_channels: 768`, `final_bias: false`	cvg-data.inf.ethz.ch

Compare under every method

Choosing¶

The grid is the first thing to sweep. 5 mm keeps the grid of the reference methods; weights from rooms were trained at 2 to 2.5 cm (--opts model.backbone.voxel_size=0.01).
Weights whose input is what the methods feed: minkunet18a-openscene was trained on three constant ones per point, so its first layer loads too (not under FineGrasp, which feeds coordinates and normals).
Pre-training on single partial frames: minkunet34c-pointcontrast, trained on pairs of RGB-D frames, the regime of one table-top view.
A permissive licence, self-supervised: spunet-gaussiancross (MIT).
Deeper and pretrained encoders usually want a lower rate and a warm-up than the recipes tuned for a 14-layer network; none of this was tuned here.

Details¶

The settings that matter

model:
  backbone:
    type: MinkUNet
    in_channels: 3            # what the method feeds; compose sets it
    out_channels: 512         # free 1x1 projection; null returns the decoder width
    voxel_size: 0.005         # the encoder's unit of length, in metres
    layers: [2, 2, 2, 2, 2, 2, 2, 2]
    planes: [32, 64, 128, 256, 128, 128, 96, 96]
    stem_kernel: 5            # 5 in MinkUNet; 3 in the Res16UNet34C checkpoints

MinkUNet is one class for the whole family of MinkowskiEngine; layers, planes and block select the member (only 18A and 34C have presets):

Member	`layers`	Decoder half of `planes` (the encoder half is 32, 64, 128, 256)	`block`
14A / 14B / 14C / 14D	`[1, 1, 1, 1, 1, 1, 1, 1]`	128-128-96-96 / 128-128-128-128 / 192-192-128-128 / 384-384-384-384	`basic`
18A / 18B / 18D	`[2, 2, 2, 2, 2, 2, 2, 2]`	as 14A / 14B / 14D	`basic`
34A / 34B / 34C	`[2, 3, 4, 6, 2, 2, 2, 2]`	256-128-64-64 / 256-128-64-32 / 256-128-96-96	`basic`
50 / 101	34's / `[2, 3, 4, 23, 2, 2, 2, 2]`	256-128-96-96	`bottleneck`

Sparse convolutions go through grasppanda.ops.sparse (spconv underneath; one spconv extra per CUDA build).

How released weights load

Name a release as pretrained: (table below) or give a file path:

model:
  backbone:
    type: SpUNet
    in_channels: 3
    out_channels: 512
    grid_size: 0.005
    pretrained: gaussiancross-scannet-lin   # a name, or a file path
    pretrained_mismatch: skip               # error (default): refuse any tensor that does not load

Only the encoder is loaded. The loader finds the prefix under which the file keeps it, ignores heads and optimiser state, and re-lays kernels where the original library stores them differently. No pickled code runs.
pretrained_mismatch: skip keeps the initialisation of the tensors that do not fit and lists them in the log. A grasp model nearly always needs it, for two layers: the stem (releases read colour and normals, the methods feed constant ones) and the final projection when out_channels differs.
A grasp checkpoint still wants the released file, because pretrained loads when the encoder is built. Keep the file, or set pretrained: null to evaluate a grasp checkpoint.
Other files of the same releases (the other GaussianCross models, the other Volt teachers, the other Mask3D checkpoints) have the same layout and are expected to load the same way; they were not downloaded. Pass such a file as a path, with the options the load report asks for.

Pitfalls met while building this

The stem is the tensor that does not load, and it reads the input. A 6-channel stem loads under FineGrasp, but nothing says its six channels mean what the release was trained on. Loading says nothing about meaning.
Licences differ inside one family. PointContrast, Mask3D and OpenScene publish code under MIT or Apache-2.0 and say nothing about the weights, which were trained on ScanNet.
SpUNet is not MinkUNet34C. Same block counts and widths, but the width changes in another layer; a checkpoint of one does not fit the other, and the loader says so rather than loading half of it.
Two kernel sizes may not share a spconv indice key. A 5³ stem sharing the key of the 3³ blocks ran on the CPU and failed on the GPU; the stem of MinkUNet has its own key.
Legacy-format checkpoints that pickle foreign objects cannot be read without executing them and are refused; re-save the state dict where it was written.

Not wrapped

Candidate	Why not
Contrastive Scene Contexts, Res16UNet34C pre-training	a legacy pickle with objects of its code base; the reader refuses to execute it.
FCAF3D / TR3D sparse backbones (MMDetection3D)	layers the sparse facade does not have, and they return a pyramid of coarse levels, not per-point features.
SPVCNN / SPVNAS	runs on TorchSparse, a second sparse library with no backend here.
PonderV2 `SpUNet-v1m3`	another module tree than `SpUNet`; needs a key map. Small follow-up.
Mosaic3D SpUNet	no licence on the model card.
OA-CNNs weights	none released; `oacnns` starts from random weights.