Scene encoders: point-based networks¶

Seed-based · 5 methods--set backbone=23 options

These encoders work on the raw point cloud: set abstraction (PointNet++ and its descendants), residual MLPs, graph convolutions and random sampling. Their radii and neighbour counts set how far a point sees. What all scene encoders share is on the overview.

Use it¶

GraspNet-baselineGSNetScale-Balanced-GraspEconomicGraspFineGrasp

grasppanda compose graspnet_baseline/graspnet-baseline_pointnet2_g1b-realsense_18e \
    --set backbone=pointnext -o graspnet-baseline_pointnext.yaml
grasppanda compose graspnet_baseline/graspnet-baseline_pointnet2_g1b-realsense_18e --list backbone

grasppanda compose gsnet/gsnet_sunet14_g1b-realsense_10e \
    --set backbone=pointnext -o gsnet_pointnext.yaml
grasppanda compose gsnet/gsnet_sunet14_g1b-realsense_10e --list backbone

grasppanda compose scale_balanced_grasp/sbg_pointnet2-transformer_g1b-realsense_18e \
    --set backbone=pointnext -o scale-balanced-grasp_pointnext.yaml
grasppanda compose scale_balanced_grasp/sbg_pointnet2-transformer_g1b-realsense_18e --list backbone

grasppanda compose economicgrasp/economicgrasp_sunet14_g1b-realsense_10e \
    --set backbone=pointnext -o economicgrasp_pointnext.yaml
grasppanda compose economicgrasp/economicgrasp_sunet14_g1b-realsense_10e --list backbone

grasppanda compose finegrasp/finegrasp_sunet14_g1b-realsense_10e \
    --set backbone=pointnext -o finegrasp_pointnext.yaml
grasppanda compose finegrasp/finegrasp_sunet14_g1b-realsense_10e --list backbone

The first command writes a checked config; the second lists every option of the slot under that method, with what was measured. Then grasppanda probe <file> tries the config on real frames and grasppanda train <file> trains it (Swap a component). Each method page lists the same: GraspNet-baseline · GSNet · Scale-Balanced-Grasp · EconomicGrasp · FineGrasp.

Options¶

A probe shows that a combination trains on real frames of GraspNet-1Billion, not how good it is. Accuracy is claimed only for a method trained and evaluated here.

Option	What it is	Runs	Trains like shipped
`assanet`	ASSANet (NeurIPS 2021): PointNet++ with separable, anisotropic set abstraction.	5/5	5/5 on a par
`dela`	DeLA (decoupled local aggregation, 2023) on a table-top grid pyramid.	5/5	4/5 on a par · 1 diverging diverging under FineGrasp
`dgcnn`	DGCNN (ACM TOG 2019), scene-segmentation form: three EdgeConv layers at full resolution, the graph rebuilt in feature space for the second and third.	5/5	0/5 on a par · 1 diverging · 4 worse
`dgcnn-static`	DGCNN with a static spatial graph and no global vector: the local, cheap variant.	5/5	1/5 on a par · 4 worse
`pointmetabase`	PointMetaBase-L (CVPR 2023) with a table-top radius.	5/5	3/5 on a par · 2 worse worse under EconomicGrasp, FineGrasp
`pointmetabase-xl`	PointMetaBase-XL (CVPR 2023) with a table-top radius.	5/5	3/5 on a par · 1 diverging · 1 worse diverging under Scale-Balanced-Grasp; worse under FineGrasp
`pointmlp`	PointMLP (ICLR 2022), dense form: residual MLPs with the geometric affine module.	5/5	1/5 on a par · 1 diverging · 3 worse
`pointnet2`	PointNet++ in the GraspNet-baseline layout: features on a 1024-point subsample.	4/4 own in GraspNet-baseline	0/4 on a par · 4 worse
`pointnet2-dense`	PointNet++ propagated back to every input point, with the points and radii of GraspNet-baseline.	5/5	2/5 on a par · 3 worse worse under GSNet, EconomicGrasp, FineGrasp
`pointnet2-msg`	PointNet++ with multi-scale grouping (MSG), features on 1024 sampled points.	5/5	2/5 on a par · 3 worse worse under GSNet, EconomicGrasp, FineGrasp
`pointnet2-msg-dense`	PointNet++ with multi-scale grouping (MSG), propagated back to every input point.	5/5	3/5 on a par · 2 worse worse under EconomicGrasp, FineGrasp
`pointnet2-transformer`	The encoder of Scale-Balanced-Grasp: two PointNet++ abstraction levels, each followed by a point-transformer layer; features on a 1024-point subsample.	4/4 own in Scale-Balanced-Grasp	1/4 on a par · 3 worse worse under GSNet, EconomicGrasp, FineGrasp
`pointnext`	PointNeXt-S (encoder + decoder) with a table-top radius.	5/5	3/5 on a par · 2 worse worse under EconomicGrasp, FineGrasp
`pointnext-b`	PointNeXt-B (encoder + decoder) with a table-top radius: the published "B" size.	5/5	3/5 on a par · 2 worse worse under EconomicGrasp, FineGrasp
`pointnext-l`	PointNeXt-L (encoder + decoder) with a table-top radius: the published "L" size.	5/5	1/5 on a par · 4 worse
`pointnext-xl`	PointNeXt-XL (encoder + decoder) with a table-top radius: the published "XL" size.	5/5	2/5 on a par · 3 worse worse under GSNet, EconomicGrasp, FineGrasp
`pointvector`	PointVector-L (CVPR 2023) with a table-top radius.	5/5	2/5 on a par · 3 worse worse under GSNet, EconomicGrasp, FineGrasp
`pointvector-xl`	PointVector-XL (CVPR 2023) with a table-top radius.	5/5	3/5 on a par · 2 worse worse under EconomicGrasp, FineGrasp
`pyg-edgeconv`	EdgeConv layers of DGCNN on a static graph, at full resolution, through PyTorch Geometric.	4/5 not FineGrasp (unstable twice in the short probe)	1/4 on a par · 3 worse
`pyg-point-transformer`	Point Transformer vector-attention layers, at full resolution, through PyTorch Geometric.	4/5 FineGrasp undecided (unstable once, not repeated)	0/4 on a par · 4 worse
`pyg-pointgnn`	Point-GNN layers, at full resolution, through PyTorch Geometric.	4/5 FineGrasp undecided (unstable once, not repeated)	0/4 on a par · 4 worse
`pyg-pointnet`	PointNet++ layers (PointNetConv), at full resolution, through PyTorch Geometric.	5/5	0/5 on a par · 5 worse
`randlanet`	RandLA-Net (CVPR 2020): random subsampling with local feature aggregation.	5/5	1/5 on a par · 3 diverging · 1 worse

What the columns say, and who is left out

Runs: out of the methods the option is a swap under, those under which its probe trained on real frames without error. A method that ships the option is named own and not counted; a method still undecided is counted and named. Measured on 20 training iterations on 8 RealSense frames of GraspNet-1Billion (one scene), seed 0, one run per cell; nothing was measured on GraspClutter6D or on Kinect frames.
Trains like shipped: out of the methods under which the swap ran, those under which every comparable loss term ended within the method's tolerance of the component it ships, with nothing rising; a method left out is named with why. Measured on 250 iterations on 8 RealSense frames of GraspNet-1Billion (one scene), seed 0, at a constant learning rate (FineGrasp: its own warm-up), beside the method as shipped trained the same way twice.
How we measure defines every word of these columns.

Who is left out, and why:

dgcnn, trains like shipped: worse under GraspNet-baseline, GSNet, EconomicGrasp, FineGrasp; diverging under Scale-Balanced-Grasp.
dgcnn-static, trains like shipped: worse under GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.
pointmlp, trains like shipped: worse under GSNet, EconomicGrasp, FineGrasp; diverging under Scale-Balanced-Grasp.
pointnet2, trains like shipped: worse under GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.
pointnext-l, trains like shipped: worse under GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.
pyg-edgeconv, trains like shipped: worse under GSNet, Scale-Balanced-Grasp, EconomicGrasp; not compared under FineGrasp: the swap did not run at level 1.
pyg-point-transformer, trains like shipped: worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp; not compared under FineGrasp: the swap did not run at level 1.
pyg-pointgnn, trains like shipped: worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp; not compared under FineGrasp: the swap did not run at level 1.
pyg-pointnet, trains like shipped: worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.
randlanet, trains like shipped: diverging under GSNet, EconomicGrasp, FineGrasp; worse under Scale-Balanced-Grasp.

Sources and licences

Code base: the upstream repository, commit and licence that THIRD_PARTY_NOTICES.md lists for the class's file (code adapted from it, or a module tree that follows it so that its weights load); written here: no third-party code. Extra: the optional install the class needs (every extra). Output: where the features live and how many channels the preset returns; compose sets a free out_channels to the width the method's other stages need. Released weights carry licences of their own.

Class	Options	Work	Code base	Extra	Output
`ASSANet`	`assanet`	ASSANet (Qian et al., NeurIPS 2021)	guochengqian/openpoints @ `bc778ec`, MIT	—	dense · 256
`DeLA`	`dela`	DeLA (Chen et al., arXiv 2308.16532)	Matrix-ASC/DeLA @ `0f427c3`, MIT	—	dense · 256
`DGCNN`	`dgcnn`, `dgcnn-static`	DGCNN (Wang et al., ACM TOG 2019)	antao97/dgcnn.pytorch @ `07d534c`, MIT	—	dense · width set by compose
`PointMetaBase`	`pointmetabase`, `pointmetabase-xl`	PointMetaBase (Lin et al., CVPR 2023)	linhaojia13/PointMetaBase @ `c364a67`, MIT	—	dense · `pointmetabase` 32, `pointmetabase-xl` 64
`PointMLP`	`pointmlp`	PointMLP (Ma et al., ICLR 2022), dense form	ma-xu/pointMLP-pytorch @ `1ab7f04`, Apache-2.0	—	dense · width set by compose
`PointNet2`	`pointnet2`	PointNet++ (Qi et al., NeurIPS 2017), GraspNet-baseline layout	written here	—	subsampled, 1024 points · 256
`PointNet2Dense`	`pointnet2-dense`	PointNet++ (Qi et al., NeurIPS 2017), segmentation layout	written here	—	dense · 128
`PointNet2MSG`	`pointnet2-msg`	PointNet++ MSG (Qi et al., NeurIPS 2017)	facebookresearch/votenet @ `2f6d6d3`, MIT	—	subsampled, 1024 points · 256
`PointNet2MSGDense`	`pointnet2-msg-dense`	PointNet++ MSG (Qi et al., NeurIPS 2017), segmentation layout	facebookresearch/votenet @ `2f6d6d3`, MIT	—	dense · 256
`PointNet2Transformer`	`pointnet2-transformer`	Scale-Balanced-Grasp (Ma and Huang, CoRL 2022)	written here	—	subsampled, 1024 points · 256
`PointNeXt`	`pointnext`, `pointnext-b`, `pointnext-l`, `pointnext-xl`	PointNeXt (Qian et al., NeurIPS 2022)	written here	—	dense · 32; `pointnext-xl`: 64
`PointVector`	`pointvector`, `pointvector-xl`	PointVector (Deng et al., CVPR 2023)	guochengqian/openpoints @ `bc778ec`, MIT	—	dense · `pointvector` 32, `pointvector-xl` 64
`PyGPointEncoder`	`pyg-edgeconv`, `pyg-point-transformer`, `pyg-pointgnn`, `pyg-pointnet`	PointNetConv / EdgeConv / PointTransformerConv / PointGNNConv via torch_geometric (Fey and Lenssen, ICLR-W 2019)	bridge to `torch_geometric`	`torch_geometric`	dense · width set by compose
`RandLANet`	`randlanet`	RandLA-Net (Hu et al., CVPR 2020)	isl-org/Open3D-ML @ `a3d0f7a`, MIT	—	dense · width set by compose

Compare under every method

Choosing¶

The PointNet++ family of the ported methods: pointnet2 and pointnet2-transformer; -dense propagates features back to every point, -msg uses two balls per level.
Modern set abstraction: pointnext in four sizes, pointmetabase (one ball query per stage, the cheapest of them) and pointvector.
Full resolution, no sub-sampling: dgcnn and the pyg-* layers; memory grows with the number of points times the neighbours.
Cheap sampling: randlanet samples at random and pools attentively.
Radii are set for a table top by analogy. The published configurations are for rooms (first radius 0.1 m); sweep the radius before comparing encoders.

Details¶

The settings that matter, encoder by encoder

Options are changed with --opts model.backbone.<option>=<value>; the path addresses the encoder wherever an adapter put it.

PointNet++ with multi-scale grouping (pointnet2-msg, pointnet2-msg-dense):

model:
  backbone:
    type: PointNet2MSG
    in_channels: 0
    npoints: [2048, 1024, 512, 256]
    radii: [[0.02, 0.04], [0.05, 0.1], [0.1, 0.2], [0.2, 0.3]]    # one list per level
    nsamples: [[32, 64], [16, 32], [16, 16], [16, 16]]

PointNet2MSG keeps the level sizes of GraspNet-baseline (1024 seeds, 256 channels) with two balls per level; the larger ball is what a single-scale network would use, the smaller helps where points are dense. Cost grows with the sum of nsamples over the scales.

PointNeXt sizes:

Preset	`blocks`	`width`	`sa_layers` / `sa_use_res`	Parameters with the official segmentation head
`pointnext` (S)	1, 1, 1, 1, 1	32	2 / true	0.792 M
`pointnext-b`	1, 2, 3, 2, 2	32	1 / false	3.828 M
`pointnext-l`	1, 3, 5, 3, 3	32	1 / false	7.125 M
`pointnext-xl`	1, 4, 7, 4, 4	64	1 / false	41.576 M

The presets set radius: 0.04 (the room value is 0.1). Every inverted residual block runs one ball query.

PointMetaBase and PointVector take the arguments of PointNeXt (blocks, width, strides, radius, nsample). PointMetaBase shares one ball query and one offset embedding per stage; the unit suite pins its sizes to the published ones. PointVector is the scene network of the official code. Their output width is width (32 or 64).

ASSANet: use_xyz: true (set by the preset) appends the coordinates to the input features. Without it, constant input features are normalised to zero and the encoder learns nothing; the class refuses that combination in training.

PointMLP: neighbourhoods by count, anchors by ratio, so nothing to convert from rooms. gmp_dim appends a max-pooled vector of the whole frame to every point (0 turns it off).

DeLA: grid_size[0] is the spacing of the input cloud and sets the unit of the offset encoding; keep the factor of 2 between levels unless you also change k. Level sizes depend on the frame, so DeLA is dense only.

DGCNN: k neighbours per point; dynamic: true rebuilds the graph in feature space (the paper), false keeps it spatial; emb_dims is the width of the frame-level vector (0 leaves it out). Keep coords_as_features: true with constant input features.

RandLA-Net: sub_sampling_ratio and dim_output per level. Sampling is random in training and a fixed permutation in evaluation, so inference is repeatable. A cloud needs at least num_neighbors × prod(ratios[:-1]) points.

PyTorch Geometric layers (extra torch_geometric): a stack of one library convolution at full resolution; only the layers are the library's, the stack is not a published network.

Preset	`conv`	Library layer
`pyg-pointnet`	`pointnet`	`torch_geometric.nn.PointNetConv`
`pyg-edgeconv`	`edge`	`torch_geometric.nn.EdgeConv`
`pyg-point-transformer`	`point_transformer`	`torch_geometric.nn.PointTransformerConv`
`pyg-pointgnn`	`pointgnn`	`torch_geometric.nn.PointGNNConv` (all `channels` equal)

Released weights for PointNet++ and PointNeXt

tools/convert_backbone_weights.py cuts the encoder out of a released checkpoint, checks it strictly against the class and writes a file whose keys start with backbone.. Nothing is downloaded by the tool.

`--format`	Source file	Fits	Trained on	Licence	SHA-256, first 16
`pointnext-s` (`-b`, `-l`, `-xl`)	the S3DIS checkpoints linked from PointNeXt `docs/modelzoo.md`	`PointNeXt(in_channels=4, ...)` of that size	S3DIS area 5, rooms at 4 cm, colour + height	code MIT; weights not stated	S (9.6 MB): `95802f5c110e9fdd`; B, L, XL not downloaded
`votenet`	the files of `votenet_pretrained_models.zip` linked from the VoteNet README	`PointNet2(in_channels=1, radii=[0.2, 0.4, 0.8, 1.2])`	SUN RGB-D single-view scans; ScanNet	code MIT; weights not stated	SUN RGB-D: `ff645ce392676cc4`; ScanNet: `22e57a2b8df1315a` (11.5 MB each)
`csc-pointnet2`, `csc-pointnet2-dense`	the PointNet++ pre-training linked from the Contrastive Scene Contexts README	`PointNet2` (the GraspNet-baseline layout) or `PointNet2Dense`, coordinates only	ScanNet frame pairs, self-supervised	code MIT; weights not stated	`a6812612720295ff` (6.4 MB)

The checksums are those of the files as downloaded on 2026-10-13; the full digests are in tests/integration/test_backbone_released_weights.py, which checks them before it loads a file.

python -m tools.convert_backbone_weights --list
python -m tools.convert_backbone_weights partition8_4096_15k_pointnet.pth csc_pointnet2.pth \
    --format csc-pointnet2
grasppanda train <method config> \
    --opts runtime.load_from=csc_pointnet2.pth runtime.allow_weight_mismatch=true

Radii are not stored in a checkpoint. These networks were trained with room radii; they load under table-top radii without complaint, and then see the table enlarged about five times. Whether that helps is untested.
Input channels are part of the weights. PointNeXt reads colour and a height above the floor, VoteNet one height channel; a camera-frame table-top cloud has no floor height, so define what you feed.
The dataset terms of S3DIS, SUN RGB-D and ScanNet apply; treat the files as research-only until you have checked. The files were loaded strictly once; activations were not compared with the upstream code.

Pitfalls met while building this

The grouper of Scale-Balanced-Grasp wants 256 channels. pointnext*, pointmetabase* and pointvector* end on 32 or 64; compose puts the projected grouper in place, and the model is then a variant of Scale-Balanced-Grasp. The encoders whose width is free are set 256 wide by their presets for this reason.
Constant input features starve encoders that read geometry through features. DGCNN, PyGPointEncoder, RandLANet and PointMLP therefore read the coordinates themselves.
Absolute coordinates. RandLA-Net's neighbourhood encoding, DGCNN, PointMLP and the bridge read camera-frame coordinates as features, so their features are not translation invariant.
Frame-level vectors. dgcnn (emb_dims) and pointmlp (gmp_dim) append a vector of the whole frame to every point, so features depend on the workspace crop; both can be switched off.
Padded balls under mean or sum. A ball query pads a sparse ball by repeating its first hit; assanet (mean) and pointvector (sum) count it several times, as upstream does.
torch_geometric without pyg-lib cannot build a k-NN graph; the bridge builds it itself.

Not wrapped

Candidate	Why not
KPConv (rigid and deformable)	needs kernel-point dispositions, radius neighbourhoods and a grid pyramid computed in its own data pipeline with compiled code; a follow-up of its own.
KPConvX / KPConvD	expected to need the same; not attempted.
RandLA-Net, authors' repository	CC BY-NC-SA 4.0; the MIT implementation of Open3D-ML was adapted instead.
Contact-GraspNet's PointNet++ layout and weights	a non-commercial licence; its three-scale layout can be written as a `PointNet2MSG` config by hand.
`XConv` (PointCNN), `GravNetConv` of PyTorch Geometric	they build their graph inside the layer, which needs `pyg-lib`.
DeepLA-Net, LinNet, PointMixer, SPoTr, X-3D, PAConv, RepSurf, PVCNN, DeepGCN	not reached, or no licence file found earlier.