Scene encoders: transformers and sequence models¶

Seed-based · 5 methods--set backbone=16 options

These encoders attend over points, voxels, octree nodes or point patches: the Point Transformer line (V1 to V3, with Sonata, Concerto and Utonia weights), LitePT, Volt, OctFormer, and the point-patch tokens of PointViT, PointMamba and Uni3D. What all scene encoders share is on the overview.

Use it¶

GraspNet-baselineGSNetScale-Balanced-GraspEconomicGraspFineGrasp

grasppanda compose graspnet_baseline/graspnet-baseline_pointnet2_g1b-realsense_18e \
    --set backbone=ptv3 -o graspnet-baseline_ptv3.yaml
grasppanda compose graspnet_baseline/graspnet-baseline_pointnet2_g1b-realsense_18e --list backbone

grasppanda compose gsnet/gsnet_sunet14_g1b-realsense_10e \
    --set backbone=ptv3 -o gsnet_ptv3.yaml
grasppanda compose gsnet/gsnet_sunet14_g1b-realsense_10e --list backbone

grasppanda compose scale_balanced_grasp/sbg_pointnet2-transformer_g1b-realsense_18e \
    --set backbone=ptv3 -o scale-balanced-grasp_ptv3.yaml
grasppanda compose scale_balanced_grasp/sbg_pointnet2-transformer_g1b-realsense_18e --list backbone

grasppanda compose economicgrasp/economicgrasp_sunet14_g1b-realsense_10e \
    --set backbone=ptv3 -o economicgrasp_ptv3.yaml
grasppanda compose economicgrasp/economicgrasp_sunet14_g1b-realsense_10e --list backbone

grasppanda compose finegrasp/finegrasp_sunet14_g1b-realsense_10e \
    --set backbone=ptv3 -o finegrasp_ptv3.yaml
grasppanda compose finegrasp/finegrasp_sunet14_g1b-realsense_10e --list backbone

The first command writes a checked config; the second lists every option of the slot under that method, with what was measured. Then grasppanda probe <file> tries the config on real frames and grasppanda train <file> trains it (Swap a component). Each method page lists the same: GraspNet-baseline · GSNet · Scale-Balanced-Grasp · EconomicGrasp · FineGrasp.

Options¶

A probe shows that a combination trains on real frames of GraspNet-1Billion, not how good it is. Accuracy is claimed only for a method trained and evaluated here.

Option	What it is	Runs	Trains like shipped
`litept`	LitePT-S (official LitePT, Pointcept LitePT-v1) on a 5 mm grid.	5/5	5/5 on a par
`litept-structured3d`	LitePT-S started from the released Structured3D weights (MIT), on a 5 mm grid.	5/5	3/5 on a par · 2 worse worse under EconomicGrasp, FineGrasp
`octformer`	OctFormer (ScanNet segmentation layout) on an octree with 5 mm leaves.	5/5	4/5 on a par · 1 worse worse under FineGrasp
`point-mamba`	PointMamba: Mamba blocks over 1024 point-patch tokens in Hilbert order.	5/5	2/5 on a par · 3 worse worse under GSNet, EconomicGrasp, FineGrasp
`point-transformer-v1`	Point Transformer (V1) U-Net, PointTransformer-Seg26 of Pointcept, as published.	5/5	1/5 on a par · 4 diverging
`point-transformer-v2`	Point Transformer V2 (Pointcept PT-v2m2) with grid pooling scaled to a table top.	5/5	1/5 on a par · 4 diverging
`point-vit`	Plain transformer over 1024 point-patch tokens (the Point-BERT / Point-MAE encoder).	5/5	0/5 on a par · 5 worse
`ptv3`	Point Transformer V3 (encoder + decoder, Pointcept PT-v3m1) on a 5 mm grid.	5/5	4/5 on a par · 1 worse worse under FineGrasp
`ptv3-encoder`	Point Transformer V3 without its decoder (encoder only, PT-v3m1), on a 5 mm grid.	2/5	2/2 on a par
`ptv3-ppt`	PTv3 started from the Point Prompt Training weights (ScanNet + S3DIS + Structured3D, MIT).	5/5	3/5 on a par · 2 diverging diverging under Scale-Balanced-Grasp, FineGrasp
`ptv3-scannet200`	Point Transformer V3 started from the released ScanNet200 weights (MIT), on a 5 mm grid.	4/5 not FineGrasp (unstable twice in the short probe)	4/4 on a par
`ptv3-sonata`	The PTv3 variant of Sonata and Concerto (Pointcept PT-v3m2), encoder only, 5 mm grid.	3/5 not GSNet, EconomicGrasp (unstable twice in the short probe)	2/3 on a par · 1 diverging
`ptv3-sonata-small`	The small Sonata encoder with its released self-supervised weights (CC BY-NC 4.0), 5 mm grid.	2/5	2/2 on a par
`ptv3-utonia`	The PTv3 variant of Utonia (Pointcept PT-v3m3): PTv3Sonata plus rotary position embedding, encoder only, 5 mm grid.	3/5 not GSNet, EconomicGrasp (unstable twice in the short probe)	1/3 on a par · 2 diverging
`uni3d`	Uni3D-S: 1024 point-patch tokens through the blocks of timm's EVA-02 small.	5/5	1/5 on a par · 1 diverging · 3 worse
`volt`	Volt-S: a plain transformer with global attention over 2.5 cm voxel-patch tokens.	5/5	2/5 on a par · 1 diverging · 2 worse

What the columns say, and who is left out

Runs: out of the methods the option is a swap under, those under which its probe trained on real frames without error. A method that ships the option is named own and not counted; a method still undecided is counted and named. Measured on 20 training iterations on 8 RealSense frames of GraspNet-1Billion (one scene), seed 0, one run per cell; nothing was measured on GraspClutter6D or on Kinect frames.
Trains like shipped: out of the methods under which the swap ran, those under which every comparable loss term ended within the method's tolerance of the component it ships, with nothing rising; a method left out is named with why. Measured on 250 iterations on 8 RealSense frames of GraspNet-1Billion (one scene), seed 0, at a constant learning rate (FineGrasp: its own warm-up), beside the method as shipped trained the same way twice.
How we measure defines every word of these columns.

Who is left out, and why:

point-transformer-v1, trains like shipped: diverging under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp.
point-transformer-v2, trains like shipped: diverging under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp.
point-vit, trains like shipped: worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.
ptv3-encoder, runs: not GSNet, EconomicGrasp, FineGrasp (unstable twice in the short probe).
ptv3-encoder, trains like shipped: not compared under GSNet, EconomicGrasp, FineGrasp: the swap did not run at level 1.
ptv3-scannet200, trains like shipped: not compared under FineGrasp: the swap did not run at level 1.
ptv3-sonata, trains like shipped: diverging under FineGrasp; not compared under GSNet, EconomicGrasp: the swap did not run at level 1.
ptv3-sonata-small, runs: not GSNet, EconomicGrasp, FineGrasp (unstable twice in the short probe).
ptv3-sonata-small, trains like shipped: not compared under GSNet, EconomicGrasp, FineGrasp: the swap did not run at level 1.
ptv3-utonia, trains like shipped: diverging under Scale-Balanced-Grasp, FineGrasp; not compared under GSNet, EconomicGrasp: the swap did not run at level 1.
uni3d, trains like shipped: worse under GSNet, EconomicGrasp, FineGrasp; diverging under Scale-Balanced-Grasp.
volt, trains like shipped: worse under Scale-Balanced-Grasp, FineGrasp; diverging under EconomicGrasp.

Sources and licences

Code base: the upstream repository, commit and licence that THIRD_PARTY_NOTICES.md lists for the class's file (code adapted from it, or a module tree that follows it so that its weights load); written here: no third-party code. Extra: the optional install the class needs (every extra). Output: where the features live and how many channels the preset returns; compose sets a free out_channels to the width the method's other stages need. Released weights carry licences of their own.

Class	Options	Work	Code base	Extra	Output
`LitePT`	`litept`, `litept-structured3d`	LitePT (Yue et al., CVPR 2026)	Pointcept/Pointcept @ `1342eda`, MIT	`spconv`	dense · 72
`OctFormer`	`octformer`	OctFormer (Wang, ACM TOG / SIGGRAPH 2023)	octree-nn/octformer @ `c2975ab`, MIT	`ocnn`	dense · width set by compose
`PointMamba`	`point-mamba`	PointMamba (Liang et al., NeurIPS 2024)	Pang-Yatian/Point-MAE @ `7445a68`, MIT; LMD0311/PointMamba @ `80858e0`, Apache-2.0	`mamba-ssm`	subsampled, 1024 points · width set by compose
`PointTransformerV1`	`point-transformer-v1`	Point Transformer (Zhao et al., ICCV 2021)	Pointcept/Pointcept @ `1342eda`, MIT	—	dense · width set by compose
`PointTransformerV2`	`point-transformer-v2`	Point Transformer V2 (Wu et al., NeurIPS 2022)	Pointcept/Pointcept @ `1342eda`, MIT	—	dense · width set by compose
`PointViT`	`point-vit`	Point-BERT (Yu et al., CVPR 2022) and Point-MAE (Pang et al., ECCV 2022) encoder	Pang-Yatian/Point-MAE @ `7445a68`, MIT	—	subsampled, 1024 points · width set by compose
`PTv3`	`ptv3`, `ptv3-encoder`, `ptv3-ppt`, `ptv3-scannet200`	Point Transformer V3 (Wu et al., CVPR 2024)	Pointcept/Pointcept @ `1342eda`, MIT	`spconv`	dense · 64; `ptv3-encoder`: 992
`PTv3Sonata`	`ptv3-sonata`, `ptv3-sonata-small`	Sonata (Wu et al., CVPR 2025), Concerto (Zhang et al., NeurIPS 2025)	Pointcept/Pointcept @ `1342eda`, MIT	`spconv`	dense · `ptv3-sonata` 1232, `ptv3-sonata-small` 992
`PTv3Utonia`	`ptv3-utonia`	Utonia (Zhang et al., ICML 2026)	Pointcept/Pointcept @ `1342eda`, MIT	`spconv`	dense · 1386
`Uni3D`	`uni3d`	Uni3D (Zhou et al., ICLR 2024)	Pang-Yatian/Point-MAE @ `7445a68`, MIT; baaivision/Uni3D @ `64e03c3`, MIT	`timm`	subsampled, 1024 points · width set by compose
`Volt`	`volt`	Volt (Yilmaz et al., ECCV 2026, arXiv:2604.19609)	YilmazKadir/Volt @ `8dcd608`, MIT	—	dense · width set by compose

Released weights that load by name

Each file was downloaded and loaded strictly into its class (every tensor of the class from the file, with its shape). Write the name as pretrained: of the encoder; building the model without the file prints the command that fetches it. A file on a hub is fetched at the revision it was loaded from (the last column). No file was evaluated for grasping.

`pretrained:`	Class	Work	Trained on	Licence	MB	Strict load with	File, revision
`ptv3-scannet200`	`PTv3`	Point Transformer V3 (Wu et al., CVPR 2024)	supervised ScanNet200 semantic segmentation	MIT	554	defaults	hub @ `5e25cfa`
`ptv3-s3dis-rpe`	`PTv3`	Point Transformer V3 (Wu et al., CVPR 2024)	supervised S3DIS semantic segmentation, relative position bias	MIT	554	`enable_rpe: true`, `enc_patch_size: [128, 128, 128, 128, 128]`, `dec_patch_size: [128, 128, 128, 128]`	hub @ `5e25cfa`
`ptv3-nuscenes`	`PTv3`	Point Transformer V3 (Wu et al., CVPR 2024)	supervised nuScenes lidar segmentation (outdoor)	MIT	554	`in_channels: 4`	hub @ `5e25cfa`
`sonata-small`	`PTv3Sonata`	Sonata (Wu et al., CVPR 2025)	Sonata self-supervised pre-training, small encoder	CC BY-NC 4.0 (NC)	154	`enc_depths: [2, 2, 2, 6, 2]`, `enc_channels: [32, 64, 128, 256, 512]`, `enc_num_head: [2, 4, 8, 16, 32]`	hub @ `df99897`
`sonata`	`PTv3Sonata`	Sonata (Wu et al., CVPR 2025)	Sonata self-supervised pre-training, base encoder	CC BY-NC 4.0 (NC)	434	defaults	hub @ `df99897`
`concerto-base-origin`	`PTv3Sonata`	Concerto (Zhang et al., NeurIPS 2025)	Concerto joint 2-D/3-D self-supervised pre-training: the model of the paper	CC BY-NC 4.0 (NC)	433	defaults	hub @ `efc1307`
`litept-small-structured3d`	`LitePT`	LitePT (Yue et al., CVPR 2026)	supervised Structured3D semantic segmentation (synthetic rooms)	MIT	152	defaults	hub @ `a8e76e9`
`litept-small-nuscenes`	`LitePT`	LitePT (Yue et al., CVPR 2026)	supervised nuScenes lidar segmentation (outdoor)	MIT	152	`in_channels: 4`	hub @ `a8e76e9`
`ptv3-ppt-s3dis`	`PTv3`	PPT (Wu et al., CVPR 2024) with Point Transformer V3	Point Prompt Training: supervised jointly on ScanNet, S3DIS and Structured3D with one set of normalisation layers per dataset (0 ScanNet, 1 S3DIS, 2 Structured3D)	MIT	446	`enable_rpe: true`, `enc_channels: [32, 64, 128, 256, 384]`, `enc_num_head: [2, 4, 8, 16, 24]`, `enc_patch_size: [128, 128, 128, 128, 128]`, `dec_channels: [64, 64, 128, 256]`, `dec_num_head: [4, 4, 8, 16]`, `dec_patch_size: [128, 128, 128, 128]`, `pretrained_condition: 0`	hub @ `5e25cfa`

Compare under every method

Choosing¶

Serialised voxels: ptv3 and litept run on the 5 mm grid of the sparse encoders and come with released weights (ptv3-scannet200, ptv3-ppt, litept-structured3d; the Sonata weights are non-commercial).
The unit of length is a setting of every encoder here: grid_sizes of point-transformer-v2, grid_size × kernel_size (the token edge) of volt, voxel_size of octformer; point-transformer-v1 sub-samples by ratio and has none.
Point-patch tokens (point-vit, point-mamba, uni3d) were designed for single objects; their tokens become the seeds, and num_group must equal the seeds of the method.
Attention cost: point-vit attends globally (quadratic in the tokens), point-mamba is linear but runs on CUDA only.
The encoders checked against their authors' code (PTv3, its Sonata and Utonia variants, LitePT) are on Encoders checked against their authors' code.

Details¶

The settings that matter

Point Transformer V2: grid_sizes is the cell edge of the four poolings, in metres; the class defaults are for rooms, the preset divides the ScanNet values by 4.

model:
  backbone:
    type: PointTransformerV2
    grid_sizes: [0.015, 0.0375, 0.09375, 0.234375]   # metres

Volt: grid_size × kernel_size is the edge of a token (2.5 cm in the preset); smaller tokens resolve finer geometry and cost quadratically more attention. embed_dim / num_heads must be 64. The authors state that Volt overfits without their training recipe (strong augmentation, distillation, EMA), which is not part of this toolbox.

OctFormer (extra ocnn): voxel_size is the edge of an octree leaf; a frame must fit a cube of voxel_size * 2**octree_depth; patch_size octree nodes form one attention group.

PointViT, PointMamba, Uni3D (extras mamba-ssm for PointMamba, timm for Uni3D):

model:
  backbone:
    type: PointViT                 # or PointMamba, or Uni3D
    num_group: 1024                # tokens per cloud = points with features = seeds
    group_size: 32                 # nearest points per token (Uni3D: 64)
    coord_scale: 2.0               # metres -> the unit-sphere scale of the object models

PTv3 and LitePT take the sparse grid (grid_size) and, with pretrained:, a release by name (below). ptv3-encoder sets enc_mode: true: no decoder, the encoder stages concatenated on every voxel. ptv3-ppt holds one set of normalisation layers per training dataset; pretrained_condition picks one. Other files of the same releases (the larger PPT model, the later Sonata file, the large Concerto model, the larger LitePT models) have the same layout and are expected to load the same way; they were not downloaded, so pass such a file as a path.

Released weights for the point-patch encoders and Volt

These load through tools/convert_encoder_checkpoint.py, which reads the config you composed and writes the keys where the encoder sits:

python -m tools.convert_encoder_checkpoint pretrain.pth point-mae.pth \
    --config <composed config>
grasppanda train <composed config> \
    --opts runtime.load_from=point-mae.pth runtime.allow_weight_mismatch=true

File	MB	Fits	Encoder options	Trained on	Licence	SHA-256, first 16
Point-MAE `pretrain.pth`	348	`PointViT`	defaults	ShapeNet objects, masked autoencoding	repository MIT	`27ded932bb0a2625`
MaskPoint `pretrain_shapenet.pth`	286	`PointViT`	`token_dim: 256`	ShapeNet objects	repository BSD-3-Clause	`8173353d4117669c`
PointMamba `pretrain.pth`	64	`PointMamba`	defaults	ShapeNet objects	repository Apache-2.0	`3d1a89683d8b135d`
Uni3D `uni3d-ti/model.pt`	13	`Uni3D`	`timm_model: eva02_tiny_patch14_224`	object collections with colour, aligned with CLIP	code MIT; the model card states none	`6ff7e821e6997cb8`
Uni3D `uni3d-s/model.pt`	46	`Uni3D`	the preset's	the same	the same	`9a6342fda2245f00`
Volt-S `joint_training_small`	381	`Volt`	`in_channels: 6`, `grid_size: 0.02`, or `--fresh tokenizer.proj` with the preset	ScanNet, ScanNet200, ScanNet++, ARKitScenes, with distillation	MIT	`2ce43d64b17999fa`

The checksums are those of the files as downloaded on 2026-10-13. All but Volt were trained on single objects scaled to the unit sphere: on a metric scene they are an initialisation of the transformer, not a feature extractor. With other encoder sizes nothing is written, and the tool names the option to change.

Pitfalls met while building this

mamba-ssm on PyPI is a source build. Prebuilt wheels per torch, CUDA and Python are on its releases page.
import mamba_ssm compiles a Triton helper and fails with cannot find -lcuda when the machine has libcuda.so.1 but no libcuda.so: point TRITON_LIBCUDA_PATH at a directory with a link of that name.
PointMamba has no CPU path; on the CPU it says so.
ocnn runs its convolutions on Triton kernels by default, which did not compile here; OctFormer sets ocnn's pure-PyTorch path (conv_method: block_gemm).
OctFormer's features depend on the batch: attention groups are counts of z-ordered nodes of the whole batch, as in the reference.
The default Concerto files are not the paper's Concerto. The model card calls them pre-release Utonia models; the paper's model is concerto-base-origin.
Object checkpoints use other sizes. MaskPoint's ShapeNet file has the Point-BERT layout (token_dim: 256); its ScanNet file is a 3DETR encoder and does not fit PointViT.

Not wrapped

Candidate	Why not
Swin3D	built on MinkowskiEngine plus its own CUDA package, neither a dependency here.
Superpoint Transformer / EZ-SP	consumes a superpoint partition computed outside the network.
Mamba3D, Point Cloud Mamba	no licence file.
PointGPT	its blocks differ from `PointViT`'s, so its checkpoints need a second block type.
ReCon, ACT, PCP-MAE, Point-JEPA, Point-BERT's own file	checkpoints behind links that were not downloaded; nothing is claimed.
Point-M2AE	a hierarchical encoder, not the shared single-scale layout.
Point-MoE, D-DITR, LAM3C weights for PTv3	a mixture-of-experts block (Point-MoE); AGPL-3.0 (D-DITR) and non-commercial (LAM3C) weights, not downloaded.