GSNet¶
Seed-based skeleton ICCV 2021 depth input equal to the reference ✓ trained here
From the point cloud of a depth image, it scores the graspness of every point, keeps 1024 seeds where it is high, picks one of 300 approach views per seed and regresses score and width over a grid of angles and depths.
Paper · Reference code (third-party) · Configs, results and notes
Run GSNet in five minutes¶
Four steps from the released checkpoint to grasps. A test parses each command and runs the demo and the Python code on a toy detector of the method's family.
1. Get the weights¶
GraspPanda trained GSNet once, but those weights are not published yet: the file to download is the checkpoint of the reference implementation (graspness_unofficial, third-party). Download minkuresunet_realsense.tar (Google Drive) and convert it once, from the root of a GraspPanda checkout:
python -m tools.convert_checkpoint minkuresunet_realsense.tar gsnet_realsense.pth \
--source-format graspness-unofficial --camera realsense
Before you use it: It loads strictly and runs, but it is far below the paper: its best-ranked grasps run into the scene, so use it with the collision filter.
Licence, checksum and what the converter writes
- Licence: academic, non-commercial (SJTU), as the reference repository; trained on GraspNet-1Billion (CC BY-NC-SA 4.0)
- SHA-256 of the download:
a0b9d85b300e40c76b66d092de62138d74757d6b7c2037f0d364f33960c54aa9 - What the converter writes:
gsnet_realsense.pth, every tensor of the downloaded file under the name of the module that plays its role here, no value changed, with a note of the file it came from. A tensor left over or missing stops the conversion with a report; nothing is ever loaded partially. The converter is a tool of the repository; it is not installed with the package. - The weights trained here (the
reproducedrows of the README): trained on GraspNet-1Billion (CC BY-NC-SA 4.0); whether and where these weights are published has not been decided.
2. Grasps on the example scene¶
grasppanda demo gsnet/gsnet_sunet14_g1b-realsense_10e \
--weights gsnet_realsense.pth --example --collision-filter
--example is a synthetic table-top scene shipped with GraspPanda: no dataset, no download. 14.8 s for the whole command, 418 grasps kept (one NVIDIA RTX A6000 shared with other jobs).
What it prints and writes, and every measured run
The command prints the config and its hash, how the weights loaded (every tensor matched, none missing), the test pipeline, the cloud, how many grasps the model predicted and how many the collision filter left (the config's variant cd: ModelFreeCollisionFilter), and the ten best grasps. It writes into work_dirs/gsnet_sunet14_g1b-realsense_10e/demo/:
grasps.npy: the grasps that are left,(G, 17)float32 in the GraspNet layout, best first, camera frame, metres;grasps.pngandgrasps.gif: the cloud with the 30 best grasps, still and swinging around the scene;example/: the scene as files (depth.png,rgb.png,mask.png,intrinsics.txt).
Measured with the released file, converted: 14.8 s for the whole command; 15,000 points to the model, sampled from 316,254 in the workspace; 1024 grasps predicted, 418 left after ModelFreeCollisionFilter (2026-10-15, commit 0ddca31, one NVIDIA RTX A6000 shared with other jobs).
Measured with the weights trained here (not published): 16.6 s for the whole command; 15,000 points to the model, sampled from 316,254 in the workspace; 1024 grasps predicted, 689 left after ModelFreeCollisionFilter (2026-10-15, commit ea6268b, one NVIDIA RTX A6000 shared with other jobs).
3. From Python, a browser or your camera¶
from grasppanda import GraspInferencer
from grasppanda.apis import example_scene
scene = example_scene() # the scene of --example
inferencer = GraspInferencer(
"gsnet/gsnet_sunet14_g1b-realsense_10e", "gsnet_realsense.pth"
)
grasps = inferencer(
depth=scene.depth, intrinsics=scene.intrinsics,
depth_scale=scene.depth_scale, mask=scene.mask,
table_from_camera=scene.table_from_camera, collision_filter=True,
)
print(len(grasps), grasps.sort_by_score().to_graspnet_array()[:3])
It runs the demo's pipeline on the demo's scene and returns its 418 grasps. mask= stands in for the segmentation the config crops the workspace with; on your camera, pass a mask or workspace=(xmin, ymin, zmin, xmax, ymax, zmax) in metres. More: From Python.
# ~/grasppanda_weights: the folder that holds gsnet_realsense.pth
grasppanda explore --weights-dir ~/grasppanda_weights
Open http://127.0.0.1:8765/ and choose GSNet in the Run tab: gsnet_realsense.pth is chosen for you by its name. Keep the example scene or upload a depth image with its intrinsics, and press Run: the page shows the picture, the best ten grasps, the counts through the post-processing and the command that does the same in a terminal. The server listens on your machine only and downloads nothing: Run in the browser.
grasppanda demo gsnet/gsnet_sunet14_g1b-realsense_10e \
--weights gsnet_realsense.pth \
--depth work_dirs/gsnet_sunet14_g1b-realsense_10e/demo/example/depth.png \
--intrinsics work_dirs/gsnet_sunet14_g1b-realsense_10e/demo/example/intrinsics.txt \
--workspace -0.25 -0.2 0.35 0.25 0.2 0.8 \
--collision-filter --out work_dirs/camera
The scene files of step 2 stand in for a camera. GSNet was trained on clouds cropped around the objects; on a camera image, --workspace (a box in the camera frame, metres) or --mask does that crop. Units, frames, the depth scale and the intrinsics: Your own camera.
4. A first number on the benchmark¶
With GraspNet-1Billion on disk (Datasets), score every 64th frame of the seen test scenes, 120 frames:
grasppanda test gsnet/gsnet_sunet14_g1b-realsense_10e \
--weights gsnet_realsense.pth \
--opts data.test.dataset.split=test_seen data.test.dataset.frame_step=64 \
evaluation.num_workers=4
Measured with the released file, converted: AP 19.23 without the collision filter and 39.75 with it (this file was not scored on the whole split).
What it prints, and every measured run against the whole split
It prints AP, AP0.8 and AP0.4 for every variant of the config, raw (the predictions as they are) and cd (with the collision filter), and writes a result record per variant into the run directory under work_dirs/. Testing needs no derived asset; the evaluator extracts a cache from the dataset's object models the first time it runs, which the times below do not include.
| Weights, provenance | Collision filter | AP, 120 frames of test_seen | AP, whole seen split | Difference | Wall time, commit |
|---|---|---|---|---|---|
the released file, converted, author-ckpt |
off | 19.23 | not scored on the whole split | — | 36.6 s, 0ddca31 |
| on | 39.75 | not scored on the whole split | — | ||
the weights trained here (not published), reproduced |
off | 67.82 | 68.83 (record) | −1.01 | 40 s, ea6268b |
| on | 70.74 | 71.23 (record) | −0.49 |
Provenance: author-ckpt, the checkpoint of the reference implementation (graspness_unofficial, third-party), evaluated by GraspPanda; reproduced, trained and evaluated by GraspPanda. Inputs: the workspace is cropped around the ground-truth segmentation (gt_assisted: workspace), as in the reference. On these 120 frames the AP is within 1.01 AP of the whole split, so the subset tells you quickly whether your installation gives what the records say; it is not a benchmark number. Measured on 2026-10-15 on one NVIDIA RTX A6000 shared with other jobs.
The machine: one NVIDIA RTX A6000 (48 GB) shared with other jobs, an AMD EPYC 7262 with OMP_NUM_THREADS=3; Python 3.12, PyTorch 2.7.1 with CUDA 11.8, spconv 2.3.8.
Numbers¶
GraspNet-1Billion, RealSense, the whole test split: AP on seen / similar / novel, from the metafile.
| Weights | Collision filter | Seen | Similar | Novel | Record |
|---|---|---|---|---|---|
| reported (paper) | off | 65.70 | 53.75 | 23.98 | — |
| reported (paper) | on | 67.12 | 54.81 | 24.31 | — |
| trained here | off | 68.83 | 60.35 | 25.20 | record |
| trained here | on | 71.23 | 62.29 | 26.42 | record |
Where these numbers come from
- The status of these numbers: trained here, a model trained by GraspPanda, scored on whole test splits.
- reported: paper.
- trained here: a model trained from scratch by GraspPanda with the config's recipe, scored the same way.
- Inputs: workspace crop from the ground-truth segmentation, as in the reference (gt_assisted: workspace).
- Other cameras, GraspClutter6D and what is known about each gap: the README; every method side by side: Model zoo.
The options of every slot¶
Every option that fits GSNet, except target-mask, was composed under it; those accepted were probed on real frames; the others say why.
A probe shows that a combination trains on real frames of GraspNet-1Billion, not how good it is. Accuracy is claimed only for a method trained and evaluated here.
| Slot | Shipped | Composes | Refused | Runs here | Explorer |
|---|---|---|---|---|---|
| Scene encoder | sparse-unet14 |
47 | 0 | 41/46 | compare |
| Seed proposal | graspness-fps |
24 | 0 | 23/23 | compare |
| View | template-view |
21 | 0 | 20/20 | compare |
| Assigner | NearestLabelAssigner |
3 | 1 | 2/3 | compare |
| Grouper | CylinderGroup |
27 | 1 | 26/27 | compare |
| Head bundle | swad |
10 | 0 | 9/9 | compare |
| Loss | its own losses | 35 | 1 | 20/20 | compare |
| Post-processing | variants cd, raw | 14 · 1 no record | 0 | 14/14 | compare |
| Optimizer | Adam |
17 | 0 | 16/17 | compare |
| Schedule | ExponentialLR |
7 | 0 | 7/7 | compare |
| Hooks | none | 6 | 1 | 6/7 | compare |
| Augmentation | its own | 3 | 0 | 3/3 | compare |
| Recipe | its own | 3 | 0 | 3/3 | compare |
Scene encoder: 47 options
--set backbone=<option>. What each option is: the scene encoder page.
| Option | Runs | Trains like shipped | Under GSNet |
|---|---|---|---|
sparse-unet14 |
2/2 own in GSNet, EconomicGrasp, FineGrasp |
2/2 on a par | shipped |
assanet |
5/5 | 5/5 on a par | ✓ |
dgcnn-static |
5/5 | 1/5 on a par · 4 worse | ✓ |
dgcnn |
5/5 | 0/5 on a par · 1 diverging · 4 worse | predicts ✗: no grasp on any frame |
dela |
5/5 | 4/5 on a par · 1 diverging diverging under FineGrasp |
predicts ✗: no grasp on any frame |
litept-structured3d |
5/5 | 3/5 on a par · 2 worse worse under EconomicGrasp, FineGrasp |
✓ |
litept |
5/5 | 5/5 on a par | ✓ |
minkunet18a-openscene |
5/5 | 4/5 on a par · 1 worse worse under EconomicGrasp |
learns ✗: objectness flat |
minkunet18a |
5/5 | 4/5 on a par · 1 worse worse under EconomicGrasp |
✓ |
minkunet34c-pointcontrast |
5/5 | 5/5 on a par | ✓ |
minkunet34c |
5/5 | 4/5 on a par · 1 worse worse under EconomicGrasp |
learns ✗: objectness flat |
oacnns |
2/5 | 2/2 on a par | unstable twice in the short probe |
octformer |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
ptv3-encoder |
2/5 | 2/2 on a par | unstable twice in the short probe |
ptv3-ppt |
5/5 | 3/5 on a par · 2 diverging diverging under Scale-Balanced-Grasp, FineGrasp |
learns ✗: objectness flat |
ptv3-scannet200 |
4/5 not FineGrasp (unstable twice in the short probe) |
4/4 on a par | ✓ |
ptv3 |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
ptv3-sonata-small |
2/5 | 2/2 on a par | unstable twice in the short probe |
ptv3-sonata |
3/5 not GSNet, EconomicGrasp (unstable twice in the short probe) |
2/3 on a par · 1 diverging | unstable twice in the short probe |
ptv3-utonia |
3/5 not GSNet, EconomicGrasp (unstable twice in the short probe) |
1/3 on a par · 2 diverging | unstable twice in the short probe |
pointmlp |
5/5 | 1/5 on a par · 1 diverging · 3 worse | ✓ |
point-mamba |
5/5 | 2/5 on a par · 3 worse worse under GSNet, EconomicGrasp, FineGrasp |
✓ |
pointmetabase-xl |
5/5 | 3/5 on a par · 1 diverging · 1 worse diverging under Scale-Balanced-Grasp; worse under FineGrasp |
predicts ✗: no grasp on any frame |
pointmetabase |
5/5 | 3/5 on a par · 2 worse worse under EconomicGrasp, FineGrasp |
✓ |
pointnext-b |
5/5 | 3/5 on a par · 2 worse worse under EconomicGrasp, FineGrasp |
predicts ✗: no grasp on any frame |
pointnext-l |
5/5 | 1/5 on a par · 4 worse | predicts ✗: no grasp on any frame |
pointnext-xl |
5/5 | 2/5 on a par · 3 worse worse under GSNet, EconomicGrasp, FineGrasp |
predicts ✗: no grasp on any frame |
pointnext |
5/5 | 3/5 on a par · 2 worse worse under EconomicGrasp, FineGrasp |
predicts ✗: no grasp on any frame |
pointnet2 |
4/4 own in GraspNet-baseline |
0/4 on a par · 4 worse | ✓ |
pointnet2-dense |
5/5 | 2/5 on a par · 3 worse worse under GSNet, EconomicGrasp, FineGrasp |
predicts ✗: no grasp on any frame |
pointnet2-msg |
5/5 | 2/5 on a par · 3 worse worse under GSNet, EconomicGrasp, FineGrasp |
✓ |
pointnet2-msg-dense |
5/5 | 3/5 on a par · 2 worse worse under EconomicGrasp, FineGrasp |
✓ |
pointnet2-transformer |
4/4 own in Scale-Balanced-Grasp |
1/4 on a par · 3 worse worse under GSNet, EconomicGrasp, FineGrasp |
predicts ✗: no grasp on any frame |
point-transformer-v1 |
5/5 | 1/5 on a par · 4 diverging | ✓ |
point-transformer-v2 |
5/5 | 1/5 on a par · 4 diverging | ✓ |
pointvector-xl |
5/5 | 3/5 on a par · 2 worse worse under EconomicGrasp, FineGrasp |
✓ |
pointvector |
5/5 | 2/5 on a par · 3 worse worse under GSNet, EconomicGrasp, FineGrasp |
predicts ✗: no grasp on any frame |
point-vit |
5/5 | 0/5 on a par · 5 worse | predicts ✗: no grasp on any frame |
pyg-edgeconv |
4/5 not FineGrasp (unstable twice in the short probe) |
1/4 on a par · 3 worse | predicts ✗: no grasp on any frame |
pyg-point-transformer |
4/5 FineGrasp undecided (unstable once, not repeated) |
0/4 on a par · 4 worse | predicts ✗: no grasp on any frame |
pyg-pointgnn |
4/5 FineGrasp undecided (unstable once, not repeated) |
0/4 on a par · 4 worse | predicts ✗: no grasp on frame 1 |
pyg-pointnet |
5/5 | 0/5 on a par · 5 worse | predicts ✗: no grasp on any frame |
randlanet |
5/5 | 1/5 on a par · 3 diverging · 1 worse | predicts ✗: no grasp on any frame |
spunet-gaussiancross |
5/5 | 3/5 on a par · 1 diverging · 1 worse worse under GSNet; diverging under Scale-Balanced-Grasp |
learns ✗: objectness flat |
spunet |
4/5 not FineGrasp (unstable twice in the short probe) |
4/4 on a par | ✓ |
uni3d |
5/5 | 1/5 on a par · 1 diverging · 3 worse | ✓ |
volt |
5/5 | 2/5 on a par · 1 diverging · 2 worse | learns ✗: objectness flat; predicts ✗: no grasp on any frame |
Who is left out, and why:
dgcnn-static, trains like shipped: worse under GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.dgcnn, trains like shipped: worse under GraspNet-baseline, GSNet, EconomicGrasp, FineGrasp; diverging under Scale-Balanced-Grasp.oacnns, runs: not GSNet, EconomicGrasp, FineGrasp (unstable twice in the short probe).oacnns, trains like shipped: not compared under GSNet, EconomicGrasp, FineGrasp: the swap did not run at level 1.ptv3-encoder, runs: not GSNet, EconomicGrasp, FineGrasp (unstable twice in the short probe).ptv3-encoder, trains like shipped: not compared under GSNet, EconomicGrasp, FineGrasp: the swap did not run at level 1.ptv3-scannet200, trains like shipped: not compared under FineGrasp: the swap did not run at level 1.ptv3-sonata-small, runs: not GSNet, EconomicGrasp, FineGrasp (unstable twice in the short probe).ptv3-sonata-small, trains like shipped: not compared under GSNet, EconomicGrasp, FineGrasp: the swap did not run at level 1.ptv3-sonata, trains like shipped: diverging under FineGrasp; not compared under GSNet, EconomicGrasp: the swap did not run at level 1.ptv3-utonia, trains like shipped: diverging under Scale-Balanced-Grasp, FineGrasp; not compared under GSNet, EconomicGrasp: the swap did not run at level 1.pointmlp, trains like shipped: worse under GSNet, EconomicGrasp, FineGrasp; diverging under Scale-Balanced-Grasp.pointnext-l, trains like shipped: worse under GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.pointnet2, trains like shipped: worse under GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.point-transformer-v1, trains like shipped: diverging under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp.point-transformer-v2, trains like shipped: diverging under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp.point-vit, trains like shipped: worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.pyg-edgeconv, trains like shipped: worse under GSNet, Scale-Balanced-Grasp, EconomicGrasp; not compared under FineGrasp: the swap did not run at level 1.pyg-point-transformer, trains like shipped: worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp; not compared under FineGrasp: the swap did not run at level 1.pyg-pointgnn, trains like shipped: worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp; not compared under FineGrasp: the swap did not run at level 1.pyg-pointnet, trains like shipped: worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.randlanet, trains like shipped: diverging under GSNet, EconomicGrasp, FineGrasp; worse under Scale-Balanced-Grasp.spunet, trains like shipped: not compared under FineGrasp: the swap did not run at level 1.uni3d, trains like shipped: worse under GSNet, EconomicGrasp, FineGrasp; diverging under Scale-Balanced-Grasp.volt, trains like shipped: worse under Scale-Balanced-Grasp, FineGrasp; diverging under EconomicGrasp.
Not counted as options (no probe of their own):
feature-propagation: an adapter: compose inserts it where the next stage needs it, so it is measured inside the cells it was inserted in; inserted in 65 cells (43 runs, 18 runs-empty, 4 refused).furthest-point-subsample: an adapter: compose inserts it where the next stage needs it, so it is measured inside the cells it was inserted in; inserted in 90 cells (54 runs, 36 runs-empty).image-painted-subsampled: point painting: a wrapper written into a config by hand (compose does not write it); no probe record.image-painted: point painting: a wrapper written into a config by hand (compose does not write it); no probe record.
Seed proposal: 24 options
--set proposal=<option>. What each option is: the seed proposal page.
| Option | Runs | Trains like shipped | Under GSNet |
|---|---|---|---|
graspness-fps |
2/2 own in GSNet, EconomicGrasp, FineGrasp |
0/2 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp |
shipped |
dense-ffps |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
dense-fps |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
dense-fpsample |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
dense-fusion-fps |
5/5 | 2/5 on a par · 3 worse | ✓ |
dense-random |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
dense-torch-cluster |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
dense-voxel |
5/5 | 3/5 on a par · 2 worse worse under Scale-Balanced-Grasp, FineGrasp |
✓ |
graspness-ffps |
5/5 | 3/5 on a par · 1 diverging · 1 worse | ✓ |
graspness-fps-fill |
5/5 | 3/5 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp |
✓ |
graspness-fps-topk |
5/5 | 2/5 on a par · 3 worse worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp |
✓ |
graspness-fpsample |
5/5 | 3/5 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp |
✓ |
graspness-fusion-fps |
5/5 | 2/5 on a par · 3 worse | ✓ |
graspness-nms |
5/5 | 0/5 on a par · 5 worse | ✓ |
graspness-random |
5/5 | 3/5 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp |
✓ |
graspness-sfps |
5/5 | 3/5 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp |
✓ |
graspness-top-fraction |
5/5 | 1/5 on a par · 4 worse | ✓ |
graspness-topk |
5/5 | 0/5 on a par · 5 worse | ✓ |
graspness-torch-cluster |
5/5 | 3/5 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp |
✓ |
graspness-voxel |
5/5 | 0/5 on a par · 1 diverging · 4 worse | ✓ |
objectness-fps |
3/5 | 2/3 on a par · 1 worse worse under FineGrasp |
✓ |
product-fps |
3/5 | 3/3 on a par | ✓ |
scored-fps |
5/5 | 3/5 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp |
✓ |
SubsampledSeeds |
3/3 own in GraspNet-baseline, Scale-Balanced-Grasp |
1/3 on a par · 2 worse worse under EconomicGrasp, FineGrasp |
✓ |
Who is left out, and why:
dense-fusion-fps, trains like shipped: worse under GraspNet-baseline, Scale-Balanced-Grasp, FineGrasp.graspness-ffps, trains like shipped: diverging under GraspNet-baseline; worse under Scale-Balanced-Grasp.graspness-fusion-fps, trains like shipped: worse under GraspNet-baseline, Scale-Balanced-Grasp, FineGrasp.graspness-nms, trains like shipped: worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.graspness-top-fraction, trains like shipped: worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp.graspness-topk, trains like shipped: worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.graspness-voxel, trains like shipped: worse under GraspNet-baseline, GSNet, EconomicGrasp, FineGrasp; diverging under Scale-Balanced-Grasp.objectness-fps, runs: not GraspNet-baseline, Scale-Balanced-Grasp (incompatible by design).product-fps, runs: not GraspNet-baseline, Scale-Balanced-Grasp (incompatible by design).
View: 21 options
--set view=<option>. What each option is: the view page.
| Option | Runs | Trains like shipped | Under GSNet |
|---|---|---|---|
template-view |
2/2 own in GSNet, EconomicGrasp, FineGrasp |
0/2 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp |
shipped |
direction-regression |
5/5 | 1/5 on a par · 1 diverging · 3 worse | ✓ |
direction-vmf |
5/5 | 1/5 on a par · 4 worse | ✓ |
healpix-192 |
5/5 | 3/5 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp |
✓ |
healpix-768 |
5/5 | 5/5 on a par | ✓ |
icosphere-162 |
5/5 | 3/5 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp |
✓ |
icosphere-642 |
5/5 | 5/5 on a par | ✓ |
lattice-angular-cost |
5/5 | 1/5 on a par · 4 worse | ✓ |
lattice-best-view |
5/5 | 1/5 on a par · 4 worse | ✓ |
lattice-fibonacci-120 |
5/5 | 5/5 on a par | ✓ |
lattice-hemisphere-60 |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
lattice-soft-argmax |
5/5 | 4/5 on a par · 1 worse worse under Scale-Balanced-Grasp |
✓ |
lattice-topk |
5/5 | 2/5 on a par · 3 worse worse under GSNet, Scale-Balanced-Grasp, FineGrasp |
✓ |
lattice-view |
5/5 | 4/5 on a par · 1 worse worse under Scale-Balanced-Grasp |
✓ |
objectness-view-graspable |
2/4 | 0/2 on a par · 2 worse worse under GraspNet-baseline, GSNet |
predicts ✗: no grasp on any frame |
objectness-view |
2/4 | 0/2 on a par · 1 diverging · 1 worse diverging under GSNet; worse under Scale-Balanced-Grasp |
✓ |
rotation-6d |
5/5 | 1/5 on a par · 1 diverging · 3 worse | learns ✗: view_frame rising |
rotation-9d |
5/5 | 1/5 on a par · 1 diverging · 3 worse | learns ✗: view_frame flat |
rotation-quaternion |
5/5 | 1/5 on a par · 4 worse | learns ✗: view_frame rose 6 % |
two-vector-baseline |
5/5 | 1/5 on a par · 4 worse | ✓ |
two-vector |
5/5 | 1/5 on a par · 4 worse | learns ✗: view_frame rising |
Who is left out, and why:
direction-regression, trains like shipped: worse under GSNet, EconomicGrasp, FineGrasp; diverging under Scale-Balanced-Grasp.direction-vmf, trains like shipped: worse under GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.lattice-angular-cost, trains like shipped: worse under GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.lattice-best-view, trains like shipped: worse under GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.objectness-view-graspable, runs: own in Scale-Balanced-Grasp; not EconomicGrasp, FineGrasp (incompatible by design).objectness-view, runs: own in GraspNet-baseline; not EconomicGrasp, FineGrasp (incompatible by design).rotation-6d, trains like shipped: worse under GSNet, EconomicGrasp, FineGrasp; diverging under Scale-Balanced-Grasp.rotation-9d, trains like shipped: worse under GSNet, EconomicGrasp, FineGrasp; diverging under Scale-Balanced-Grasp.rotation-quaternion, trains like shipped: worse under GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.two-vector-baseline, trains like shipped: worse under GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.two-vector, trains like shipped: worse under GraspNet-baseline, GSNet, EconomicGrasp, FineGrasp.
Assigner: 4 options
--set assigner=<option>. What each option is: the assigner page.
| Option | Runs | Trains like shipped | Under GSNet |
|---|---|---|---|
NearestLabelAssigner |
2/4 | 0/1 on a par · 1 diverging | shipped |
compact-label |
0/3 own in EconomicGrasp, FineGrasp; incompatible by design |
not compared | refused (incompatible by design): CompactLabelAssigner returns 'compact' matched labels, but SWADHead is supervised with 'dense' ones. |
nearest-label-object-seeds |
2/4 | 1/1 on a par not comparable under Scale-Balanced-Grasp |
✓ |
scale-balanced |
2/4 | 0/1 on a par · 1 worse not comparable under GraspNet-baseline; worse under GSNet |
✓ |
Who is left out, and why:
NearestLabelAssigner, runs: own in GSNet; not EconomicGrasp, FineGrasp (incompatible by design).NearestLabelAssigner, trains like shipped: not comparable under GraspNet-baseline; diverging under Scale-Balanced-Grasp.nearest-label-object-seeds, runs: own in GraspNet-baseline; not EconomicGrasp, FineGrasp (incompatible by design).scale-balanced, runs: own in Scale-Balanced-Grasp; not EconomicGrasp, FineGrasp (incompatible by design).
Grouper: 28 options
--set grouper=<option>. What each option is: the grouper page.
| Option | Runs | Trains like shipped | Under GSNet |
|---|---|---|---|
CylinderGroup |
4/4 own in GSNet |
2/4 on a par · 2 worse worse under GraspNet-baseline, FineGrasp |
shipped |
ball |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
box |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
cylinder-attentive |
5/5 | 4/5 on a par · 1 worse worse under EconomicGrasp |
✓ |
cylinder-edgeconv |
5/5 | 4/5 on a par · 1 diverging diverging under FineGrasp |
✓ |
cylinder-invres |
5/5 | 3/5 on a par · 1 diverging · 1 worse worse under GraspNet-baseline; diverging under GSNet |
✓ |
cylinder-vector-attention |
5/5 | 4/5 on a par · 1 diverging diverging under GSNet |
✓ |
depth-ball |
5/5 | 3/5 on a par · 2 worse worse under Scale-Balanced-Grasp, FineGrasp |
✓ |
depth-box |
5/5 | 5/5 on a par | ✓ |
depth-cylinder-attentive |
5/5 | 5/5 on a par | ✓ |
depth-cylinder-multi-scale |
4/4 own in Scale-Balanced-Grasp |
4/4 on a par | ✓ |
depth-cylinder-projection |
5/5 | 5/5 on a par | ✓ |
depth-embed |
2/5 not GSNet, EconomicGrasp, FineGrasp (incompatible by design) |
1/2 on a par · 1 worse worse under GraspNet-baseline |
refused (incompatible by design): DepthEmbedGroup emits features with axes ('seed', 'depth', 'channel'), but SWADHead expects ('seed', 'channel'). |
depth-knn |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
depth-pool |
3/5 | 2/3 on a par · 1 worse worse under FineGrasp |
✓ |
knn |
5/5 | 3/5 on a par · 2 worse worse under GraspNet-baseline, FineGrasp |
✓ |
multi-scale-linear |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
multi-scale |
4/4 own in FineGrasp |
2/4 on a par · 1 diverging · 1 worse | ✓ |
pyg-edge-conv |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
pyg-gatv2-conv |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
pyg-point-transformer-conv |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
pyg-pointnet-conv |
5/5 | 2/5 on a par · 3 worse worse under GraspNet-baseline, EconomicGrasp, FineGrasp |
✓ |
pyg-transformer-conv |
5/5 | 4/5 on a par · 1 diverging diverging under GSNet |
✓ |
roi-aware |
5/5 | 4/5 on a par · 1 diverging diverging under GSNet |
✓ |
roi-grid |
5/5 | 5/5 on a par | ✓ |
voxel-set-abstraction |
3/5 | 3/3 on a par | ✓ |
CylinderInteractionGroup |
4/4 own in EconomicGrasp |
2/4 on a par · 2 worse worse under GraspNet-baseline, FineGrasp |
✓ |
DepthCylinderGroup |
4/4 own in GraspNet-baseline |
2/4 on a par · 2 worse worse under Scale-Balanced-Grasp, FineGrasp |
✓ |
Who is left out, and why:
depth-pool, runs: not GraspNet-baseline, Scale-Balanced-Grasp (incompatible by design).multi-scale, trains like shipped: worse under GraspNet-baseline; diverging under Scale-Balanced-Grasp.voxel-set-abstraction, runs: not GraspNet-baseline, Scale-Balanced-Grasp (not supported yet).
Not an option by its type name: DepthEmbedGroup, DepthPoolGroup, MultiRadiusGroup, MultiScaleGroup need an option only a preset supplies (refused before any contract is compared); the presets are listed.
Head bundle: 10 options
--set head=<option>. What each option is: the head bundle page.
| Option | Runs | Trains like shipped | Under GSNet |
|---|---|---|---|
swad |
4/4 own in GSNet |
3/4 on a par · 1 diverging diverging under EconomicGrasp |
shipped |
decoupled-grid-bins |
5/5 | 4/5 on a par · 1 diverging diverging under EconomicGrasp |
learns ✗: angle flat |
decoupled-grid-class |
5/5 | 3/5 on a par · 1 diverging · 1 worse worse under GSNet; diverging under EconomicGrasp |
learns ✗: angle flat |
decoupled-grid-regression |
5/5 | 2/5 on a par · 3 diverging diverging under GSNet, EconomicGrasp, FineGrasp |
learns ✗: angle flat |
decoupled-grid |
5/5 | 3/5 on a par · 1 diverging · 1 worse worse under GSNet; diverging under EconomicGrasp |
learns ✗: angle flat |
economic-focal |
3/5 | 3/3 on a par | ✓ |
economic |
1/3 | GSNet: on a par | ✓ |
operation-objectness |
4/4 own in Scale-Balanced-Grasp |
1/4 on a par · 1 diverging · 2 worse worse under GSNet, EconomicGrasp; diverging under FineGrasp |
✓ |
operation |
4/4 own in GraspNet-baseline |
2/4 on a par · 2 worse worse under GSNet, EconomicGrasp |
✓ |
swad-focal |
5/5 | 5/5 on a par | ✓ |
Who is left out, and why:
economic-focal, runs: not GraspNet-baseline, Scale-Balanced-Grasp (incompatible by design).economic, runs: own in EconomicGrasp, FineGrasp; not GraspNet-baseline, Scale-Balanced-Grasp (incompatible by design).
Loss: 19 of 20 presets
--set <stage>.<term>_loss=<option>. What each option is: the loss page.
| Option | Runs | Trains like shipped | Under GSNet |
|---|---|---|---|
focal |
5/5 | 5/5 on a par | proposal.objectness_loss, head.score_loss: ✓; head.width_loss: refused (incompatible by design): SWADHead.width_loss is a value term (a prediction and a target of equal shape, the target an unbounded quantity), but FocalLoss fits class or unit terms. |
quality-focal |
3/4 not GraspNet-baseline (incompatible by design) |
3/3 on a par | head.score_loss: ✓ |
balanced-l1 |
5/5 | 3/5 on a par · 2 worse worse under EconomicGrasp, FineGrasp |
head.score_loss, head.width_loss: ✓ |
softmax-focal |
5/5 | 5/5 on a par | proposal.objectness_loss: ✓ |
varifocal |
3/3 | 2/3 on a par · 1 diverging diverging under GSNet |
head.score_loss: ✓ |
huber |
5/5 | 5/5 on a par | head.score_loss, head.width_loss: ✓ |
label-smoothing |
5/5 | 5/5 on a par | proposal.objectness_loss: ✓ |
bce |
3/3 | 2/3 on a par · 1 diverging diverging under FineGrasp |
head.score_loss: ✓ |
ghm-r |
5/5 | 2/5 on a par · 1 diverging · 2 worse worse under GSNet, EconomicGrasp; diverging under FineGrasp |
head.score_loss, head.width_loss: ✓ |
poly1 |
5/5 | 4/5 on a par · 1 worse worse under GraspNet-baseline |
proposal.objectness_loss: ✓ |
ghm-c |
5/5 | 0/5 on a par · 1 diverging · 4 worse | proposal.objectness_loss, head.score_loss: ✓ |
charbonnier |
5/5 | 5/5 on a par | head.score_loss, head.width_loss: ✓ |
ohem |
5/5 | 4/5 on a par · 1 worse worse under GSNet |
proposal.objectness_loss: ✓ |
ohem-smooth-l1 |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
head.score_loss, head.width_loss: ✓ |
cauchy |
5/5 | 5/5 on a par | head.score_loss, head.width_loss: ✓ |
class-balanced |
5/5 | 5/5 on a par | proposal.objectness_loss: ✓ |
dice |
4/5 not Scale-Balanced-Grasp (incompatible by design) |
4/4 on a par | proposal.objectness_loss: ✓ |
lovasz-softmax |
4/5 not Scale-Balanced-Grasp (incompatible by design) |
3/4 on a par · 1 worse worse under GSNet |
proposal.objectness_loss: ✓ |
uncertainty-weighting |
5/5 | 5/5 on a par | proposal.objectness_loss, head.score_loss, head.width_loss: ✓ |
Who is left out, and why:
ghm-c, trains like shipped: worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp; diverging under FineGrasp.
Also probed under GSNet: 6 cells that replace several terms at once (focal+quality-focal+balanced-l1, softmax-focal+varifocal+huber, label-smoothing+bce+ghm-r, poly1+ghm-c+charbonnier, ohem+ohem-smooth-l1+cauchy, uncertainty-weighting); 6 of them run.
distribution-focal has no cell of its own: fits bins terms only, which no shipped method has; probed as the width loss of the head preset decoupled-grid-bins (stage cells).
Post-processing: 15 options
--set post=<option>. What each option is: the post-processing page.
| Option | Runs | Trains like shipped | Under GSNet |
|---|---|---|---|
approach-cone |
6/6 | — | ✓ |
center-gripper |
6/6 | — | ✓ |
collision-strict |
6/6 | — | ✓ |
collision |
6/6 | — | ✓ |
furthest-sampling |
6/6 | — | ✓ |
gripper-limits |
6/6 | — | ✓ |
mesh-collision |
6/6 | — | ✓ |
nms |
6/6 | — | ✓ |
score-threshold |
6/6 | — | ✓ |
table-plane |
6/6 | — | ✓ |
to-base-frame |
6/6 | — | ✓ |
top-k |
6/6 | — | ✓ |
width-from-cloud |
6/6 | — | ✓ |
workspace |
6/6 | — | ✓ |
target-mask |
no record reads observation.masks, which no shipped dataset fills |
— | no record |
Trains like shipped: — (post-processing trains nothing: it changes predictions, not the training).
Training recipe: 37 options (optimizer, schedule, hooks, augmentation, recipe)
Optimizer, 17 options: --set optimizer=<option>. What each option is: the optimizer page.
| Option | Runs | Trains like shipped | Under GSNet |
|---|---|---|---|
adamw-no-decay-1d |
6/6 | 4/6 on a par · 2 worse worse under FineGrasp, HGGD |
✓ |
adamw |
6/6 | 3/6 on a par · 3 worse worse under Scale-Balanced-Grasp, FineGrasp, HGGD |
✓ |
ademamix |
6/6 | 4/6 on a par · 2 worse worse under FineGrasp, HGGD |
✓ |
encoder-lr-0.1 |
5/5 | 1/5 on a par · 1 diverging · 3 worse | ✓ |
layer-decay |
5/5 | 2/5 on a par · 1 diverging · 2 worse worse under GSNet, EconomicGrasp; diverging under FineGrasp |
✓ |
prodigy |
6/6 | 4/6 on a par · 2 worse worse under Scale-Balanced-Grasp, FineGrasp |
✓ |
schedulefree-adamw |
6/6 | not compared | ✓ |
schedulefree-radam |
6/6 | not compared | ✓ |
schedulefree-sgd |
3/6 | not compared | unstable twice in the short probe |
soap |
5/6 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
stableadamw |
6/6 | 3/6 on a par · 3 worse worse under GraspNet-baseline, FineGrasp, HGGD |
✓ |
timm-adan |
6/6 | 2/6 on a par · 1 diverging · 3 worse | ✓ |
timm-adopt |
6/6 | 3/6 on a par · 3 worse worse under EconomicGrasp, FineGrasp, HGGD |
✓ |
timm-lamb |
6/6 | 5/6 on a par · 1 diverging diverging under FineGrasp |
✓ |
timm-lion |
6/6 | 0/6 on a par · 1 diverging · 5 worse | ✓ |
timm-madgrad |
6/6 | 3/6 on a par · 1 diverging · 2 worse | ✓ |
timm-mars |
6/6 | 4/6 on a par · 2 worse worse under FineGrasp, HGGD |
✓ |
Who is left out, and why:
encoder-lr-0.1, trains like shipped: worse under GraspNet-baseline, GSNet, EconomicGrasp; diverging under FineGrasp.schedulefree-adamw, trains like shipped: not compared under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp, HGGD: it trained on a schedule of its own, not on that of the method as shipped.schedulefree-radam, trains like shipped: not compared under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp, HGGD: it trained on a schedule of its own, not on that of the method as shipped.schedulefree-sgd, runs: not GSNet, EconomicGrasp, FineGrasp (unstable twice in the short probe).schedulefree-sgd, trains like shipped: not compared under GraspNet-baseline, Scale-Balanced-Grasp, HGGD: it trained on a schedule of its own, not on that of the method as shipped; not compared under GSNet, EconomicGrasp, FineGrasp: the swap did not run at level 1.soap, runs: not HGGD (not verified: one CUDA illegal memory access on a shared GPU).timm-adan, trains like shipped: worse under GSNet, EconomicGrasp, HGGD; diverging under FineGrasp.timm-lion, trains like shipped: worse under GraspNet-baseline, GSNet, EconomicGrasp, FineGrasp, HGGD; diverging under Scale-Balanced-Grasp.timm-madgrad, trains like shipped: diverging under GraspNet-baseline; worse under Scale-Balanced-Grasp, FineGrasp.
Schedule, 7 options: --set schedule=<option>. What each option is: the schedule page.
| Option | Runs | Trains like shipped | Under GSNet |
|---|---|---|---|
constant |
5/6 not FineGrasp (unstable twice in the short probe) |
— | ✓ |
cosine-restarts |
5/6 not FineGrasp (unstable twice in the short probe) |
— | ✓ |
cosine |
5/6 not FineGrasp (unstable twice in the short probe) |
— | ✓ |
multistep-warmup |
5/6 not FineGrasp (unstable twice in the short probe) |
— | ✓ |
onecycle |
6/6 | — | ✓ |
poly |
5/6 not FineGrasp (unstable twice in the short probe) |
— | ✓ |
warmup-cosine |
5/6 not FineGrasp (unstable twice in the short probe) |
— | ✓ |
Trains like shipped: — (level 2 holds the schedule of the method as shipped fixed (a constant rate; FineGrasp: its own warm-up), so a schedule swap is not compared).
Hooks, 7 options: --set hooks=<option>. What each option is: the hooks page.
| Option | Runs | Trains like shipped | Under GSNet |
|---|---|---|---|
accumulate-2 |
5/5 own in HGGD |
2/5 on a par · 3 worse worse under GraspNet-baseline, GSNet, EconomicGrasp |
✓ |
agc |
6/6 | 6/6 on a par | ✓ |
amp |
0/6 not supported yet |
not compared | refused (not supported yet): train.amp: mixed precision is not supported by GSNet yet: its loss is not written for half-precision tensors (the detector does not declare amp_ready = True). |
ema |
6/6 | 6/6 on a par | ✓ |
freeze-encoder |
4/5 not FineGrasp (unstable twice in the short probe) |
4/4 on a par | learns ✗: objectness flat; graspness rose 8 % |
grad-clip |
6/6 | 5/6 on a par · 1 worse worse under FineGrasp |
✓ |
swa |
4/6 | 4/4 on a par | ✓ |
Who is left out, and why:
freeze-encoder, trains like shipped: not compared under FineGrasp: the swap did not run at level 1.swa, runs: not GraspNet-baseline, Scale-Balanced-Grasp (not verified: the probe is shorter than the epoch at which averaging starts).
Augmentation, 3 options: --set augment=<option>. What each option is: the augmentation page.
| Option | Runs | Trains like shipped | Under GSNet |
|---|---|---|---|
dropout |
5/6 | 5/5 on a par | ✓ |
jitter |
5/6 not HGGD (not supported yet) |
5/5 on a par | ✓ |
shift |
5/6 not HGGD (not supported yet) |
3/5 on a par · 2 worse worse under EconomicGrasp, FineGrasp |
✓ |
Who is left out, and why:
dropout, runs: not FineGrasp (fails: the transform order of the composed config is wrong (DepthNormals must run first)).
Recipe, 3 options: --set recipe=<option>. What each option is: the recipe page.
| Option | Runs | Trains like shipped | Under GSNet |
|---|---|---|---|
adamw-onecycle |
6/6 | not compared | ✓ |
finetune-encoder |
4/5 not FineGrasp (unstable twice in the short probe) |
not compared | learns ✗: objectness flat; graspness rose 152 % |
ptv3-adamw-onecycle |
5/5 | not compared | ✓ |
Who is left out, and why:
adamw-onecycle, trains like shipped: not compared under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp, HGGD: it trained on a schedule of its own, not on that of the method as shipped.finetune-encoder, trains like shipped: not compared under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp: it trained on a schedule of its own, not on that of the method as shipped; not compared under FineGrasp: the swap did not run at level 1.ptv3-adamw-onecycle, trains like shipped: not compared under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp: it trained on a schedule of its own, not on that of the method as shipped.
How to swap, and how to read these lists
Swap one option in with a single line, or list the options of a slot:
grasppanda compose gsnet/gsnet_sunet14_g1b-realsense_10e \
--set <slot>=<option> -o swap.yaml
grasppanda compose gsnet/gsnet_sunet14_g1b-realsense_10e --list <slot>
The summary table:
- Composes: the options the config check accepts under GSNet, the one it ships included. needs consent: an option it holds back until you accept the code it downloads (the row says how), neither accepted nor refused. no record: never composed; the row says why.
- Refused: the options the config check refuses under GSNet; each row gives the first sentence of the reason. For losses, both columns count the terms (or sets of terms) a preset was tried on.
- Runs here: the swaps whose probe trained on real frames under GSNet without an error, out of every option tried there as a swap: a refusal counts, the option the method ships does not, and a loss preset is one option. Does it run? counts the same way.
The lists of each slot:
- Runs: the methods under which the swap trained on real frames without an error, out of every method it is a swap under; the methods that ship the option are own and not counted. Who is left out, and why, stands under the chip, or below the table when it is long. Measured: 20 training iterations on 8 RealSense frames of GraspNet-1Billion (one scene), seed 0, one run per cell; nothing was measured on GraspClutter6D or on Kinect frames.
- Trains like shipped: trained 250 iterations beside the component the method ships, its losses fall as theirs do, out of the methods it was compared under; a method under which the swap did not run, or where it trained on a schedule of its own, is named and not counted. Level 2 needs runs, not learns: a term flat after 20 iterations can fall by 250. Measured: 250 iterations on 8 RealSense frames of GraspNet-1Billion (one scene), seed 0, at a constant learning rate (FineGrasp: its own warm-up), beside the method as shipped trained the same way twice.
- Under GSNet: ✓ when it reached every level here; level ? (as shipped) when GSNet as shipped does not reach that level either (a note below the table says why), so the level says nothing about the option; else the level it missed and why (composes, runs, learns, predicts), or why it was refused.
The levels, the refusal kinds and the thresholds: How we measure; every record: Does it run?; every comparison at level 2: Does it train like the shipped component?.
How this port differs from the reference
- The reference implementation is third-party; no official training code exists.
- The reference test script enables its collision filter by default, while the published numbers are without it. Both variants are always evaluated here.
- Point-wise graspness labels must come from
grasppanda prepare. Label files made by other generators are row-aligned to a preprocessing they do not record.
How it was checked, stage by stage: tests/integration/test_gsnet_parity.py and the README.
Next¶
- Swap a component: change one slot of GSNet with a single line.
- Open GSNet in the explorer: every option of every slot, with what was measured.
- All six methods side by side, and the model zoo.