GraspNet-baseline¶
Seed-based skeleton CVPR 2020 depth input equal to the reference ✓ authors' weights scored here
From the point cloud of a depth image, it keeps 1024 PointNet++ seeds, scores 300 approach views per seed and predicts score, in-plane angle, width and tolerance at four depths along the best one.
Paper · Reference code · Configs, results and notes
Run GraspNet-baseline in five minutes¶
Four steps from the authors' checkpoint to grasps. A test parses each command and runs the demo and the Python code on a toy detector of the method's family.
1. Get the weights¶
GraspPanda publishes no weights for GraspNet-baseline: the file to download is the authors' checkpoint, under its licence. Download checkpoint-rs.tar (Google Drive) and convert it once, from the root of a GraspPanda checkout:
python -m tools.convert_checkpoint checkpoint-rs.tar baseline_realsense.pth \
--source-format graspnet-baseline --camera realsense
Licence, checksum and what the converter writes
- Licence: GraspNet-Baseline Software License Agreement (non-commercial research use only)
- SHA-256 of the download:
60680087c61cba2b6791614fef1519071e294f6dcaf99b3f581bb95f7c51a868 - What the converter writes:
baseline_realsense.pth, every tensor of the downloaded file under the name of the module that plays its role here, no value changed, with a note of the file it came from. A tensor left over or missing stops the conversion with a report; nothing is ever loaded partially. The converter is a tool of the repository; it is not installed with the package.
2. Grasps on the example scene¶
grasppanda demo graspnet_baseline/graspnet-baseline_pointnet2_g1b-realsense_18e \
--weights baseline_realsense.pth --example --collision-filter
--example is a synthetic table-top scene shipped with GraspPanda: no dataset, no download. 10.5 s for the whole command, 285 grasps kept (one NVIDIA RTX A6000 shared with other jobs).
What it prints and writes, and every measured run
The command prints the config and its hash, how the weights loaded (every tensor matched, none missing), the test pipeline, the cloud, how many grasps the model predicted and how many the collision filter left (the config's variant cd: ModelFreeCollisionFilter), and the ten best grasps. It writes into work_dirs/graspnet-baseline_pointnet2_g1b-realsense_18e/demo/:
grasps.npy: the grasps that are left,(G, 17)float32 in the GraspNet layout, best first, camera frame, metres;grasps.pngandgrasps.gif: the cloud with the 30 best grasps, still and swinging around the scene;example/: the scene as files (depth.png,rgb.png,mask.png,intrinsics.txt).
Measured with the authors' checkpoint, converted: 10.5 s for the whole command; 20,000 points to the model, sampled from 316,254 in the workspace; 880 grasps predicted, 285 left after ModelFreeCollisionFilter (2026-10-15, commit ea6268b, one NVIDIA RTX A6000 shared with other jobs).
3. From Python, a browser or your camera¶
from grasppanda import GraspInferencer
from grasppanda.apis import example_scene
scene = example_scene() # the scene of --example
inferencer = GraspInferencer(
"graspnet_baseline/graspnet-baseline_pointnet2_g1b-realsense_18e",
"baseline_realsense.pth")
grasps = inferencer(
depth=scene.depth, intrinsics=scene.intrinsics,
depth_scale=scene.depth_scale, mask=scene.mask,
table_from_camera=scene.table_from_camera, collision_filter=True,
)
print(len(grasps), grasps.sort_by_score().to_graspnet_array()[:3])
It runs the demo's pipeline on the demo's scene and returns its 285 grasps. mask= stands in for the segmentation the config crops the workspace with; on your camera, pass a mask or workspace=(xmin, ymin, zmin, xmax, ymax, zmax) in metres. More: From Python.
# ~/grasppanda_weights: the folder that holds baseline_realsense.pth
grasppanda explore --weights-dir ~/grasppanda_weights
Open http://127.0.0.1:8765/ and choose GraspNet-baseline in the Run tab: baseline_realsense.pth is chosen for you by its name. Keep the example scene or upload a depth image with its intrinsics, and press Run: the page shows the picture, the best ten grasps, the counts through the post-processing and the command that does the same in a terminal. The server listens on your machine only and downloads nothing: Run in the browser.
grasppanda demo graspnet_baseline/graspnet-baseline_pointnet2_g1b-realsense_18e \
--weights baseline_realsense.pth \
--depth work_dirs/graspnet-baseline_pointnet2_g1b-realsense_18e/demo/example/depth.png \
--intrinsics work_dirs/graspnet-baseline_pointnet2_g1b-realsense_18e/demo/example/intrinsics.txt \
--workspace -0.25 -0.2 0.35 0.25 0.2 0.8 \
--collision-filter --out work_dirs/camera
The scene files of step 2 stand in for a camera. GraspNet-baseline was trained on clouds cropped around the objects; on a camera image, --workspace (a box in the camera frame, metres) or --mask does that crop. Units, frames, the depth scale and the intrinsics: Your own camera.
4. A first number on the benchmark¶
With GraspNet-1Billion on disk (Datasets), score every 64th frame of the seen test scenes, 120 frames:
grasppanda test graspnet_baseline/graspnet-baseline_pointnet2_g1b-realsense_18e \
--weights baseline_realsense.pth \
--opts data.test.dataset.split=test_seen data.test.dataset.frame_step=64 \
evaluation.num_workers=4
Measured with the authors' checkpoint, converted: AP 32.60 without the collision filter and 48.67 with it, within 0.77 AP of the whole split.
What it prints, and every measured run against the whole split
It prints AP, AP0.8 and AP0.4 for every variant of the config, raw (the predictions as they are) and cd (with the collision filter), and writes a result record per variant into the run directory under work_dirs/. Testing needs no derived asset; the evaluator extracts a cache from the dataset's object models the first time it runs, which the times below do not include.
| Weights, provenance | Collision filter | AP, 120 frames of test_seen | AP, whole seen split | Difference | Wall time, commit |
|---|---|---|---|---|---|
the authors' checkpoint, converted, author-ckpt |
off | 32.60 | 32.64 (record) | −0.04 | 35 s, ea6268b |
| on | 48.67 | 47.90 (record) | +0.77 |
Provenance: author-ckpt, the authors' checkpoint, evaluated by GraspPanda. Inputs: the workspace is cropped around the ground-truth segmentation (gt_assisted: workspace), as in the reference. On these 120 frames the AP is within 0.77 AP of the whole split, so the subset tells you quickly whether your installation gives what the records say; it is not a benchmark number. Measured on 2026-10-15 on one NVIDIA RTX A6000 shared with other jobs.
The machine: one NVIDIA RTX A6000 (48 GB) shared with other jobs, an AMD EPYC 7262 with OMP_NUM_THREADS=3; Python 3.12, PyTorch 2.7.1 with CUDA 11.8, spconv 2.3.8.
Numbers¶
GraspNet-1Billion, RealSense, the whole test split: AP on seen / similar / novel, from the metafile.
| Weights | Collision filter | Seen | Similar | Novel | Record |
|---|---|---|---|---|---|
| reported (reference repository README) | on | 47.47 | 42.27 | 16.61 | — |
| reported (paper) | not stated | 27.56 | 26.11 | 10.55 | — |
| authors' weights | off | 32.64 | 27.48 | 10.42 | record |
| authors' weights | on | 47.90 | 42.70 | 16.98 | record |
Where these numbers come from
- The status of these numbers: authors' weights scored here, the authors' checkpoint, converted and scored on whole test splits; nothing trained here yet.
- reported: reference repository README, "In repo" (collision threshold 0.01); paper, as quoted in the reference repository README ("In paper").
- authors' weights, scored here: the authors' checkpoint, converted and run through
grasppanda teston every frame of the test split; the record holds the per-scene values, the config hash and the hashes of code and weights. - Inputs: workspace crop from the ground-truth segmentation, as in the reference (gt_assisted: workspace).
- Other cameras, GraspClutter6D and what is known about each gap: the README; every method side by side: Model zoo.
The options of every slot¶
Every option that fits GraspNet-baseline, except target-mask, was composed under it; those accepted were probed on real frames; the others say why.
A probe shows that a combination trains on real frames of GraspNet-1Billion, not how good it is. Accuracy is claimed only for a method trained and evaluated here.
| Slot | Shipped | Composes | Refused | Runs here | Explorer |
|---|---|---|---|---|---|
| Scene encoder | pointnet2 |
47 | 0 | 46/46 | compare |
| Seed proposal | SubsampledSeeds |
22 | 2 | 21/23 | compare |
| View | objectness-view |
21 | 0 | 20/20 | compare |
| Assigner | nearest-label-object-seeds |
3 | 1 | 2/3 | compare |
| Grouper | DepthCylinderGroup |
26 | 2 | 25/27 | compare |
| Head bundle | operation |
8 | 2 | 7/9 | compare |
| Loss | its own losses | 19 | 1 | 17/18 | compare |
| Post-processing | variants cd, raw | 14 · 1 no record | 0 | 14/14 | compare |
| Optimizer | Adam |
17 | 0 | 17/17 | compare |
| Schedule | MultiStepLR |
7 | 0 | 7/7 | compare |
| Hooks | BNMomentumHook |
6 | 1 | 5/7 | compare |
| Augmentation | its own | 3 | 0 | 3/3 | compare |
| Recipe | its own | 3 | 0 | 3/3 | compare |
Scene encoder: 47 options
--set backbone=<option>. What each option is: the scene encoder page.
| Option | Runs | Trains like shipped | Under GraspNet-baseline |
|---|---|---|---|
pointnet2 |
4/4 own in GraspNet-baseline |
0/4 on a par · 4 worse | shipped |
assanet |
5/5 | 5/5 on a par | ✓ |
dgcnn-static |
5/5 | 1/5 on a par · 4 worse | ✓ |
dgcnn |
5/5 | 0/5 on a par · 1 diverging · 4 worse | ✓ |
dela |
5/5 | 4/5 on a par · 1 diverging diverging under FineGrasp |
✓ |
litept-structured3d |
5/5 | 3/5 on a par · 2 worse worse under EconomicGrasp, FineGrasp |
✓ |
litept |
5/5 | 5/5 on a par | ✓ |
minkunet18a-openscene |
5/5 | 4/5 on a par · 1 worse worse under EconomicGrasp |
✓ |
minkunet18a |
5/5 | 4/5 on a par · 1 worse worse under EconomicGrasp |
✓ |
minkunet34c-pointcontrast |
5/5 | 5/5 on a par | ✓ |
minkunet34c |
5/5 | 4/5 on a par · 1 worse worse under EconomicGrasp |
✓ |
oacnns |
2/5 | 2/2 on a par | ✓ |
octformer |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
ptv3-encoder |
2/5 | 2/2 on a par | ✓ |
ptv3-ppt |
5/5 | 3/5 on a par · 2 diverging diverging under Scale-Balanced-Grasp, FineGrasp |
✓ |
ptv3-scannet200 |
4/5 not FineGrasp (unstable twice in the short probe) |
4/4 on a par | ✓ |
ptv3 |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
ptv3-sonata-small |
2/5 | 2/2 on a par | ✓ |
ptv3-sonata |
3/5 not GSNet, EconomicGrasp (unstable twice in the short probe) |
2/3 on a par · 1 diverging | ✓ |
ptv3-utonia |
3/5 not GSNet, EconomicGrasp (unstable twice in the short probe) |
1/3 on a par · 2 diverging | ✓ |
pointmlp |
5/5 | 1/5 on a par · 1 diverging · 3 worse | ✓ |
point-mamba |
5/5 | 2/5 on a par · 3 worse worse under GSNet, EconomicGrasp, FineGrasp |
✓ |
pointmetabase-xl |
5/5 | 3/5 on a par · 1 diverging · 1 worse diverging under Scale-Balanced-Grasp; worse under FineGrasp |
✓ |
pointmetabase |
5/5 | 3/5 on a par · 2 worse worse under EconomicGrasp, FineGrasp |
✓ |
pointnext-b |
5/5 | 3/5 on a par · 2 worse worse under EconomicGrasp, FineGrasp |
predicts ✗: no grasp on frame 2 |
pointnext-l |
5/5 | 1/5 on a par · 4 worse | predicts ✗: no grasp on any frame |
pointnext-xl |
5/5 | 2/5 on a par · 3 worse worse under GSNet, EconomicGrasp, FineGrasp |
✓ |
pointnext |
5/5 | 3/5 on a par · 2 worse worse under EconomicGrasp, FineGrasp |
✓ |
pointnet2-dense |
5/5 | 2/5 on a par · 3 worse worse under GSNet, EconomicGrasp, FineGrasp |
✓ |
pointnet2-msg |
5/5 | 2/5 on a par · 3 worse worse under GSNet, EconomicGrasp, FineGrasp |
✓ |
pointnet2-msg-dense |
5/5 | 3/5 on a par · 2 worse worse under EconomicGrasp, FineGrasp |
✓ |
pointnet2-transformer |
4/4 own in Scale-Balanced-Grasp |
1/4 on a par · 3 worse worse under GSNet, EconomicGrasp, FineGrasp |
✓ |
point-transformer-v1 |
5/5 | 1/5 on a par · 4 diverging | predicts ✗: no grasp on any frame |
point-transformer-v2 |
5/5 | 1/5 on a par · 4 diverging | predicts ✗: no grasp on any frame |
pointvector-xl |
5/5 | 3/5 on a par · 2 worse worse under EconomicGrasp, FineGrasp |
✓ |
pointvector |
5/5 | 2/5 on a par · 3 worse worse under GSNet, EconomicGrasp, FineGrasp |
✓ |
point-vit |
5/5 | 0/5 on a par · 5 worse | predicts ✗: no grasp on any frame |
pyg-edgeconv |
4/5 not FineGrasp (unstable twice in the short probe) |
1/4 on a par · 3 worse | ✓ |
pyg-point-transformer |
4/5 FineGrasp undecided (unstable once, not repeated) |
0/4 on a par · 4 worse | ✓ |
pyg-pointgnn |
4/5 FineGrasp undecided (unstable once, not repeated) |
0/4 on a par · 4 worse | ✓ |
pyg-pointnet |
5/5 | 0/5 on a par · 5 worse | ✓ |
randlanet |
5/5 | 1/5 on a par · 3 diverging · 1 worse | ✓ |
spunet-gaussiancross |
5/5 | 3/5 on a par · 1 diverging · 1 worse worse under GSNet; diverging under Scale-Balanced-Grasp |
✓ |
spunet |
4/5 not FineGrasp (unstable twice in the short probe) |
4/4 on a par | ✓ |
sparse-unet14 |
2/2 own in GSNet, EconomicGrasp, FineGrasp |
2/2 on a par | ✓ |
uni3d |
5/5 | 1/5 on a par · 1 diverging · 3 worse | ✓ |
volt |
5/5 | 2/5 on a par · 1 diverging · 2 worse | ✓ |
Who is left out, and why:
pointnet2, trains like shipped: worse under GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.dgcnn-static, trains like shipped: worse under GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.dgcnn, trains like shipped: worse under GraspNet-baseline, GSNet, EconomicGrasp, FineGrasp; diverging under Scale-Balanced-Grasp.oacnns, runs: not GSNet, EconomicGrasp, FineGrasp (unstable twice in the short probe).oacnns, trains like shipped: not compared under GSNet, EconomicGrasp, FineGrasp: the swap did not run at level 1.ptv3-encoder, runs: not GSNet, EconomicGrasp, FineGrasp (unstable twice in the short probe).ptv3-encoder, trains like shipped: not compared under GSNet, EconomicGrasp, FineGrasp: the swap did not run at level 1.ptv3-scannet200, trains like shipped: not compared under FineGrasp: the swap did not run at level 1.ptv3-sonata-small, runs: not GSNet, EconomicGrasp, FineGrasp (unstable twice in the short probe).ptv3-sonata-small, trains like shipped: not compared under GSNet, EconomicGrasp, FineGrasp: the swap did not run at level 1.ptv3-sonata, trains like shipped: diverging under FineGrasp; not compared under GSNet, EconomicGrasp: the swap did not run at level 1.ptv3-utonia, trains like shipped: diverging under Scale-Balanced-Grasp, FineGrasp; not compared under GSNet, EconomicGrasp: the swap did not run at level 1.pointmlp, trains like shipped: worse under GSNet, EconomicGrasp, FineGrasp; diverging under Scale-Balanced-Grasp.pointnext-l, trains like shipped: worse under GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.point-transformer-v1, trains like shipped: diverging under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp.point-transformer-v2, trains like shipped: diverging under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp.point-vit, trains like shipped: worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.pyg-edgeconv, trains like shipped: worse under GSNet, Scale-Balanced-Grasp, EconomicGrasp; not compared under FineGrasp: the swap did not run at level 1.pyg-point-transformer, trains like shipped: worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp; not compared under FineGrasp: the swap did not run at level 1.pyg-pointgnn, trains like shipped: worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp; not compared under FineGrasp: the swap did not run at level 1.pyg-pointnet, trains like shipped: worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.randlanet, trains like shipped: diverging under GSNet, EconomicGrasp, FineGrasp; worse under Scale-Balanced-Grasp.spunet, trains like shipped: not compared under FineGrasp: the swap did not run at level 1.uni3d, trains like shipped: worse under GSNet, EconomicGrasp, FineGrasp; diverging under Scale-Balanced-Grasp.volt, trains like shipped: worse under Scale-Balanced-Grasp, FineGrasp; diverging under EconomicGrasp.
Not counted as options (no probe of their own):
feature-propagation: an adapter: compose inserts it where the next stage needs it, so it is measured inside the cells it was inserted in; inserted in 65 cells (43 runs, 18 runs-empty, 4 refused).furthest-point-subsample: an adapter: compose inserts it where the next stage needs it, so it is measured inside the cells it was inserted in; inserted in 90 cells (54 runs, 36 runs-empty).image-painted-subsampled: point painting: a wrapper written into a config by hand (compose does not write it); no probe record.image-painted: point painting: a wrapper written into a config by hand (compose does not write it); no probe record.
Seed proposal: 24 options
--set proposal=<option>. What each option is: the seed proposal page.
| Option | Runs | Trains like shipped | Under GraspNet-baseline |
|---|---|---|---|
SubsampledSeeds |
3/3 own in GraspNet-baseline, Scale-Balanced-Grasp |
1/3 on a par · 2 worse worse under EconomicGrasp, FineGrasp |
shipped |
dense-ffps |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
dense-fps |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
dense-fpsample |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
dense-fusion-fps |
5/5 | 2/5 on a par · 3 worse | ✓ |
dense-random |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
dense-torch-cluster |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
dense-voxel |
5/5 | 3/5 on a par · 2 worse worse under Scale-Balanced-Grasp, FineGrasp |
✓ |
graspness-ffps |
5/5 | 3/5 on a par · 1 diverging · 1 worse | learns ✗: objectness flat |
graspness-fps-fill |
5/5 | 3/5 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp |
learns ✗: objectness flat |
graspness-fps-topk |
5/5 | 2/5 on a par · 3 worse worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp |
learns ✗: objectness flat |
graspness-fps |
2/2 own in GSNet, EconomicGrasp, FineGrasp |
0/2 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp |
learns ✗: objectness flat |
graspness-fpsample |
5/5 | 3/5 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp |
✓ |
graspness-fusion-fps |
5/5 | 2/5 on a par · 3 worse | learns ✗: objectness flat |
graspness-nms |
5/5 | 0/5 on a par · 5 worse | learns ✗: objectness flat; angle flat |
graspness-random |
5/5 | 3/5 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp |
✓ |
graspness-sfps |
5/5 | 3/5 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp |
✓ |
graspness-top-fraction |
5/5 | 1/5 on a par · 4 worse | ✓ |
graspness-topk |
5/5 | 0/5 on a par · 5 worse | learns ✗: objectness rose 9 % |
graspness-torch-cluster |
5/5 | 3/5 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp |
✓ |
graspness-voxel |
5/5 | 0/5 on a par · 1 diverging · 4 worse | ✓ |
objectness-fps |
3/5 | 2/3 on a par · 1 worse worse under FineGrasp |
refused (incompatible by design): ScoredSeeds and ObjectnessViewStage both predict and supervise the 'objectness' of a seed. |
product-fps |
3/5 | 3/3 on a par | refused (incompatible by design): ScoredSeeds and ObjectnessViewStage both predict and supervise the 'objectness' of a seed. |
scored-fps |
5/5 | 3/5 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp |
✓ |
Who is left out, and why:
dense-fusion-fps, trains like shipped: worse under GraspNet-baseline, Scale-Balanced-Grasp, FineGrasp.graspness-ffps, trains like shipped: diverging under GraspNet-baseline; worse under Scale-Balanced-Grasp.graspness-fusion-fps, trains like shipped: worse under GraspNet-baseline, Scale-Balanced-Grasp, FineGrasp.graspness-nms, trains like shipped: worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.graspness-top-fraction, trains like shipped: worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp.graspness-topk, trains like shipped: worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.graspness-voxel, trains like shipped: worse under GraspNet-baseline, GSNet, EconomicGrasp, FineGrasp; diverging under Scale-Balanced-Grasp.objectness-fps, runs: not GraspNet-baseline, Scale-Balanced-Grasp (incompatible by design).product-fps, runs: not GraspNet-baseline, Scale-Balanced-Grasp (incompatible by design).
View: 21 options
--set view=<option>. What each option is: the view page.
| Option | Runs | Trains like shipped | Under GraspNet-baseline |
|---|---|---|---|
objectness-view |
2/4 | 0/2 on a par · 1 diverging · 1 worse diverging under GSNet; worse under Scale-Balanced-Grasp |
shipped |
direction-regression |
5/5 | 1/5 on a par · 1 diverging · 3 worse | learns ✗: score rose 11 % |
direction-vmf |
5/5 | 1/5 on a par · 4 worse | ✓ |
healpix-192 |
5/5 | 3/5 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp |
✓ |
healpix-768 |
5/5 | 5/5 on a par | ✓ |
icosphere-162 |
5/5 | 3/5 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp |
✓ |
icosphere-642 |
5/5 | 5/5 on a par | ✓ |
lattice-angular-cost |
5/5 | 1/5 on a par · 4 worse | ✓ |
lattice-best-view |
5/5 | 1/5 on a par · 4 worse | ✓ |
lattice-fibonacci-120 |
5/5 | 5/5 on a par | ✓ |
lattice-hemisphere-60 |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
lattice-soft-argmax |
5/5 | 4/5 on a par · 1 worse worse under Scale-Balanced-Grasp |
✓ |
lattice-topk |
5/5 | 2/5 on a par · 3 worse worse under GSNet, Scale-Balanced-Grasp, FineGrasp |
✓ |
lattice-view |
5/5 | 4/5 on a par · 1 worse worse under Scale-Balanced-Grasp |
✓ |
objectness-view-graspable |
2/4 | 0/2 on a par · 2 worse worse under GraspNet-baseline, GSNet |
predicts ✗: no grasp on frame 1 |
rotation-6d |
5/5 | 1/5 on a par · 1 diverging · 3 worse | learns ✗: view_frame rose 6 % |
rotation-9d |
5/5 | 1/5 on a par · 1 diverging · 3 worse | learns ✗: view_frame rose 35 % |
rotation-quaternion |
5/5 | 1/5 on a par · 4 worse | learns ✗: view_frame rose 14 % |
template-view |
2/2 own in GSNet, EconomicGrasp, FineGrasp |
0/2 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp |
learns ✗: angle flat |
two-vector-baseline |
5/5 | 1/5 on a par · 4 worse | learns ✗: view_frame rising; score rose 31 % |
two-vector |
5/5 | 1/5 on a par · 4 worse | learns ✗: view_frame rose 30 %; score rose 91 % |
Who is left out, and why:
objectness-view, runs: own in GraspNet-baseline; not EconomicGrasp, FineGrasp (incompatible by design).direction-regression, trains like shipped: worse under GSNet, EconomicGrasp, FineGrasp; diverging under Scale-Balanced-Grasp.direction-vmf, trains like shipped: worse under GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.lattice-angular-cost, trains like shipped: worse under GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.lattice-best-view, trains like shipped: worse under GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.objectness-view-graspable, runs: own in Scale-Balanced-Grasp; not EconomicGrasp, FineGrasp (incompatible by design).rotation-6d, trains like shipped: worse under GSNet, EconomicGrasp, FineGrasp; diverging under Scale-Balanced-Grasp.rotation-9d, trains like shipped: worse under GSNet, EconomicGrasp, FineGrasp; diverging under Scale-Balanced-Grasp.rotation-quaternion, trains like shipped: worse under GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.two-vector-baseline, trains like shipped: worse under GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.two-vector, trains like shipped: worse under GraspNet-baseline, GSNet, EconomicGrasp, FineGrasp.
Assigner: 4 options
--set assigner=<option>. What each option is: the assigner page.
| Option | Runs | Trains like shipped | Under GraspNet-baseline |
|---|---|---|---|
nearest-label-object-seeds |
2/4 | 1/1 on a par not comparable under Scale-Balanced-Grasp |
shipped |
compact-label |
0/3 own in EconomicGrasp, FineGrasp; incompatible by design |
not compared | refused (incompatible by design): CompactLabelAssigner returns 'compact' matched labels, but OperationHead is supervised with 'dense' ones. |
scale-balanced |
2/4 | 0/1 on a par · 1 worse not comparable under GraspNet-baseline; worse under GSNet |
✓ |
NearestLabelAssigner |
2/4 | 0/1 on a par · 1 diverging | ✓ |
Who is left out, and why:
nearest-label-object-seeds, runs: own in GraspNet-baseline; not EconomicGrasp, FineGrasp (incompatible by design).scale-balanced, runs: own in Scale-Balanced-Grasp; not EconomicGrasp, FineGrasp (incompatible by design).NearestLabelAssigner, runs: own in GSNet; not EconomicGrasp, FineGrasp (incompatible by design).NearestLabelAssigner, trains like shipped: not comparable under GraspNet-baseline; diverging under Scale-Balanced-Grasp.
Grouper: 28 options
--set grouper=<option>. What each option is: the grouper page.
| Option | Runs | Trains like shipped | Under GraspNet-baseline |
|---|---|---|---|
DepthCylinderGroup |
4/4 own in GraspNet-baseline |
2/4 on a par · 2 worse worse under Scale-Balanced-Grasp, FineGrasp |
shipped |
ball |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
box |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
cylinder-attentive |
5/5 | 4/5 on a par · 1 worse worse under EconomicGrasp |
✓ |
cylinder-edgeconv |
5/5 | 4/5 on a par · 1 diverging diverging under FineGrasp |
✓ |
cylinder-invres |
5/5 | 3/5 on a par · 1 diverging · 1 worse worse under GraspNet-baseline; diverging under GSNet |
✓ |
cylinder-vector-attention |
5/5 | 4/5 on a par · 1 diverging diverging under GSNet |
✓ |
depth-ball |
5/5 | 3/5 on a par · 2 worse worse under Scale-Balanced-Grasp, FineGrasp |
✓ |
depth-box |
5/5 | 5/5 on a par | ✓ |
depth-cylinder-attentive |
5/5 | 5/5 on a par | ✓ |
depth-cylinder-multi-scale |
4/4 own in Scale-Balanced-Grasp |
4/4 on a par | ✓ |
depth-cylinder-projection |
5/5 | 5/5 on a par | ✓ |
depth-embed |
2/5 not GSNet, EconomicGrasp, FineGrasp (incompatible by design) |
1/2 on a par · 1 worse worse under GraspNet-baseline |
✓ |
depth-knn |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
depth-pool |
3/5 | 2/3 on a par · 1 worse worse under FineGrasp |
refused (incompatible by design): DepthPoolGroup emits features with axes ('seed', 'channel'), but OperationHead expects ('seed', 'depth', 'channel'). |
knn |
5/5 | 3/5 on a par · 2 worse worse under GraspNet-baseline, FineGrasp |
learns ✗: angle flat |
multi-scale-linear |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
multi-scale |
4/4 own in FineGrasp |
2/4 on a par · 1 diverging · 1 worse | ✓ |
pyg-edge-conv |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
pyg-gatv2-conv |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
pyg-point-transformer-conv |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
✓ |
pyg-pointnet-conv |
5/5 | 2/5 on a par · 3 worse worse under GraspNet-baseline, EconomicGrasp, FineGrasp |
✓ |
pyg-transformer-conv |
5/5 | 4/5 on a par · 1 diverging diverging under GSNet |
✓ |
roi-aware |
5/5 | 4/5 on a par · 1 diverging diverging under GSNet |
✓ |
roi-grid |
5/5 | 5/5 on a par | ✓ |
voxel-set-abstraction |
3/5 | 3/3 on a par | refused (not supported yet): VoxelSetAbstractionGroup emits features with axes ('seed', 'channel'), but OperationHead expects ('seed', 'depth', 'channel'). |
CylinderGroup |
4/4 own in GSNet |
2/4 on a par · 2 worse worse under GraspNet-baseline, FineGrasp |
✓ |
CylinderInteractionGroup |
4/4 own in EconomicGrasp |
2/4 on a par · 2 worse worse under GraspNet-baseline, FineGrasp |
✓ |
Who is left out, and why:
depth-pool, runs: not GraspNet-baseline, Scale-Balanced-Grasp (incompatible by design).multi-scale, trains like shipped: worse under GraspNet-baseline; diverging under Scale-Balanced-Grasp.voxel-set-abstraction, runs: not GraspNet-baseline, Scale-Balanced-Grasp (not supported yet).
Not an option by its type name: DepthEmbedGroup, DepthPoolGroup, MultiRadiusGroup, MultiScaleGroup need an option only a preset supplies (refused before any contract is compared); the presets are listed.
Head bundle: 10 options
--set head=<option>. What each option is: the head bundle page.
| Option | Runs | Trains like shipped | Under GraspNet-baseline |
|---|---|---|---|
operation |
4/4 own in GraspNet-baseline |
2/4 on a par · 2 worse worse under GSNet, EconomicGrasp |
shipped |
decoupled-grid-bins |
5/5 | 4/5 on a par · 1 diverging diverging under EconomicGrasp |
✓ |
decoupled-grid-class |
5/5 | 3/5 on a par · 1 diverging · 1 worse worse under GSNet; diverging under EconomicGrasp |
✓ |
decoupled-grid-regression |
5/5 | 2/5 on a par · 3 diverging diverging under GSNet, EconomicGrasp, FineGrasp |
✓ |
decoupled-grid |
5/5 | 3/5 on a par · 1 diverging · 1 worse worse under GSNet; diverging under EconomicGrasp |
✓ |
economic-focal |
3/5 | 3/3 on a par | refused (incompatible by design): ObjectnessViewStage computes its targets from 'dense' matched labels, but CompactLabelAssigner returns 'compact' ones. |
economic |
1/3 | GSNet: on a par | refused (incompatible by design): ObjectnessViewStage computes its targets from 'dense' matched labels, but CompactLabelAssigner returns 'compact' ones. |
operation-objectness |
4/4 own in Scale-Balanced-Grasp |
1/4 on a par · 1 diverging · 2 worse worse under GSNet, EconomicGrasp; diverging under FineGrasp |
✓ |
swad-focal |
5/5 | 5/5 on a par | ✓ |
swad |
4/4 own in GSNet |
3/4 on a par · 1 diverging diverging under EconomicGrasp |
✓ |
Who is left out, and why:
economic-focal, runs: not GraspNet-baseline, Scale-Balanced-Grasp (incompatible by design).economic, runs: own in EconomicGrasp, FineGrasp; not GraspNet-baseline, Scale-Balanced-Grasp (incompatible by design).
Loss: 17 of 20 presets
--set <stage>.<term>_loss=<option>. What each option is: the loss page.
| Option | Runs | Trains like shipped | Under GraspNet-baseline |
|---|---|---|---|
ohem-smooth-l1 |
5/5 | 4/5 on a par · 1 worse worse under FineGrasp |
head.tolerance_loss: ✓ |
ghm-r |
5/5 | 2/5 on a par · 1 diverging · 2 worse worse under GSNet, EconomicGrasp; diverging under FineGrasp |
head.tolerance_loss: ✓ |
ghm-c |
5/5 | 0/5 on a par · 1 diverging · 4 worse | head.angle_loss: ✓ |
balanced-l1 |
5/5 | 3/5 on a par · 2 worse worse under EconomicGrasp, FineGrasp |
head.tolerance_loss: ✓ |
charbonnier |
5/5 | 5/5 on a par | head.tolerance_loss: ✓ |
uncertainty-weighting |
5/5 | 5/5 on a par | head.angle_loss, head.tolerance_loss: ✓ |
quality-focal |
3/4 not GraspNet-baseline (incompatible by design) |
3/3 on a par | head.score_loss: refused (incompatible by design): OperationHead.score_loss is a value term (a prediction and a target of equal shape, the target an unbounded quantity), but QualityFocal fits unit terms. |
cauchy |
5/5 | 5/5 on a par | head.tolerance_loss: ✓ |
class-balanced |
5/5 | 5/5 on a par | head.angle_loss: learns ✗: angle flat |
dice |
4/5 not Scale-Balanced-Grasp (incompatible by design) |
4/4 on a par | head.angle_loss: learns ✗: angle flat |
focal |
5/5 | 5/5 on a par | head.angle_loss: ✓ |
huber |
5/5 | 5/5 on a par | head.tolerance_loss: ✓ |
label-smoothing |
5/5 | 5/5 on a par | head.angle_loss: ✓ |
lovasz-softmax |
4/5 not Scale-Balanced-Grasp (incompatible by design) |
3/4 on a par · 1 worse worse under GSNet |
head.angle_loss: learns ✗: angle flat |
ohem |
5/5 | 4/5 on a par · 1 worse worse under GSNet |
head.angle_loss: ✓ |
poly1 |
5/5 | 4/5 on a par · 1 worse worse under GraspNet-baseline |
head.angle_loss: ✓ |
softmax-focal |
5/5 | 5/5 on a par | head.angle_loss: ✓ |
Who is left out, and why:
ghm-c, trains like shipped: worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp; diverging under FineGrasp.
Also probed under GraspNet-baseline: 2 cells that replace several terms at once (ohem-smooth-l1+ghm-r+ghm-c+balanced-l1+charbonnier, uncertainty-weighting); 2 of them run.
Not tried under GraspNet-baseline: varifocal, bce fit unit terms only, and GraspNet-baseline has none (its terms are of kind class, value).
distribution-focal has no cell of its own: fits bins terms only, which no shipped method has; probed as the width loss of the head preset decoupled-grid-bins (stage cells).
Post-processing: 15 options
--set post=<option>. What each option is: the post-processing page.
| Option | Runs | Trains like shipped | Under GraspNet-baseline |
|---|---|---|---|
approach-cone |
6/6 | — | ✓ |
center-gripper |
6/6 | — | ✓ |
collision-strict |
6/6 | — | ✓ |
collision |
6/6 | — | ✓ |
furthest-sampling |
6/6 | — | ✓ |
gripper-limits |
6/6 | — | ✓ |
mesh-collision |
6/6 | — | ✓ |
nms |
6/6 | — | ✓ |
score-threshold |
6/6 | — | ✓ |
table-plane |
6/6 | — | ✓ |
to-base-frame |
6/6 | — | ✓ |
top-k |
6/6 | — | ✓ |
width-from-cloud |
6/6 | — | ✓ |
workspace |
6/6 | — | ✓ |
target-mask |
no record reads observation.masks, which no shipped dataset fills |
— | no record |
Trains like shipped: — (post-processing trains nothing: it changes predictions, not the training).
Training recipe: 37 options (optimizer, schedule, hooks, augmentation, recipe)
Optimizer, 17 options: --set optimizer=<option>. What each option is: the optimizer page.
| Option | Runs | Trains like shipped | Under GraspNet-baseline |
|---|---|---|---|
adamw-no-decay-1d |
6/6 | 4/6 on a par · 2 worse worse under FineGrasp, HGGD |
✓ |
adamw |
6/6 | 3/6 on a par · 3 worse worse under Scale-Balanced-Grasp, FineGrasp, HGGD |
✓ |
ademamix |
6/6 | 4/6 on a par · 2 worse worse under FineGrasp, HGGD |
✓ |
encoder-lr-0.1 |
5/5 | 1/5 on a par · 1 diverging · 3 worse | ✓ |
layer-decay |
5/5 | 2/5 on a par · 1 diverging · 2 worse worse under GSNet, EconomicGrasp; diverging under FineGrasp |
✓ |
prodigy |
6/6 | 4/6 on a par · 2 worse worse under Scale-Balanced-Grasp, FineGrasp |
learns ✗: angle flat |
schedulefree-adamw |
6/6 | not compared | learns ✗: score rose 19 %; angle flat |
schedulefree-radam |
6/6 | not compared | learns ✗: score flat; angle flat |
schedulefree-sgd |
3/6 | not compared | ✓ |
soap |
5/6 | 4/5 on a par · 1 worse worse under FineGrasp |
learns ✗: score flat; angle flat |
stableadamw |
6/6 | 3/6 on a par · 3 worse worse under GraspNet-baseline, FineGrasp, HGGD |
✓ |
timm-adan |
6/6 | 2/6 on a par · 1 diverging · 3 worse | ✓ |
timm-adopt |
6/6 | 3/6 on a par · 3 worse worse under EconomicGrasp, FineGrasp, HGGD |
✓ |
timm-lamb |
6/6 | 5/6 on a par · 1 diverging diverging under FineGrasp |
learns ✗: score rising; angle flat |
timm-lion |
6/6 | 0/6 on a par · 1 diverging · 5 worse | learns ✗: score flat; angle flat |
timm-madgrad |
6/6 | 3/6 on a par · 1 diverging · 2 worse | ✓ |
timm-mars |
6/6 | 4/6 on a par · 2 worse worse under FineGrasp, HGGD |
✓ |
Who is left out, and why:
encoder-lr-0.1, trains like shipped: worse under GraspNet-baseline, GSNet, EconomicGrasp; diverging under FineGrasp.schedulefree-adamw, trains like shipped: not compared under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp, HGGD: it trained on a schedule of its own, not on that of the method as shipped.schedulefree-radam, trains like shipped: not compared under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp, HGGD: it trained on a schedule of its own, not on that of the method as shipped.schedulefree-sgd, runs: not GSNet, EconomicGrasp, FineGrasp (unstable twice in the short probe).schedulefree-sgd, trains like shipped: not compared under GraspNet-baseline, Scale-Balanced-Grasp, HGGD: it trained on a schedule of its own, not on that of the method as shipped; not compared under GSNet, EconomicGrasp, FineGrasp: the swap did not run at level 1.soap, runs: not HGGD (not verified: one CUDA illegal memory access on a shared GPU).timm-adan, trains like shipped: worse under GSNet, EconomicGrasp, HGGD; diverging under FineGrasp.timm-lion, trains like shipped: worse under GraspNet-baseline, GSNet, EconomicGrasp, FineGrasp, HGGD; diverging under Scale-Balanced-Grasp.timm-madgrad, trains like shipped: diverging under GraspNet-baseline; worse under Scale-Balanced-Grasp, FineGrasp.
Schedule, 7 options: --set schedule=<option>. What each option is: the schedule page.
| Option | Runs | Trains like shipped | Under GraspNet-baseline |
|---|---|---|---|
constant |
5/6 not FineGrasp (unstable twice in the short probe) |
— | ✓ |
cosine-restarts |
5/6 not FineGrasp (unstable twice in the short probe) |
— | ✓ |
cosine |
5/6 not FineGrasp (unstable twice in the short probe) |
— | ✓ |
multistep-warmup |
5/6 not FineGrasp (unstable twice in the short probe) |
— | ✓ |
onecycle |
6/6 | — | ✓ |
poly |
5/6 not FineGrasp (unstable twice in the short probe) |
— | ✓ |
warmup-cosine |
5/6 not FineGrasp (unstable twice in the short probe) |
— | ✓ |
Trains like shipped: — (level 2 holds the schedule of the method as shipped fixed (a constant rate; FineGrasp: its own warm-up), so a schedule swap is not compared).
Hooks, 7 options: --set hooks=<option>. What each option is: the hooks page.
| Option | Runs | Trains like shipped | Under GraspNet-baseline |
|---|---|---|---|
accumulate-2 |
5/5 own in HGGD |
2/5 on a par · 3 worse worse under GraspNet-baseline, GSNet, EconomicGrasp |
✓ |
agc |
6/6 | 6/6 on a par | ✓ |
amp |
0/6 not supported yet |
not compared | refused (not supported yet): train.amp: mixed precision is not supported by GraspNetBaseline yet: its loss is not written for half-precision tensors (the detector does not declare amp_ready = True). |
ema |
6/6 | 6/6 on a par | ✓ |
freeze-encoder |
4/5 not FineGrasp (unstable twice in the short probe) |
4/4 on a par | ✓ |
grad-clip |
6/6 | 5/6 on a par · 1 worse worse under FineGrasp |
✓ |
swa |
4/6 | 4/4 on a par | not verified: the probe is shorter than the epoch at which averaging starts |
Who is left out, and why:
freeze-encoder, trains like shipped: not compared under FineGrasp: the swap did not run at level 1.swa, runs: not GraspNet-baseline, Scale-Balanced-Grasp (not verified: the probe is shorter than the epoch at which averaging starts).
Augmentation, 3 options: --set augment=<option>. What each option is: the augmentation page.
| Option | Runs | Trains like shipped | Under GraspNet-baseline |
|---|---|---|---|
dropout |
5/6 | 5/5 on a par | ✓ |
jitter |
5/6 not HGGD (not supported yet) |
5/5 on a par | ✓ |
shift |
5/6 not HGGD (not supported yet) |
3/5 on a par · 2 worse worse under EconomicGrasp, FineGrasp |
✓ |
Who is left out, and why:
dropout, runs: not FineGrasp (fails: the transform order of the composed config is wrong (DepthNormals must run first)).
Recipe, 3 options: --set recipe=<option>. What each option is: the recipe page.
| Option | Runs | Trains like shipped | Under GraspNet-baseline |
|---|---|---|---|
adamw-onecycle |
6/6 | not compared | ✓ |
finetune-encoder |
4/5 not FineGrasp (unstable twice in the short probe) |
not compared | ✓ |
ptv3-adamw-onecycle |
5/5 | not compared | ✓ |
Who is left out, and why:
adamw-onecycle, trains like shipped: not compared under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp, HGGD: it trained on a schedule of its own, not on that of the method as shipped.finetune-encoder, trains like shipped: not compared under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp: it trained on a schedule of its own, not on that of the method as shipped; not compared under FineGrasp: the swap did not run at level 1.ptv3-adamw-onecycle, trains like shipped: not compared under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp: it trained on a schedule of its own, not on that of the method as shipped.
How to swap, and how to read these lists
Swap one option in with a single line, or list the options of a slot:
grasppanda compose graspnet_baseline/graspnet-baseline_pointnet2_g1b-realsense_18e \
--set <slot>=<option> -o swap.yaml
grasppanda compose graspnet_baseline/graspnet-baseline_pointnet2_g1b-realsense_18e --list <slot>
The summary table:
- Composes: the options the config check accepts under GraspNet-baseline, the one it ships included. needs consent: an option it holds back until you accept the code it downloads (the row says how), neither accepted nor refused. no record: never composed; the row says why.
- Refused: the options the config check refuses under GraspNet-baseline; each row gives the first sentence of the reason. For losses, both columns count the terms (or sets of terms) a preset was tried on.
- Runs here: the swaps whose probe trained on real frames under GraspNet-baseline without an error, out of every option tried there as a swap: a refusal counts, the option the method ships does not, and a loss preset is one option. Does it run? counts the same way.
The lists of each slot:
- Runs: the methods under which the swap trained on real frames without an error, out of every method it is a swap under; the methods that ship the option are own and not counted. Who is left out, and why, stands under the chip, or below the table when it is long. Measured: 20 training iterations on 8 RealSense frames of GraspNet-1Billion (one scene), seed 0, one run per cell; nothing was measured on GraspClutter6D or on Kinect frames.
- Trains like shipped: trained 250 iterations beside the component the method ships, its losses fall as theirs do, out of the methods it was compared under; a method under which the swap did not run, or where it trained on a schedule of its own, is named and not counted. Level 2 needs runs, not learns: a term flat after 20 iterations can fall by 250. Measured: 250 iterations on 8 RealSense frames of GraspNet-1Billion (one scene), seed 0, at a constant learning rate (FineGrasp: its own warm-up), beside the method as shipped trained the same way twice.
- Under GraspNet-baseline: ✓ when it reached every level here; level ? (as shipped) when GraspNet-baseline as shipped does not reach that level either (a note below the table says why), so the level says nothing about the option; else the level it missed and why (composes, runs, learns, predicts), or why it was refused.
The levels, the refusal kinds and the thresholds: How we measure; every record: Does it run?; every comparison at level 2: Does it train like the shipped component?.
How this port differs from the reference
- One model instance trains and predicts.
- Losses are means over the supervised entries (the reference adds 1e-6 to the count), and zero, not NaN, when nothing is supervised.
- Validation uses the
val_devscenes, nottest_seen.
How it was checked, stage by stage: tests/integration/test_baseline_parity.py and the README.
Next¶
- Swap a component: change one slot of GraspNet-baseline with a single line.
- Open GraspNet-baseline in the explorer: every option of every slot, with what was measured.
- All six methods side by side, and the model zoo.