Seed proposal¶

Seed-based · 5 methods--set proposal=24 options

The seed proposal picks the points of the scene at which grasps are predicted, the same number in every frame. It is the only stage that decides where the rest of the model looks.

Use it¶

GraspNet-baselineGSNetScale-Balanced-GraspEconomicGraspFineGrasp

grasppanda compose graspnet_baseline/graspnet-baseline_pointnet2_g1b-realsense_18e \
    --set proposal=graspness-sfps -o graspnet-baseline_graspness-sfps.yaml
grasppanda compose graspnet_baseline/graspnet-baseline_pointnet2_g1b-realsense_18e --list proposal

grasppanda compose gsnet/gsnet_sunet14_g1b-realsense_10e \
    --set proposal=graspness-sfps -o gsnet_graspness-sfps.yaml
grasppanda compose gsnet/gsnet_sunet14_g1b-realsense_10e --list proposal

grasppanda compose scale_balanced_grasp/sbg_pointnet2-transformer_g1b-realsense_18e \
    --set proposal=graspness-sfps -o scale-balanced-grasp_graspness-sfps.yaml
grasppanda compose scale_balanced_grasp/sbg_pointnet2-transformer_g1b-realsense_18e --list proposal

grasppanda compose economicgrasp/economicgrasp_sunet14_g1b-realsense_10e \
    --set proposal=graspness-sfps -o economicgrasp_graspness-sfps.yaml
grasppanda compose economicgrasp/economicgrasp_sunet14_g1b-realsense_10e --list proposal

grasppanda compose finegrasp/finegrasp_sunet14_g1b-realsense_10e \
    --set proposal=graspness-sfps -o finegrasp_graspness-sfps.yaml
grasppanda compose finegrasp/finegrasp_sunet14_g1b-realsense_10e --list proposal

The first command writes a checked config; the second lists every option of the slot under that method, with what was measured. Then grasppanda probe <file> tries the config on real frames and grasppanda train <file> trains it (Swap a component). Each method page lists the same: GraspNet-baseline · GSNet · Scale-Balanced-Grasp · EconomicGrasp · FineGrasp.

Options¶

A probe shows that a combination trains on real frames of GraspNet-1Billion, not how good it is. Accuracy is claimed only for a method trained and evaluated here.

Score the points, then sample · 16Sample without a score · 8

Option	What it is	Runs	Trains like shipped
`graspness-ffps`	Graspable points, then furthest point sampling in feature space (F-FPS of 3DSSD, CVPR 2020): seeds that differ in what the encoder sees instead of where they are.	5/5	3/5 on a par · 1 diverging · 1 worse
`graspness-fps`	1024 seeds by furthest point sampling among the points predicted graspable (GraspnessFPS).	2/2 own in GSNet, EconomicGrasp, FineGrasp	0/2 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp
`graspness-fps-fill`	Graspable points, then furthest point sampling; a frame with too few graspable points is filled with the best-scored other points as real seeds instead of padding (GSNet's selection pads).	5/5	3/5 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp
`graspness-fps-topk`	Graspable points; half of the seeds by furthest point sampling for coverage, the other half the best-scored remaining points (the selection rule of Contact-GraspNet, ICRA 2021).	5/5	2/5 on a par · 3 worse worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp
`graspness-fpsample`	Graspable points, then exact furthest point sampling on the CPU with the bucket-based QuickFPS of the `fpsample` library (pip install fpsample).	5/5	3/5 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp
`graspness-fusion-fps`	Graspable points, then fusion sampling (FS of 3DSSD, CVPR 2020): half of the seeds by feature- space, half by coordinate-space furthest point sampling.	5/5	2/5 on a par · 3 worse
`graspness-nms`	Graspable points in order of graspness, none closer than 1 cm to an accepted one (greedy suppression on points as in S4G, CoRL 2019).	5/5	0/5 on a par · 5 worse
`graspness-random`	Graspable points, then uniformly random sampling without replacement: the cheapest sampler there is, and the baseline of the sampling ablation in the GSNet paper.	5/5	3/5 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp
`graspness-sfps`	Graspable points, then semantics-guided furthest point sampling (S-FPS of SASA, AAAI 2022) with graspness as the weight: spread the seeds, prefer graspable points.	5/5	3/5 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp
`graspness-top-fraction`	The best tenth of every frame's points by graspness are the candidates, whatever their values (a rank cut as described for DexGraspNet 2.0), then furthest point sampling.	5/5	1/5 on a par · 4 worse
`graspness-topk`	The 1024 points with the highest predicted graspness, wherever they are (top-k selection as in IA- SSD, CVPR 2022, and GraNet, IROS 2023): no spatial spread.	5/5	0/5 on a par · 5 worse
`graspness-torch-cluster`	Graspable points, then furthest point sampling with `torch_cluster.fps`, the kernel of PyTorch Geometric (pip install torch-cluster).	5/5	3/5 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp
`graspness-voxel`	Graspable points, then one seed per occupied voxel (grid subsampling): seeds on a regular lattice, the best-scored point of every cell.	5/5	0/5 on a par · 1 diverging · 4 worse
`objectness-fps`	Seeds by furthest point sampling over the predicted foreground: objectness alone selects, graspness is still predicted and supervised but does not cut.	3/5	2/3 on a par · 1 worse worse under FineGrasp
`product-fps`	Seeds by furthest point sampling over the points whose objectness probability times graspness exceeds 0.1 (the candidate rule of MultiGraspNet).	3/5	3/3 on a par
`scored-fps`	Points on an object (arg-max) with graspness above 0.1, then plain furthest point sampling: ScoredSeeds with its defaults. GSNet's selection without the two quirks of its sampling kernel, and the file to copy when you write a proposal of your own.	5/5	3/5 on a par · 2 worse worse under GraspNet-baseline, Scale-Balanced-Grasp

Option	What it is	Runs	Trains like shipped
`dense-ffps`	1024 seeds by furthest point sampling in feature space (F-FPS of 3DSSD, CVPR 2020) over all points of a dense encoder.	5/5	4/5 on a par · 1 worse worse under FineGrasp
`dense-fps`	SampledSeeds with its defaults: 1024 seeds by furthest point sampling over all points of a dense encoder, no score and no loss.	5/5	4/5 on a par · 1 worse worse under FineGrasp
`dense-fpsample`	1024 seeds by exact furthest point sampling on the CPU with the bucket-based QuickFPS of the `fpsample` library (pip install fpsample), over all scene points.	5/5	4/5 on a par · 1 worse worse under FineGrasp
`dense-fusion-fps`	Fusion sampling (FS of 3DSSD, CVPR 2020) over all points of a dense encoder: half of the seeds by feature distance, half by coordinate distance.	5/5	2/5 on a par · 3 worse
`dense-random`	1024 uniformly random scene points as seeds (random sampling as in RandLA-Net, CVPR 2020): the cheapest proposal, most seeds on the table.	5/5	4/5 on a par · 1 worse worse under FineGrasp
`dense-torch-cluster`	1024 seeds by furthest point sampling with `torch_cluster.fps`, the kernel of PyTorch Geometric (pip install torch-cluster), over all scene points.	5/5	4/5 on a par · 1 worse worse under FineGrasp
`dense-voxel`	One seed per occupied 2 cm voxel of the scene, thinned to 1024 by furthest point sampling: seeds on a regular lattice.	5/5	3/5 on a par · 2 worse worse under Scale-Balanced-Grasp, FineGrasp
`SubsampledSeeds`	The backbone's subsample is the seed set; all seeds are real.	3/3 own in GraspNet-baseline, Scale-Balanced-Grasp	1/3 on a par · 2 worse worse under EconomicGrasp, FineGrasp

What the columns say, and who is left out

Runs: out of the methods the option is a swap under, those under which its probe trained on real frames without error. A method that ships the option is named own and not counted; a method still undecided is counted and named. Measured on 20 training iterations on 8 RealSense frames of GraspNet-1Billion (one scene), seed 0, one run per cell; nothing was measured on GraspClutter6D or on Kinect frames.
Trains like shipped: out of the methods under which the swap ran, those under which every comparable loss term ended within the method's tolerance of the component it ships, with nothing rising; a method left out is named with why. Measured on 250 iterations on 8 RealSense frames of GraspNet-1Billion (one scene), seed 0, at a constant learning rate (FineGrasp: its own warm-up), beside the method as shipped trained the same way twice.
How we measure defines every word of these columns.

Who is left out, and why:

graspness-ffps, trains like shipped: diverging under GraspNet-baseline; worse under Scale-Balanced-Grasp.
graspness-fusion-fps, trains like shipped: worse under GraspNet-baseline, Scale-Balanced-Grasp, FineGrasp.
graspness-nms, trains like shipped: worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.
graspness-top-fraction, trains like shipped: worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp.
graspness-topk, trains like shipped: worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp.
graspness-voxel, trains like shipped: worse under GraspNet-baseline, GSNet, EconomicGrasp, FineGrasp; diverging under Scale-Balanced-Grasp.
objectness-fps, runs: not GraspNet-baseline, Scale-Balanced-Grasp (incompatible by design).
product-fps, runs: not GraspNet-baseline, Scale-Balanced-Grasp (incompatible by design).
dense-fusion-fps, trains like shipped: worse under GraspNet-baseline, Scale-Balanced-Grasp, FineGrasp.

Sources and extras

The work each option follows; extra: the optional install it needs (every extra).

Source	Options
GSNet (Wang et al., ICCV 2021) scorer with the selection rules of MultiGraspNet (product), DexGraspNet 2.0 (top fraction) and a swappable sampler	`graspness-ffps`, `graspness-fps-fill`, `graspness-fps-topk`, `graspness-fpsample`, `graspness-fusion-fps`, `graspness-nms`, `graspness-random`, `graspness-sfps`, `graspness-top-fraction`, `graspness-topk`, `graspness-torch-cluster`, `graspness-voxel`, `objectness-fps`, `product-fps`, `scored-fps`
GSNet (Wang et al., ICCV 2021)	`graspness-fps`
Dense-feature seed sampling as in GraspNet-baseline (Fang et al., CVPR 2020) with the samplers of 3DSSD (Yang et al., CVPR 2020) and others as options	`dense-ffps`, `dense-fps`, `dense-fpsample`, `dense-fusion-fps`, `dense-random`, `dense-torch-cluster`, `dense-voxel`
GraspNet-baseline (Fang et al., CVPR 2020)	`SubsampledSeeds`

Compare under every method

Choosing¶

Start from the method's own proposal, marked own in the table, and change one thing.
Spread or rank. graspness-sfps and graspness-fps-topk keep the seeds spread over the scene; graspness-topk takes the best-scored points wherever they are.
No score at all. The dense-* options sample among all scene points; they predict nothing and need no labels of their own.
A score needs its labels. The scoring options read point-wise graspness labels: grasppanda prepare graspnet1b --graspness --camera <camera> --config <config> builds them once.
Seeds feed the grouper. A grouper that crops among the seeds sees their density, so check it again after a change here.

Details¶

How a proposal is built: five choices

Every published proposal is the same five choices, and each is an option here.

Choice	What it decides	Where it is set
scorer	what is predicted per scene point (objectness, graspness, nothing)	the proposal class
selecting score	which score cuts and ranks	`select_by`
candidate rule	threshold, product threshold or rank cut	`graspness_threshold`, `objectness_threshold`, `top_fraction`
sampler	which candidates become seeds	`sampler: {type: ...}`, a component of its own
too few points	what a frame with fewer candidates than seeds gets	`too_few`

The four proposal classes:

Class	Scorer	Requires	Predicts
`GraspnessFPS`	objectness + graspness	dense features	graspness, objectness; selection and sampler fixed, with the two quirks of the reference sampling kernel
`ScoredSeeds`	objectness + graspness (the same targets and losses)	dense features	graspness, objectness; selecting score, candidate rule, sampler and padding are options
`SubsampledSeeds`	none	a sub-sampling encoder	nothing; the encoder's subsample is the seed set
`SampledSeeds`	none	dense features	nothing; a sampler picks among all scene points

Input and output

proposal(feats, observation) -> (SeedSet, scores) and proposal.loss(scores, supervision) -> dict.

	Type	Content
in	`PointFeatures`	`(N, C)` features of the encoder, bound to the points of the whole batch (frames told apart by `points.batch_offsets`); `C` is injected (`in_channels`).
in	`Observation`	`points`: the scene, `(N, 3)` in metres, camera frame.
out	`SeedSet`	`B × num_seeds` seeds, each a scene point (`parent_index`); their features, still attached to the encoder's graph; named per-seed scores (`graspness`, `objectness`), detached; `valid`, false for padding; `source`, how they were chosen.
out	scores	what the proposal's own `loss` needs (`PointScores` for a scoring proposal, `None` otherwise).

What the neighbours require:

Every frame yields exactly num_seeds seeds, so later stages work on dense (B, M, …) tensors. Padding is marked valid = False and never becomes a grasp; the share of real seeds is logged as metric/real_seeds.
Seeds are scene points. Assigners match labels through parent_index; no option invents positions.
Seeds are chosen from the predictions in training too, so a sampler changes which examples the view stage and the head learn from, from the first iteration.

The contract attributes, compared by grasppanda compose before anything is built:

Attribute	Values	Rule
`requires`	`dense`, `subsampled`, `any`	must equal the encoder's `output`; compose inserts the adapter (`FeaturePropagation` or `FurthestPointSubsample`) and says so.
`seed_scores`	the scores it predicts	one owner per score. Next to `ObjectnessViewStage`, which predicts objectness too, compose makes a scoring proposal select by graspness only and says so; one that selects by objectness is refused.
`sampler_inputs` against the sampler's `needs`	`scores`, `feats`	a sampler that reads scores is refused inside a proposal that predicts none.

Samplers: model.proposal.sampler

A sampler has no parameters, so there are no weights to load. grasppanda components --slot samplers lists them.

Sampler	What it is	Paper	Code	Reads	Extra
`FPS`	furthest point sampling on coordinates; start `first` or `random`	PointNet++, NeurIPS 2017	`grasppanda.ops.furthest_point_sample`	coordinates	none
`FeatureFPS`	furthest point sampling in feature space (F-FPS)	3DSSD, CVPR 2020	written from the definition; distances as in 3DSSD@8bc7605 (MIT)	features	none
`FusionFPS`	part of the seeds by F-FPS, the rest by coordinates (FS)	3DSSD, CVPR 2020	as above	features	none
`ScoreWeightedFPS`	semantics-guided FPS (S-FPS): maximise score^γ × distance	SASA, AAAI 2022	written from the definition; kernel read in SASA@cd79f60 (Apache-2.0)	scores	none
`TopK`	the best-scored points, no spread	IA-SSD, CVPR 2022; the last step of GraNet, IROS 2023	`torch.sort`	scores	none
`RandomSampler`	uniform, without replacement, own seeded generator	the sampling ablation of GSNet (ICCV 2021); RandLA-Net (CVPR 2020)	`torch.randperm`	coordinates	none
`VoxelGridSampler`	one point per occupied voxel (the best-scored one when scores exist)	regular subsampling as in KPConv, ICCV 2019	`grasppanda.ops.voxelize`	coordinates	none
`FPSThenTopK`	FPS for coverage, then the best-scored of the rest	the selection rule of Contact-GraspNet, ICRA 2021	written from the description (its code is under a non-commercial licence and was not copied)	scores	none
`ScoreNMS`	best score first, no two seeds closer than a radius	Algorithm 1 of S4G, CoRL 2019	written from the paper (the repository has no licence file)	scores	none
`FpsampleFPS`	CPU furthest point sampling, bucket-based QuickFPS and four other methods	QuickFPS (IEEE TCAD 2023, as the library cites it)	bridge to `fpsample` (MIT)	coordinates	`fpsample`
`TorchClusterFPS`	the furthest point sampling kernel of PyTorch Geometric	PointNet++, NeurIPS 2017	bridge to `torch_cluster.fps` (MIT)	coordinates	`torch-cluster`

The settings that matter

Any sampler goes inside either proposal that has one:

model:
  proposal:
    type: ScoredSeeds
    num_seeds: 1024
    select_by: graspness_and_objectness   # graspness | objectness | product
    graspness_threshold: 0.1
    objectness_threshold: 0.5
    top_fraction: null                    # e.g. 0.1: a rank cut instead of the thresholds
    too_few: pad                          # best_rest: fill with the best other points
    sampler: {type: ScoreWeightedFPS, gamma: 1.0}

Setting	How to choose
`select_by`	`graspness_and_objectness` is GSNet's rule. `graspness` drops the objectness test (objectness is still trained). `objectness` makes every predicted object point a candidate. `product` cuts `P(object) × graspness`, MultiGraspNet's rule (arXiv 2602.06504).
`graspness_threshold`	belongs to the label definition: 0.1 for graspness normalised to [0, 1] per frame (GSNet) or per instance (FineGrasp). Change the label, change the threshold.
`top_fraction`	a rank cut: the best share of each frame's points, whatever their values (DexGraspNet 2.0 keeps the top 1 % before FPS). With 15 000 points a frame it must be at least 0.07 to leave 1024 candidates; the preset uses 0.1.
`too_few`	`pad`: a frame without candidates has no real seed and trains the proposal only, as in the first iterations of every run. `best_rest`: such a frame gets the best-scored other points as real seeds.
`sampler.gamma` (`ScoreWeightedFPS`)	0 is FPS started at the best point, 1 is SASA's reference, large values approach top-k. `distance: squared` is the released kernel, `euclidean` the paper's formula.
`sampler.metric`, `sampler.coord_weight` (`FeatureFPS`, `FusionFPS`)	`concat` is the released 3DSSD code, `sum` is Eq. 1 of the paper. With 512 feature channels the coordinates hardly count; raise `coord_weight` or set `normalize: true` when the seeds cluster.
`sampler.voxel_size`, `sampler.min_distance`	decide how many real seeds a scene yields: read `metric/real_seeds` from a probe and adapt.
`sampler.seed`	an integer makes two evaluations pick the same seeds.

A checkpoint of GSNet, EconomicGrasp or FineGrasp loads strictly into a model whose proposal is ScoredSeeds (the scorer is the same layer), so a sampler can also be swapped at test time on a trained model. That is a different experiment from training with it; say in the config which one it is.

Pitfalls met while building this

fpsample's bucket methods do not begin at start_idx. vanilla returns exactly the indices of grasppanda.ops.furthest_point_sample; bucket spreads as well but returns another seed set; the npdu methods assume points in scan order and spread worse than random sampling on a shuffled cloud. Use method: vanilla to match the reference.
The CPU bridges are not faster here. They are for CPU-only inference and for the same kernel as another code base.
torch_cluster.fps takes a ratio, not a count, and starts at a random point. The bridge converts the count, trims the result and sets random_start: false.
Three distances are called F-FPS. The 3DSSD paper, its released code and SASA's code differ; metric and coord_weight name the convention.
S-FPS: paper and kernel differ by a factor of two in γ. The kernel multiplies the squared distance by score^γ; distance names the convention.
3DSSD's fusion sampling returns twice the points. A proposal has a fixed seed count, so FusionFPS splits it between the two kinds (feature_fraction).
A score above 1 must not be clamped. Predicted graspness is not bounded by 1; clamping tied every value above 1 and top-k fell back to the first k points. The selecting score is now only kept non-negative.
Feature-space sampling over the whole scene is the most expensive option here. Picks stay on the device and a feature distance is one matrix-vector product per pick.

Not wrapped

Candidate	Why not
Object-balanced seeds (a quota per instance), as in Scale-Balanced-Grasp and DexGraspNet 2.0	needs an instance mask at test time, which no dataset or transform fills yet; the sampler side would be a small wrapper.
GraNet's cascaded selection (GraNet@d9992e1, MIT)	brings its own graph encoder and a ten-class label no dataset here builds; its last step is `graspness-topk`.
MultiGraspNet's graspness loss	a loss, not a proposal; its candidate rule is `product-fps`.
RandLA-Net, S4G and Contact-GraspNet code	licences (CC BY-NC-SA, none, NVIDIA); the small rules were written from the papers.
PyTorch3D, Open3D and mmcv samplers	need compiled builds; a third and fourth spelling of the same FPS.
Voted seeds (VoteGrasp), mask queries (M2T2), image heat-map centres (HGGD)	they emit virtual seeds or a variable number of regions; the skeleton needs scene points.
A sampler inside the encoders, where 3DSSD, IA-SSD and SASA place theirs	a change to the encoders, not to this slot; the `SAMPLERS` registry is ready for it.