Heads and assigners¶

Seed-based · 5 methods--set head=--set assigner=14 options

The head bundle predicts the grasp at every seed: in-plane rotation, depth, width and score. The assigner matches every seed with the grasp labels it is trained on.

Use it¶

GraspNet-baselineGSNetScale-Balanced-GraspEconomicGraspFineGrasp

grasppanda compose graspnet_baseline/graspnet-baseline_pointnet2_g1b-realsense_18e \
    --set head=swad-focal -o graspnet-baseline_swad-focal.yaml
grasppanda compose graspnet_baseline/graspnet-baseline_pointnet2_g1b-realsense_18e --list head

grasppanda compose gsnet/gsnet_sunet14_g1b-realsense_10e \
    --set head=swad-focal -o gsnet_swad-focal.yaml
grasppanda compose gsnet/gsnet_sunet14_g1b-realsense_10e --list head

grasppanda compose scale_balanced_grasp/sbg_pointnet2-transformer_g1b-realsense_18e \
    --set head=swad-focal -o scale-balanced-grasp_swad-focal.yaml
grasppanda compose scale_balanced_grasp/sbg_pointnet2-transformer_g1b-realsense_18e --list head

grasppanda compose economicgrasp/economicgrasp_sunet14_g1b-realsense_10e \
    --set head=swad-focal -o economicgrasp_swad-focal.yaml
grasppanda compose economicgrasp/economicgrasp_sunet14_g1b-realsense_10e --list head

grasppanda compose finegrasp/finegrasp_sunet14_g1b-realsense_10e \
    --set head=swad-focal -o finegrasp_swad-focal.yaml
grasppanda compose finegrasp/finegrasp_sunet14_g1b-realsense_10e --list head

The first command writes a checked config; the second lists every option of the slot under that method, with what was measured. Then grasppanda probe <file> tries the config on real frames and grasppanda train <file> trains it (Swap a component). Each method page lists the same: GraspNet-baseline · GSNet · Scale-Balanced-Grasp · EconomicGrasp · FineGrasp.

An assigner is set the same way, --set assigner=<option>.

Options¶

A probe shows that a combination trains on real frames of GraspNet-1Billion, not how good it is. Accuracy is claimed only for a method trained and evaluated here.

Head bundles

Option	What it is	Runs	Trains like shipped
`decoupled-grid`	DecoupledGridHead: a score grid as in SWAD, the in-plane angle and the depth as two classifiers of their own, one regressed width per depth bin.	5/5	3/5 on a par · 1 diverging · 1 worse worse under GSNet; diverging under EconomicGrasp
`decoupled-grid-bins`	DecoupledGridHead with the width as a distribution over 16 bins of the gripper opening, trained with the distribution focal loss and decoded as its expectation.	5/5	4/5 on a par · 1 diverging diverging under EconomicGrasp
`decoupled-grid-class`	DecoupledGridHead with everything but the score classified: angle, depth and the width, the latter into 16 bins of the gripper opening.	5/5	3/5 on a par · 1 diverging · 1 worse worse under GSNet; diverging under EconomicGrasp
`decoupled-grid-regression`	DecoupledGridHead with depth and width regressed: a continuous insertion depth and one width per depth bin; only the in-plane angle is classified.	5/5	2/5 on a par · 3 diverging diverging under GSNet, EconomicGrasp, FineGrasp
`economic`	EconomicHead as EconomicGrasp and FineGrasp use it: angle, depth and quality classes plus a width per seed.	1/3	GSNet: on a par
`economic-focal`	EconomicHead with a softmax focal loss on its angle, depth and quality classes and the arg-max restricted to the supervised bins.	3/5	3/3 on a par
`operation`	OperationHead as GraspNet-baseline uses it: score, angle class, width and tolerance per (angle, depth bin).	4/4 own in GraspNet-baseline	2/4 on a par · 2 worse worse under GSNet, EconomicGrasp
`operation-objectness`	OperationHead as Scale-Balanced-Grasp uses it: the score is supervised on every depth bin of a supervised seed, and the decoded score is also multiplied by P(object).	4/4 own in Scale-Balanced-Grasp	1/4 on a par · 1 diverging · 2 worse worse under GSNet, EconomicGrasp; diverging under FineGrasp
`swad`	A score and a width regressed per (angle, depth bin) of every seed; one decoded grasp per seed (SWADHead).	4/4 own in GSNet	3/4 on a par · 1 diverging diverging under EconomicGrasp
`swad-focal`	SWADHead with a quality focal loss on its score grid: the focal loss for graded targets, with score targets normalised by a fixed friction range.	5/5	5/5 on a par

Assigners

Option	What it is	Runs	Trains like shipped
`compact-label`	CompactLabelAssigner as EconomicGrasp and FineGrasp use it: one labelled grasp per seed at its chosen view, and only for a seed within 5 mm of a label point.	0/3 own in EconomicGrasp, FineGrasp; incompatible by design	not compared
`nearest-label-object-seeds`	NearestLabelAssigner as GraspNet-baseline uses it: only seeds on an object are supervised, and training crops sit at the matched label point.	2/4	1/1 on a par not comparable under Scale-Balanced-Grasp
`NearestLabelAssigner`	Nearest annotated grasp point per seed, labels sliced at the chosen view.	2/4	0/1 on a par · 1 diverging
`scale-balanced`	ScaleBalancedAssigner as Scale-Balanced-Grasp uses it: the matching of GraspNet-baseline plus a loss weight per seed, larger for rare grasp widths.	2/4	0/1 on a par · 1 worse not comparable under GraspNet-baseline; worse under GSNet

What the columns say, and who is left out

Runs: out of the methods the option is a swap under, those under which its probe trained on real frames without error. A method that ships the option is named own and not counted; a method still undecided is counted and named. Measured on 20 training iterations on 8 RealSense frames of GraspNet-1Billion (one scene), seed 0, one run per cell; nothing was measured on GraspClutter6D or on Kinect frames.
Trains like shipped: out of the methods under which the swap ran, those under which every comparable loss term ended within the method's tolerance of the component it ships, with nothing rising; a method left out is named with why. Measured on 250 iterations on 8 RealSense frames of GraspNet-1Billion (one scene), seed 0, at a constant learning rate (FineGrasp: its own warm-up), beside the method as shipped trained the same way twice.
How we measure defines every word of these columns.

Who is left out, and why:

economic, runs: own in EconomicGrasp, FineGrasp; not GraspNet-baseline, Scale-Balanced-Grasp (incompatible by design).
economic-focal, runs: not GraspNet-baseline, Scale-Balanced-Grasp (incompatible by design).
nearest-label-object-seeds, runs: own in GraspNet-baseline; not EconomicGrasp, FineGrasp (incompatible by design).
NearestLabelAssigner, runs: own in GSNet; not EconomicGrasp, FineGrasp (incompatible by design).
NearestLabelAssigner, trains like shipped: not comparable under GraspNet-baseline; diverging under Scale-Balanced-Grasp.
scale-balanced, runs: own in Scale-Balanced-Grasp; not EconomicGrasp, FineGrasp (incompatible by design).

Sources and extras

The work each option follows; extra: the optional install it needs (every extra).

Source	Options
After the grasp head of FGC-GraspNet (Lu et al., ICRA 2022), written here; width bins with the distribution focal loss of Generalized Focal Loss (Li et al., NeurIPS 2020)	`decoupled-grid`, `decoupled-grid-bins`, `decoupled-grid-class`, `decoupled-grid-regression`
EconomicGrasp (Wu et al., ECCV 2024)	`economic`, `economic-focal`, `compact-label`
GraspNet-baseline (Fang et al., CVPR 2020)	`operation`, `operation-objectness`
GSNet (Wang et al., ICCV 2021)	`swad`, `swad-focal`
Label matching of GraspNet-baseline and GSNet	`nearest-label-object-seeds`, `NearestLabelAssigner`
Scale-Balanced-Grasp (Ma and Huang, CoRL 2022)	`scale-balanced`

Compare under every method: head bundles · assigners

Choosing¶

A head is swapped whole. Its network, target encoder, loss and decoder share one way of writing a grasp. Its loss terms are swapped one by one on Losses.
Head, assigner and labels go together. A head is trained on dense label grids or on compact best grasps; compose switches the assigner and the dataset section to the head's label set and says so.
The crop decides the head. A head reads one feature per seed or one per depth bin; Grouper lists which crop gives which.
Classify or regress depth and width: decoupled-grid-class and decoupled-grid-regression are the two ends; decoupled-grid-bins predicts the width as a distribution.
Scores of two heads are not comparable. A score threshold fitted to one head does not carry over to another.

Details¶

Input and output of a head bundle

class HeadBundle(nn.Module):
    def forward(self, crops: GroupFeatures) -> Predictions: ...        # raw outputs, with gradients
    def encode(self, matched: MatchedLabels, ...) -> Targets: ...      # labels -> targets
    def loss(self, predictions, targets, ...) -> dict[str, Tensor]:    # named, weighted terms
    def decode(self, raw, seeds, approach, gripper) -> GraspGroup: ... # one grasp per seed


in: crop features	from the grouper, one row per valid seed, with the grouper's `out_axes`: `('seed', 'channel')` or `('seed', 'depth', 'channel')`.
in: matched labels	from the assigner, reduced to the chosen view: `dense` (a friction and a width per in-plane angle and depth bin) or `compact` (one best grasp per seed).
out, training	named loss terms, each multiplied by its `weight`; the trainer sums them and logs each as `loss/<name>`.
out, prediction	a `GraspGroup` with one grasp per seed: translation (the seed), rotation (approach from the view stage, in-plane angle from the head), depth, width (metres), score.

Attribute	Meaning	Rule
`in_axes`	axes of the crop features it expects	must equal the grouper's `out_axes`
`label_format`	`dense` or `compact`	must equal the assigner's (and the dataset must serve that label set)
`score_kind`	meaning of the decoded score	scores of different kinds are not comparable
`applies_seed_weights`	whether its loss honours a weight per seed	an assigner that returns weights needs a head that applies them
`loss_terms`	loss option → kind of term	every loss node is checked against it (Losses)

The assigner

The assigner runs in training only. It matches every seed with the labels of the label point it sits on and hands them, reduced to the chosen view, to the view stage and the head. So it is chosen with the head and the label set, not on its own.

NearestLabelAssigner (its defaults, no preset): every seed takes the labels of the nearest label point of its object; every seed is supervised.
nearest-label-object-seeds: the same matching; only seeds on an object are supervised, and training crops sit at the matched label point.
compact-label: one labelled grasp per seed at its chosen view, only within 5 mm of a label point; it brings the dataset section compact_labels.
scale-balanced: the matching above plus a loss weight per seed, larger for rare grasp widths. It needs the width histogram of the training split (grasppanda prepare graspnet1b --scale-statistic; configs/scale_balanced_grasp/README.md); compose adds the dataset section scale_statistic and says so.

grasppanda compose --show assigner=scale-balanced prints a preset with what must be re-tuned.

The settings of DecoupledGridHead

model:
  head:
    type: DecoupledGridHead
    num_angle: 12
    num_depth: 4
    depth_step: 0.01
    depth: class                  # class | regression (a continuous depth between the bins)
    width: bins                   # regression | class (bin centre) | bins (expectation)
    width_bins: 16                # for class and bins: 16 over 0.1 m is a bin every 6.7 mm
    score_target: log_friction_fixed_range
    friction_range: [0.1, 1.0]    # the range of the GraspNet-1Billion labels
    width_loss: {type: DistributionFocal, weight: 1.0}

The kind of depth_loss and width_loss follows the options depth and width, so a loss must change with them (the presets do).
The class terms start near ln 12 and ln 4, a regressed width is a distance in metres × 10: watch loss/angle and loss/depth beside loss/score and re-tune the weights.
width_decode_gain: 1.2 opens the gripper 20 % wider than learned, as the other heads of the family do; 1.0 decodes what was learned.
A head and a loss combine: --set head=decoupled-grid --set head.score_loss=varifocal.

DecoupledGridHead is written here after the idea of the grasp head of FGC-GraspNet (arXiv 2206.11141, ICRA 2022), without its code. No head has released weights of its own.

Pitfalls met while building this

A label set is a contract. Compose switches the labels and the assigner for a head of the other label set, and the dense grids then come from the label store of grasppanda prepare <dataset> --labels. A config written by hand with a mismatch is refused:

✗ assigner · head: CompactLabelAssigner returns 'compact' matched labels, but
  DecoupledGridHead is supervised with 'dense' ones. A head bundle and the assigner
  that feeds it must agree on the label set; the dataset must serve that label set too.

A logit loss changes what the score is. A score threshold fitted on the unbounded score of swad does not carry over to swad-focal, whose score lies in [0, 1].
SWADHead with score_target: log_friction_batch_norm (the GSNet reference) normalises targets with the extrema of the batch; log_friction_fixed_range gives a target that means the same in every batch (swad-focal sets it).
The extra class of EconomicHead. With unmatched_class: true every classifier has one output that is never supervised; label smoothing gives it target mass. Use decode_argmax: supervised (as economic-focal does) or unmatched_class: false.
Seed weights. A stage that receives the weights of ScaleBalancedAssigner must apply them (applies_seed_weights); a loss that scores the supervised set as a whole cannot, so it is refused there (Losses).

Not wrapped

Candidate	Why not
The grasp head of FGC-GraspNet as code	no licence file, and it derives from graspnet-baseline (non-commercial); its idea is `DecoupledGridHead`, written here.
Heads of graspnet-baseline and its descendants as code	a non-commercial licence; the ported heads are written from papers and behaviour.
Contact-GraspNet's per-point contact head	an NVIDIA non-commercial licence.
A decoupled head for depth-bin crops or for compact labels	`DecoupledGridHead` reads dense label grids and one feature per seed; under the other methods compose changes their labels or crop for it.