Losses

Seed-based · 5 methods--set <stage>.<term>_loss=20 options

Every loss term of a stage is a small config node of its own and is swapped on its own, for example --set head.score_loss=quality-focal. A loss fits only the kinds of term it accepts; HGGD computes its losses inside the detector, so this slot is not HGGD's.

Use it

GraspNet-baselineGSNetScale-Balanced-GraspEconomicGraspFineGrasp

grasppanda compose graspnet_baseline/graspnet-baseline_pointnet2_g1b-realsense_18e \
    --set head.angle_loss=focal -o graspnet-baseline_focal.yaml
grasppanda compose graspnet_baseline/graspnet-baseline_pointnet2_g1b-realsense_18e --list loss

grasppanda compose gsnet/gsnet_sunet14_g1b-realsense_10e \
    --set proposal.objectness_loss=focal -o gsnet_focal.yaml
grasppanda compose gsnet/gsnet_sunet14_g1b-realsense_10e --list loss

grasppanda compose scale_balanced_grasp/sbg_pointnet2-transformer_g1b-realsense_18e \
    --set head.angle_loss=focal -o scale-balanced-grasp_focal.yaml
grasppanda compose scale_balanced_grasp/sbg_pointnet2-transformer_g1b-realsense_18e --list loss

grasppanda compose economicgrasp/economicgrasp_sunet14_g1b-realsense_10e \
    --set head.score_loss=focal -o economicgrasp_focal.yaml
grasppanda compose economicgrasp/economicgrasp_sunet14_g1b-realsense_10e --list loss

grasppanda compose finegrasp/finegrasp_sunet14_g1b-realsense_10e \
    --set head.score_loss=focal -o finegrasp_focal.yaml
grasppanda compose finegrasp/finegrasp_sunet14_g1b-realsense_10e --list loss

The first command writes a checked config; the second lists every option of the slot under that method, with what was measured. Then grasppanda probe <file> tries the config on real frames and grasppanda train <file> trains it (Swap a component). Each method page lists the same: GraspNet-baseline · GSNet · Scale-Balanced-Grasp · EconomicGrasp · FineGrasp.

The left side of --set is <stage>.<term>, as the table of terms below the options lists it; the right side is a preset or a component type (--set head.width_loss=Huber builds it with its defaults). A preset carries no weight: compose keeps the weight of the loss it replaces and says so.

Options

A probe shows that a combination trains on real frames of GraspNet-1Billion, not how good it is. Accuracy is claimed only for a method trained and evaluated here.

For class terms · 9For quality and value terms · 9For bins, or any term · 2

Option	What it is	Runs	Trains like shipped
class-balanced fits class	Class-balanced cross-entropy: every class is weighted by the inverse of its effective number of samples.	5/5	5/5 on a par
dice fits class	Dice loss through kornia: one minus the overlap of predicted and labelled classes over the supervised set.	4/5 not Scale-Balanced-Grasp (incompatible by design)	4/4 on a par
focal fits class, unit	Sigmoid focal loss (RetinaNet) through torchvision: cross-entropy that turns down elements the network already gets right.	5/5	5/5 on a par
ghm-c fits unit, class	Gradient harmonising, classification form (GHM-C): sigmoid cross-entropy in which every range of gradient norms counts alike.	5/5	0/5 on a par · 1 diverging · 4 worse
label-smoothing fits class	Cross-entropy with label smoothing: a tenth of the target mass is spread over all classes.	5/5	5/5 on a par
lovasz-softmax fits class	Lovasz-Softmax loss through kornia: a surrogate of the mean intersection-over-union of the supervised set.	4/5 not Scale-Balanced-Grasp (incompatible by design)	3/4 on a par · 1 worse worse under GSNet
ohem fits class	Online hard example mining on a cross-entropy: only the hardest quarter of the supervised elements counts.	5/5	4/5 on a par · 1 worse worse under GSNet
poly1 fits class	Poly-1 loss (PolyLoss): the cross-entropy plus epsilon times (1 - p_t).	5/5	4/5 on a par · 1 worse worse under GraspNet-baseline
softmax-focal fits class	Softmax focal loss through kornia: (1 - p_t)^gamma times the cross-entropy over classes that exclude each other.	5/5	5/5 on a par

Option	What it is	Runs	Trains like shipped
balanced-l1 fits value, unit	Balanced L1 (Libra R-CNN): a Smooth-L1 whose small errors keep a larger gradient.	5/5	3/5 on a par · 2 worse worse under EconomicGrasp, FineGrasp
bce fits unit	Sigmoid cross-entropy between a logit and a soft or binary target in [0, 1].	3/3	2/3 on a par · 1 diverging diverging under FineGrasp
cauchy fits value, unit	Cauchy penalty through kornia: log(d^2 / 2 + 1), which grows only logarithmically for large errors.	5/5	5/5 on a par
charbonnier fits value, unit	Charbonnier penalty through kornia: sqrt(d^2 + 1) - 1, quadratic for small errors and linear for large ones.	5/5	5/5 on a par
ghm-r fits value, unit	Gradient harmonising, regression form (GHM-R): a smooth L1 in which every range of gradient norms counts alike.	5/5	2/5 on a par · 1 diverging · 2 worse worse under GSNet, EconomicGrasp; diverging under FineGrasp
huber fits value, unit	Huber penalty: quadratic below delta, linear with slope delta above it.	5/5	5/5 on a par
ohem-smooth-l1 fits value, unit	Online hard example mining on a Smooth-L1: only the hardest quarter of the supervised cells counts.	5/5	4/5 on a par · 1 worse worse under FineGrasp
quality-focal fits unit	Quality focal loss (Generalized Focal Loss): sigmoid cross-entropy to a soft target, focused on the cells whose prediction is far off.	3/4 not GraspNet-baseline (incompatible by design)	3/3 on a par
varifocal fits unit	Varifocal loss (VarifocalNet): cells with a grasp are weighted by their target quality, empty cells are focused.	3/3	2/3 on a par · 1 diverging diverging under GSNet

Option	What it is	Runs	Trains like shipped
distribution-focal fits bins	Distribution focal loss (Generalized Focal Loss): cross-entropy to the two bins around a continuous target.	5/5 measured in decoupled-grid-bins	not compared
uncertainty-weighting fits class, unit, value, bins	Uncertainty weighting (Kendall et al., CVPR 2018) of one loss term: a learned factor on the term, which pays for being small.	5/5	5/5 on a par

What the columns say, and who is left out

• Runs: out of the methods the option is a swap under, those under which its probe trained on real frames without error. A method that ships the option is named own and not counted; a method still undecided is counted and named. Measured on 20 training iterations on 8 RealSense frames of GraspNet-1Billion (one scene), seed 0, one run per cell; nothing was measured on GraspClutter6D or on Kinect frames.
• Trains like shipped: out of the methods under which the swap ran, those under which every comparable loss term ended within the method's tolerance of the component it ships, with nothing rising; a method left out is named with why. Measured on 250 iterations on 8 RealSense frames of GraspNet-1Billion (one scene), seed 0, at a constant learning rate (FineGrasp: its own warm-up), beside the method as shipped trained the same way twice.
• How we measure defines every word of these columns.

Who is left out, and why:

• ghm-c, trains like shipped: worse under GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp; diverging under FineGrasp.
• distribution-focal, runs: fits only the width term of the head bundle decoupled-grid-bins; measured with it.

The loss terms of every method, and their kinds

• class: (N, C) logits against (N,) class indices.
• unit: a prediction and a target of equal shape, the target a quality in [0, 1].
• value: a prediction and a target of equal shape, the target an unbounded quantity.
• bins: (N, C) logits over bins against (N,) fractional bin positions.

Method	Term and its kind
GraspNet-baseline	view.objectness_loss (class), view.view_loss (value), head.score_loss (value), head.angle_loss (class), head.width_loss (value), head.tolerance_loss (value)
GSNet	proposal.objectness_loss (class), proposal.graspness_loss (unit), view.loss (unit), head.score_loss (unit), head.width_loss (value)
Scale-Balanced-Grasp	view.objectness_loss (class), view.view_loss (value), head.score_loss (value), head.angle_loss (class), head.width_loss (value), head.tolerance_loss (value)
EconomicGrasp	proposal.objectness_loss (class), proposal.graspness_loss (unit), view.loss (unit), head.angle_loss (class), head.depth_loss (class), head.score_loss (class), head.width_loss (value)
FineGrasp	proposal.objectness_loss (class), proposal.graspness_loss (unit), view.loss (unit), head.angle_loss (class), head.depth_loss (class), head.score_loss (class), head.width_loss (value)

Compare under every method

Choosing

• Objectness (class, two classes, many easy points): focal (try alpha: -1 first), softmax-focal, class-balanced, ohem, or the set losses dice and lovasz-softmax.
• Angle, depth and quality classes (class, classes exclude each other): softmax-focal, label-smoothing, poly1, class-balanced, ohem.
• Graspness, view scores, score grids (unit): quality-focal, varifocal, ghm-c and bce train a logit; balanced-l1, huber, ghm-r, charbonnier, cauchy and ohem-smooth-l1 keep the output a value.
• Widths, tolerances, log-friction scores (value): the value losses above; set their knee (delta, beta, mu) in the units of the target.
• Balance the terms against each other: uncertainty-weighting on every term that takes part.

Details

Input and output of a loss

class Loss(nn.Module):
    def forward(self, prediction, target, mask=None, weights=None) -> Tensor  # weight x mean over the supervised elements

A stage declares the kind of every loss term it has, a loss the kinds it accepts, and grasppanda compose refuses a loss in a term of another kind, naming the losses that fit. Two more attributes of a loss matter:

• prediction: value or logit. A loss that trains the output of a unit term as a logit (QualityFocal, Varifocal, GHMC, FocalLoss, BinaryCrossEntropy) changes what the raw output means; the stages read it through the sigmoid where they consume it.
• applies_weights: false for a loss that scores the supervised set as a whole (DiceLoss, LovaszSoftmaxLoss). It is refused in a term that receives a weight per seed from the assigner.

grasppanda components <Loss> prints both and the accepted kinds.

What compose writes

For --set head.score_loss=quality-focal, the composed config holds the new node with the weight of the loss it replaced:

model:
  head:
    score_loss:
      type: QualityFocal
      beta: 2.0                   # exponent of |target - sigmoid(prediction)|
      weight: 15.0                # kept from the SmoothL1 it replaces: re-tune

By hand, --opts model.head.score_loss.type=FocalLoss on any command that takes --opts changes the type and keeps the other keys of the node, so it works only when the new loss has the options of the old one. Otherwise replace the whole node in a YAML file.

Wrappers: mining, class balance, multi-task weighting

TopK, ClassBalanced and UncertaintyWeighted take another loss as loss:.

model:
  proposal:
    objectness_loss:
      type: TopK
      fraction: 0.25              # share of the supervised elements that is kept
      loss: {type: FocalLoss, weight: 1.0, alpha: -1, gamma: 2.0}

uncertainty-weighting is a preset without a loss of its own: compose wraps the loss that is in the term, with its weight. A term that is not wrapped keeps its fixed weight. The learned value is the parameter <stage>.<term>.log_variance; exclude it from weight decay. Wrapped terms, and so the total loss, can become negative.

Sources and licences

Loss	Paper	Code
focal	Lin et al., ICCV 2017, arXiv 1708.02002	bridge to torchvision.ops.sigmoid_focal_loss (BSD-3-Clause)
softmax-focal, dice, lovasz-softmax, charbonnier, cauchy	focal loss; V-Net (arXiv 1606.04797); Berman et al., CVPR 2018; Barron, CVPR 2019	bridges to kornia.losses (Apache-2.0)
quality-focal, distribution-focal, varifocal, ghm-c, ghm-r, balanced-l1	Generalized Focal Loss (NeurIPS 2020), VarifocalNet (CVPR 2021), GHM (AAAI 2019), Libra R-CNN (CVPR 2019)	adapted from open-mmlab/mmdetection @ cfd5d3a (Apache-2.0)
huber, bce, label-smoothing	—; —; Szegedy et al., CVPR 2016	PyTorch
poly1	PolyLoss (arXiv 2204.12511)	the Poly-1 formula on PyTorch's cross-entropy, written here
ohem, ohem-smooth-l1	online hard example mining (Shrivastava et al., CVPR 2016)	written here (a topk over the penalties)
class-balanced	Cui et al., CVPR 2019	the weight formula of the paper, written here
uncertainty-weighting	Kendall et al., CVPR 2018	the formula as in the UW weighting of LibMTL (MIT), written here

No loss has weights. Without its library a config still loads; building the model raises an error that names the extra.

Pitfalls met while building this

• A loss in the wrong kind of term is refused, with the alternatives, for example a focal loss on a width (--set head.width_loss=focal under GSNet):

  ✗ assigner · head [structural]: SWADHead.width_loss is a `value` term (a prediction and a target
    of equal shape, the target an unbounded quantity), but FocalLoss fits `class` or `unit` terms.
    Losses for a `value` term: BalancedL1, GHMR, Huber, L1, MSE, RobustLoss, SmoothL1, TopK,
    UncertaintyWeighted.
  

• The scores of the GraspNet-baseline family are not qualities in [0, 1]. The score and view targets of OperationHead and ObjectnessViewStage are log-friction scores up to ln 10, so they are value terms and the sigmoid losses are refused there.

• Weights are kept, scales are not. A focal term is much smaller than the Smooth-L1 or cross-entropy it replaces; compare the logged loss/... values of both runs before trusting a weight.
• GHM with momentum > 0 keeps running counts as a buffer, saved with the checkpoint.
• Adding a constructor option to a loss the shipped configs use changes their config hash, and with it every measured record. Label smoothing is therefore a class of its own (LabelSmoothingCrossEntropy), and CrossEntropy keeps its constructor.

Not wrapped

Candidate	Why not
Gradient surgery between terms (PCGrad, Jacobian descent)	needs one backward pass per term inside the training step: a trainer hook, not a loss.
Ranking losses (allRank, torchsort)	a loss over the order of the grasps of a scene needs frame boundaries a loss does not see.
Ordinal regression of the quality classes (CORAL)	needs another output layer in EconomicHead: a new term kind and a head option.
Label and feature distribution smoothing for imbalanced regression	needs the label statistics of the training set per term.
Rotated-box losses (mmrotate)	they compare oriented image boxes; no seed-based stage predicts one.
Loss terms for HGGD	its loss terms are functions inside the detector, not config nodes.