Does it train like the shipped component?¶
An option that runs (Does it run?) may still train badly. Here each swap that ran trains 250 iterations beside the method as shipped (the reference below, not the authors' code), and the losses are compared term by term. Other pages call this check trains like shipped.
A probe shows that a combination trains on real frames of GraspNet-1Billion, not how good it is. Accuracy is claimed only for a method trained and evaluated here.
One run per cell, 250 iterations on 8 frames of one scene. An outcome is about training dynamics, not an accuracy measurement: a cell on a par may still detect worse grasps, and one that is worse here may catch up over a full schedule.
The answer, kind by method¶
| Kind | GraspNet-baseline | GSNet | Scale-Balanced-Grasp | EconomicGrasp | FineGrasp | HGGD |
|---|---|---|---|---|---|---|
| HGGD image encoders | — | — | — | — | — | 23 12 1 |
| Scene encoders | 39 5 2 | 20 18 3 | 27 10 9 | 11 26 4 | 9 24 5 | — |
| Seed proposals | 6 14 1 | 18 5 | 5 15 1 | 18 5 | 10 13 | — |
| View stages | 15 5 | 8 11 1 | 5 12 3 | 9 9 | 7 11 | — |
| Assigners | 2 | 1 1 | 1 1 | — | — | — |
| Groupers | 18 7 | 22 4 | 22 2 1 | 24 2 | 11 14 1 | — |
| Head bundles | 7 | 4 4 1 | 7 | 2 2 5 | 7 2 | — |
| Losses | 14 2 | 14 4 1 | 13 1 | 16 3 | 14 2 3 | — |
| Training recipe | 17 4 1 | 18 5 | 18 3 1 | 16 7 | 5 12 4 | 8 8 |
A cell counts the options of a kind under that method: on a par worse diverging or did not train not comparable. — : the method has no such kind. Every option is below; what is not counted is at the end.
How a cell is judged¶
- The same settings for a cell and its reference: 250 iterations on 8 RealSense frames of GraspNet-1Billion (one scene), seed 0, at a constant learning rate (FineGrasp: its own warm-up), beside the method as shipped trained the same way twice.
- Per loss term, the mean of the last 25 iterations of the cell is divided by the reference's.
- On a par: every comparable term ends within the method's tolerance, nothing rises, every value is finite and a frame has a grasp. Worse: a term ends above the tolerance, or a part got no gradient. Diverging: a term rises at the end, a value became non-finite, or no grasp was predicted.
- A term the option redefines is not compared (a loss preset, a head bundle, an assigner, a stage with its own loss code); the cell is judged on the other terms.
- Counted only where the comparison holds: a swap that did not run at level 1, and a cell trained on a schedule other than its reference's, are not counted; Not on this page names them.
- The tolerance of a method is 1.25 unless its two reference runs (seed 0 and 1) differ by more; then it is the next step of 0.05 above that spread.
- FineGrasp runs under
--schedule recipe, its reference and its cells alike: at the constant 0.002 without the recipe's 800-iteration warm-up, the two post-norm transformer layers that fuse FineGrasp's four crop scales give every seed one feature (the reference implementation's fusion does the same), so the head learns the label distribution and every grasp gets one score until about iteration 530; under the warm-up the fusion recovers by iteration 80.
Thresholds, measures and outcomes in full
- mean: the mean of a loss term over the last 25 of the 250 iterations (reported, finite values; the total
lossis not judged, it sums terms whose definitions may differ). - ratio: the cell's mean divided by the reference's mean of the same term.
- rise: over the last half of the run (iterations 126 to 250), the mean of its second half divided by the mean of its first half, minus 1 (the middle iteration of an odd count in neither half): +0.30 means the term ended 30 % higher than it was half-way.
- tolerance: per method: 1.25, or the first multiple of 0.05 above the largest ratio between two seeds of its reference (any term but the total, either way round), when that is larger.
- rising: a rise above the rising threshold of the method that also exceeds the reference's rise of the term by more than it (the rise alone for a term the reference does not report); the threshold is 0.1, or the first multiple of 0.05 above the largest difference between the rises of one term under two seeds of the reference, when that is larger.
- lower: a comparable term below 1/tolerance of the reference's: reported, not judged.
Outcomes, in order of precedence:
| Outcome | Meaning |
|---|---|
fails |
the run did not finish: an exception, a crash, a timeout, or no real frame read. A run that died of GPU memory is not recorded (it says nothing about the config) and is run again |
no reference |
the method as shipped has no finished run under the same settings: nothing to compare with |
diverging |
a loss term is rising over the last half of the run (see rising), a loss or a weight became non-finite, or no grasp was predicted on any of the 4 frames while the reference predicts some |
worse |
not diverging, and a comparable loss term ends above the tolerance of the method times the reference's (mean of the last 25 iterations), or a part of the model that holds trainable parameters received no gradient in any iteration |
not comparable |
not diverging, nothing worse, but the cell changes the definition of every term the reference has: there is no term to compare |
on a par |
every comparable loss term ends within the tolerance of the method, at least one term is comparable, nothing rises, every value is finite and a frame has a grasp. A term far below the reference's is reported (lower), not judged |
A term is not comparable when the cell changes its definition; the cell is judged on the others:
- loss: a loss preset replaces the function of the term it is put in.
- head: a head bundle defines the targets and the loss of every head term (score, width, angle, depth, tolerance).
- assigner: an assigner changes the labels the view stage and the head see (a view stage sees the labels through the assigner only): every view and head term.
- stage code: a proposal or view stage owns its targets and its loss: one whose class computes them with other code (its own
lossor target methods, not inherited from the reference's stage) defines every term of that stage anew; one that computes them as the reference's does, with the same loss options, keeps them (a proposal that changes only how it samples). - presence: a term only one of the two runs reports.
- sign: a term whose mean is not positive (a weighted loss can go below zero).
The tolerance and the rising threshold of a method. A cell and its reference are two runs, so a ratio away from 1 means something only beyond what two runs of the very same config show. Every reference was therefore trained twice, under seed 0 and seed 1 (The noise floor). Under the five seed-based methods, a term that has settled (it rose or fell by less than 10 % over the last half) agrees within 7 % between the seeds; a term still falling steeply at the end differs more (EconomicGrasp's graspness by 1.32x), and HGGD at its peak rate differs by up to 1.55x (angle, offsets). One tolerance for all would be blind under the stable methods or trigger-happy under HGGD, so each method gets its own: 1.25 (the plan's value, more than three times the spread of a settled term) unless its references spread more, then the first multiple of 0.05 above their largest ratio. The rising threshold follows the same rule from the rises. The spread comes from one pair of runs per method: a third seed would firm it up.
| Method | Largest ratio between two seeds | Tolerance | Largest difference of rise | Rising threshold |
|---|---|---|---|---|
| GraspNet-baseline | 1.117 | 1.25 | 0.051 | 0.1 |
| GSNet | 1.088 | 1.25 | 0.192 | 0.2 |
| Scale-Balanced-Grasp | 1.178 | 1.25 | 0.055 | 0.1 |
| EconomicGrasp | 1.174 | 1.25 | 0.108 | 0.15 |
| FineGrasp | 1.058 | 1.25 | 0.034 | 0.1 |
| HGGD | 1.548 | 1.55 | 0.159 | 0.2 |
Option by option¶
One chip per option and method: ✓ on a par, worse, diverging, did not train, n/c not comparable. Hover a chip for the ratios to the reference; the list under a table says why a cell is not on a par. A loss preset put in several terms shows the worst of them.
HGGD image encoders¶
36 options compared 23 on a par under every method compared · 37 of 37 selected cells run. 1 not counted (why).
HGGD image encoders: one row per option
| Option | HGGD |
|---|---|
| convnextv2-atto | worse |
| convnextv2-tiny | ✓ |
| dinov2-small-reg-depth-stem | ✓ |
| dinov2-small-reg | worse |
| dinov3-convnext-tiny | ✓ |
| dinov3-vits16 | diverging |
| efficientnetv2-s | ✓ |
| efficientvit-b1 | worse |
| eva02-tiny | worse |
| fastvit-sa12 | ✓ |
| fastvit-t8 | ✓ |
| hf-convnextv2-tiny | ✓ |
| hf-depth-anything-v2-small | worse |
| hf-dinov2-small-reg | ✓ |
| hf-dinov2-small | worse |
| hf-hiera-tiny | worse |
| hf-siglip2-base | ✓ |
| hf-swinv2-tiny | ✓ |
| hiera-small-abswin | ✓ |
| maxvit-tiny | ✓ |
| mobilenetv4-conv-small | ✓ |
| mobilenetv4-hybrid-medium | ✓ |
| pe-spatial-tiny | worse |
| repvit-m1 | worse |
| resnet18 | ✓ |
| resnet34-rgbd | ✓ |
| resnet34 | ✓ |
| resnet50 | ✓ |
| sam2-hiera-tiny | worse |
| siglip2-base | ✓ |
| swinv2-tiny | worse |
| tv-convnext-tiny | worse |
| tv-efficientnetv2-s | ✓ |
| tv-mobilenetv3-large | ✓ |
| tv-regnet-y-800mf | ✓ |
| tv-resnet34 | ✓ |
Not on a par, and why:
convnextv2-atto: HGGD: worse (heatmap 1.62x)dinov2-small-reg: HGGD: worse (anchor 2.05x)dinov3-vits16: HGGD: diverging (anchor rises +72 % over the last half)efficientvit-b1: HGGD: worse (anchor 1.59x)eva02-tiny: HGGD: worse (anchor 1.67x)hf-depth-anything-v2-small: HGGD: worse (anchor 1.71x)hf-dinov2-small: HGGD: worse (anchor 2.01x)hf-hiera-tiny: HGGD: worse (offsets 1.69x)pe-spatial-tiny: HGGD: worse (anchor 2.27x)repvit-m1: HGGD: worse (anchor 1.63x)sam2-hiera-tiny: HGGD: worse (anchor 1.59x)swinv2-tiny: HGGD: worse (anchor 1.78x)tv-convnext-tiny: HGGD: worse (anchor 1.63x)
Scene encoders¶
47 options compared 9 on a par under every method compared · 230 of 230 selected cells run. 18 not counted (why).
Scene encoders: one row per option
| Option | GraspNet-baseline | GSNet | Scale-Balanced-Grasp | EconomicGrasp | FineGrasp |
|---|---|---|---|---|---|
| assanet | ✓ | ✓ | ✓ | ✓ | ✓ |
| dgcnn-static | ✓ | worse | worse | worse | worse |
| dgcnn | worse | worse | diverging | worse | worse |
| dela | ✓ | ✓ | ✓ | ✓ | diverging |
| litept-structured3d | ✓ | ✓ | ✓ | worse | worse |
| litept | ✓ | ✓ | ✓ | ✓ | ✓ |
| minkunet18a-openscene | ✓ | ✓ | ✓ | worse | ✓ |
| minkunet18a | ✓ | ✓ | ✓ | worse | ✓ |
| minkunet34c-pointcontrast | ✓ | ✓ | ✓ | ✓ | ✓ |
| minkunet34c | ✓ | ✓ | ✓ | worse | ✓ |
| oacnns | ✓ | — | ✓ | — | — |
| octformer | ✓ | ✓ | ✓ | ✓ | worse |
| ptv3-encoder | ✓ | — | ✓ | — | — |
| ptv3-ppt | ✓ | ✓ | diverging | ✓ | diverging |
| ptv3-scannet200 | ✓ | ✓ | ✓ | ✓ | — |
| ptv3 | ✓ | ✓ | ✓ | ✓ | worse |
| ptv3-sonata-small | ✓ | — | ✓ | — | — |
| ptv3-sonata | ✓ | — | ✓ | — | diverging |
| ptv3-utonia | ✓ | — | diverging | — | diverging |
| pointmlp | ✓ | worse | diverging | worse | worse |
| point-mamba | ✓ | worse | ✓ | worse | worse |
| pointmetabase-xl | ✓ | ✓ | diverging | ✓ | worse |
| pointmetabase | ✓ | ✓ | ✓ | worse | worse |
| pointnext-b | ✓ | ✓ | ✓ | worse | worse |
| pointnext-l | ✓ | worse | worse | worse | worse |
| pointnext-xl | ✓ | worse | ✓ | worse | worse |
| pointnext | ✓ | ✓ | ✓ | worse | worse |
| pointnet2 | — | worse | worse | worse | worse |
| pointnet2-dense | ✓ | worse | ✓ | worse | worse |
| pointnet2-msg | ✓ | worse | ✓ | worse | worse |
| pointnet2-msg-dense | ✓ | ✓ | ✓ | worse | worse |
| pointnet2-transformer | ✓ | worse | — | worse | worse |
| point-transformer-v1 | diverging | diverging | diverging | diverging | ✓ |
| point-transformer-v2 | diverging | diverging | diverging | diverging | ✓ |
| pointvector-xl | ✓ | ✓ | ✓ | worse | worse |
| pointvector | ✓ | worse | ✓ | worse | worse |
| point-vit | worse | worse | worse | worse | worse |
| pyg-edgeconv | ✓ | worse | worse | worse | — |
| pyg-point-transformer | worse | worse | worse | worse | — |
| pyg-pointgnn | worse | worse | worse | worse | — |
| pyg-pointnet | worse | worse | worse | worse | worse |
| randlanet | ✓ | diverging | worse | diverging | diverging |
| spunet-gaussiancross | ✓ | worse | diverging | ✓ | ✓ |
| spunet | ✓ | ✓ | ✓ | ✓ | — |
| sparse-unet14 | ✓ | — | ✓ | — | — |
| uni3d | ✓ | worse | diverging | worse | worse |
| volt | ✓ | ✓ | worse | diverging | worse |
Not on a par, and why:
dgcnn-static: GSNet: worse (objectness 2.86x); Scale-Balanced-Grasp: worse (objectness 1.27x); EconomicGrasp: worse (objectness 2.72x); FineGrasp: worse (objectness 2.74x)dgcnn: GraspNet-baseline: worse (objectness 1.26x); GSNet: worse (objectness 2.99x); Scale-Balanced-Grasp: diverging (no grasp on any of the 4 frames); EconomicGrasp: worse (objectness 3.44x); FineGrasp: worse (objectness 2.60x)dela: FineGrasp: diverging (no grasp on any of the 4 frames)litept-structured3d: EconomicGrasp: worse (objectness 1.26x); FineGrasp: worse (objectness 1.35x)minkunet18a-openscene: EconomicGrasp: worse (objectness 1.52x)minkunet18a: EconomicGrasp: worse (objectness 2.27x)minkunet34c: EconomicGrasp: worse (objectness 1.32x)octformer: FineGrasp: worse (objectness 1.42x)ptv3-ppt: Scale-Balanced-Grasp: diverging (width rises +43 % over the last half); FineGrasp: diverging (objectness rises +42 % over the last half)ptv3: FineGrasp: worse (graspness 2.10x)ptv3-sonata: FineGrasp: diverging (angle rises +49 % over the last half)ptv3-utonia: Scale-Balanced-Grasp: diverging (width rises +40 % over the last half); FineGrasp: diverging (objectness rises +47 % over the last half)pointmlp: GSNet: worse (objectness 2.21x); Scale-Balanced-Grasp: diverging (tolerance rises +13 % over the last half); EconomicGrasp: worse (objectness 2.14x); FineGrasp: worse (objectness 1.97x)point-mamba: GSNet: worse (objectness 1.31x); EconomicGrasp: worse (objectness 1.83x); FineGrasp: worse (objectness 1.52x)pointmetabase-xl: Scale-Balanced-Grasp: diverging (width rises +33 % over the last half); FineGrasp: worse (objectness 1.30x)pointmetabase: EconomicGrasp: worse (objectness 1.50x); FineGrasp: worse (objectness 1.57x)pointnext-b: EconomicGrasp: worse (objectness 1.71x); FineGrasp: worse (objectness 1.54x)pointnext-l: GSNet: worse (objectness 1.62x); Scale-Balanced-Grasp: worse (tolerance 1.32x); EconomicGrasp: worse (objectness 2.26x); FineGrasp: worse (objectness 1.54x)pointnext-xl: GSNet: worse (objectness 1.43x); EconomicGrasp: worse (objectness 1.86x); FineGrasp: worse (objectness 1.40x)pointnext: EconomicGrasp: worse (objectness 1.66x); FineGrasp: worse (objectness 1.50x)pointnet2: GSNet: worse (objectness 1.92x); Scale-Balanced-Grasp: worse (tolerance 1.46x); EconomicGrasp: worse (objectness 2.85x); FineGrasp: worse (objectness 2.39x)pointnet2-dense: GSNet: worse (objectness 1.68x); EconomicGrasp: worse (objectness 2.27x); FineGrasp: worse (objectness 2.20x)pointnet2-msg: GSNet: worse (objectness 1.37x); EconomicGrasp: worse (objectness 2.06x); FineGrasp: worse (objectness 2.07x)pointnet2-msg-dense: EconomicGrasp: worse (objectness 1.41x); FineGrasp: worse (objectness 1.40x)pointnet2-transformer: GSNet: worse (objectness 1.33x); EconomicGrasp: worse (objectness 2.18x); FineGrasp: worse (objectness 2.13x)point-transformer-v1: GraspNet-baseline: diverging (no grasp on any of the 4 frames); GSNet: diverging (no grasp on any of the 4 frames); Scale-Balanced-Grasp: diverging (no grasp on any of the 4 frames); EconomicGrasp: diverging (no grasp on any of the 4 frames)point-transformer-v2: GraspNet-baseline: diverging (no grasp on any of the 4 frames); GSNet: diverging (no grasp on any of the 4 frames); Scale-Balanced-Grasp: diverging (no grasp on any of the 4 frames); EconomicGrasp: diverging (no grasp on any of the 4 frames)pointvector-xl: EconomicGrasp: worse (objectness 1.54x); FineGrasp: worse (objectness 1.42x)pointvector: GSNet: worse (objectness 1.64x); EconomicGrasp: worse (objectness 2.43x); FineGrasp: worse (objectness 1.56x)point-vit: GraspNet-baseline: worse (objectness 1.32x); GSNet: worse (objectness 1.79x); Scale-Balanced-Grasp: worse (objectness 1.41x); EconomicGrasp: worse (objectness 2.65x); FineGrasp: worse (objectness 1.71x)pyg-edgeconv: GSNet: worse (objectness 3.33x); Scale-Balanced-Grasp: worse (objectness 1.51x); EconomicGrasp: worse (objectness 3.06x)pyg-point-transformer: GraspNet-baseline: worse (objectness 1.37x); GSNet: worse (objectness 3.44x); Scale-Balanced-Grasp: worse (objectness 1.61x); EconomicGrasp: worse (objectness 3.08x)pyg-pointgnn: GraspNet-baseline: worse (objectness 1.37x); GSNet: worse (objectness 3.45x); Scale-Balanced-Grasp: worse (objectness 1.60x); EconomicGrasp: worse (objectness 3.10x)pyg-pointnet: GraspNet-baseline: worse (objectness 1.34x); GSNet: worse (objectness 3.37x); Scale-Balanced-Grasp: worse (objectness 1.57x); EconomicGrasp: worse (objectness 3.00x); FineGrasp: worse (objectness 2.61x)randlanet: GSNet: diverging (no grasp on any of the 4 frames); Scale-Balanced-Grasp: worse (objectness 1.34x); EconomicGrasp: diverging (no grasp on any of the 4 frames); FineGrasp: diverging (no grasp on any of the 4 frames)spunet-gaussiancross: GSNet: worse (score 1.33x); Scale-Balanced-Grasp: diverging (width rises +26 % over the last half)uni3d: GSNet: worse (objectness 1.65x); Scale-Balanced-Grasp: diverging (width rises +29 % over the last half); EconomicGrasp: worse (objectness 2.77x); FineGrasp: worse (objectness 1.66x)volt: Scale-Balanced-Grasp: worse (tolerance 1.34x); EconomicGrasp: diverging (no grasp on any of the 4 frames); FineGrasp: worse (objectness 2.08x)
Seed proposals¶
24 options compared 1 on a par under every method compared · 111 of 111 selected cells run.
Seed proposals: one row per option
| Option | GraspNet-baseline | GSNet | Scale-Balanced-Grasp | EconomicGrasp | FineGrasp |
|---|---|---|---|---|---|
| dense-ffps | ✓ | ✓ | ✓ | ✓ | worse |
| dense-fps | ✓ | ✓ | ✓ | ✓ | worse |
| dense-fpsample | ✓ | ✓ | ✓ | ✓ | worse |
| dense-fusion-fps | worse | ✓ | worse | ✓ | worse |
| dense-random | ✓ | ✓ | ✓ | ✓ | worse |
| dense-torch-cluster | ✓ | ✓ | ✓ | ✓ | worse |
| dense-voxel | ✓ | ✓ | worse | ✓ | worse |
| graspness-ffps | diverging | ✓ | worse | ✓ | ✓ |
| graspness-fps-fill | worse | ✓ | worse | ✓ | ✓ |
| graspness-fps-topk | worse | worse | worse | ✓ | ✓ |
| graspness-fps | worse | — | worse | — | — |
| graspness-fpsample | worse | ✓ | worse | ✓ | ✓ |
| graspness-fusion-fps | worse | ✓ | worse | ✓ | worse |
| graspness-nms | worse | worse | worse | worse | worse |
| graspness-random | worse | ✓ | worse | ✓ | ✓ |
| graspness-sfps | worse | ✓ | worse | ✓ | ✓ |
| graspness-top-fraction | worse | worse | worse | worse | ✓ |
| graspness-topk | worse | worse | worse | worse | worse |
| graspness-torch-cluster | worse | ✓ | worse | ✓ | ✓ |
| graspness-voxel | worse | worse | diverging | worse | worse |
| objectness-fps | — | ✓ | — | ✓ | worse |
| product-fps | — | ✓ | — | ✓ | ✓ |
| scored-fps | worse | ✓ | worse | ✓ | ✓ |
| SubsampledSeeds | — | ✓ | — | worse | worse |
Not on a par, and why:
dense-ffps: FineGrasp: worse (angle 1.53x)dense-fps: FineGrasp: worse (width 1.53x)dense-fpsample: FineGrasp: worse (width 1.51x)dense-fusion-fps: GraspNet-baseline: worse (width 1.31x); Scale-Balanced-Grasp: worse (tolerance 1.33x); FineGrasp: worse (angle 1.56x)dense-random: FineGrasp: worse (width 1.58x)dense-torch-cluster: FineGrasp: worse (width 1.51x)dense-voxel: Scale-Balanced-Grasp: worse (tolerance 1.69x); FineGrasp: worse (width 1.58x)graspness-ffps: GraspNet-baseline: diverging (view rises +13 % over the last half); Scale-Balanced-Grasp: worse (objectness 2.33x)graspness-fps-fill: GraspNet-baseline: worse (view 1.82x); Scale-Balanced-Grasp: worse (objectness 2.38x)graspness-fps-topk: GraspNet-baseline: worse (view 2.74x); GSNet: worse (score 1.35x); Scale-Balanced-Grasp: worse (objectness 1.92x)graspness-fps: GraspNet-baseline: worse (view 1.77x); Scale-Balanced-Grasp: worse (objectness 2.31x)graspness-fpsample: GraspNet-baseline: worse (objectness 1.25x); Scale-Balanced-Grasp: worse (objectness 2.18x)graspness-fusion-fps: GraspNet-baseline: worse (view 1.69x); Scale-Balanced-Grasp: worse (objectness 2.28x); FineGrasp: worse (width 1.30x)graspness-nms: GraspNet-baseline: worse (view 2.58x); GSNet: worse (objectness 1.27x); Scale-Balanced-Grasp: worse (objectness 1.51x); EconomicGrasp: worse (graspness 1.57x); FineGrasp: worse (objectness 1.70x)graspness-random: GraspNet-baseline: worse (view 2.00x); Scale-Balanced-Grasp: worse (objectness 2.24x)graspness-sfps: GraspNet-baseline: worse (view 2.10x); Scale-Balanced-Grasp: worse (objectness 2.12x)graspness-top-fraction: GraspNet-baseline: worse (view 2.72x); GSNet: worse (score 1.38x); Scale-Balanced-Grasp: worse (objectness 2.54x); EconomicGrasp: worse (depth 1.42x)graspness-topk: GraspNet-baseline: worse (view 3.06x); GSNet: worse (score 1.56x); Scale-Balanced-Grasp: worse (objectness 2.20x); EconomicGrasp: worse (depth 1.48x); FineGrasp: worse (angle 1.39x)graspness-torch-cluster: GraspNet-baseline: worse (view 1.89x); Scale-Balanced-Grasp: worse (objectness 2.26x)graspness-voxel: GraspNet-baseline: worse (view 1.79x); GSNet: worse (graspness 1.42x); Scale-Balanced-Grasp: diverging (width rises +34 % over the last half); EconomicGrasp: worse (width 1.25x); FineGrasp: worse (objectness 1.27x)objectness-fps: FineGrasp: worse (width 1.28x)scored-fps: GraspNet-baseline: worse (view 1.86x); Scale-Balanced-Grasp: worse (objectness 2.11x)SubsampledSeeds: EconomicGrasp: worse (width 1.26x); FineGrasp: worse (width 1.47x)
View stages¶
21 options compared 3 on a par under every method compared · 96 of 96 selected cells run.
View stages: one row per option
| Option | GraspNet-baseline | GSNet | Scale-Balanced-Grasp | EconomicGrasp | FineGrasp |
|---|---|---|---|---|---|
| direction-regression | ✓ | worse | diverging | worse | worse |
| direction-vmf | ✓ | worse | worse | worse | worse |
| healpix-192 | worse | ✓ | worse | ✓ | ✓ |
| healpix-768 | ✓ | ✓ | ✓ | ✓ | ✓ |
| icosphere-162 | worse | ✓ | worse | ✓ | ✓ |
| icosphere-642 | ✓ | ✓ | ✓ | ✓ | ✓ |
| lattice-angular-cost | ✓ | worse | worse | worse | worse |
| lattice-best-view | ✓ | worse | worse | worse | worse |
| lattice-fibonacci-120 | ✓ | ✓ | ✓ | ✓ | ✓ |
| lattice-hemisphere-60 | ✓ | ✓ | ✓ | ✓ | worse |
| lattice-soft-argmax | ✓ | ✓ | worse | ✓ | ✓ |
| lattice-topk | ✓ | worse | worse | ✓ | worse |
| lattice-view | ✓ | ✓ | worse | ✓ | ✓ |
| objectness-view-graspable | worse | worse | — | — | — |
| objectness-view | — | diverging | worse | — | — |
| rotation-6d | ✓ | worse | diverging | worse | worse |
| rotation-9d | ✓ | worse | diverging | worse | worse |
| rotation-quaternion | ✓ | worse | worse | worse | worse |
| template-view | worse | — | worse | — | — |
| two-vector-baseline | ✓ | worse | worse | worse | worse |
| two-vector | worse | worse | ✓ | worse | worse |
Not on a par, and why:
direction-regression: GSNet: worse (score 1.75x); Scale-Balanced-Grasp: diverging (width rises +32 % over the last half); EconomicGrasp: worse (graspness 1.69x); FineGrasp: worse (angle 1.47x)direction-vmf: GSNet: worse (graspness 1.38x); Scale-Balanced-Grasp: worse (tolerance 1.70x); EconomicGrasp: worse (graspness 1.40x); FineGrasp: worse (graspness 1.41x)healpix-192: GraspNet-baseline: worse (width 1.29x); Scale-Balanced-Grasp: worse (tolerance 1.60x)icosphere-162: GraspNet-baseline: worse (width 1.31x); Scale-Balanced-Grasp: worse (tolerance 1.42x)lattice-angular-cost: GSNet: worse (graspness 1.39x); Scale-Balanced-Grasp: worse (tolerance 2.01x); EconomicGrasp: worse (graspness 1.33x); FineGrasp: worse (graspness 2.86x)lattice-best-view: GSNet: worse (graspness 1.30x); Scale-Balanced-Grasp: worse (tolerance 2.43x); EconomicGrasp: worse (graspness 1.37x); FineGrasp: worse (graspness 2.47x)lattice-hemisphere-60: FineGrasp: worse (depth 1.26x)lattice-soft-argmax: Scale-Balanced-Grasp: worse (tolerance 1.33x)lattice-topk: GSNet: worse (score 1.92x); Scale-Balanced-Grasp: worse (tolerance 1.44x); FineGrasp: worse (angle 1.45x)lattice-view: Scale-Balanced-Grasp: worse (tolerance 1.33x)objectness-view-graspable: GraspNet-baseline: worse (view 2.87x); GSNet: worse (objectness 2.60x)objectness-view: GSNet: diverging (graspness rises +24 % over the last half); Scale-Balanced-Grasp: worse (objectness 1.69x)rotation-6d: GSNet: worse (objectness 1.50x); Scale-Balanced-Grasp: diverging (tolerance rises +17 % over the last half); EconomicGrasp: worse (graspness 1.74x); FineGrasp: worse (objectness 1.46x)rotation-9d: GSNet: worse (objectness 1.32x); Scale-Balanced-Grasp: diverging (width rises +101 % over the last half); EconomicGrasp: worse (graspness 1.72x); FineGrasp: worse (objectness 1.39x)rotation-quaternion: GSNet: worse (objectness 1.27x); Scale-Balanced-Grasp: worse (tolerance 1.36x); EconomicGrasp: worse (graspness 1.78x); FineGrasp: worse (graspness 1.62x)template-view: GraspNet-baseline: worse (tolerance 1.31x); Scale-Balanced-Grasp: worse (tolerance 1.33x)two-vector-baseline: GSNet: worse (graspness 2.42x); Scale-Balanced-Grasp: worse (tolerance 1.83x); EconomicGrasp: worse (objectness 1.25x); FineGrasp: worse (graspness 1.44x)two-vector: GraspNet-baseline: worse (width 1.43x); GSNet: worse (objectness 1.72x); EconomicGrasp: worse (graspness 1.84x); FineGrasp: worse (objectness 1.38x)
Assigners¶
3 options compared 1 on a par under every method compared · 6 of 6 selected cells run.
Assigners: one row per option
| Option | GraspNet-baseline | GSNet | Scale-Balanced-Grasp |
|---|---|---|---|
| nearest-label-object-seeds | — | ✓ | n/c |
| scale-balanced | n/c | worse | — |
| NearestLabelAssigner | n/c | — | diverging |
Not on a par, and why:
scale-balanced: GSNet: worse (graspness 1.29x)NearestLabelAssigner: Scale-Balanced-Grasp: diverging (no grasp on any of the 4 frames)
Groupers¶
28 options compared 6 on a par under every method compared · 128 of 128 selected cells run.
Groupers: one row per option
| Option | GraspNet-baseline | GSNet | Scale-Balanced-Grasp | EconomicGrasp | FineGrasp |
|---|---|---|---|---|---|
| ball | ✓ | ✓ | ✓ | ✓ | worse |
| box | ✓ | ✓ | ✓ | ✓ | worse |
| cylinder-attentive | ✓ | ✓ | ✓ | worse | ✓ |
| cylinder-edgeconv | ✓ | ✓ | ✓ | ✓ | diverging |
| cylinder-invres | worse | diverging | ✓ | ✓ | ✓ |
| cylinder-vector-attention | ✓ | diverging | ✓ | ✓ | ✓ |
| depth-ball | ✓ | ✓ | worse | ✓ | worse |
| depth-box | ✓ | ✓ | ✓ | ✓ | ✓ |
| depth-cylinder-attentive | ✓ | ✓ | ✓ | ✓ | ✓ |
| depth-cylinder-multi-scale | ✓ | ✓ | — | ✓ | ✓ |
| depth-cylinder-projection | ✓ | ✓ | ✓ | ✓ | ✓ |
| depth-embed | worse | — | ✓ | — | — |
| depth-knn | ✓ | ✓ | ✓ | ✓ | worse |
| depth-pool | — | ✓ | — | ✓ | worse |
| knn | worse | ✓ | ✓ | ✓ | worse |
| multi-scale-linear | ✓ | ✓ | ✓ | ✓ | worse |
| multi-scale | worse | ✓ | diverging | ✓ | — |
| pyg-edge-conv | ✓ | ✓ | ✓ | ✓ | worse |
| pyg-gatv2-conv | ✓ | ✓ | ✓ | ✓ | worse |
| pyg-point-transformer-conv | ✓ | ✓ | ✓ | ✓ | worse |
| pyg-pointnet-conv | worse | ✓ | ✓ | worse | worse |
| pyg-transformer-conv | ✓ | diverging | ✓ | ✓ | ✓ |
| roi-aware | ✓ | diverging | ✓ | ✓ | ✓ |
| roi-grid | ✓ | ✓ | ✓ | ✓ | ✓ |
| voxel-set-abstraction | — | ✓ | — | ✓ | ✓ |
| CylinderGroup | worse | — | ✓ | ✓ | worse |
| CylinderInteractionGroup | worse | ✓ | ✓ | — | worse |
| DepthCylinderGroup | — | ✓ | worse | ✓ | worse |
Not on a par, and why:
ball: FineGrasp: worse (angle 1.30x)box: FineGrasp: worse (objectness 1.27x)cylinder-attentive: EconomicGrasp: worse (graspness 1.31x)cylinder-edgeconv: FineGrasp: diverging (objectness rises +18 % over the last half)cylinder-invres: GraspNet-baseline: worse (objectness 1.26x); GSNet: diverging (score rises +50 % over the last half)cylinder-vector-attention: GSNet: diverging (score rises +48 % over the last half)depth-ball: Scale-Balanced-Grasp: worse (tolerance 2.88x); FineGrasp: worse (depth 1.52x)depth-embed: GraspNet-baseline: worse (objectness 1.27x)depth-knn: FineGrasp: worse (width 1.32x)depth-pool: FineGrasp: worse (depth 1.27x)knn: GraspNet-baseline: worse (objectness 1.28x); FineGrasp: worse (angle 1.44x)multi-scale-linear: FineGrasp: worse (width 1.32x)multi-scale: GraspNet-baseline: worse (objectness 1.26x); Scale-Balanced-Grasp: diverging (tolerance rises +17 % over the last half)pyg-edge-conv: FineGrasp: worse (angle 1.30x)pyg-gatv2-conv: FineGrasp: worse (depth 1.27x)pyg-point-transformer-conv: FineGrasp: worse (width 1.41x)pyg-pointnet-conv: GraspNet-baseline: worse (objectness 1.25x); EconomicGrasp: worse (graspness 1.45x); FineGrasp: worse (graspness 1.32x)pyg-transformer-conv: GSNet: diverging (score rises +45 % over the last half)roi-aware: GSNet: diverging (score rises +43 % over the last half)CylinderGroup: GraspNet-baseline: worse (objectness 1.27x); FineGrasp: worse (graspness 1.34x)CylinderInteractionGroup: GraspNet-baseline: worse (objectness 1.27x); FineGrasp: worse (width 1.26x)DepthCylinderGroup: Scale-Balanced-Grasp: worse (tolerance 2.24x); FineGrasp: worse (depth 1.27x)
Head bundles¶
10 options compared 3 on a par under every method compared · 41 of 41 selected cells run.
Head bundles: one row per option
| Option | GraspNet-baseline | GSNet | Scale-Balanced-Grasp | EconomicGrasp | FineGrasp |
|---|---|---|---|---|---|
| decoupled-grid-bins | ✓ | ✓ | ✓ | diverging | ✓ |
| decoupled-grid-class | ✓ | worse | ✓ | diverging | ✓ |
| decoupled-grid-regression | ✓ | diverging | ✓ | diverging | diverging |
| decoupled-grid | ✓ | worse | ✓ | diverging | ✓ |
| economic-focal | — | ✓ | — | ✓ | ✓ |
| economic | — | ✓ | — | — | — |
| operation-objectness | ✓ | worse | — | worse | diverging |
| operation | — | worse | ✓ | worse | ✓ |
| swad-focal | ✓ | ✓ | ✓ | ✓ | ✓ |
| swad | ✓ | — | ✓ | diverging | ✓ |
Not on a par, and why:
decoupled-grid-bins: EconomicGrasp: diverging (score rises +19 % over the last half)decoupled-grid-class: GSNet: worse (graspness 1.28x); EconomicGrasp: diverging (score rises +24 % over the last half)decoupled-grid-regression: GSNet: diverging (score rises +40 % over the last half); EconomicGrasp: diverging (score rises +21 % over the last half); FineGrasp: diverging (score rises +22 % over the last half)decoupled-grid: GSNet: worse (graspness 1.25x); EconomicGrasp: diverging (score rises +31 % over the last half)operation-objectness: GSNet: worse (objectness 1.35x); EconomicGrasp: worse (graspness 1.51x); FineGrasp: diverging (tolerance rises +24 % over the last half)operation: GSNet: worse (objectness 1.33x); EconomicGrasp: worse (objectness 1.26x)swad: EconomicGrasp: diverging (score rises +26 % over the last half)
Losses¶
19 options compared 10 on a par under every method compared · 130 of 130 selected cells run. 11 not counted (why).
Losses: one row per option
| Option | GraspNet-baseline | GSNet | Scale-Balanced-Grasp | EconomicGrasp | FineGrasp |
|---|---|---|---|---|---|
| focal | ✓ | ✓ | ✓ | ✓ | ✓ |
| quality-focal | — | ✓ | — | ✓ | ✓ |
| balanced-l1 | ✓ | ✓ | ✓ | worse | worse |
| softmax-focal | ✓ | ✓ | ✓ | ✓ | ✓ |
| varifocal | — | diverging | — | ✓ | ✓ |
| huber | ✓ | ✓ | ✓ | ✓ | ✓ |
| label-smoothing | ✓ | ✓ | ✓ | ✓ | ✓ |
| bce | — | ✓ | — | ✓ | diverging |
| ghm-r | ✓ | worse | ✓ | worse | diverging |
| poly1 | worse | ✓ | ✓ | ✓ | ✓ |
| ghm-c | worse | worse | worse | worse | diverging |
| charbonnier | ✓ | ✓ | ✓ | ✓ | ✓ |
| ohem | ✓ | worse | ✓ | ✓ | ✓ |
| ohem-smooth-l1 | ✓ | ✓ | ✓ | ✓ | worse |
| cauchy | ✓ | ✓ | ✓ | ✓ | ✓ |
| class-balanced | ✓ | ✓ | ✓ | ✓ | ✓ |
| dice | ✓ | ✓ | — | ✓ | ✓ |
| lovasz-softmax | ✓ | worse | — | ✓ | ✓ |
| uncertainty-weighting | ✓ | ✓ | ✓ | ✓ | ✓ |
Not on a par, and why:
balanced-l1: EconomicGrasp: worse (graspness 1.59x); EconomicGrasp: worse (graspness 1.71x); FineGrasp: worse (objectness 1.41x); FineGrasp: worse (graspness 1.31x)varifocal: GSNet: diverging (score rises +43 % over the last half)bce: FineGrasp: diverging (graspness rises +24 % over the last half)ghm-r: GSNet: worse (objectness 1.35x); EconomicGrasp: worse (graspness 2.10x); EconomicGrasp: worse (objectness 1.27x); FineGrasp: worse (objectness 1.99x); FineGrasp: diverging (graspness rises +15 % over the last half)poly1: GraspNet-baseline: worse (width 1.47x)ghm-c: GraspNet-baseline: worse (width 1.35x); GSNet: worse (graspness 1.33x); GSNet: worse (objectness 3.50x); Scale-Balanced-Grasp: worse (width 1.50x); EconomicGrasp: worse (objectness 1.37x); EconomicGrasp: worse (objectness 4.03x); FineGrasp: diverging (depth rises +10 % over the last half); FineGrasp: worse (objectness 5.96x)ohem: GSNet: worse (graspness 1.92x)ohem-smooth-l1: FineGrasp: worse (graspness 1.29x)lovasz-softmax: GSNet: worse (graspness 1.27x)
Training recipe¶
23 options compared 6 on a par under every method compared · 204 of 204 selected cells run. 77 not counted (why).
Training recipe: one row per option
| Option | GraspNet-baseline | GSNet | Scale-Balanced-Grasp | EconomicGrasp | FineGrasp | HGGD |
|---|---|---|---|---|---|---|
| optimizer=adamw-no-decay-1d | ✓ | ✓ | ✓ | ✓ | worse | worse |
| optimizer=adamw | ✓ | ✓ | worse | ✓ | worse | worse |
| optimizer=ademamix | ✓ | ✓ | ✓ | ✓ | worse | worse |
| optimizer=encoder-lr-0.1 | worse | worse | ✓ | worse | diverging | — |
| optimizer=layer-decay | ✓ | worse | ✓ | worse | diverging | — |
| optimizer=prodigy | ✓ | ✓ | worse | ✓ | worse | ✓ |
| optimizer=soap | ✓ | ✓ | ✓ | ✓ | worse | — |
| optimizer=stableadamw | worse | ✓ | ✓ | ✓ | worse | worse |
| optimizer=timm-adan | ✓ | worse | ✓ | worse | diverging | worse |
| optimizer=timm-adopt | ✓ | ✓ | ✓ | worse | worse | worse |
| optimizer=timm-lamb | ✓ | ✓ | ✓ | ✓ | diverging | ✓ |
| optimizer=timm-lion | worse | worse | diverging | worse | worse | worse |
| optimizer=timm-madgrad | diverging | ✓ | worse | ✓ | worse | ✓ |
| optimizer=timm-mars | ✓ | ✓ | ✓ | ✓ | worse | worse |
| hooks=accumulate-2 | worse | worse | ✓ | worse | ✓ | — |
| hooks=agc | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| hooks=ema | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| hooks=freeze-encoder | ✓ | ✓ | ✓ | ✓ | — | — |
| hooks=grad-clip | ✓ | ✓ | ✓ | ✓ | worse | ✓ |
| hooks=swa | — | ✓ | — | ✓ | ✓ | ✓ |
| augment=dropout | ✓ | ✓ | ✓ | ✓ | — | ✓ |
| augment=jitter | ✓ | ✓ | ✓ | ✓ | ✓ | — |
| augment=shift | ✓ | ✓ | ✓ | worse | worse | — |
Not on a par, and why:
optimizer=adamw-no-decay-1d: FineGrasp: worse (objectness 1.70x); HGGD: worse (angle 2.08x)optimizer=adamw: Scale-Balanced-Grasp: worse (width 1.27x); FineGrasp: worse (objectness 1.64x); HGGD: worse (angle 2.11x)optimizer=ademamix: FineGrasp: worse (objectness 1.61x); HGGD: worse (angle 1.94x)optimizer=encoder-lr-0.1: GraspNet-baseline: worse (objectness 1.33x); GSNet: worse (objectness 2.24x); EconomicGrasp: worse (objectness 3.62x); FineGrasp: diverging (angle rises +29 % over the last half)optimizer=layer-decay: GSNet: worse (graspness 1.32x); EconomicGrasp: worse (objectness 1.38x); FineGrasp: diverging (angle rises +23 % over the last half)optimizer=prodigy: Scale-Balanced-Grasp: worse (objectness 3.90x); FineGrasp: worse (objectness 1.42x)optimizer=soap: FineGrasp: worse (objectness 5.93x)optimizer=stableadamw: GraspNet-baseline: worse (width 1.44x); FineGrasp: worse (objectness 1.62x); HGGD: worse (angle 2.05x)optimizer=timm-adan: GSNet: worse (objectness 1.27x); EconomicGrasp: worse (objectness 1.32x); FineGrasp: diverging (view rises +11 % over the last half); HGGD: worse (angle 2.21x)optimizer=timm-adopt: EconomicGrasp: worse (objectness 1.60x); FineGrasp: worse (objectness 1.49x); HGGD: worse (angle 2.91x)optimizer=timm-lamb: FineGrasp: diverging (angle rises +16 % over the last half)optimizer=timm-lion: GraspNet-baseline: worse (tolerance 1.43x); GSNet: worse (objectness 3.99x); Scale-Balanced-Grasp: diverging (tolerance rises +27 % over the last half); EconomicGrasp: worse (objectness 3.75x); FineGrasp: worse (objectness 7.16x); HGGD: worse (heatmap 1.81x)optimizer=timm-madgrad: GraspNet-baseline: diverging (width rises +18 % over the last half); Scale-Balanced-Grasp: worse (width 1.28x); FineGrasp: worse (graspness 1.70x)optimizer=timm-mars: FineGrasp: worse (objectness 1.45x); HGGD: worse (angle 4.22x)hooks=accumulate-2: GraspNet-baseline: worse (tolerance 1.59x); GSNet: worse (objectness 1.29x); EconomicGrasp: worse (objectness 1.32x)hooks=grad-clip: FineGrasp: worse (graspness 1.44x)augment=shift: EconomicGrasp: worse (objectness 1.31x); FineGrasp: worse (objectness 1.41x)
The noise floor: the references twice¶
Each reference trained under seed 0 and again under seed 1, nothing else changed: the spread two runs of one config show, which sets each method's tolerance. Over 36 terms (the total left out), the largest ratio away from 1 is 1.55x and the root mean square 1.114x.
The references, and every term under both seeds
| Reference | Loss terms (mean of the last window) | Grasps | Median score | Run |
|---|---|---|---|---|
| GraspNet-baseline | objectness 0.374, view 0.0298, score 0.0225, angle 0.334, width 0.0129, tolerance 0.0201 | 413, 249, 490, 498 | 0.205, 0.125, 0.232, 0.202 | 98 s |
| GSNet | objectness 0.156, graspness 0.076, view 0.848, score 0.0313, width 0.101 | 1024, 1024, 1024, 1024 | 0.0722, 0.0725, 0.0748, 0.0754 | 164 s |
| Scale-Balanced-Grasp | objectness 0.198, view 0.0824, score 0.0142, angle 0.313, width 0.0115, tolerance 0.00707 | 8, 16, 3, 5 | 0.0499, 0.0513, 0.0167, 0.0104 | 142 s |
| EconomicGrasp | objectness 0.126, graspness 0.0634, view 0.83, angle 1.54, depth 0.778, score 1.31, width 0.574 | 1024, 1024, 1024, 1024 | -2.02, -1.83, -1.79, -0.849 | 160 s |
| FineGrasp | objectness 0.0899, graspness 0.0472, view 0.733, angle 0.937, depth 0.436, score 0.812, width 0.315 | 1024, 1024, 1024, 1024 | -3.37, -2.23, -3.24, -3.21 | 669 s |
| HGGD | heatmap 1.41, angle 0.658, offsets 0.216, anchor 0.768, anchor_offset 0.194 | 477, 474, 475, 428 | 0.206, 0.207, 0.206, 0.201 | 204 s |
Run is the wall time of the probe on a shared machine: an order of magnitude.
ratio is seed 1 over seed 0 (mean of the last window); rise is each run's rise over the last half.
| Method | Again with | Term | Seed 0 | Other seed | Ratio | Rise (seed 0 / other) |
|---|---|---|---|---|---|---|
| GraspNet-baseline | seed 1 | loss/objectness |
0.3739 | 0.3587 | 0.96 | -0.066 / -0.09 |
| GraspNet-baseline | seed 1 | loss/view |
0.02976 | 0.02988 | 1 | -0.036 / -0.051 |
| GraspNet-baseline | seed 1 | loss/score |
0.02246 | 0.02063 | 0.918 | -0.05 / -0.061 |
| GraspNet-baseline | seed 1 | loss/angle |
0.3337 | 0.3301 | 0.989 | -0.049 / -0.063 |
| GraspNet-baseline | seed 1 | loss/width |
0.01289 | 0.01439 | 1.12 | -0.18 / -0.23 |
| GraspNet-baseline | seed 1 | loss/tolerance |
0.02008 | 0.02065 | 1.03 | -0.36 / -0.33 |
| GraspNet-baseline | seed 1 | loss |
0.7927 | 0.7744 | 0.977 | -0.072 / -0.089 |
| GSNet | seed 1 | loss/objectness |
0.1562 | 0.149 | 0.954 | -0.22 / -0.41 |
| GSNet | seed 1 | loss/graspness |
0.07598 | 0.08055 | 1.06 | -0.37 / -0.34 |
| GSNet | seed 1 | loss/view |
0.8476 | 0.7787 | 0.919 | 0.21 / 0.14 |
| GSNet | seed 1 | loss/score |
0.0313 | 0.03024 | 0.966 | 0.2 / 0.24 |
| GSNet | seed 1 | loss/width |
0.101 | 0.1005 | 0.995 | -0.058 / -0.093 |
| GSNet | seed 1 | loss |
1.212 | 1.139 | 0.94 | 0.039 / -0.065 |
| Scale-Balanced-Grasp | seed 1 | loss/objectness |
0.1975 | 0.1846 | 0.935 | -0.092 / -0.13 |
| Scale-Balanced-Grasp | seed 1 | loss/view |
0.08244 | 0.07993 | 0.97 | -0.016 / -0.037 |
| Scale-Balanced-Grasp | seed 1 | loss/score |
0.01418 | 0.01409 | 0.994 | 0.004 / -0.024 |
| Scale-Balanced-Grasp | seed 1 | loss/angle |
0.3132 | 0.3111 | 0.993 | -0.062 / -0.078 |
| Scale-Balanced-Grasp | seed 1 | loss/width |
0.01146 | 0.009845 | 0.859 | 0.14 / 0.19 |
| Scale-Balanced-Grasp | seed 1 | loss/tolerance |
0.007067 | 0.008327 | 1.18 | -0.23 / -0.25 |
| Scale-Balanced-Grasp | seed 1 | loss |
0.6258 | 0.6078 | 0.971 | -0.064 / -0.087 |
| EconomicGrasp | seed 1 | loss/objectness |
0.1259 | 0.1147 | 0.911 | -0.14 / -0.24 |
| EconomicGrasp | seed 1 | loss/graspness |
0.06338 | 0.05403 | 0.852 | -0.43 / -0.44 |
| EconomicGrasp | seed 1 | loss/view |
0.8303 | 0.8172 | 0.984 | 0.035 / 0.018 |
| EconomicGrasp | seed 1 | loss/angle |
1.543 | 1.599 | 1.04 | 0.032 / 0.061 |
| EconomicGrasp | seed 1 | loss/depth |
0.7779 | 0.8287 | 1.06 | 0.026 / 0.061 |
| EconomicGrasp | seed 1 | loss/score |
1.31 | 1.373 | 1.05 | 0.036 / 0.075 |
| EconomicGrasp | seed 1 | loss/width |
0.5737 | 0.5985 | 1.04 | -0.12 / -0.11 |
| EconomicGrasp | seed 1 | loss |
5.224 | 5.386 | 1.03 | -0.003 / 0.016 |
| FineGrasp | seed 1 | loss/objectness |
0.08986 | 0.09015 | 1 | -0.17 / -0.2 |
| FineGrasp | seed 1 | loss/graspness |
0.04722 | 0.04895 | 1.04 | -0.28 / -0.25 |
| FineGrasp | seed 1 | loss/view |
0.7334 | 0.7759 | 1.06 | -0.025 / -0.003 |
| FineGrasp | seed 1 | loss/angle |
0.9369 | 0.9471 | 1.01 | -0.16 / -0.15 |
| FineGrasp | seed 1 | loss/depth |
0.4363 | 0.4352 | 0.997 | -0.14 / -0.13 |
| FineGrasp | seed 1 | loss/score |
0.8125 | 0.8199 | 1.01 | -0.13 / -0.12 |
| FineGrasp | seed 1 | loss/width |
0.3151 | 0.3147 | 0.999 | -0.22 / -0.23 |
| FineGrasp | seed 1 | loss |
3.371 | 3.432 | 1.02 | -0.13 / -0.13 |
| HGGD | seed 1 | loss/heatmap |
1.405 | 1.771 | 1.26 | -0.027 / -0.095 |
| HGGD | seed 1 | loss/angle |
0.6582 | 1.019 | 1.55 | -0.16 / -0.14 |
| HGGD | seed 1 | loss/offsets |
0.2157 | 0.2554 | 1.18 | -0.16 / -0.031 |
| HGGD | seed 1 | loss/anchor |
0.7677 | 0.7178 | 0.935 | -0.06 / -0.22 |
| HGGD | seed 1 | loss/anchor_offset |
0.1935 | 0.1942 | 1 | -0.085 / -0.24 |
| HGGD | seed 1 | loss |
3.24 | 3.957 | 1.22 | -0.078 / -0.14 |
Not on this page¶
- Post-processing: post-processing trains nothing, so a training run says nothing about it; its check is to compare the grasps in and out of the chain (and the evaluator) on 20 frames of a trained method, which is not part of this page yet.
- Sets of stages, and requests that replace several slots: several slots replaced at once: no single stage of the reference to compare with.
- Schedules (42 cells): level 2 holds the schedule of the method as shipped fixed (a constant rate; FineGrasp: its own warm-up), so a schedule swap is not compared.
- On a schedule of its own (30 cells): it trained on a schedule of its own, not on that of the method as shipped (a schedule-free optimizer, a recipe preset with a scheduler).
- Did not run at level 1 (23 cells): the swap did not run at level 1; it stays named on every page.
- Several loss terms at once (11 cells): a loss preset is read from its cells of one term.
- The method's own component (1 cell): it is the method as shipped, not a swap.
The cells not counted, option by option
| Option | Under | Why |
|---|---|---|
schedule=constant |
GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp, HGGD | Schedules |
schedule=cosine-restarts |
GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp, HGGD | Schedules |
schedule=cosine |
GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp, HGGD | Schedules |
schedule=multistep-warmup |
GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp, HGGD | Schedules |
schedule=onecycle |
GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp, HGGD | Schedules |
schedule=poly |
GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp, HGGD | Schedules |
schedule=warmup-cosine |
GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp, HGGD | Schedules |
optimizer=schedulefree-adamw |
GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp, HGGD | On a schedule of its own |
optimizer=schedulefree-radam |
GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp, HGGD | On a schedule of its own |
optimizer=schedulefree-sgd |
GraspNet-baseline, Scale-Balanced-Grasp, HGGD | On a schedule of its own |
recipe=adamw-onecycle |
GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp, HGGD | On a schedule of its own |
recipe=finetune-encoder |
GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp | On a schedule of its own |
backbone=ptv3, recipe=ptv3-adamw-onecycle |
GraspNet-baseline, GSNet, Scale-Balanced-Grasp, EconomicGrasp, FineGrasp | On a schedule of its own |
oacnns |
GSNet, EconomicGrasp, FineGrasp | Did not run at level 1: unstable twice in the short probe |
ptv3-encoder |
GSNet, EconomicGrasp, FineGrasp | Did not run at level 1: unstable twice in the short probe |
ptv3-scannet200 |
FineGrasp | Did not run at level 1: unstable twice in the short probe |
ptv3-sonata-small |
GSNet, EconomicGrasp, FineGrasp | Did not run at level 1: unstable twice in the short probe |
ptv3-sonata |
GSNet, EconomicGrasp | Did not run at level 1: unstable twice in the short probe |
ptv3-utonia |
GSNet, EconomicGrasp | Did not run at level 1: unstable twice in the short probe |
pyg-edgeconv |
FineGrasp | Did not run at level 1: unstable twice in the short probe |
pyg-point-transformer |
FineGrasp | Did not run at level 1: unstable once, not repeated |
pyg-pointgnn |
FineGrasp | Did not run at level 1: unstable once, not repeated |
spunet |
FineGrasp | Did not run at level 1: unstable twice in the short probe |
optimizer=schedulefree-sgd |
GSNet, EconomicGrasp, FineGrasp | Did not run at level 1: unstable twice in the short probe |
hooks=freeze-encoder |
FineGrasp | Did not run at level 1: unstable twice in the short probe |
recipe=finetune-encoder |
FineGrasp | Did not run at level 1: unstable twice in the short probe |
proposal.objectness_loss=focal, head.score_loss=quality-focal, head.width_loss=balanced-l1 |
GSNet | Several loss terms at once |
proposal.objectness_loss=softmax-focal, head.score_loss=varifocal, head.width_loss=huber |
GSNet | Several loss terms at once |
proposal.objectness_loss=label-smoothing, proposal.graspness_loss=bce, head.width_loss=ghm-r |
GSNet | Several loss terms at once |
proposal.objectness_loss=poly1, head.score_loss=ghm-c, head.width_loss=charbonnier |
GSNet | Several loss terms at once |
proposal.objectness_loss=ohem, head.score_loss=ohem-smooth-l1, head.width_loss=cauchy |
GSNet | Several loss terms at once |
proposal.objectness_loss=dice, proposal.graspness_loss=quality-focal, head.angle_loss=softmax-focal, head.depth_loss=label-smoothing, head.score_loss=class-balanced, head.width_loss=cauchy |
EconomicGrasp | Several loss terms at once |
proposal.objectness_loss=lovasz-softmax, proposal.graspness_loss=bce, view.loss=varifocal, head.angle_loss=poly1, head.depth_loss=ohem, head.score_loss=focal |
FineGrasp | Several loss terms at once |
view.view_loss=ohem-smooth-l1, head.score_loss=ghm-r, head.angle_loss=ghm-c, head.width_loss=balanced-l1, head.tolerance_loss=charbonnier |
GraspNet-baseline | Several loss terms at once |
view.view_loss=huber, head.angle_loss=class-balanced, head.width_loss=huber |
Scale-Balanced-Grasp | Several loss terms at once |
proposal.objectness_loss=uncertainty-weighting, proposal.graspness_loss=uncertainty-weighting, view.loss=uncertainty-weighting, head.score_loss=uncertainty-weighting, head.width_loss=uncertainty-weighting |
GSNet | Several loss terms at once |
view.objectness_loss=uncertainty-weighting, view.view_loss=uncertainty-weighting, head.score_loss=uncertainty-weighting, head.angle_loss=uncertainty-weighting, head.width_loss=uncertainty-weighting, head.tolerance_loss=uncertainty-weighting |
GraspNet-baseline | Several loss terms at once |
hggd-residual |
HGGD | The method's own component: the preset writes out HGGD's own image encoder, parameter for parameter |
Accuracy of any kind is on the model zoo. The noise floor above is the only repeat: every cell is one run.
The records¶
- learns_like_reference.json: every cell with its outcome, its reasons, the ratio of every term and its two commands.
- learns_like_reference/losses_per_iteration.json: the losses of every run, the mean of every 5 iterations.
Run it again
python -m tools.learn_check references --seeds 0 1 --scale-statistic-sha <SHA-256>
python -m tools.learn_check run --kind encoder --scale-statistic-sha <SHA-256>
python -m tools.learn_check status # selected, run and pending cells; time left
python -m tools.learn_check export # compare -> learns_like_reference.json
python -m tools.learn_check page # this page
The runs are those of tools/probe_grid.py (--iters 250 --predict-frames 4), one run directory per kind, so an interrupted run continues where it stopped. The SHA-256 is that of the imported width histogram of Scale-Balanced-Grasp.