Skip to content

Components

GraspPanda builds a grasp detector from stages, and every stage is a slot: a place in the config where one component can be swapped for another with one line, --set <slot>=<option>. Five methods share the seed-based skeleton and its slots; HGGD reads images and has one slot of its own, the image encoder. Post-processing and the training recipe apply to all six methods. In all, 244 options you can --set.

Two detector families fed by GraspNet-1Billion and GraspClutter6D: the slots of the seed-based skeleton, HGGD's image encoder and fixed stages, and the post-processing and training recipe all methods share Two detector families fed by GraspNet-1Billion and GraspClutter6D: the slots of the seed-based skeleton, HGGD's image encoder and fixed stages, and the post-processing and training recipe all methods share

Open the figure at full size: light · dark.

Which method has which slot

Slot GraspNet-baseline GSNet Scale-Balanced-Grasp EconomicGrasp FineGrasp HGGD
Image encoder — — — — — ✓
Scene encoder ✓ ✓ ✓ ✓ ✓ —
Seed proposal ✓ ✓ ✓ ✓ ✓ —
View ✓ ✓ ✓ ✓ ✓ —
Assigner ✓ ✓ ✓ ✓ ✓ —
Grouper ✓ ✓ ✓ ✓ ✓ —
Head bundle ✓ ✓ ✓ ✓ ✓ —
Loss ✓ ✓ ✓ ✓ ✓ —
Post-processing ✓ ✓ ✓ ✓ ✓ ✓
Optimizer ✓ ✓ ✓ ✓ ✓ ✓
Schedule ✓ ✓ ✓ ✓ ✓ ✓
Hooks ✓ ✓ ✓ ✓ ✓ ✓
Augmentation ✓ ✓ ✓ ✓ ✓ ✓
Recipe ✓ ✓ ✓ ✓ ✓ ✓

The slots

  • Image encoder

    Turns the colour image into feature maps; HGGD predicts its grasp heat-map from them.

    HGGD only 38 options

    --set image_encoder=

  • Scene encoder

    One feature vector per point of the depth cloud: what a seed knows before any crop. By family: point-based · sparse-voxel · transformers.

    Seed-based · 5 methods 47 options

    --set backbone=

  • Seed proposal

    Picks the points of the scene at which grasps are predicted.

    Seed-based · 5 methods 24 options

    --set proposal=

  • View

    Chooses the approach direction of every seed.

    Seed-based · 5 methods 21 options

    --set view=

  • Grouper

    Gathers the points around a seed into the feature the head reads.

    Seed-based · 5 methods 28 options

    --set grouper=

  • Heads and assigners

    The head predicts the grasp at a seed; the assigner matches seeds with labels in training.

    Seed-based · 5 methods 10 head bundles · 4 assigners

    --set head= --set assigner=

  • Losses

    One term of the loss of a stage, replaced on its own.

    Seed-based · 5 methods 20 options

    --set <stage>.<term>_loss=

  • Post-processing

    Filters, selects and corrects predicted grasps before a robot or an evaluator.

    All 6 methods 15 options

    --set post=

  • Training recipe

    How weights are updated, how the rate moves, what runs around the loop, how a sample is perturbed.

    All 6 methods 17 optimizers · 7 schedules · 7 hooks · 3 augmentations · 3 whole recipes

    --set optimizer= --set schedule= …

Find your way

Words used on these pages

Seed, preset, contract, adapter, label set
  • Seed: a point of the scene at which a grasp is predicted; the seed proposal picks them (1024 per frame in each of the five seed-based methods).
  • Preset: a small YAML file with a component and settings for table-top depth clouds; its name is what follows --set <slot>=.
  • Contract: what a component declares about its input and output. grasppanda compose compares contracts before anything is built and refuses a mismatch with the fix.
  • Adapter: a wrapper compose puts around a scene encoder so that it fits the next stage.
  • Label set: the grasp labels a head is trained on, dense (every view, angle and depth of a labelled point) or compact (the best grasp per view). A head, its assigner and the dataset section use the same one.