Structures

The typed structures stages exchange, from Observation to GraspGroup. Which detectors use which: Data contracts.

grasppanda.structures

Typed data contracts. Plain dataclasses holding tensors; no dependency on any model.

ContractError

Bases: ValueError

Data handed between stages violates a declared contract.

Gripper dataclass

Parallel-jaw gripper geometry in metres.

Defaults are the GraspNet-1Billion benchmark gripper, the same constants its evaluator uses for collision checking.

Mask dataclass

A boolean mask together with its provenance.

Provenance

Bases: str, Enum

Where a piece of inference-time input came from.

SampleKey dataclass

Identity of one evaluation unit.

CompactGraspLabels dataclass

The best grasp of every (annotated point, view) of one object instance in a frame.

The dense label set holds a grid of in-plane angles and depths per point and view (:class:~grasppanda.structures.ObjectGraspLabels). Here that grid is collapsed to its best cell, which leaves four small numbers per point and view; this is the "economic" supervision of EconomicGrasp. Points without any good grasp are absent.

Values are stored the way the label asset stores them, as small integers, and are turned into metric quantities by :meth:quality and :meth:metres. The integers travel through the data loader; only the rows a batch matches are ever converted.

Attributes:

Name	Type	Description
object_id	int	Dataset id of the object model.
pose	Tensor	(4, 4) object-to-camera transform for this frame.
points	Tensor	(P, 3) annotated grasp points in the object frame.
view_graspness	Tensor	(P, V) target view scores of every point, already normalised over its views; the views are in the order of the label lattice.
angle	Tensor	(P, V) in-plane angle bin of the kept grasp.
depth	Tensor	(P, V) depth bin of the kept grasp.
score	Tensor	(P, V) quality level of the kept grasp in 0 … SCORE_LEVELS. Zero means the view has no usable grasp at this point; otherwise higher is better (the label asset derives it from the friction coefficient).
width	Tensor	(P, V) gripper width of the kept grasp in 1 / WIDTH_UNITS metres.
view_max_friction	float	Friction coefficient up to which a grasp counted towards view_graspness when the asset was generated.

quality(score) classmethod

Stored score levels → quality in [0, 1] (float32).

metres(width) classmethod

Stored widths → metres (float32).

GraspGroup dataclass

A set of 6-DoF parallel-jaw grasps for one sample.

Geometry follows the GraspNet-1Billion convention. All lengths are metres.

Attributes:

Name	Type	Description
translation	Tensor	(G, 3) grasp point in frame.
rotation	Tensor	(G, 3, 3) proper rotations. Column 0 is the approach axis, column 1 the closing axis, column 2 the finger-height axis.
width	Tensor	(G,) jaw opening along the closing axis.
depth	Tensor	(G,) how far the fingers reach past the grasp point along the approach axis. This is an insertion depth, not a camera depth.
score	Tensor	(G,) ranking score, higher is better. score_kind names what it means; it is not assumed to be a calibrated probability.
height	Tensor | None	(G,) finger thickness; defaults to the gripper's.
object_id	Tensor | None	(G,) instance association, -1 when unknown.
frame	str	Reference frame of translation and rotation.
gripper	Gripper	Gripper geometry the widths and depths refer to.
score_kind	str	Semantics of score, declared by the head bundle.
key	SampleKey | None	Which sample these grasps belong to.

empty(*, device='cpu', **meta) classmethod

A valid group with no grasps. An empty prediction is a result, not an error.

cat(groups) classmethod

Concatenate groups that share frame, gripper and score semantics.

sort_by_score(*, descending=True)

Stable sort, so equal scores keep their current order on every platform.

validate(*, atol=0.0001)

Raise :class:ContractError unless every grasp is physically meaningful.

transform(matrix, *, frame)

Apply a rigid (4, 4) transform and relabel the reference frame.

to_graspnet_array(dtype=np.float32)

Export to the GraspNet (G, 17) layout. Use only at the I/O boundary.

from_graspnet_array(array, **meta) classmethod

Import from the GraspNet (G, 17) layout. Use only at the I/O boundary.

Batch dataclass

An observation and, during training, its supervision.

A dataset returns a batch of one frame; :func:collate merges them.

ObjectGraspLabels dataclass

Dense grasp annotations of one object instance in a frame.

The grids are stored in one of two ways. Dense: scores and widths (and, when present, collision and tolerance) are tensors. Deferred: deferred reads rows from the label store when they are asked for, and the four tensors are None. Code that consumes labels reads them through :func:gather_grids (many objects in one pass) or the gather_* methods (one object), which return the same values for both storages; :meth:dense turns deferred labels into tensors.

Attributes:

Name	Type	Description
object_id	int	Dataset id of the object model.
pose	Tensor	(4, 4) object-to-camera transform for this frame.
points	Tensor	(P, 3) annotated grasp points in the object frame.
scores	Tensor | None	(P, V, A, D) minimum friction coefficient for force closure. Lower is better and values ≤ 0 are invalid. These are labels, not confidences.
widths	Tensor | None	(P, V, A, D) gripper widths in metres.
collision	Tensor | None	(P, V, A, D) true where the grasp collides in this scene.
tolerance	Tensor | None	(P, V, A, D) grasp tolerance in metres: how far the grasp point may move over the object surface while the same grasp stays good. A derived label; None unless the dataset is configured to serve it.
deferred	DeferredGrids | None	Reader of the grids of the P label points, in place of the four tensors above.

shapes()

Shape of every grid the labels hold, by name, whichever way it is stored.

gather_scores(rows)

(K, V, A, D) friction scores of the label points rows, on their device.

gather_widths(rows, views=None)

Gripper widths of the label points rows, on their device.

(K, V, A, D) without views; with views the (K, A, D) grid of label point rows[k] at view views[k].

gather_collision(rows)

(K, V, A, D) collision mask of rows; None when the labels carry none (because collisions were ignored, or already written into the scores).

gather_tolerance(rows, views=None)

Grasp tolerance of the label points rows, shaped as :meth:gather_widths.

dense()

These labels with every grid as a tensor, on the device of points.

Reads all rows of deferred labels, which is what deferring them avoids; meant for inspection and for code written against the tensors.

to(device)

Move the tensors; a deferred reader stays where its store is.

Observation dataclass

Everything available at inference time for a batch of frames.

predict(observation) gets this object and nothing else. Any input derived from ground truth must say so, either as a :class:Mask with that provenance or as an entry of provenance; it is then reported by :meth:gt_assisted and stamped on the results of the run.

Attributes:

Name	Type	Description
keys	list[SampleKey]	One :class:SampleKey per frame of the batch.
intrinsics	Tensor | None	(B, 3, 3) camera matrices.
depth	Tensor | None	(B, H, W) metres; zero where invalid.
rgb	Tensor | None	(B, 3, H, W) in [0, 1].
points	PointSet | None	Scene points the model consumes, camera frame, metres.
point_feats	Tensor | None	(N, C) input features aligned with points.
full_points	PointSet | None	The cloud before sub-sampling, for post-processors such as a collision filter. Never an input of the network.
masks	dict[str, Mask]	Named per-point masks aligned with points (e.g. instance).
provenance	dict[str, Provenance]	Where inputs that shaped this observation came from, by name. A workspace crop computed from ground-truth segmentation is recorded here as {"workspace": Provenance.GROUND_TRUTH}.

gt_assisted()

Names of inputs that were derived from ground truth.

provides()

Names of the inputs that are present.

Supervision dataclass

Training-only information. Never passed to predict.

Attributes:

Name	Type	Description
objects	list[list[ObjectGraspLabels]]	Per frame, the grasp labels of every object instance in it.
point_labels	dict[str, Tensor]	Named (N, ...) labels aligned with observation.points.
extras	dict[str, list[Any]]	Method-specific targets, one list entry per frame. Compact grasp labels travel here under COMPACT_LABELS.

PointFeatures dataclass

Features that cannot exist without the point set they describe.

expect(points, *, user)

Assert these features belong to points; user names who is asking.

dense()

Return (B, N, C); raises if samples differ in size.

PointSet dataclass

Points of a batch, concatenated, with an identity.

Attributes:

Name	Type	Description
coords	Tensor	(N, 3) coordinates in metres.
batch_offsets	Tensor | None	(B + 1,) boundaries; sample b owns coords[o[b]:o[b+1]]. None means a single sample.
frame	str	Reference frame of the coordinates.
parent_id	int | None	id of the set this one was sampled from, if any.
parent_index	Tensor | None	(N,) index into the parent when this set is a true subset.
id	int	Unique identity. Features are bound to it, so a feature tensor can never be silently reused with a different set.

sizes property

(B,) number of points of each sample.

batch_index property

(N,) sample index of every point.

from_dense(coords, *, frame='camera') classmethod

Build from (B, N, 3) clouds of equal size.

cat(sets) classmethod

Concatenate point sets of the same frame into one batch with a new identity.

dense()

Return (B, N, 3); raises if samples differ in size.

subset(index, batch_offsets=None)

A child set made of coords[index], remembering where it came from.

to(device)

Move to a device. Identity is preserved: it is still the same set.

ApproachSet dataclass

One approach direction per seed.

Attributes:

Name	Type	Description
direction	Tensor	(M, 3) unit vectors pointing from the gripper towards the object.
rotation	Tensor | None	(M, 3, 3) crop frame whose first column is the approach axis.
view_index	Tensor | None	(M,) index into a view template, when directions are discrete.
logits	Tensor | None	Raw view scores, kept for the loss.

GroupFeatures dataclass

Local features per seed, with named axes.

axes names every dimension of feats, e.g. ("seed", "channel") or ("seed", "depth", "channel"). A head states the axes it expects, so a depth axis is never squeezed or invented implicitly.

expect(axes, *, user)

Assert the axis layout a consumer was written for.

SeedSet dataclass

Candidate grasp locations.

Attributes:

Name	Type	Description
points	PointSet	The seeds, usually a child of the scene point set.
feats	Tensor | None	(M, C) features, if the stage produced any.
scores	dict[str, Tensor]	Named per-seed scores with declared meaning, e.g. graspness or objectness. No score is mandatory: a method without graspness leaves it out.
valid	Tensor | None	(M,) mask. Padding added to reach a fixed seed count is marked invalid rather than passed on as a real candidate.
source	str	How the seeds were chosen, for records and debugging.

collate(items)

Merge batches into one. Point clouds may differ in size; images may not.