Learning-based approaches to understanding and manipulating 3D objects can handle a wide variety of objects, but may be unable to generalize all sets of objects. We will study the generalization of deep learning models in two subdomains of 3D objects: grasping and object descriptors.

In the first part of the thesis, we show how objects can be designed to be adversarial to Dex-Net, a neural network based grasping policy. We define an "adversarial grasp object" as an object that is visually similar to an original object but decreases the predicted graspability resulting from a robot grasping policy. We propose a method for synthesizing adversarial grasp objects by removing antipodal face pairs via systematic face rotations. We also explore a method that maintains local convexity and a deep-learning based method. Experiments suggest that all three algorithms consistently reduce graspability. Physical experiments demonstrate that the adversarial objects generated by the convexity preserving analytic algorithm decrease the grasp success rate by at least 87%. In simulation, the analytic rotation algorithm is able to reduce the graspability metric by 66%, 57%, and 63% on intersected cylinders, intersected prisms, and ShapeNet bottles.

In the second part of the thesis, we explore the potential application of dense object descriptors to mechanical search, where we attempt to find a target object in clutter. We find that dense object descriptors can generalize reasonably well and learn a consistent representation for unseen classes of objects. Feature matching methods can be combined with the descriptors to search for target objects in a heap.