Self-supervised Transparent Liquid Segmentation for Robotic Pouring

Robots able to pour liquids would assistance in tasks like cooking or watering our crops. Even so, transparent liquids are difficult to perceive in photographs. A recent paper printed on proposes a approach for perceiving transparent liquid inside clear containers.

Transparent liquid in a transparent container.

Clear liquid in a clear container. Graphic credit rating: Piqsels, CC0 General public Domain

The approach employs a generative product that learns to translate pictures of colored liquid into artificial photos of a clear liquid, which can be used to coach a clear liquid segmentation model. It does not require labeled correspondence amongst coloured images and transparent photographs and therefore permits automatic and highly efficient dataset assortment.

Scientists construct a robotic pouring process to reveal the utility of the transparent liquid segmentation product. In addition, quite a few dataset augmentation experiments are carried out to exhibit the probable of the proposed strategy to generalize to numerous scenes.

Liquid condition estimation is significant for robotics jobs these kinds of as pouring however, estimating the state of transparent liquids is a demanding challenge. We suggest a novel segmentation pipeline that can segment clear liquids this sort of as water from a static, RGB picture without the need of requiring any manual annotations or heating of the liquid for instruction. As an alternative, we use a generative model that is capable of translating photos of colored liquids into synthetically generated clear liquid pictures, properly trained only on an unpaired dataset of coloured and transparent liquid photos. Segmentation labels of colored liquids are acquired quickly employing background subtraction. Our experiments display that we are equipped to precisely predict a segmentation mask for clear liquids with no demanding any manual annotations. We reveal the utility of clear liquid segmentation in a robotic pouring job that controls pouring by perceiving the liquid peak in a clear cup.

Study paper: Narayan Narasimhan, G., Zhang, K., Eisner, B., Lin, X., and Held, D., “Self-supervised Transparent Liquid Segmentation for Robotic Pouring”, 2022. Link: muscles/2203.01538