**Curve plane detection and multi-image recognition for label scanning**

I am interested in using Niantic Lightship to create an AR experience that allows users to scan wine labels (a bit like 19crimes) and see the person on the label come to life and tell their story. I have a few questions about curve plane detection and multi-image recognition:

  • Is it possible to use Lightship to scan and recognize curved images, such as images on wine bottles?
  • Is it possible to use Lightship to recognize and track multiple images at the same time? For example, if I have two bottles next to each other with different faces on them, could Lightship recognize both faces and track them as the user moves the bottles around?

Would be great if someone can confirm if this is possible or not.


Hi Yvo,

In general, image detection is meant to detect flat images, not curved images but let me look into this a little further and see what I can dig up. Do you plan on using ARDK 2.5 or the ARDK 3 Beta?

For your second question, yes, Lightship can track multiple images at once even while the user is moving around.

We are not bound to a specific version of ARDK and will use what will best suits our needs.

Hi Yvo,

So with ARDK, the image does have to be a flat 2D image. If there is a curve, the image detection may fail. You can possibly overcome this limitation by using a smaller portion of the bottle for image detection to avoid such a pronounced curve.

If you need to use Unity, maybe the ARDK 3 beta would be your best bet. Since it integrates with AR Foundation, you might be able to find some resources online for ARFoundation that could potentially assist with detecting a curved image, though we wouldn’t be able to provide support if you encountered any issues since image detection only supports flat images.

Another option is using 8th Wall, but it’s not Unity-based; it’s WebAR, so it will run in the browser instead. 8th Wall lets you set a curvature for image detection to take into account, which makes it possible to detect an image that wraps around a bottle. Below are links to their website and their support page along with a resource on curved image targets. 8th Wall is owned by Niantic, but they still run their own support through Slack.