Semantic Segmentation only on specific Objects

I'm Stuck
1

Include the following details (edit as applicable):

  • Issue category: Semantic Segmentation / Multiplayer / Real-time Mapping-Depth / ARDK Documentation / Unity Example Package / Sample App aka ‘AR Voyage’ / ARDK Virtual Studio Tools / Developer Tools / Networking / VPS
  • Device type & OS version: Android / iOS / Other Ex. iPhone 8+ on iOS 13
  • Host machine & OS version: Mac / Windows / Linux / Other Ex. Mac on Big Sur x.x
  • Issue Environment : Unity Remote / Unity Mock / On Device / Dev Portal
  • Xcode version:
  • ARDK version: 2.1 and 2.4
  • Unity version:

Hey there,

I use the semantic segmentation feature to display far-away objects in the sky layer. This works fine, but I want to turn off the segmentation for specific game objects.

Has anyone implemented such a feature, and knows if it is possible/other solutions exist?

Bests,
Samuel

2

I think this could be achieved by doing the following:

  • have a base camera with depth occlusion for objects close by or on the ground
  • have an overlay camera that renders objects on the horizon or in the sky, add segmentation manager to this one.
  • finally add the camera as an overlay to the base camera or do the reverse based on what you want to render on top

In this way you can use depth for objects in the vicinity of the user and segmentation for large scale objects. The far away objects will still be occluded by objects close by, just in a lower quality (segmented instead of depth tested). Haven’t tested this myself yet but i think it can be done.

3

Hello Samuel,

I would recommend looking into the solution Merijn provided and let us know how it goes. If it doesn’t work for you we can look into your question further.

Thanks for helping the community Merijn!

4

The problem I am facing is slightly different, as I need to turn the segmentation off, for certain gameobjects.

Another approach might be to inverse the segmentation mask that gets returned: eg all pixels where sky is not present. Does anybody have experience with this problem?

Bests,
Samuel