Unstable bilinear depth detection

I'm Stuck
1
  • Issue category: Occlusion
  • Device type & OS version: iOS
  • Issue Environment : On Device
  • ARDK version: 2.3.0
  • Unity version: 2021.3.11f

Hey all!
Got a question regarding the prefer smooth edges toggle in the ar depth manager. As soon as i enable it, the occlusion is acting very unstable on device. Setting the interpolation mode (smooth/balanced/none) or adding an AR depth interpolation adapter doesn’t make a difference. I am not using the occlusion stabilisation yet though i could give that a shot.
Is this a common occurrence to see, has anyone else seen/reported this before?


In this footage you can see me switching between point and bilinear sampling, which is what the ar depth manager does when toggling.

Thanks in advance!

2

Hi Marijn,

May I also know the version of iOS and the device you’re building to? Has it been working but suddenly started acting unstable?

3

I’m running this on an iPad pro (2021) with iOS 16. There were some build issues when updating to iOS 16 but i’ve got it building now.

When looking at it some more i can identify 2 issues with the depth map, for me these have always been there in ARDK 2.3:

  1. the depth changes based on the angle i’m looking at
  2. the depth manager looks like it’s estimating to much depth for occlusion, meanwhile hit-testing still seems to give the same results. This also occurs in editor.

I will test some more possible solutions today, if i find something i’ll let you know.

4

I will get back to you as soon as I can. We don’t officially support iPad so that might be a source of error but you’re also saying that it happens on Editor so I will have to see what I find there.

Since it’s also happening on Editor, can you also provide me the computer and OS you’re using for your development?

5

Hey Jesus,

Apologies for the delay, i have been running some tests with multiple devices and had the same results, this includes a Oneplus phone and an iPhone.

I’m building on a Macbook (M1 chip) and using URP as render-pipeline.
Hope you can help me out somehow, if you need some more footage im glad to provide it, in the meantime i’ll try out a new project without URP.

6

Hi Merijn,

We are still looking into this but wanted to note that for Android, I was able to more or less reproduce what you have in your video but only when the camera feed has multiple objects at varying distances. When the view only had a few objects in it, things were pretty stable. Is that more or less how your results on Android stack up or was there instability all around?

I will get back to you with results for M1 Macs and iOS soon.

7

Hey i did some research and got it working properly.
Turns out this problem occurs when i set my build to a linear color space, switching to gamma fixes this. I don’t understand why this could be though, does the color space affect the camera feed interpretation maybe?

8

Hi Merijn,

I’m not aware of these color spaces causing any form of instability or performance issues but Linear color space did have an issue with showing a more washed out color as opposed to Gamma which shows more vibrant color. The issue may be manifesting itself differently when in this mode since it’s more of a grayscale.

Thank you for bringing this up, I will let our team know to see if the instability might just be due to how frames are rendered when on a Linear color space.

9

Hi Merijn,

I’ve confirmed that this isn’t an issue with performance or stability. The example does technically work but the reason you’re seeing the graphical issues you’ve noted is due to the color space. The workaround at the moment is to not use the smooth edges option. This should be addressed in the 2.5 release of ARDK but I do not have a timeline or concrete answer for when that will be released.