Help with getting the Background Camera Feed

I'm Stuck
1

Include the following details (edit as applicable):

  • Issue category: ARDK Documentation
  • Device type & OS version: Galaxy S20FE Android 12
  • Host machine & OS version: Windows 10
  • Issue Environment : On Device
  • ARDK version: 2.3.0
  • Unity version: 2020.3.25f1

Description of the issue:

I’m trying to migrate my application, that uses AR Foundation with Meshing, to the ARDK however our stack uses ML Object Detection and we need the “Clean” image, also referred to as Background Image, or Camera feed.

In the AR Foundation this problem was solved using two cameras at the same time, one for displaying to the user with all the AR Props, and another one with the culling mask turn to Nothing rendering to a RendererTexture, but on the Lightship this same solution does not work, atleast what I tried, so I’m asking for help with that.

2

Hey,
Can you give me more details on this? The ARDK approach might be completely different, so I need to know what the end result you are expecting is.
Also, When you say that it doesn’t work, what do you mean? What do you see and what were you expecting? Does it break and throws errors or does it build and just doesn’t work as you expected?

3

Hi there, so the end result that I want is the Camera Feed only so that can be a Input on the ML Inference, the applying a RendererTexture on a second camera option didnt worked for me because the texture itself was blank and not useful, but it did compile and didnt showed any errors or crashed.

The ways I tried so far to get the camera:

  • Tried the two cameras option that I explained earlier;
  • Using the GPUTexture and CPUTexture from the ARRenderingManager, it didn`t worked, both was showing a all black texture/image;
  • Using the ARFrame properties from CurrentFrame CapturedImageTextures and trying to transform to Texture2D using LoadRawTextureData, this throws a error " LoadRawTextureData: not enough data provided" the size of the buffer was ScreenWidth * ScreenHeight * 4 and the type was BGRA
  • Using the CapturedImageBuffer and trying to transform to Texture2D using LoadRawTextureData,
    this crashed the Application

And thats what I done so far, sorry if my first post seemed rushed, so if you need any more explanation just ask, and thank you for your time