Include the following details (edit as applicable):
Issue category: Semantic Segmentation / ARDK Documentation / Unity Example Package / ARDK Virtual Studio Tools / Developer Tools
Device type & OS version: Android
Host machine & OS version: Windows
ARDK version: 2.3.0
Unity version: 2021.3
Description of the issue:
I want to access live camera feed from ARDK ARSceneCamera and pass it to my native Android plugin. Can anyone help me access the live camera feed from ARDK?
You can use either IARSession.CurrentFrame or the IARSession.FrameUpdated event in IARSession to access the camera feed. Note that for Android, the camera feed is one ARGB32 texture while for iOS it’s two textures that form a YCbCr image.
You can reference our ARKitFrameRenderer.cs script and ARCoreFrameRendere.cs script (found in /Assets/ARDK/Rendering/FrameRenderer/) for iOS and Android respectively to see how these textures are used.
A few things to note are that there can be added latency when using these textures through a pipeline with additional plugins, and ARDK can write over the previous data, since this information can be updated every frame, so make sure to save a copy of it if you plan on working on the camera feed data.
Lastly, we do not provide support for mixing ARDK with other plugins so it may be possible that you will have to find a way to let this 3rd party plugin interpret ARDK camera feed data in the event that it doesn’t accept it as it is.