Contextual Awareness Features not working on a Android 12 device that meets requirements

  • Issue category: Real-time Mapping-Depth
  • Device type & OS version: Android 12
  • Host machine & OS version: Windows
  • Issue Environment : On Device

Hi.

First of all, I would like to say that I think ARDK technology is amazing. Thank you very much for providing this kind of tool to the community. I believe and hope that great things are going to be done with it.

About my issue: I’ve been doing some LightShip testing with an Android 12 device that, according to the ARDK System Requirements, supports Contextual Awareness features ( Development and Runtime Requirements of ARDK — Niantic Lightship Augmented Reality Developer Kit release-2.1 documentation). It is a OnePlus Nord2 equipped with a Mali-G77 GPU, released one year ago approx (OnePlus Nord 2: Price, specs and best deals)

Despite meeting ARDK requirements, the phone failed the following device support checks in all ARDK versions I’ve worked with:

  • ARWorldTrackingConfigurationFactory.CheckDepthEstimationSupport().
  • ARWorldTrackingConfigurationFactory.CheckMeshingSupport().

However, _ARConfigurationValidator.cs script states that “ARDK’s device support checks serve as recommendations, not a hard block. Devices are still able to try to run unsupported features.”. Therefore, I have tested the meshing functionality with several ARDK versions. These are the results:

  • ARDK 1.1.0: Impossible to use. A few mesh chunks are rendered on the screen, but they aren’t accurate at all, and drift continuously. (Sorry, but I don’t have a video for this version. I can make one if it helps).

  • ARDK 1.3.0 and ARDK 2.0.0: meshing is not very accurate, and sometimes it drifts a little. For example, an Android 11 device with an Adreno 650 GPU (it has a similar computational power) offers much more accurate and stable results. Still, the mesh is good enough for my use case, and the multiplayer experience is OK as well. I could work with that. You can see a scan with 2.0.0 in the following video:

  • ARDK 2.1.0: from my point of view, there has been a regression. Accuracy is not as bad as with ARDK 1.1.0, but definitely worse than 2.0.0. The mesh also drifts noticeably. Impossible to use. Please, check two scan attempts in the following video:

The main issues I see here:

  1. A relatively recent device, with a well-known GPU that meets the minimum requirements, cannot run contextual awareness features consistently.
  2. The current minimum requirements stated in the documentation cannot be used as a guide as to whether or not a device can run ARDK’s contextual awareness functions. I think the only option to know for sure is for the user to install the app and wait for the device to fail at runtime.
  3. There may be regressions between ARDK versions.

Related questions:

  • I think the device checks in “_ARConfigurationValidator.cs” should always be used as a hard-block in case the app needs ARDK’s contextual awareness features, don’t you agree? On the one hand, there may be devices that fail the tests but can still run the features (like this OnePlus using versions 1.3.0 and 2.0.0). By using the device checks as a hard-block, we would be preventing those users from trying the app, which isn’t good. On the other hand, I think it is even worse to let them run the app and eventually have a bad experience (like the OnePlus in version 2.1.0).
  • Could you provide a reliable list of capable devices (or device features) that the users could check before installing the app, or that we could display in our apps as a recommendation in case a device fails the checks (or even if the device doesn’t even support ARCore)?
  • Are you working on trying to extend ARDK support to more GPUs? Do you know if these issues are occurring with other theoretically supported GPUs, or could this device just be an exception?
  • Regarding the regressions between versions, well, I am just concerned about them. Any thoughts on that would be appreciated.

Thank you very much in advance for your support!!

Hello Chankeiro,

I have some information for you regarding the issues you are experiencing:

  1. The video you posted is showing tracking failing, which is why you see the mesh drifting. This means that Google’s ARCore tracking on the device is not correctly providing poses to ARDK in order for it to align the mesh. This typically happens on lower spec’d devices that struggle to keep up with the processing load.
  2. ARDK 2.1.0 also brought with it the use of a newer version of ARCore as required by google so this may play into some of the performance differences you are seeing, but we cannot be sure at this time.
  3. Currently there is no easy fix for this, but we are looking at a number of android specific optimisations for future ARDK releases which may improve this in the future.

Please let us know if you have any further questions we can answer for you.

Hi @Jason_Rose1 ,

Thank you very much for the information. Some comments:

  • “This tends to happen on low-spec devices that have difficulty keeping up with the processing load”: I don’t think this is the case with the OnePlus Nord2, as it meets Lightship’s contextual awareness requirements by quite a margin. In fact, I have been able to run contextual awareness functions on devices with much worse specs than the OnePlus. For example, the Xiaomi in the screenshot below is an old device with much lower specs than the OnePlus and doesn’t even meet the requirements (it has an Adreno 620 GPU), but you can get pretty accurate meshes with it.

  • “We are looking at a number of android-specific optimizations for future versions of ARDK”: this is great news. In the meantime, could you share a list of capable devices (or device features) that we can use as a guide?

Hello Chankeiro,

We don’t have a list of capable devices. All of the device features you can use are listed here: Android Devices and Contextual Awareness