ARDK 2.x -> 3.4 Upgrade: Lightship seems to break Image Tracking

  • Issue category: ARDK
  • Device type & OS version: Huawei P30 Pro (Android 12), Samsung Galaxy S20 FE (Android 12), Samsung Galaxy S20 5G (Android 13)
  • Host machine & OS version: Windows 11
  • Issue Environment: On Device
  • Xcode version: n/a
  • ARDK version: freshly upgraded from 2.3.0 to 3.4.0-2404081151
  • Unity version: 2021.3.3f1, have also tried 2022.3.10f1, 2022.3.16f1

Bug reproduction steps:
Upgrade an existing Unity project in 2021.3.3f1 with ARDK 2.3.0 to ARDK 3.4.0, following the steps outlined in the documentation here: Migration Guide | Niantic Lightship

Description of the issue
I’ve been trying to upgrade an existing project to ARDK3, but Image Tracking isn’t working whenever I have Lightship enabled.

It’s not an issue with my code or image markers; I’ve been testing with a fresh scene with just Unity base components, the Unity sample image library, and a simple debug log script.

No prefab spawns when the camera is pointed at the image marker, and ARTrackedImageManager.trackedImagesChanged can be subscribed to but does not seem to trigger, regardless of how I position the image marker (distance, angle, printed, on screen, etc).

Screen recordings, LogCat messages, and test APK here: ARDK3 Image Tracking issues - Google Drive (APK is just a blank image tracking scene with 1 script for debug logs, using Unity’s test image library. Debug log script is also in that folder under Unity_SetupScreenshots)

Worth noting:

  • I have two other projects using the exact same dev environments, also both upgraded from 2.x, in which image tracking is working perfectly. Even when I match the settings to those projects, the issue persists.
  • If I uninstall Lightship and use just AR Core, Image Tracking works perfectly in the problem-project.
  • I don’t want to uninstall Lightship as we’re using Semantic Segmentation in other scenes (which are working perfectly).
  • AR initialises & the AR Foundation debug menu says that Image Tracking is available in this configuration, but trackedImagesChanged never fires from AR Foundation.
  • Creating a fresh project to get around the issue is not an option; this is a well established project.

Possibly not related but mentioning just in case it’s relevant:

  • We don’t have the SharedAR package installed.
  • When I make a change to the XR config (eg disable Semantic Segmentation, which isn’t used in this scene anyway), the apk often crashes on load if I don’t completely delete the previous version of it off my phone first.
  • In order to delete this project’s Library I have to use force-delete due to Windows permissions (despite being an Admin account); in other projects I can just move to trash.

Things I’ve tried:
:question: Is one of the other features part of the problem? Maybe! Object detection being enabled seems to prevent AR from initialising at all, but it’s enabled in another project that works just fine
:x: Check Lightship version is the same as our working projects. They’re the same version, as is AR Foundation/ARKit/ARCore
:x: Check Android LogCat for error messages that aren’t showing up in ConsoleLogs. Nothing that looks related to image tracking to me - screenshots in the link above.
:x: Disable custom manifest no difference
:x: Rebuild the library no difference
:x: Use the Android Dependency Resolver no difference
:x: Uninstall and Reinstall Lightship I noticed AR Foundation/ARCore/ARKit were visible in the package manager. I uninstalled them and Lightship, then reinstalled Lightship only. This time around AR Foundation didn’t show up in the Package Manager in 2021.3.3f1 (it does in 2022.3.10/16), though the packages are visible in the Project>Packages files. First test build after the uninstall/reinstall crashed straight after I enabled camera permissions. Rebuilt the library again and the next build didn’t crash but image tracking still didn’t work.
:x: Upgrade to Unity 2022.3.10f1 & 2022.3.16f1 no difference

I’m deeply at a loss on what else to try to resolve this. I understand from the other threads that this is near impossible to repro, so happy trial and error on request (my timezone is Wellington (GMT+12) so I may be async-timing).

Other people having the same/similar issue:

Hello Jess,

Thanks for your patience. I was looking at the information provided along with the other users’ support requests. I’m still looking into this, it looks like you and another user are currently experiencing this but I’m yet to be able to reproduce this issue, image detection is working for me.

Can you send a screenshot of your Tracked Image Manager and if you’re instantiating objects through code once the image is detected can you show me how you’re writing said code?

Yeah, as mentioned it’s fine in my other projects - only this project that seems to be having the issue.

I stripped everything back to a test scene when I first ran into the problem, so I’m not instantiating anything through code at present. Just using the Tracked Image Prefab in the AR Tracked Image Manager + a simple debug log script (below).

You can see the debug logs at the end of this video: https://drive.google.com/file/d/1U9dyqurSfWizUdcQT2r1xrayERBMAC2c/view?usp=sharing

trackedImagesChanged gets subscribed to successfully, but doesn’t seem to be called when I point the camera at an image marker - and given that the Tracked Image Manager doesn’t instantiate the prefab, I infer that the marker is not being detected rather than it being a code issue. (The same code also works fine in my other projects)

Screenshot 2024-05-09 100127
Screenshot 2024-05-09 100139

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.XR.ARFoundation;
using static UnityEngine.CullingGroup;

public class ImageTrackDebug : MonoBehaviour
{
    [SerializeField] private ARTrackedImageManager _arTrackedImageManager;
    private bool _sessionHasInitialized = false;

    void Start()
    {
        ARSession.stateChanged += OnStateChanged;
        Debug.Log("Subscribed to ARSession.stateChanged");
    }

    private void OnDestroy()
    {
        ARSession.stateChanged -= OnStateChanged;
        _arTrackedImageManager.trackedImagesChanged -= OnTrackedImagesChanged;
    }

    private void OnStateChanged(ARSessionStateChangedEventArgs args)
    {
        if ((args.state == ARSessionState.SessionTracking || args.state == ARSessionState.Ready || args.state == ARSessionState.SessionInitializing) && !_sessionHasInitialized)
        {
            _arTrackedImageManager.trackedImagesChanged += OnTrackedImagesChanged;
            Debug.Log("Subscribed to arTrackedImageManager.trackedImagesChanged");
            _sessionHasInitialized = true;
        }
        if (args.state != ARSessionState.SessionTracking && _sessionHasInitialized) Debug.Log($"Session is not tracking. Reason: {ARSession.notTrackingReason}");
    }

    private void OnTrackedImagesChanged(ARTrackedImagesChangedEventArgs eventArgs)
    {
        Debug.Log("Starting OnTrackedImagesChanged");

        foreach (var newImage in eventArgs.added)
        {
            Debug.Log("Image added: " + newImage.name);
        }

        foreach (var updatedImage in eventArgs.updated)
        {
            Debug.Log("Image updated: " + updatedImage.name);
        }
    }
}

Can you try subscribing to the events in OnEnable? Similar to how it’s done in the Unity documentation: https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@5.1/manual/features/image-tracking.html

By the looks of your output log everything looks good I mean you’re getting confirmation that you’re subscribing to these events but just to rule out the slight possibility that maybe something down the chain isn’t initializing correctly, try using OnEnable instead.

I see some errors related to depth in the LogCat but I don’t think that should interfere with an image being tracked.

At this stage, every possible quirk is worth trying! :slight_smile: Unfortunately subscribing in OnEnable makes no difference.

I tried disabling & reenabling the TrackedImageManager at runtime on the off-chance that would kick it into gear, but that didn’t make a difference.

And I had the thought “what if the reference library is somehow empty”, but that’s coming back as having the expected Count (2).

Can you go into the Play Store and search for Google Play Services for AR?
Check to see if it’s up to date, if it’s not, update it.
If it is up to date, see if you can uninstall it and reinstall it.

We had a user this week that had issues getting some AR features to run and this resolved it for them

It was up to date, yeah. Just tried uninstalling and reinstalling, and no change to Image Tracking, unfortunately.

Given that image tracking was working in this project with Lightship uninstalled/just AR Foundation installed, I’m not surprised this didn’t do the trick for me though.

Ok, let’s try this. Can you send in logcats for the vanilla ARCore version where image tracking does work and can you also send a new set for Lighship + ARCore? I’m not seeing anything in the logcat that could imply any sort of error so hopefully comparing the logs of the version that works to the version that doesn’t give some clues.

I appreciate your patience with this issue, I’m hoping we can get it resolved as soon as possible :slight_smile:

No worries, I really appreciate the help!

I’ve done two sets of logs for each, all linked in the Drive folder below. The folders contain Warn level screenshots, and the videos are screen records from Debug level. (Too many logs to screenshot at that level - you might need to download the videos rather than preview in browser to get a legible video resolution.)

https://drive.google.com/drive/folders/1iCNREPtV9L9J-JzV2s_qnZxqDtHJ04zJ?usp=sharing

I’ve got both APKs installed under different names now, so if you want me to do new LogCats with any specific filters, let me know.