Detect image RE-entering view

Include the following details (edit as applicable):

  • Issue category: ARDK Documentation
  • Device type & OS version: Android / iOS
  • Host machine & OS version: Mac / Windows
  • Issue Environment : On Device
  • Xcode version: current
  • ARDK version: current
  • Unity version: 2021.3.4.f1

Description of the issue:
We want to use images as event triggers in our project (not necessarily as anchor points).
I have successfully used OnAnchorsAdded to do this, but it only works for the first detection of the image in the camera. We need it to trigger EVERY time it is visible.
I’ve been looking into OnAnchorsUpdated but the documentation says that this will fire when the anchors move or if the initial estimation is improved. The images we will use do not move, they are stationary images attached to walls or display plinths so I can’t rely on that event firing every time the image comes into camera.

How can I detect a subsequent appearance of a previously detected image in the current camera frame?

(My only thought is to add an invisible GameObject to the anchor point, then detect if that is in frame… but that will trigger from all angles and through walls which is undesirable).

As a general note, it feels like there are nowhere near enough event types for the APK to be flexible. I would expect the _session to include a ton more options. For this activity a simple OnImageDetected would be wonderful, and is a surprising omission.

Regards,
Pete Baron

Hi Peter,

There are two things to try to solve your issue.

  1. Remove the anchor (IARSession.RemoveAnchor) once it goes out of frame, then see if they’ll get a new IARSession.AnchorsAdded when it appears in frame again.
  2. Try listening to the AnchorsUpdated event, which should be firing frequently enough (particularly if the anchor goes in and out of frame) even if the image is stationary.
    I will also pass this information along to the teams regarding your suggestions. Thanks.
1 Like

Thanks for the reply @Moranda_Nalls1

  1. To be able to detect when the image goes out of frame (there’s no event for this) I’ll need to use that trick of adding a gameobject at the anchor point and test it against the camera. As mentioned previously, that trick won’t work in the case of walls blocking the line to the anchor object - it will believe it is still visible.

  2. I’ll definitely try this and see how often it fires with stationary objects.
    The documentation for this event strongly implies that it should not fire unless the image moves or the detection changes, but if it’s using high accuracy for the position and isn’t using a threshold on the distance, then yes I would imagine it will fire quite frequently (even constantly). I would be concerned about that being corrected in future ARDK updates though, because we would be relying on a bug never being fixed.

I wonder, would it be ok for me to always remove the anchors 0.25 seconds after they are detected? That way, if suggestion #1 works and a new anchor is added, I’ll have confirmation that the image is still in view four times per second, but it might make the XR system work very hard.

Please keep me informed if the team does implement any feature that will allow us do this more cleanly.
I will update here after I have run those tests.

@Moranda_Nalls1 apologies, I marked this as solved but further testing has found problems with both of the suggested solutions.

  • suggestion 2: the anchor update event fires continually, even when the image is not in view
  • suggestion 1: removing the anchor results in it being added again, also when the image is not in view

Can you please escalate this to the team as it currently appears to be impossible to detect when an image remains visible in the camera frame.
I think that the continual firing of anchor update when the image is not view is a bug, and is also needlessly consuming system resources.

Peter,

I’ll escalate this to the team. Please provide the detailed steps so we’re able to reproduce the bug you’re experiencing. Thanks.

Thanks Moranda. I’m just wrapping up our tests right now and then I’ll try to build simple examples of the issues I’ve encountered for a detailed report.

EDIT: suggestion 2, above:
I can confirm that the AnchorUpdate fires constantly for an enabled tracking image, even when the image is not in view. To do this we only need to add a Debug.Log(“Anchor Updated”); in the ARDK Example “ImageDetectionExampleManager.cs” script:

        private void OnAnchorsUpdated(AnchorsArgs args)
        {
            Debug.Log("Anchor Updated");

This could be by design, but it isn’t required for animation etc (Unity Update already has that covered) and it prevents the Anchors from being used as effective Image trackers (because we can’t tell when they are visible or not).

Expected behaviour:
OnAnchorsUpdated should be called only when the anchor is in view.

I’ll update again when I’ve checked the anchors being replaced issue.

1 Like

Tracking down the behaviour for the first suggestion (remove the anchor) took longer. The problem was not what I originally thought it was. I have made a video on device which shows the full test.

Using the Example code for Image Detection again:

  • add logging for anchor added, updated and removed, which includes the image name
                IARImageAnchor imageAnchor = (IARImageAnchor)anchor;
                string imageName = imageAnchor.ReferenceImage.Name;
                Debug.Log("Anchor Added " + imageName);
  • set up the yeti and the crowd images ready to be viewed
  • start the AR and move the device so the yeti image can be seen
  • the log will show added (yeti) and then spam updated (yeti)
  • move the phone so the yeti is not in view
  • updated (yeti) will continue (this is the issue I reported above)
  • press button ‘deactivate yeti anchor’
  • the update spam will stop as expected
    I repeated the same process with the crowd image, and was surprised to see that pressing ‘activate yeti’ or ‘deactivate yeti’ will toggle the crowd image update spam on and off (despite the code only specifying _yetiImage in the calls).

My conclusion is that anchor update will fire for all previously detected images, until ARImageDetectionManager.RaiseConfigurationChanged() is called. Only that function causes the ARDK to reassess what is visible and what is not visible.
Digging further down I see that the code ends in _observerRequestedCollection = true; which is picked up by the next Tick in RerunIfRequested() to run a new ARSession.

So instead of triggering a Remove Anchor to reset the continuous update spam which doesn’t work, calling ARImageDetectionManager.RemoveImage() will do what we need… at the cost of creating a new ARSession each time.

Can you tell me if doing things this way will break other running AR activities?

I see in the code there is this comment above the _arSession.Run():

        // Keep the ARConfiguration from the last run, but have a fresh slate for run options.
        // Just because a dev wanted to reset tracking on the previous run doesn't mean they want
        // changing plane detection, for example, to reset tracking again.

However Image Detection resets for ALL images when it is reset for any individual image and I’m concerned that creating a new session will cause other systems to reset too.

The image detection is only a small part of the planned project, it’s the trigger for a scene switch to a different ‘experience’. Each experience will be using a variety of other AR features, while the image detection continues to run and try to detect if the user has found a new marker or returned to a previously seen one.

Peter,
Thanks for your feedback. I’ll reach out to you as soon as I have an update.