Device type & OS version: Android / iOS / Other Ex. iPhone 8+ on iOS 13
Host machine & OS version: Mac / Windows / Linux / Other Ex. Mac on Big Sur x.x
Issue Environment : Unity Remote / Unity Mock / On Device / Dev Portal
Xcode version:
ARDK version:
Unity version:
Description of the issue:
Hi, I am trying to use hand tracking with ARDK 3.3, but I can’t find the file at the sample folder, just like at previous ARDK versions. Where could I get to install it to ARDK 3.3? Thanks
ARDK 3 does have palm detection as part of the Object Detection experimental feature but please do note that it doesn’t necessarily track the palm, only detects it.
Not through ARDK but you’re free to try other 3rd party plugins. One we’ve heard of in the past is Mano Motion. We just can’t provide support for issues that users run into when working with other libraries/ plugins. At the moment ARDK hasn’t been officially validated to work with other 3rd party plugins but since it’s combined with Unity’s ARFoundation, the chances of other libraries working with ARDK are better than they used to be
I wanted to follow up with you to ask what your use case for palm tracking is. What are you trying to achieve? Object detection is still an experimental feature which means that it’s still being worked on. We’d love to consider the use cases of developers as this feature is further developed. Please do note that this doesn’t guarantee that a specific feature or use case is guaranteed to be added on.
My goal is the app can track my hand to make an object to appear on it. It was possible before, I lend you this video if you wish. I has the example at the beginning.
Thank you for your patience. So unfortunately at this time, hand tracking is not a feature of ARDK 3. You can detect a hand as you’ve seen with object detection but you can’t track it. Palm detection and tracking was only an experimental feature in ARDK 2, it hadn’t yet become an full official feature of ARDK so it wasn’t included in ARDK 3.
There is a potential workaround but it would require you to essentially implement your own tracking and it might not work if multiple hands are detected. What you can do is the following:
Use Object Detection as normal to detect a human hand and also include an Occlusion manager for depth.
With both Object Detection and depth, you’d be able to sample the midpoint of a hand in 3D space but you would need to continue sampling it every frame since a hand can move.
As mentioned above, the hand can move so the sampled midpoint in one frame can be different from the midpoint of another frame. Therefore you might want to assume that if the points are close enough then it’s the same hand. You’ll need to experiment with your project to see what distance between two points you could consider to be “close enough”
Lastly, you’ll want to interpolate the position of the 3d Object being placed on the hand based off of the current sampled midpoint and the midpoint of the previous frame. Assuming the hand won’t be doing any large motions, the interpolation should be able to keep the object on the hand and track it as smoothly as possible