Cloud Recognition
The documentation for cloud recognition is split into two parts
- Documentation of the server-side component (Studio and Studio API)
- Documentation of the SDK side implementation, which follows below in more detail
Make sure to read the documentation about Studio and Studio API when using cloud recognition feature.
This example shows how to recognize images on a cloud server and then overlay it with augmentations utilizing the ImageTracker
and CloudRecognitionService
classes.
For a better understanding, here are some terms that will be used in the following and other sections of this documentation related to vision-based augmented reality.
Target: An image and its associated extracted data that is used to recognize an image.
Target Collection: A group of
targets
that are searched together. Think of it as a directory, which contains all your images you want to search. The Wikitude SDK can work with two different sorts ofTarget Collections
- On-device Target Collection: a static
wtc
file containing the extracted data of your images. Can consist of up to 1,000 images. - Cloud Target Collection: A target collection stored on the Wikitude server. See
Cloud Archive
below. Can consist of up to 50,000 images.
- On-device Target Collection: a static
Cloud Archive: An archive stored on the server that is optimized for cloud-based recognition. It is generated from a
Target Collection
and is used in combination withCloudRecognitionService
.CloudRecognitionService: Instead of analysing and computing the live camera feed directly on the device, the
CloudRecognitionService
will send the image(s) taken by the camera to the Wikitude Cloud Recognition server. The server will then do the hard work of trying to match the image with your targets in the specified cloud archive. Beside the benefit of searching in large image database, using theCloudRecognitionService
has also a positive impact on the general performance in most cases. Especially when using a large target collection and on older devices.
Cloud Recognition Sample
For both Cloud Recognition samples below we will use external rendering. If you don't know what that means please go through the section on rendering before starting here.
The CloudRecognitionService
is able to run in two modes, called on-click and continuous. In On-Click mode a single recognition cycle will be executed, while in continuous mode the recognition will be run repeatedly with a variable interval.
Both sample pages are located in NativeSDKExamples\NativeSDKExamples\src\Views\CloudTracking\
.
On-Click Cloud Tracking
On-Click Cloud Tracking
sample works almost the same way as Simple Image Recognition
, we will create a ImageTracker
with a CloudRecognitionService
in place of the TargetCollectionResource
.
First a CloudRecognitionService
by callingTrackerManager::CreateCloudRecognitionService
with the client token, group id and target collection id as parameters. Optionally, we can pass a CloudRecognitionServiceConfigurator
delegate to obtain a CloudRecognitionServiceConfiguration
to configure the cloud recognition server and the target conflict solver.
page->_cloudRecognitionService = page->_sdk->getTrackerManager()->createCloudRecognitionService(
"b277eeadc6183ab57a83b07682b3ceba",
"B1QL5CTCZ",
"54e4b9fe6134bb74351b2aa3",
ref new CloudRecognitionServiceConfigurator([](CloudRecognitionServiceConfiguration^ configuration_) {
configuration_->setCloudRecognitionServerURL(CloudServerRegion::Europe());
}));
We can then create the Image Tracker
by calling the TrackerManager::createImageTracker
method with the CloudRecognitionService
as the first parameter.
page->_imageTracker = page->_sdk->getTrackerManager()->createImageTracker(page->_cloudRecognitionService,nullptr);
To start a single recognition, we call the CloudRecognitionService::recognize
method. We pass a RecognitionHandler
delegate as the first handler. We make that call in response of a button press.
void NativeSDKExamples::SingleRecognitionPage::recognizeButton_Click(Platform::Object^ sender, Windows::UI::Xaml::RoutedEventArgs^ e)
{
if (_cloudRecognitionService) {
_cloudRecognitionService->recognize(ref new RecognitionHandler([this](CloudRecognitionServiceResponse^ response_, Error^ error_){
_dispatcher->RunAsync(CoreDispatcherPriority::Normal, ref new DispatchedHandler([=]() {
if (error_) {
auto toastXml = ToastNotificationManager::GetTemplateContent(ToastTemplateType::ToastText02);
auto toastNodeList = toastXml->GetElementsByTagName("text");
auto header = toastNodeList->Item(0);
header->AppendChild(toastXml->CreateTextNode(error_->getDomain()));
auto content = toastNodeList->Item(1);
content->AppendChild(toastXml->CreateTextNode("Recognition failed - " + error_->getDescription()));
auto toastNode = toastXml->SelectSingleNode("/toast");
auto audio = toastXml->CreateElement("audio");
audio->SetAttribute("src", "ms-winsoundevent:Notification.SMS");
auto toast = ref new ToastNotification(toastXml);
ToastNotificationManager::CreateToastNotifier()->Show(toast);
}
else if (response_ && response_->isRecognized()) {
dropDownAlert->Visibility = Windows::UI::Xaml::Visibility::Collapsed;
cloudTrackingInfoField->Text = response_->getTargetInformations()->getName();
cloudTrackingInfoBar->Visibility = Windows::UI::Xaml::Visibility::Visible;
}
else {
dropDownAlert->Visibility = Windows::UI::Xaml::Visibility::Visible;
cloudTrackingInfoField->Text = "Recognition failed - Please try again";
cloudTrackingInfoBar->Visibility = Windows::UI::Xaml::Visibility::Visible;
}
}));
}));
}
}
Please note that only a recognize call is made if the cloud recognition service finished initializing.
In case the server communication was successful, the first parameter of the delegate is a valid object reference of type CloudRecognitionServiceResponse
that represents the server result. This object contains information if a image target was found or not, its name and, if specified, its associated meta data.
In case of an error, the second parameter will contain a valid Error
object reference which contains detailed information about why the communication failed.
Since both simple image tracking and this use the ImageTracker
events, all tracking related methods to draw a augmentation around the recognized target are available as well.
Continuous Cloud Tracking
On-Click recognition is useful in some particular cases, but more often than not you probably want to use continuous recognition. For continuous cloud recognition we set an interval in which the CloudRecognitionService
automatically calls the recognize function.
To start continuous recognition, simply call the CloudRecognitionService::startContinuousRecognitionWithInterval
method once it finished initializing.
_cloudRecognitionService->startContinuousRecognitionWithInterval(_recognitionInterval,
ref new InterruptionHandler([wr](unsigned int suggestedIntervalInMilliseconds_) {
ContinuousRecognitionPage ^ page = wr.Resolve<ContinuousRecognitionPage>();
page->_recognitionInterval = suggestedIntervalInMilliseconds_;
}),
ref new RecognitionHandler(
[wr](wikitude::sdk::uwp::CloudRecognitionServiceResponse^ response_, wikitude::sdk::uwp::Error^ error_) {
if (response_->isRecognized()) {
ContinuousRecognitionPage ^ page = wr.Resolve<ContinuousRecognitionPage>();
page->_dispatcher->RunAsync(CoreDispatcherPriority::Normal, ref new DispatchedHandler([wr, response_]() {
ContinuousRecognitionPage ^ page = wr.Resolve<ContinuousRecognitionPage>();
if (page->_continuousRecognitionEnabled) {
page->stopContinuousRecognition();
}
page->dropDownAlert->Visibility = Windows::UI::Xaml::Visibility::Collapsed;
page->cloudTrackingInfoField->Text = response_->getTargetInformations()->getName();
page->cloudTrackingInfoBar->Visibility = Windows::UI::Xaml::Visibility::Visible;
}));
}
}));
As before, this method is called once a Button
was pressed.
The first parameter defines in which interval new camera frames should be sent to the server for processing. The second parameter is again a response handler with information about the recognized target and behaves the same way as in the case of On-Click recognition.
To stop a continuous recognition session, e.g. once a target was found, simply call the CloudRecognitionService::stopContinuousRecognition
method.