The goal of analytics is to understand the user journey in the application and their interaction with it to improve the user experience. To gather the necessary data, Smartlook follows user actions and records what users see on the screen. This presents a challenge for protecting user privacy – out of decency, law requirements, and also because the data itself is rarely needed for analytical purposes.
What analytics uses is metadata.
Smartlook protects user privacy when gathering the information it needs in several ways. Smartlook masks native visual elements that typically contain private information. The data is replaced by neutral color boxes during the recording process so that sensitive information never makes it out of the users' device. Moreover, Smartlook offers a set of methods that enable application developers to avoid recording unnecessary information.
Sensitive elements are hidden locally on the device. No sensitive data is transferred over the network or stored in the dashboard.
Potentially, any UI element could contain sensitive information that should not be recorded. However, some elements that are not recorded by default could contain harmless, yet vital data for analytics. Smartlook offers a comprehensive set of methods to handle various scenarios.
In Smartlook, controlling UI elements recording is done by setting their sensitivity.
In order to provide robust user privacy protection by default, some UI elements, namely text inputs and web views (that typically contain sensitive user data) are set sensitive by the default Smartlook setup. An application author must take active steps to remove these elements from the sensitive elements list in order to record their content.
There are three levels of granularity established for handling the sensitive data recording:
- individual UI elements recording
- UI element types recording
- whole screen recording
The support for individual sensitive data protecting methods depends heavily on the technology used for creating the particular user interface. For further details, see your platform in the API documentation.
Individual elements sensitivity
Gaming (OpenGL, Metal) frameworks
Sensitivity can set to three possible states: sensitive, insensitive or default. These states are typically represented as an optional boolean value (
The default state specifies that more general rules apply, i.e., a view instance with default sensitivity takes its sensitivity from its class.
When evaluating an UI element sensitivity, the more specific setting is always used, e.g. an individual instance sensitivity overrides the respective class sensitivity.
When just one or few individual UI elements' sensitivity must be adjusted, the respective elements references are used to set their individual sensitivity.
On some platforms, Smartlook provides dedicated extensions, accessible as a
slSensitive property (usage e.g.
myView.slSensitive = false). When this is not possible or practical,
Smartlook.instance.sensitivity methods can be used to manipulate individual component sensitivity.
Please refer to each platform API documentation for further details.
In some use-cases, setting sensitivity on individual views is not practical, e.g., in an application that heavily uses web views for presenting some insensitive content, or if the application defines its own subclasses of visual components that present sensitive user data.
To cover this use-case, Smartlook offers on some platforms a convenient way to set sensitivity of all descendants of some class or visual elements that conform to a protocol.
Please refer to each platform API documentation for further details.
Native WebView classes are handled as sensitive by default. Its instances, or the whole class, must be set insensitive in order to record its content. However, each web view can still contain HTML elements with sensitive content, and Smartlook thus does not record the standard HTML elements where the user enters data.
In order to change the default behavior, html elements that are not recorded by default can be set insensitive by assigning
smartlook-show css class to them. Any html element that should not be recorded can be set sensitive by assigning
smartlook-hide css class to it.
<input type="text" class='smartlook-show'> <img class='smartlook-hide'>
This functionality is fully available only on some platforms. Please consult the respective platform API reference what rendering modes are available for a particular development toolchain.
There can be scenarios when direct recording of the screen is not appropriate or necessary to understand the user behavior. In such cases, Smartlook can be instructed to use (permanently or just temporarily) one of its wireframe rendering modes.
When a wireframe mode is on, Smartlook draws a sketch of the screens instead of copying its content literally into the video. The principal UI elements and user interaction with them are still clearly distinguishable, but their actual look and content (texts, images) are replaced by an abstract representation.
The default rendering mode for an application is set in the dashboard.
When debugging or experimenting to find the most suitable one for a particular application, the default rendering mode can also be set within the setup options. In production releases, this option should not be used.
The application can change the rendering mode at any time, e.g., to temporarily hide the sensitive user content on some screens. It is also possible to obtain the currently used values.
More about rendering modes could be found in the each platform API references.
It is not just the visual elements that can contain sensitive information. Sometimes, a clue to sensitive user data can be found in user interaction events. Consider that, e.g., location of click events on a keyboard area could be used to guess a user’s password even when the keyboard itself is obfuscated.
Click events in the area of the system keyboard are never recorded by Smartlook. User interaction with custom input elements, e.g., custom PIN-Pad, must be filtered out programmatically by the application developer.
The default setting is to track all events (with the exception of events on the keyboard, for obvious user privacy reasons). When this is not desired, the application may fine-tune the level of tracked events.
Typically, the default
Full Tracking mode is desired, as it gives a detailed picture of user activities.
On the other hand, on some screens the event location of touches on an otherwise sensitive view can reveal sensitive private information (e.g., custom input view for PIN). In such a case, touches should not be recorded, and the application should switch temporarily to the
Ignore User Interaction Events mode.
In specific cases, some apps that handle complex private information on particular screens may prefer stopping event recording altogether, by switching to the
No Tracking mode for the screens.
Event tracking mode cannot be set on a per-UI-element basis, but only for the whole application.
More about event tracking modes can be found in each platform API reference.
Updated 3 months ago