Overview
Whatfix's ScreenSense technology is powered by our patented element detection algorithm - the engine responsible for accurately identifying UI elements within your applications. When you create content, ScreenSense captures and remembers elements so your guidance appears exactly where it should, every time.
To improve element detection in complex scenarios, Whatfix is augmenting ScreenSense with AI to solve scenarios that are difficult to resolve using traditional methods.
Note:
AI-powered Element Detection is disabled by default and is enabled only upon explicit customer approval. This capability is not automatically enabled for any account.
To enable AI powered Element Detection, contact your Customer Success Manager or support@whatfix.com.
What are the common Element Detection challenges?
When creating content for complex enterprise applications, you may encounter the following challenges:
Elements that do not reliably attach: Some tooltips or guidance may not appear on the intended element despite multiple attempts to identify the element accurately.
Time spent adding CSS selectors: You may need to manually add CSS selectors or work with the Whatfix Support team to improve element identification.
Multilingual applications: Content that works correctly in English may break when the application switches to Spanish, French, or any other language.
These challenges can impact content reliability and increase the effort required to maintain guidance across applications.
The challenge of multilingual applications? Finally solved.
One of the key enhancements in ScreenSense AI-powered Element Detection is its ability to understand the meaning and context of element text across languages.
For example, ScreenSense recognizes that “Purchase Order” and “Orden de Compra” refer to the same element. Content created in one language can continue to identify the correct element when the application language changes.
This capability is based on semantic understanding rather than direct translation.
As a result, organizations and content authors can:
Reduce the need for language-specific CSS selectors.
Create content that works across multiple application languages.
Deliver more consistent guidance experiences for global users.
What's new?
ScreenSense also improves element detection by analyzing your application's HTML structure and generating more reliable selectors when needed.
The enhancement works in the background and does not require any changes to the content creation workflow.
How does the AI powered Element Detection work?
Create content as usual
You create content using the standard point-and-click experience in Whatfix Studio.AI-powered analysis runs in the background
ScreenSense analyzes elements that may be difficult to identify and evaluates their surrounding context.Automatic enhancement
Robust selectors are automatically generated and applied asynchronously.Improved content reliability
Your content benefits from improved element detection reliability without requiring additional authoring effort.
Note:
All processing occurs in the background and does not impact the authoring experience or end-user workflows.
How does this benefit you?
Improved self-serviceability
Tasks that previously required technical configuration, such as creating CSS selectors or contacting support for assistance, can now be handled automatically in the background.
This enables authors to spend more time creating content and less time troubleshooting element detection issues.
Reliable content
ScreenSense AI-powered element detection analyzes application HTML patterns and adapts to different scenarios.
This helps reduce content attachment issues, minimizes maintenance effort, and improves confidence that guidance will appear where users expect it.
Semantic understanding, locale agnostic guidance
Organizations can deploy guidance to global workforces without worrying about language changes affecting content performance.
A single content set can work across multiple supported languages with less configuration and maintenance effort.
How does Whatfix ensure data security?
Built in PII protection
ScreenSense automatically detects and removes sensitive information before processing occurs.
The system identifies and masks information such as:
Email addresses
Social Security Numbers (SSNs) and other personal identifiers
Credit card and financial information
IP addresses and system identifiers
Customer data is used only to improve content reliability and is not used for model training or any other purpose.
Frequently Asked Questions (FAQs)
Will this affect my existing content?
No. The enhancements apply only to new content created after the capability is enabled. Existing production content continues to work exactly as it does today.
What if the AI cannot find or construct a good selector?
This capability enhances existing element detection and does not replace it.
If the AI cannot generate a reliable selector, the content automatically falls back to Whatfix's standard element detection mechanism. This ensures content continues to function without interruption.
Does the AI processing slow down content creation?
No. All AI processing occurs asynchronously in the background after content is saved.
Authors continue to experience the same content creation workflow while ScreenSense AI-powered element detection performs its analysis in the background.
Which applications benefit most?
Applications with complex or dynamic HTML structures typically see the greatest benefits, including applications that:
Change frequently
Contain dynamic content
Support multiple languages
Have limited unique identifiers or stable attributes
Do we need to retrain our content authors?
No. Authors continue to use the same Whatfix Studio experience.
ScreenSense AI-powered element detection works behind the scenes to improve content reliability without changing existing authoring workflows.
Can we disable the capability if needed?
Yes. This capability can be enabled or disabled at the account level.
Does the capability impact end-user workflows?
No. ScreenSense AI-powered element detection operates in the background and does not affect application performance or the way end users interact with your applications. End users continue to use the application as usual while benefiting from more reliable Whatfix guidance.