Communication within the metaverse is very difficult. Have the right conversations, detect the right emotions, build empathy – how can a startup imagine breaking down technical components and building end-to-end experiences? Harold DumarFounder and CEO OVAa member of the Microsoft for Startups Founders Hub StellarX, the metaverse builder.
What is Stella X?
“Welcome. I business, is a virtual (non-human) learning coach. ”
This introduction comes from an immersive environment created with StellarX, an intuitive metaverse builder for OVAs. Desjardins Lab.
This project fuses AI with spatial computing, machine learning, and user interface technology. We have created an intuitive and immersive experience to teach and test critical communication skills in the context of financial and insurance customer service.
Call agents interact with smart virtual assistants through scripted conversations based on real-world situations. The virtual trainer evolves contextually to match the professional development of the agent.
How does context evolution work?
We dive into the world of Natural Language Processing (NLP), Azure cognitive services Their NLP framework named LUIS (Language Understanding)By leveraging the tools provided by Microsoft and those tools, bot frameworkthe OVA was able to design conversation flows that would later be integrated into the Metaverse building platform.
“The purpose of virtual assistants is to bring call agents into real-world situations through real-time assistance. For example, virtual assistants can monitor and analyze performance and provide valuable feedback,” said Director of R&D at OVA. said Pierre-Luc Lapointe,
AI-driven OVA has been building patent-pending visual scripting tools for several years now integrated into StellarX. The goal is to allow the user to co-create with his AI interactions. Combining discovery, Azure cognitive, and visual scripting tools in this project, the team aims for a system called “intent-to-code.” Once deployed, this system will allow users to create interactions in the metaverse based on voice commands.
This coaching evolution is thanks to an auto-learning algorithm that feeds your virtual assistant. Therefore, it can explore experiential personalization, predictive analytics, and recommend products and services.
How important is emotional connection in AI?
“I lost my credit card” sounds completely different than “I lost my credit card @#$%&!” In fact, it’s very easy to identify the emotional intent behind the sentence. Here, employees embody avatars in a virtual space and interact with other avatars belonging to customers. A third avatar, an AI coach, monitors sessions and collects data for future feedback.
In this context, a key challenge was to approach real-world conversations by identifying emotions. So when we classify emotions, how exactly can we classify them?
We needed to build an effective model that automatically labels positive, neutral, and negative emotions in customer-agent relationships. We used Azure Cognitive Services for integration simplicity, reliability, and time savings. We also used Azure Cognitive Services for natural language understanding tools to research and develop both virtual assistants and code writing capabilities.
The accuracy of automatic labeling of emotions was based on two different contexts:
- Spoken neutral linguistic expressions were analyzed in text form using NLU and NLP features.
- Speech emotion recognition scrutinized the exact words, but this time it was the emotional intonation of the voice.
Here, Azure Cognitive Services acted as a complete AI solution, analyzing the words under these call center conversations and correlating them with sentiment.
The question remained: How can we introduce essential soft skills for agents, such as adaptability, empathy, and resilience? I have to be able to. Therefore, the accuracy of emotion detection, identification and virtual rendering was essential.
“I chose to express my emotions in emoji style under a scale bar of five colors, moving from extreme anger (red) to utter satisfaction (green) gradually. Colors and behaviors change according to emotions, and naturally emotional changes are in sync with the conversation,” Lapointe explains.
This emotional side helped OVA integrate a virtual assistant into our platform, StellarX.
What is your vision for the Metaverse Emotion Trainer?
A wealth of literature on soft skills has been published specifically to illustrate the relevance of soft skills in the workplace. Comprehensive leadership, emotional intelligence, and good communication skills are sophisticated traits that are hard to detect and even harder to teach.
Theory and practice are two different things. Constant exercise with valuable feedback is not a luxury most companies can afford. Employees typically get bogged down in best practice manuals with little opportunity to practice, while managers don’t have enough time to share valuable insights. That is until AI-guided VR becomes a breakthrough training tool.
VR is a fully immersive and interactive medium. It helps you develop a deep and wide range of emotions thanks to your intrinsic presence. for example, PwC’s soft skills training report shows:“VR learners felt 3.75 times more emotional connection Content rather than classroom learners 2.3x more connectivity than eLearner” VR can be adapted to endless soft skills scenarios, such as managers and sales training sessions.
One of the goals of this project was to develop a smart coach (AI avatar companion) where AI can provide learning methods. With a wealth of data collected from training scenarios and real-world calls, you can constantly improve your digital educational memory. This behavioral data library translates into better self-learning methods and more sophisticated analysis of participant performance.
Finally, building confidence is one of the most noticeable outcomes of such VR training projects. According to the same PwC report, “40% of VR Learners Experience Increased Confidence Compared to classroom learners, 35% better than eLearner It’s about putting into action what you’ve learned after training in VR. ”
Why did you choose Azure when building your AI project?
During the course of this AI machine learning project, we found that using Azure made the creation process easier and faster from start to finish. As a startup, delegating external tasks and focusing on your expertise is paramount. Azure’s solution allowed us to focus on what was essential: creating.
“Azure Cognitive has sped up the transition from experimentation to production thanks to its user-friendly integrations. “It’s become an important R&D tool for us,” says Lapointe.
Our XR creation tool, StellarX, enables anyone to build virtual experiences without coding skills. We believe this tool will be the bridge to the metaverse from training sessions to research, brainstorming and collaboration.
We are on a mission to create ethical, human and immersive experiences that are accessible to everyone. Thanks to StellarX, we’re democratizing the creation of the metaverse, enabling user-generated content to revolutionize entire industries such as healthcare, construction, and defense.
For more AI startup launch tips, sign up today for the Microsoft for Startups Founders Hub.