I conducted interviews to understand the first-time use experience and test usability within an online forum.
Siemens Industrial Edge (IE) is an edge computing platform aimed at enabling efficiency among industrial plant machines, systems, and processes. Mendix, as a part of Siemens, offers a low-code application (app) development platform to enterprises. This study was part of a larger effort to improve first-time use experiences across both Mendix and Siemens platforms, to increase user acquisition and engagement.
Our team wanted to increase the number of users registering and engaging with the Siemens IE community forum. I therefore wanted to observe users interacting with the forum for the first time, understand how they perceived it, and take note of where they encountered friction.
My research goal was to determine opportunities for product improvements and/or further research within the Siemens IE community registration and initial onboarding experiences. Research questions included: what are first-time IE community users’ expectations during registration and initial onboarding, and what prevents and/or hinders them from completing tasks during registration and onboarding?
Research goal
Research question example
I conducted five 45-minute interviews with participants who were not currently users of the Mendix platform, but who were familiar with Siemens IE. This was to reduce positive bias resulting from familiarity with the Mendix platform and community. Familiarity with Siemens IE was important to enable participants to make sense of IE community content.
During the interviews, I was primarily interested in qualitative data uncovering what users do and why. I tested three usability scenarios, asked participants to imagine themselves in those scenarios, and then had them narrate their actions as they navigated through unassisted. I asked questions related to motivation and attention as needed, such as:
Scenarios were designed to help participants get a sense of the community, create a new account, and update their user profile. For each scenario, I created an optimal path to compare with real paths taken by each participant.
To close each interview, I also captured metrics like perceived user effort and perceived user satisfaction by asking questions using a 5-point likert scale aloud.
Tools
I used Confluence for the qualitative data collection, Zoom to remotely observe and interview, and Dovetail to analyze and synthesize data.
Results showed that although participants oriented themselves quickly to the community space in the Siemens IE forum, there were several usability issues (some severe) causing navigation friction. These included getting lost, trouble navigating drop down menus, and finding the interface mildly disorienting overall. I concluded that users were unlikely to use this community forum in its current state since they already had a working alternative (another internal Siemens tool).
I recommended actions to improve the forum experience, including usability fixes, renaming components, and improving functionality with filtering. I also recommended for my stakeholders to consider ways to differentiate the Siemens IE space from the alternative tool in terms of value and complementary functionality.
As a result of ongoing research and improvements related to first-time use experiences, our unit saw incremental increases in Mendix account registrations of around 3-5% for the following quarters. Our team continued applying what we learned in this project to other areas of the Mendix platform, too.
