Passwordless authentication and PWA installation : what we learned about our usability tests

It had been a while since we last ran usability tests. It's one of our favourite parts of the job, though. With summer finally here and temperatures climbing, we figured it was the perfect time to get back into it.

We rolled up our sleeves and put together a series of test scenarios around two topics we've been particularly curious about: passwordless authentication and Progressive Web Apps (PWAs). Both promise to make the user experience simpler, but do they actually feel intuitive to users? What comes naturally, and what catches them off guard? Here's what we learned from an afternoon of testing.

For each testing session, we recruit a small group of participants, usually four or five, with a variety of backgrounds. This time, we specifically looked for people with different levels of digital confidence.

Participants are asked to complete a series of tasks on a product without any guidance from us. Our job is simply to observe. Where do they hesitate? Where do they get stuck? What feels obvious, and what doesn't? Once the sessions are over, we analyse everything we observed and turn those findings into practical insights.

Our five participants ranged in age from 27 to 70, and their confidence with technology varied just as much. Some considered themselves very comfortable with digital tools, while others described themselves as much less confident.

It's also worth remembering that usability testing is never a completely natural situation. Having two people quietly taking notes while you use a product can be intimidating. Personally, if a colleague watches me do something on my computer, I suddenly forget how to use it. Thankfully, our participants handled the experience remarkably well. Every session took place on a mobile device to reflect the way people most commonly use these products.

I was especially looking forward to this scenario. Personally, I'm a big fan of passwordless authentication. But as UX designers, we know our own experience is never representative of everyone else's.

For this test, we used IKEA's passwordless login flow, which has become increasingly common across many platforms. Users simply enter their email address, receive a one time verification code, and enter that code to sign in.

I was curious to see whether anyone would notice the absence of a password field. As it turns out, almost nobody did. Participants simply followed the process. For most of them, receiving a code by email felt like a perfectly normal way to log in.

The entire flow was even smoother than I had expected. Participants opened their email, retrieved the code and completed the login process with very little difficulty. I expected digitally confident users to move through it quickly, but even those who described themselves as less comfortable with technology had no real trouble.

One participant briefly hesitated before realising they needed to retrieve a code from their inbox. The email took a few moments to arrive, but once it did, the rest of the process went smoothly.

During the discussion afterwards, several participants told us they were already familiar with this type of authentication. Some even said they preferred it to traditional passwords. The reasons were fairly consistent:

In the end, we realised that passwordless authentication wasn't new at all for several of our participants.

A Progressive Web App (PWA) is a website that can be installed on your phone much like a native app. It appears on your home screen, opens in full screen and generally provides a smoother mobile experience than a traditional responsive website. The goal is to offer the benefits of an app without requiring users to download it from an app store.

For this scenario, we chose a news website that prompts visitors to install its PWA as soon as they arrive. After the encouraging results from the passwordless test, this one told a very different story.

As soon as the page loaded, participants were greeted by a large cookie banner. Several were already showing signs of frustration. The moment they dismissed it, a second popup immediately appeared, asking them to install the app.

Most participants instinctively closed this second dialog. We wondered whether they had actually noticed it or simply dismissed it automatically. The results were mixed. Out of five participants, three read the message and understood what it was about. The other two closed it without reading anything.

That raised several questions. Is the concept of a PWA itself being rejected? Do users even know what a PWA is? More importantly, do they see any value in installing one?

The discussion after the test revealed a clear pattern. Participants weren't rejecting PWAs as a concept. Instead, they explained that they usually install apps only when they know they'll genuinely need them or use them regularly. During a first visit to a website, that value simply hasn't been established yet.

One participant, for example, had previously installed a PWA to quickly access a digital textbook used for her studies. In that situation, the benefit was immediate and obvious.

On Android, the installation process was fairly straightforward thanks to the built in Install button.

On iOS, however, the instructions were incomplete. One essential step was missing, which meant some participants simply couldn't install the PWA at all.

And that wasn't the only surprise.

On two Android devices used during testing, the installation didn't happen immediately. Everything appeared to work correctly, but when we asked participants to find the newly installed app, it simply wasn't there.

The following night, one participant emailed us to say the app had finally appeared. Considering the test took place the previous afternoon, that's far too long.

With an unclear installation flow on iOS and delayed installation on Android, it's difficult to know whether the issue comes from the technology itself, the browser or the operating system. One thing is certain: if the promise is that an app can be installed in just a few seconds, the experience has to be flawless.

Passwordless authentication, which still felt relatively new to us, has already become part of everyday life for many of our participants. Hardly anyone questioned where their password had gone. The shift in mindset had already happened.

PWAs, on the other hand, weren't rejected because of the concept itself. Most participants understood the potential value, but not at the point when it was presented to them. A useful feature alone isn't enough. The quality of the experience matters just as much. Differences between Android and iOS, browser behavior and even installation delays can quickly turn a well designed experience into a frustrating one.

That's exactly why we keep running usability tests. They allow us to challenge our assumptions with real user insights. And, as is often the case, the most valuable insights are the ones we never saw coming. Now it's our turn to figure out how to make these experiences even better.