How We Ran Usability A|B testing for ULB's New Student App Design

It is always great when universities invest in improving their students' experience on campus. This has become especially easy to do with digital transformation of processes. Doesn't it sound cool if students can have access to all university-related information just in one application? It sure does! But how should the perfect student app look like? How to make sure the application design is user-friendly and that students will use it before investing a substantial budget in the final product? — This is something ULB and weave.ly figured out with the help of prototype testing.

ULB's New Student App Idea

The Université libre de Bruxelles (ULB) is one of the biggest universities in Belgium with over 40000 students and faculty members. Over the years it has accumulated dozens of different online services catering to the various needs of its 4 campuses. Some of them are restaurant menus, library occupancy and sporting events.

Thanks to the CampusLife project supported by the European Regional Development Fund, ULB sought to combine the student-facing services in a single mobile application. This way, each student is able to customise the application to their needs choosing which services to include on their app’s home screen.

On top of the existing services, the team decided to add a new online service in the Campus Life app: Fix My Campus. This feature lets students report problems with campus infrastructure directly to campus facility services in a few clicks. Description of the issue are also combined with precise data such as pictures and geo-location.

After the ULB team came up with a concept, the app development had to follow. The design was of great importance for the university and they decided to focus on its usability testing to ensure a good quality of the product.

Quantitatively A|B Testing Designs of the App

Two versions of the app design for the usability test

At the next stage it was up to what kind of interface is more preferred by the students. Two designs were on the table. The first design enables customisation of the home screen by long pressing on any service tile. By dragging it around you can change its placement in the home screen or remove it.

The second design suggests going to a setting menu and edit the home screen by dragging and dropping tiles.

At this testing stage, weave.ly has entered. Prototypes were created for each design to run a usability test among students.

Two groups of around 20 students completed certain tasks in two separate prototypes. These prototypes automatically tracked a number of usability metrics like click heat maps or  time spent on each screen.

usability metrics of the prototype

As a result, the collected metrics and discussions with the students successfully determined which design suited its end users best! In addition, the experiment enabled the team to justify this decision towards all stakeholders too.

Qualitative Validating Core Ideas in the Field

The ULB team came up with the idea of the “FixMyCampus” service but had not validated its value proposition with its end users: students and campus facility staff members.

The team decided to setup a real-life experiment involving both students and campus facility staff. Thanks to weave.ly's plugin, a fully functional first prototype of FixMyCampus was built. A team of 10 students used it for half a day walking around campus and making reports on infrastructure issues. (e.g. overflowing trash cans or broken benches)

Student of Brussels Libre University testing prototype design of the app

We then presented the reports to the campus facility staff.  If you look at the screenshot below, you can see the collected data that verifies the validity and usefulness of the research.

Validity of the collected data of the experiment presented to the university staff

This real-life user test enabled the team to validate a crucial assumption. We proved that students would be willing to issue reports and that facility staff members would find these reports useful. Moreover, it also brought a number of unforeseen concerns to the surface. For example, student privacy regarding taking pictures on campus, creation of fake “prank” reports by students, etc.

Following this user-research phase an initial version of the CampusLife application was implemented and deployed at the start of the 2021-2022 academic year. It’s available for download for both iOS and Android.

Also want your prototypes to do the validation work for you?

Weave.ly is a no-code prototyping and user research platform. It enables you to turn existing Figma designs into fully functional prototypes that automatically collect the usability data you need to support your user research and innovation efforts. Wondering how to use weave.ly in your next project?

Weave.ly turns your Figma design into working web apps