Addressing the trove of feedback we got from WAN Show listeners

Our personalized product recommendation startup, PerfectRec, was featured at the top of the WAN Show broadcast “I Think Apple Lied” on July 15 for research we published about Apple’s implausible iPhone satisfaction claims. Following the segment, we got a surge of new users to the site and a ton of helpful feedback. 

At a lot of companies, user feedback disappears into an abyss. But at PerfectRec, we take feedback very seriously and are trying to substantively address as much of it as we can. I wanted to thank you for your feedback and share an update about what we’ve addressed, what we are working on, and what we’ve decided to table for now. I hope this sheds some light on how we use feedback to build better recommender systems and maybe even inspires you to try the site again and share your thoughts.

In the days following the WAN show appearance, we received 235 feedback form submissions, including 168 substantive written comments. This represented both a huge surge of feedback but also much more technical feedback than we were typically getting due to tech-savvy, enthusiast nature of the WAN Show audience. 

The feedback fell into four main categories: 

• Feedback on recommendation quality

• Requests to consider additional products

• Requests for new features

• Requests for a headphone jack filter 

The challenge addressing product preference feedback

PerfectRec is a personalized product recommendation site. We combine human product expertise with machine learning and an interactive questionnaire to make personalized recommendations that are better and faster than what you could find with traditional research. Our goal is to do this well enough that “the gadget person” in every family might refer people to PerfectRec when they get questions about what [phone / TV / laptop / headphones / etc] to buy.

From a user experience standpoint, that means we are catering to two audiences at the same time. We need to be fast, accessible and easy for non-technical users but also have the details and features that enthusiasts might be looking for. 

To give one concrete example, let’s take a look at headphone jack support on phones. Adding a filter for phones with headphone jacks was the single most common feedback from LTT users.  As we all know, many phones no longer support the 3.5mm audio jack. When we first launched our Phone Recommendation Engine, we included a filter option labeled “Headphone jack” in our questionnaire. Our assumption was that users who really valued having a headphone jack would select this filter, but the majority of users skip it. We were wrong about that. It turned out that many less technical users would see the list of filter options, which includes preferences like Dual Sim, SD Card, 120+ hertz refresh rate, recognize the term “headphone jack” and select it, feeling like they should choose something. 

The result was that users would go from having recognizable phone recommendations: iPhone 14 / Galaxy S22 etc, to all of a sudden being recommended less common phones like the Xperia I V, which I think we can all agree isn’t a great recommendation for someone who would otherwise be looking at an iPhone. 

We addressed this problem initially by removing the headphone jack filter. That seemed to work for a time with our typical users, but the single most common feedback we received from WAN Show listeners was “I want a headphone jack option.” 

To strike a balance between the user feedback from WAN Show listeners and the problems we observed with less technical users, we decided to add back the headphone jack filter, but rename it 3.5mm jack. We also added a notice indicating how many phones have the selected feature. Our theory, which seems to have proven correct, is that by using the slightly more technical name we could make the filter available for users who really wanted it while avoiding the “I recognize that word” problem we observed earlier. 

We think we have done a reasonable job striking a balance between the needs of typical users and power users when addressing feedback, but we’d love to hear your take! We’ve added hundreds of new skus, and made lots of UI and recommender model improvements since June 15. The LTT community’s feedback was essential to making the site better, and we’d love to know how you think we’ve done and what more we can still improve. 

Feedback Checklist

Below is a summary of all the feedback we received from the LTT community, divided into a few tables.

• Addressed - We believe our solution fully addresses the feedback. 

• Partially Addressed - We’ve implemented a solution that helps with the issue, but might not fully address it. Examples of this include adding additional stats - such as whether or not a phone has a camera notch – to our compare view, but not asking a question about it directly. 

• In Progress - These are items we are working on, but haven’t launched yet. 

• Outstanding - These are items we would like to address, but aren’t at the top of our list right now since we are prioritizing projects we think will have a larger impact for users. 

• Not Addressing - These are items we don’t plan to address in the near future.

Addressed Issues

Partially Addressed Issues

In Progress Issues

Outstanding Issues

Not Addressing