Asking the Right Answers [#44]

I have been taking part in Google Rewards for over a year now. For the most part, I complete the various surveys to feed an ongoing habit without feeling like I’m being too indulgent or wasting money. It’s a fast and easy way to make a bit of completely disposable income and, honestly, the service works well.

Broadly, the surveys I get fall into three categories: store feedback, google reviews and marketing surveys. Store feedback is usually a case of confirming that I visited a given location and then rating them out of five. It’s quick, interesting enough to see which businesses feel the service is worthwhile and lets me provide some limited feedback. I don’t really imagine that the data is all that worthwhile, but enough stores do it, some of which having done so for an entire year at this point, that they must get something from the results.

Google reviews are a little more tedious but also have a higher reward, so I quite enjoy receiving them. I’m one of those people that routinely reviews online purchases, fills out in-store questionnaires and generally says “yes” when asked if I have a minute. I totally understand why most people ignore these types of things, but I try to do them whenever I have spare time for two main reasons. The first is that I’ve worked retail, I’ve been the person with the clipboard and I am fully aware how much that role sucks. I literally spent two months, for 4-5 hours a day, wandering around Durham trying to get people interested in taking a flyer for a store I worked for, and that was difficult enough. Getting people to actually engage with you for longer than ten seconds… that sounds like hell on Earth. The second reason is that I like having a record of my opinions, which should be fairly obvious from this website (and elsewhere), and that extends out to the services I’ve used and the items I’ve purchased.

So, the first two groups are easy for me to understand and pretty common. But once every month or so I’ll get a survey from group three: marketing research. Not market research, but questioning me on the adverts that I remember having seen or my awareness of brands. I imagine most of these are Google trying to gauge how well its own advertising algorithms are, something which is totally apparent when I get a survey like the one I received this morning.

That survey was incredibly quick and began by showing me a thumbnail of a Youtube video by Philip DeFranco. The video was several years old (I could see the uploaded date on the image) and the survey wanted to know if I had watched it. Now, I’ve been subscribed to Phil since I first created a Youtube account back in 2009 and had already been watching him for over a year before that. I quite literally created my account just to be able to track which of his back catalogue of videos I had watched. As a result, I could say with pretty high certainty that I had watched the video they were showing me. I also assume, considering that Youtube is tied to my Google account, that they already knew that I had watched the video. The first question on these surveys tend to request confirmation of known information, so that made sense.

But then they did something which I don’t understand, at all. I think what they were trying to do was refine their suggested videos algorithm but the way they went about it was just weird. There were two more questions to the survey and both showed another thumbnail of one of Phil’s videos from over a year ago. Both asked me to rate, out of five, how useful these would be as suggested videos on Youtube. Now, I don’t propose to understand the exact results or answers Google are looking for here, but I can imagine that they’re hoping to confirm that, yes, someone who wants to watch a video on current affairs would like to watch more videos on current affairs. The problem, though, is that their survey is completely ignoring my own video watching history. I am subscribed to Phil’s channel; I have watched every video he’s uploaded in the past decade. I don’t need to have his old videos suggested to me because I’ve already seen them. However, none of that information has been requested by the survey, so from the perspective of the questions I’ve been asked then, yes, based on the fact I enjoyed watching the first video I would want the other two videos to be suggested.

Yesterday I was reading an A List Apart article on why asking the right questions in user testing is key to not screwing up. Perhaps because that was on my mind, this survey through me round a loop. On a personal level, completely honestly, those videos are useless suggestions to me and I would have liked to rate them 0 out of 5 (which is, irritatingly, never an option). However, I’m a huge fan of Phil and want his channel to keep growing. Saying “Yes, I watched that one video of his and never want to watch another” seems wrong. I don’t want Google to take that message away from this survey. On the other hand, I hate how my current suggested videos feed is full of videos I’ve already seen and content from channels I’m already subscribed to. It’s a personal pet peeve of the current Youtube setup because it makes that page incredibly pointless, so I really don’t want to reinforce that behaviour and say that these are good suggestions.

At this point, I’m definitely over analysing what’s going on, but you would hope a company the size of Google would understand that the way they present a survey will have differing impacts. The questions are needlessly broad and non-specific, leaving the interpretation open to the user, but the subject matter leaves me stuck trying to guess what data Google actually want from me. Do they want me to know if I like those types of videos or do they want me to ‘confirm’ that suggesting other videos from channels I’ve watched before is a good thing? Unfortunately, I don’t know which it is, which means I don’t really know what the question is, and if I don’t know that, how can I answer it?

In the end, I just stuck them both at 4/5 stars. Typing this up now I feel that was probably the wrong thing to do, but oh well. At the end of the day, Google asked what seems like a fairly innocuous question, but one which has two wildly different answers. I doubt I’m the only person getting that question but I’ll probably be an outlier in my response. Still, it’s a prime example of where the phrasing, setting and simplicity of a question can leave it horribly ambiguous. The result will likely go on to inform some form of policy at Youtube, which is a shame, because no matter what question they thought they were asking I doubt it’s the one they’re actually having answered.

Who Watches the Users?

How do you decide which use cases you should support and which you shouldn’t? This question has been hovering in the back of my mind for quite a while now, because it seems to be increasingly forgotten. When apps or websites are launched, they tend to have a specific goal in mind, a solution to a problem the developers have identified. But does this take centre stage during a redesign? What if your users believe you’re providing a different solution to the one you had intended?

These are very real problems and I don’t think tech firms are treating them with, well, any care at all. Facebook’s constant redesigns have largely worked but I also find myself looking for ways to refine what appears on my wall. Adverts I can manage, but scrolling through an entire page of content about people I’ve never heard of (friends-of-friends and liked activity) is boring, and boredom means my time on the service declines. Facebook appears to have forgotten its core goal: to connect you to your friends and make communication simple. Now it just wants to connect you, period, which ultimately just results in a lot of noise.

Similarly, Youtube have just redesigned their smart TV app. I like the redesign, but Subscriptions are now hidden behind a menu rather than being on the home screen. Youtube may think that Search or Recommendations are more important, but that’s not how I use their service. Hiding the content I want behind two menus (i.e. two clicks away) is bad practice on a PC, but ultimately manageable. On a smart TV, with clunky controls and latency issues, every click counts. A barrier has been placed in between me and what I want to do. Far from solving my problem, Youtube have just created another one.

On the flipside, Instagram recently implemented the ability to control multiple accounts from within their app. When Instagram first launched it enabled you to take a photograph and share it with your friends; over time though, it has become a portfolio platform that people run businesses from. The inclusion of account switching confirms that Instagram are actually listening to their users and, crucially, how those users are actually using the service. They may even end up enticing some of those business users to setup secondary, personal accounts as a result, furthering their initial concept. They’ve managed to branch out their service to solve not just one, but two problems, simultaneously.

And yet… despite this positive step, searching your own photos on Instagram is practically impossible. I can’t be the only one wanting this functionality (though I imagine my own personal use case is a little… unusual) because an entire industry has evolved to fit this niche. Yet Instagram themselves aren’t listening. There is a disconnect here between how I want to use the service and the way the developers think the service is used. Right now, I’m willing to put up with that and hope for a better future. But given time, as with Facebook, doubts will begin to gnaw. And unlike Facebook, I have no intimate connection with Instagram; cutting that cord would be fairly painless.

Drawing the UI for the IoT? There’s an App for that!

Forget UI design, let the user decide instead.

I absolutely love this concept from Marc Exposito which combines the best of IFTTT and augmented reality to allow you to quickly, simply and visually control Internet-of-Things enabled devices. I’ve yet to find any IoT gadgets that actually warrant a purchase (personally) but as the field expands and becomes more ubiquitous, apps such as Drawit will in turn become more and more powerful. These are the types of problems and subtle interactions that I can see the IoT, tablets, wearables etc. really embracing in the near future.

Toshl Finance

Toshl is one of those weird little apps that, on paper, appear extremely useful but which I’ve never quite clicked with. On at least three separate occasions over the past year I’ve signed up for a free account, started setting it up, hit a road block and ultimately never returned. Every time, it goes like this: I’ll get an email regarding some new feature, think little of it and move on. Then, a couple of weeks later, I’ll decide I need to sort out my life and *bing*, up pops Toshl in my recent memory. “Perfect” I think, restart my account and… well, ultimately, remember some reason why the service isn’t right for me and abandon it again.

It happened again over the weekend. My partner and I are in the planning phase of a big holiday, unlike anything we’ve done together since officially entering “adulthood” and it’s becoming apparent that we’ll need to start budgeting and tracking our finances in order that we pull it all off. We were discussing this on Saturday when Toshl, obviously, popped into my head. It was only about a week ago that I got an email from them about some new feature (genuinely no idea what), so it seemed like fate. I fired up my account, set up a couple of payments, recorded all our monthly outgoings and began to get a decent overview of what we were spending money on. Recording every little purchase was a bit of a pain though, as it requires “adding” each one individually, which involves a number of dropdown boxes and deciding on stuff like categories (which didn’t always work: car, for example, should not be a “tag” or subgroup), and simply didn’t seem time effective. It was much quicker to create a Google Sheet, import our bank statements and quickly move cells around – plus we can both work on it simultaneously without having to share passwords, logins etc. However, this worked nicely in conjunction with Toshl, so was more of a speed bump than a roadblock.

As a result, it looked like I’d finally found a use for Toshl. The inbuilt calculators and setup, though a little fiddly, are genuinely great for recurring amounts of money (both in and out of accounts) and allowed us to quickly see how much it genuinely cost for us to live. Once set up, it was also easy to see break downs of what portion of our outgoings were flexible and which were not (e.g. taxes and rent). Excited and ready to start putting all this neatly sorted data to good use, I began setting up some savings pots. That’s when I hit the roadblock.

A free Toshl account can only have 2 “Accounts”, i.e. bank accounts, and 2 “Budgets”, which we were going to use as savings pots. Only two of the latter, however, was too little to even get a feel for how they worked. I do understand why companies restrict certain features and offer “Pro” or premium experiences at a cost, especially data heavy cloud services like Toshl. However, in this instance, the “free” version feels too much like a chore to actually use and restricts you from what I would consider its “core” feature: the ability to set and monitor savings targets.

So here we are again. My latest liaison with Toshl is already over and the account is now permanently closed. We moved over the monthly outgoing/incoming calculations to the spreadsheet, added some new functionality to mimic savings pots and everything’s working pretty great. Is our solution as shiny and well designed as Toshl? No, absolutely not. But it’s a lot more flexible, gives us exactly what we need and makes it very easy to expand upon, both from the perspective of functionality and the data being stored.

To be clear, this is definitely not meant as a take-down of Toshl, its services or methodology. I think it’s a genuinely good service and, especially outside of the US, is one of the best variants of this business model out there. Ultimately, though, it doesn’t do enough of what I want it to do and I’m not willing to spend extra money on something without finding it indispensable, which Toshl just doesn’t achieve. Oh well, maybe I’ll find another use for it in after another few months have passed.