Asking the Right Answers [#43]

I have been taking part in Google Rewards for over a year now. For the most part, I complete the various surveys to feed an ongoing habit without feeling like I’m being too indulgent or wasting money. It’s a fast and easy way to make a bit of completely disposable income and, honestly, the service works well.

Broadly, the surveys I get fall into three categories: store feedback, google reviews and marketing surveys. Store feedback is usually a case of confirming that I visited a given location and then rating them out of five. It’s quick, interesting enough to see which businesses feel the service is worthwhile and lets me provide some limited feedback. I don’t really imagine that the data is all that worthwhile, but enough stores do it, some of which having done so for an entire year at this point, that they must get something from the results.

Google reviews are a little more tedious but also have a higher reward, so I quite enjoy receiving them. I’m one of those people that routinely reviews online purchases, fills out in-store questionnaires and generally says “yes” when asked if I have a minute. I totally understand why most people ignore these types of things, but I try to do them whenever I have spare time for two main reasons. The first is that I’ve worked retail, I’ve been the person with the clipboard and I am fully aware how much that role sucks. I literally spent two months, for 4-5 hours a day, wandering around Durham trying to get people interested in taking a flyer for a store I worked for, and that was difficult enough. Getting people to actually engage with you for longer than ten seconds… that sounds like hell on Earth. The second reason is that I like having a record of my opinions, which should be fairly obvious from this website (and elsewhere), and that extends out to the services I’ve used and the items I’ve purchased.

So, the first two groups are easy for me to understand and pretty common. But once every month or so I’ll get a survey from group three: marketing research. Not market research, but questioning me on the adverts that I remember having seen or my awareness of brands. I imagine most of these are Google trying to gauge how well its own advertising algorithms are, something which is totally apparent when I get a survey like the one I received this morning.

That survey was incredibly quick and began by showing me a thumbnail of a Youtube video by Philip DeFranco. The video was several years old (I could see the uploaded date on the image) and the survey wanted to know if I had watched it. Now, I’ve been subscribed to Phil since I first created a Youtube account back in 2009 and had already been watching him for over a year before that. I quite literally created my account just to be able to track which of his back catalogue of videos I had watched. As a result, I could say with pretty high certainty that I had watched the video they were showing me. I also assume, considering that Youtube is tied to my Google account, that they already knew that I had watched the video. The first question on these surveys tend to request confirmation of known information, so that made sense.

But then they did something which I don’t understand, at all. I think what they were trying to do was refine their suggested videos algorithm but the way they went about it was just weird. There were two more questions to the survey and both showed another thumbnail of one of Phil’s videos from over a year ago. Both asked me to rate, out of five, how useful these would be as suggested videos on Youtube. Now, I don’t propose to understand the exact results or answers Google are looking for here, but I can imagine that they’re hoping to confirm that, yes, someone who wants to watch a video on current affairs would like to watch more videos on current affairs. The problem, though, is that their survey is completely ignoring my own video watching history. I am subscribed to Phil’s channel; I have watched every video he’s uploaded in the past decade. I don’t need to have his old videos suggested to me because I’ve already seen them. However, none of that information has been requested by the survey, so from the perspective of the questions I’ve been asked then, yes, based on the fact I enjoyed watching the first video I would want the other two videos to be suggested.

Yesterday I was reading an A List Apart article on why asking the right questions in user testing is key to not screwing up. Perhaps because that was on my mind, this survey through me round a loop. On a personal level, completely honestly, those videos are useless suggestions to me and I would have liked to rate them 0 out of 5 (which is, irritatingly, never an option). However, I’m a huge fan of Phil and want his channel to keep growing. Saying “Yes, I watched that one video of his and never want to watch another” seems wrong. I don’t want Google to take that message away from this survey. On the other hand, I hate how my current suggested videos feed is full of videos I’ve already seen and content from channels I’m already subscribed to. It’s a personal pet peeve of the current Youtube setup because it makes that page incredibly pointless, so I really don’t want to reinforce that behaviour and say that these are good suggestions.

At this point, I’m definitely over analysing what’s going on, but you would hope a company the size of Google would understand that the way they present a survey will have differing impacts. The questions are needlessly broad and non-specific, leaving the interpretation open to the user, but the subject matter leaves me stuck trying to guess what data Google actually want from me. Do they want me to know if I like those types of videos or do they want me to ‘confirm’ that suggesting other videos from channels I’ve watched before is a good thing? Unfortunately, I don’t know which it is, which means I don’t really know what the question is, and if I don’t know that, how can I answer it?

In the end, I just stuck them both at 4/5 stars. Typing this up now I feel that was probably the wrong thing to do, but oh well. At the end of the day, Google asked what seems like a fairly innocuous question, but one which has two wildly different answers. I doubt I’m the only person getting that question but I’ll probably be an outlier in my response. Still, it’s a prime example of where the phrasing, setting and simplicity of a question can leave it horribly ambiguous. The result will likely go on to inform some form of policy at Youtube, which is a shame, because no matter what question they thought they were asking I doubt it’s the one they’re actually having answered.

Stickers, Eclipses and Lighthouses [#35]

Today is a day for another round-up of interesting pieces from across the web. Nothing too special, but hopefully a little intriguing.

First up is Google Lighthouse, one of the many branches of the Alphabet behemoth and a pretty interesting little project. I haven’t actually managed to get it up and running, but I’ll definitely be trying it out on theAdhocracy some time soon (and probably weeping at the result). I don’t need to test it, though, to see it will be a very useful tool in battling the increasingly problematic issue of internet lag.

Second is the article which led me to Lighthouse in the first place: AMPersan, by Ethan Marcotte. Not much to add to this one, just another voice adding weight to my uneasiness with the idea of AMP and similar projects. Well worth a read if you’re interested in the open web.

In third place is a collection of ‘achievement’ stickers doing the rounds of the blogosphere right now. Originally designed by Jeremy Nguyen, published on The New Yorker and personally discovered via TheLogoSmith, the stickers are a humorous look at the pitfalls of being self employed. They’re specifically designed for freelance designers, but I feel a lot of them are applicable across disciplines. If you work from home, you’ll probably find yourself smiling and nodding.

Fourth on the list is a simple article from Martian Craft outlining “The Importance of Routine“. The post is aimed at remote works and is far from news to me, but it is a well written example of how to apply this kind of thinking. I’m saving it here more to try and force myself into setting something like this up for my own free time.

Finally, I was blown away by the “Lifetime Eclipse Predictor” visualisation created for The Washington Post (discovered via Source). In the wake of the recent total eclipse in the US, along with reading various posts on the rarity of such events, I’ve been left with a real urge to try and make sure at my path eventually coincides with a path of totality. It is a ridiculously awesome coincidence that our moon’s diameter and planet’s solar distance align so accurately. I mean, even if there are other life-hosting planets out there, we’re certainly one of an incredibly small number that can witness this phenomenon. That makes it practically a responsibility to see a total eclipse, at least once.

That Anti-Diversity Googler & Self Introspection [#31]

Standard workday, standard work lunch catching up on RSS feeds. Of course, quite a few of them are discussing the leaked “Anti-Diversity” manifesto from the, now infamous, ex-Google employee (name forgotten and ultimately unimportant). It’s been an interesting view into a very specific bubble of the tech sphere, but one which has helped elucidate the issue, if only a little.

Of particular note is the response from Adactio, which is easily understood by the title of the piece: “Intolerable“. I will hold my hand up right now and say that I find the whole issue a lot more complex than Jeremy Keith outlines, but I cannot argue with his conclusion. Nor can I argue with the incredibly diverse and well-written sources he links to, each of which is definitely worth a read.

That becomes particularly true if you’re anything like me: someone whose gut instinct was “this is utterly wrong”, but who found themselves wondering if, beneath the anger, fear and sexism, a valid point was lurking. Having now read through the links (linked below) I feel a little more confident in my gut reaction, which is a nice feeling.

Just to clarify my use of the phrase “valid point”, it is not valid that one gender is in any way better or worse at being involved in the tech sector (or any sector, for that matter). Instead, it’s more of an issue of how we go about addressing the very real disparities between both job prospects and job uptake by any dissuaded minority group (and yes, women are not a literal minority, but they are in tech due to centuries of discrimination, so I feel it a valid term within context). I have a personal distaste for anything that borders on “positive discrimination”. All it creates, long term, is embitterment and injustice, in my opinion. However, having read the links below I feel a lot more at ease that the diversity programmes at Google and similar companies are not going down this route, instead focusing on making the workplace a more attractive environment for everybody. That’s something I can get behind.

If there is one element of Keith’s article that I will find fault with, it’s the blanket tone of dismissal. I understand where he’s coming from and it’s a tricky thing to call out, because it’s an opinion I find myself feeling towards other subjects. I simply don’t feel the world is ever black and white enough to make a statement like:

I refuse to debate this. Does that make me inflexible? Yep, sure does.

But, hypocritically, I also find myself agreeing with the directly following statement:

But, y’know, not everything is worthy of debate. When the very premise of the discussion is harmful, all appeals to impartiality ring hollow.

As an example, earlier this week the BBC came under fire for featuring Lord Lawson on a program about climate science. The argument for his presence is that it provides “the other side of the debate” and that the BBC have a mandate to be as impartial as possible. The issue with their reasoning is that it implies there is a debate to be had. In terms of scientific consensus, the degree to which man-made climate change is refuted is utterly negligible. The debate has been settled for decades and continuing to present it in any other way is directly harmful. It is akin, though less instantly vitriolic, to claiming that the BBC needs to include a Holocaust denier in documentaries on WWII. Yes, there are some people out there who believe that the vast majority of historians are wrong, but no organisation in their right-mind would claim that there is an actual debate soliciting both sides being heard.

Perhaps, then, it is I who is wrong on the Anti-Diversity Manifesto. Perhaps Keith is right and any discussion of non-diversity is, by its nature, only destructive and harmful because that debate, too, has been settled. Still, I can’t help but feel that claiming so and shouting it so loudly only serves to reinforce the opinions of dissenters. It’s hypocritical of me, but I don’t feel that shutting down people with these opinions is the right course of action. Perhaps, in time, that will change. For now, I’m just happy to see that the discussion being had is largely positive.

Reading List:

A Brief History of Women in Computing – Faruk Ates

So About This Googlers Manifesto – Yonatan Zunger

Dissecting the Google Employees Anti-Diversity Manifesto – Ether Alali