Why augmented reality is garbage, and how to make it transcendent and good and funny, OR a lesson in wielding the magic of socio-technical ergonomics

Once again we are on the cusp of a new world; the only big tech company that gives a damn about human-centred design will soon bring their next revolutionary device to market. Yes, that’s Apple; and yes, it’s their forthcoming augmented reality headset.

I’ve been here before, writing on this side of the reality shear; in the lead-up to the original iPad reveal, when every tablet computer was still garbage, I had spent several months in phenomenological enquiry. That means I had unaccountably been walking around with weighted mockups of various form factors for imaginary tablets in my pockets and satchels, to feel their heft and discover the affordances and constraints they would reveal; to sense the range of possibilities before it was too late.

Because as I predicted at the time, at the moment of the iPad release the probability waveforms collapsed. We are now in a world where it’s nearly impossible to imagine tablet computing working any other way. Every competitor now apes the iPad, presenting a grid of icons in a magazine-sized glass rectangle with capacitive touch interaction. The only substantial improvement has been Apple’s own introduction of a stylus. Microsoft’s innovative Duo book concept impacted on the surface and was never seen again. Maybe one day someone will make a good folding one.

The situation today is similar: none of the augmented reality devices so far have been good enough, because no-one is being serious about human-centred design.

That wasn’t always true – many of the smaller experiments, born too early to survive, had incredible promise. Daqri with their smart helmets for construction sites; Steve Mann’s deeply considered longitudinal experiments in cybernetic augmentation; the many beautiful thoughtful bespoke systems that pop up at the conference on Tangible and Embedded Interaction or in a demo by some curious Japanese research lab at SIGGRAPH Emerging Technologies.

It is however sadly true of the big, well funded projects we’re seeing today. The Microsoft Hololens is technically impressive but unfocused – who is it meant to be for? Do you expect me to run Office apps on this thing? Why does Bing keep throwing a huge 2D browser window up at me? Do you people really think I should have to poke my corporate password into a floating keyboard before you’ll let me display an object? And where's the support for shared views? I want this to work so badly, but that's exactly how it does work.

The other great hope – Magicleap – figured they could get there with a commitment to beauty and learning, which would have worked if we lived in a just world.

But no, this is harder than that. You need to get the design right, but you also need to have chosen the right design. And just like last time, it’s all about the form factor.

The form factor is how the object presents itself to the user; how it fits their body and their life. It’s the size, weight, texture, behaviour under different conditions of light and noise and atmosphere. Fit and finish.

Until Bas Ording demonstrated the interaction potential of capacitive touch to Steve Jobs there could be no iPhone. And as the device went into development his collaborator Imran Chaudhri (now co-founder of the mysterious and compelling hu.ma.ne) maintained a relentless insistence that this aspect of the experience could not be compromised. As Ken Kocienda relates in the masterful Creative Selection, he would place a sheet of paper on a glass table and push it around with his finger. Like this! No perceptible delay, no break in the illusion of direct manipulation, not for anything.

The size and shape of the device was no less important. A deck of cards could have any form factor, but it’s a 4-inch diagonal rounded rectangle, because that’s what works for humans. The iPhone, like the iPod, is a deck of cards. The iPad is a sheet of paper.

What the hell is this?

Humans have chosen the form factors for things to put on our heads. Hats. We like hats. Helmets. Headbands and earmuffs and headphones. Goggles if necessary. Spectacles, if they’re light enough.

One thing we have never chosen is a vice. We don’t enjoy putting our head in a clamp and screwing it tight. Now, I can tell that what they think they’re going for here is spectacles. Because that’s what people look through, right?

No. Stop it. You’re doing this wrong.

We do not have the technology to make something lightweight enough to be acceptable as spectacles. The Hololens is over 500g. Let’s say Apple can engineer that right down to a quarter of the weight: 125g. Still no.

So – are there items between 125g and 250g that people are willing to wear on their heads?

Next, let’s talk about fashion.

Google Glass was launched with a fashion-forward strategy. It all seems like a fever dream now but I seem to remember Sergei Brin parachuting onto a stage at Fashion Week with a bevy of models all bedecked in cybernetic headsets. Because I guess that’s how the kids at Google reckon fashion works. Famous beautiful people wear things at high profile events, and everyone follows along, right?

Well. You think you know fashion, but fashion’s a stranger. You think fashion’s your friend; my friend – fashion is danger.

Everything I know about fashion I learned from Flight of the Conchords, so that’s not much. But I know a little about cultural creativity. The famous high-profile people usher it onto the stage, but they don’t make it happen. Fashion, like all creativity, emerges from a cultural milieu. Coco Chanel was working in a Paris where women who wanted to challenge gender and sexuality were already retailoring men’s uniforms for themselves. Her brand caught on because the moment was right.

Now to the magic spell. The simple incantation that can only be cast by those who are willing to grapple with the eldritch relationship between ergonomics, form factor and the Rogers Curve that describes the Diffusion of Innovation.

There are some known routes for a new kind of clothing to get into the mainstream. For example: street wear, dance wear, sportswear.

Of these, the most overwhelmingly correct path from a niche into the mainstream for augmented reality is sportswear.

People will wear all kinds of things in a sports or fitness context. Padding, lycra, helmets, visors, even fins. Fitness is virtuous and pragmatic; these things lower the threshold of acceptability. And then once people get used to it, the new form makes its way into everyday clothing.

Running changed shoes. Yoga changed pants. Augmented reality fitness is going to change the hat.

/eof

In December 1999 I walked in to the office of my boss, Tom Kennedy – CEO of Brainwaave, a pioneering multimedia agency. My partner and I had our first child on the way, and home life was suddenly feeling very important. So I asked for a four-day week at full pay. I'd been at Brainwaave for a couple of years and had been helping deliver some important projects. I pitched it as an alternative to giving me a raise, and they went for it.

I wish I could say that I stuck with this new level of work-life balance, but no – my next move after Brainwaave was to launch a virtual earth startup with dear friend and colleague Chris Thorne. Self-employed and trying to do something unprecedented, we felt driven to spend every waking moment and every cent of investors' money trying to work magic. It took four years and the purchase of our key competitor by Google before I understood that putting in the hours is really, really not what makes or breaks a startup. Rather, it's more likely to break the team.

I'm far from alone in this realisation, or in understanding that the hours don't make or break profitability for most companies in our knowledge economy. The buzz of the four-day week movement is becoming a clamour.

But there's no need to stop there. Robots are taking our jobs, and I say we should let them. Automation can provide a leisure society, if we choose to structure society to allow it.

As you can see productivity just keeps going up. This should be unsurprising:  John Maynard Keynes predicted in 1930 that the ongoing process of automation would deliver this result, and that it ought to deliver us a leisure society. What is surprising is that real wages haven't gone up in the same way (and we know that full-time hours have not decreased substantially either). What's going on? Well, since around 1971 the developed world has largely chosen to deliver the rewards of that productivity growth entirely to the owners of capital.

It's time for a correction. And I think the best framing for this is not less work, but more leisure. A four day week is a necessary first step, but the long weekend is the point.

In Australia we should begin by immediately instituting more national holidays. First a freebie to mark the beginning of the shift. Then to celebrate important milestones like the signing of a treaty between the Crown and our First Nations, and then for the birth of a just Republic; another to celebrate our Bill of Rights, our leadership of the global Climate Treaty, and so on. Keep going until every existing weekend runs for three days, and the old three-day weekends are four days. And don't stop. When we tip the balance to three days for work and four for leisure, family, friends, volunteering and personal projects, then I reckon we'll be getting somewhere.

We can start by getting the conversation going.

I started young, a nine-year-old kid teaching myself 6502 assembly language so that I could make the lights move on the KIM-1 microcomputer that my Dad brought home, a gift from a friend of his who worked at the CSIRO*. In those days, everything came with schematics and a programming manual. The wave caught me and I rode with it for the next forty years, and I hope to ride it for at least forty more.

It's been a visual and conceptual time, so it seemed appropriate to start mapping it all out in a giant zooming canvas. Here it is; a living document.

*And as you might expect I work there today, at Australia's Commonwealth Science and Industrial Research Organisation.

When you have a moment to be advised by someone who leads the world in a field of enquiry, it's good to listen closely. You might have been able to access the same wisdom from a peer or even from within yourself, but would you have listened as intently?

So earlier this month I had such a moment. I was chatting to Professor Lian Loke in a break at the Tangible and Embedded Interaction (TEI) conference that she was co-chairing, and keynote speaker Professor Hiroshi Ishii came over to say hello.

For me, this is not a casual matter. Hiroshi Ishii is the leading light in the world of tangible digital interaction, "giving physical form to digital information and computation". He produces groundbreaking, influential and beautiful work, and leads the Tangible Media group that he founded at the MIT Media Lab after he joined them in 1995.

I was saddened by the revelations of MIT’s ties to Jeffrey Epstein, including the secret acceptance of his money and influence at the Media Lab – in part because the Media Lab had meant so much to me. Stories from its early days inspired me to pursue my own interest in design and computing, back when exploring that intersection was considered eccentric at best. It's a storied place, and Professor Ishii's work is a shining thread in that story.

Lian asked Hiroshii a question. It was about the highly produced gallery exhibitions that he stages of his work. She wanted to know why he put so much time and effort into them, rather than focusing on books (or perhaps as is more usual for academics today, sheer volume of published papers). Lian is an artist herself so I am sure she had her own ideas. But here is the answer he gave.

"When you do you research", he set out, "it is most important to communicate. This is why I spend a lot of time and effort on the communication. But also when you communicate", and he tapped his chest twice – "you need to reach the heart".

So that is what I will take with me from TEI. To pause after completing the work, to spend the time to communicate it well, to understand who I am speaking to and to do that with focus and intent.

At this year's REMIX festival I had the pleasure of being interviewed by my dear friend Mark Pesce (alongside the marvellous Simone Chua from the light sculpture and immersive design studio Amigo and Amigo) on creative entrepreneurship – here is what I wish I'd known when I started out, and what I'm still trying to figure out.

For a while in the middle of the last decade I had the honour of leading the wonderful R&D team at the ABC, our national broadcaster. Now, one of the lovely things about working at the ABC is that internal any team has access to their peerless video production capability. When I said I wanted to shoot a design fiction my brilliant producer Astrid Scott got in touch with the production resources team, who knew exactly who to assign. We got a creative and thoughtful videographer and sound recordist who worked collaboratively with us to produce exactly the feel and story we were after.

This story is for an imaginary product – an ABC news feed delivered with your morning coffee, personalised just for you and printed on your receipt. We called it ABC Ticker.

To make the reactions authentic (and as a bit of a research probe) I put out the casting call on Twitter and didn't tell people what to expect. Then we actually built a custom ticker for each of the people based on what we could glean from their social media activity (which it turns out was quite a lot). We stayed up late the night before the shoot updating them so they would be fresh. We asked them to order a coffee and filmed their reactions in real time. So the delight you see in this video is the real thing.

The ABC Ticker never made it into the real world, but what we learnt went into a cycle of research that informed the ABC's approach to personalised news. In a later research phase we built Spoke – a series of experimental apps – launched  in controlled trials around the country. Spoke delivered hyper-localised and personalised news from multiple providers, and showed that it was OK to break a few rules along the way.

The ABC took a lot of that on board and I'm proud to see resonances of Spoke in their approach to news today. But there are still more lessons in that research, and in the spirit of open knowledge sharing we published it. So if you're at the ABC or elsewhere thinking about local and personal news, here's the full report: News, Place and Relevance.

Let’s start with 場.

In Japanese, 場 is pronounced “Ba”. I've spent some time in Japan so when I see that symbol, that's the sound it makes in my head. The character holds the same meaning in Chinese; in Mandarin it is pronounced “Chang” using the third tone. Dictionaries say it means “place” or “field”. However this is a very sparse translation.

In English we have various words like Place, with different meanings. The trinity are Location (and geolocation and coordinates and lat/long and locstat), Space and Place.

In architectural theory we might say that a place is a space with a history. But what about the present? What do we call a place where history is being created? A rich place full of interaction and possibility, a place for emerging relationships?

場 is useful because it holds a little more meaning than “place”. It connotes a dynamic, inhabited place; a place for doing something. Somewhere to hold a festival, or work on a project together. A collaborative place. Ba.

This “Ba-principle” is beautifully enunciated in a paper by Hiroshi Shimizu entitled “Ba-Principle: New Logic for the Real-time Emergence of Information,” [Holonics, 5/1 (1995):67-69]. Shigeki Amitani, a colleague at the Creativity and Cognition Studios, introduced me to the concept and to Shimizu’s paper. The article is not available online as far as I can tell, even in the academic databases. Shigeki assecuted a copy through inter-library loan, which I treasure. However there is a good website on ba.

I'm interested in 場 as a lever to help us understand the power of collaborative environments across our real and virtual worlds. There are plenty of good tools for synchronous and asynchronous communication between members of distributed teams. For asynchronous communication we have email, blogs, letters, grafitti and so on. For synchronous communication – phone, videoconferencing, instant messaging etc. This last (IM) is interesting because it also supports pervasive presence; you can see when your associates are online, and perhaps also something about their state. This affords opportunities for ad-hoc meetings, a key to creative collaboration. But chat is not collaboration – you can meet and talk, but you can’t do anything.

At the same time there is a flowering of asynchronous tools for distributed collaboration (the next step beyond communication). Google has bought a swathe of them over the years. So, what about synchronous collaboration? Of course we still have Second Life; I would say that this is a 場. So is World of Warcraft and any other MMPOG – and especially the constructive ones like Minecraft and Roblox. It is clear that millions of us are happy to switch on an immersive virtual world from time to time, and collaborate with others who are also telepresent in a shared space. There are also plenty of research papers showing that CVEs (Collaborative Virtual Environments) are effective as spaces for collaboration outside of games. Still, they’re not widely adopted for this purpose.

I have an abiding interest in these questions and how to address them. I was first trained as a designer so my early ideas focused on user experience and task flow. Usability and accessibility are important; it doesn’t matter how pretty your virtual world is if your users can’t get their tasks done in it. As I've started to lead and assemble creative teams I've become more interested in socio-technical approaches, particularly affordances and framing.

However we approach it, finding ways to create the sense of a living, dynamic place full of possibility is the heart of the problem of group creativity. Hence, 場.

After three years at the ABC, first as a UX and Innovation Strategy consultant and then as creative director of R&D, I’m moving on – taking on consulting work and writing up my PhD. And it’s wonderful to see how many amazing things are going on in the world. Sydney is flourishing: startups and innovation labs are sparking up all over the place, and virtual worlds are back in style – this time with mixed and augmented reality firmly in the mix.

I’m already consulting to three intriguing startups and giving a keynote at the SBS Leader Innovation Day. As I say, this city is not standing still.

So to my first appointment: I’m joining the fledgling UTS Innovation and Creative Intelligence unit to help figure out how we build adaptive resilience into our organisations, and to map the new landscape of collaborative creativity. It’s a truly incredible cross-disciplinary group, bringing together people from all across UTS and beyond. This is the kind of liminal zone where great things are possible.In my first week I’ll be participating in a 2-day Innovation and Creative Intelligence (ICI) Lab entitled Responding to Turbulence – Building Adaptive Resilience. How very apposite.

And then I’ll be wanting to talk to you about how you are adapting to and taking advantage of change. Seriously, I want to talk to you. Drop me a line.

Back in 2008 I presented a paper at OZCHI in Cairns, and met a few of my research heroes. I documented the experience on Twitter, but despite my optimism at the time there is no longer any trace of those tweets [2018 update: they're back at a new URL].

So this year when I attended the workshop on Public Art, Evaluation and HCI I left twitter behind and posted on alpha instead. So for now at least, you can read my observations of our various contributions there.

[2018 update: sadly the Twitter challenger Alpha is gone entirely - but the Internet Archive has it.]

Some notes from a brief interview on Radio National’s Future Tense.

Recently our national broadcaster hosted Edward Jung of Intellectual Ventures, a company that "has earned a special brand of hatred in the business world as the ultimate patent troll". IV have become infamous for perfecting the practice of buying up patents and demanding settlements while limiting their own exposure to countersuit by never making anything. 

This makes them a non-practicing patent-asserting entity (PAE), or "patent troll". A pejorative that was coined – in a special irony – by one of their own founders while working the other side of the courtroom for Intel.

Quoth the EFF:

Though it claims to promote innovation, Intellectual Ventures is behind some of the most outrageous troll campaigns in recent years. Famous for hiding behind thousands of shell companies, it spawned Lodsys, the troll that harassed small app developers, and the Oasis Research litigation featured in This American Life.

It struck me as unusual that the ABC didn't cover this most well-known aspect of IV's work, instead allowing Jung to frame himself as an expert on creativity and innovation, and to perpetuate some myths – both about how creativity really works, and also about how the patent system treats it. So this week they kindly allowed me to present a brief counterpoint:

Edward Jung described creativity as this process where there is a person sitting alone in a room and an idea comes to them and they send it out to the world. Now, that’s a very outmoded idea of creativity and it’s very odd, but it is a lot like the way the patent system sees creativity.

As other speakers on your show mentioned, creativity is actually a social process, it happens within and among people. Every idea is built upon the huge substructure of other ideas. As Newton said, he stood on the shoulders of giants. As do we all.

So there’s this social process going on, and when something is invented somewhere in the world it’s very likely that it is being simultaneously invented by other people.

[full transcript here - yes, on the wrong page and you must scroll down and hit TRANSCRIPT to reveal it] and [full episode audio here]

Intellectual property has a purpose. That purpose is clearest when you see it used to regulate commercial disputes between large and similarly armed conglomerates, and most self-evidently broken when deployed by powerful entitities against weaker ones. Patent trolls reveal a particular bug in the system – non-practicing entities are stronger because they have no attack surface. Today's products and services contain huge numbers of innovations within them, so if they put any of their patents into actual products then they could be counter-sued by other patent holders. This gives them an incentive to hoard, the very opposite of promoting "the Progress of Science and useful Arts".

The huge edifice of IP law hasn't stopped innovation. It has become so complex and interwoven with industry that I can't tell, today, if it is a net gain or a net loss. But one thing is sure: in practice it is deeply skewed towards some actors more than others, without strong correlation to their usefulness to society or to innovation. It could do with some radical unskewing.

Just back from Creativity and Cognition 2011, which was truly ace. I gave the paper I wrote with my co-supervisor, Prof. Ernest Edmonds, which people seemed to like. Saw some thought-provoking presentations and met a number of inspiring and wonderful people. Everything one could wish for in a conference, really.

Guy Claxton gave a truly thoughtful keynote. Creative-Mindedness: When Technology Helps and When It Hinders. He pointed out that formal education as it’s currently instituted systematically destroys the creative habits of mind. In response to a question on how precisely it does this, he referred to his chart of those habits. For example, one creative habit is inquisitiveness, which is damaged by the focus in structured curricula on requiring students to study questions they have not asked. Another is creative stamina & resilience (exemplified by Einstein, who said that it was not so much that he was especially clever, but more that he stayed with problems for longer). This is damaged by the scheduling of classes that require every problem to be solved in an hour.

The papers continued through the next few days – but there were also a  lot of excellent posters. Apparently as there was only a single track for papers, the organisers could not accept some submissions that were actually very good, so those people were encouraged to resubmit as posters. Which meant that the quality of work in the posters was pretty impressive.

Of course, it’s Creativity and Cognition so there was also room for art – my favourite works were Matt Ruby’s Sympathy for Pacman and Jack Stenner & Patrick LeMieux’s Open House: Interaction as Critical Reflection. To top it off, the conference was held at Atlanta’s High Museum of Art, and we were permitted an after-hours tour. As well as some tragically unmoving Calder mobiles (which really don’t belong in temperature controlled rooms), there on a wall was perhaps my favourite artwork of all time: Duchamp’s L.H.O.O.Q. Shaved. Yes, you have to know the story for this one to work properly.

So finally: a few people asked for my slides, so after the break I’ll embed a Quicktime movie of them. Thank you everyone at C&C 2011, and especially the erstwhile organisers for providing such a great atmosphere for collaboration and creativity.

Participants:

• Tim McLaughlin – Texas A&M University
• Tommy Burnette – Lucasfilm Singapore
• Tim Fields – Certain Affinity
• Jonathan Gibbs – DreamWorks Animation
• David Parrish – Reel FX Creative Studios

People have different communication styles: some do well face to face, others work better remotely – perhaps they are better at written communication, or on webcam.

If we’re in the same room, it’s easier to correct misapprehensions: “splinters become much bigger wounds”. We form tribes, so be careful to make it one tribe instead of us (here) vs them (remote). During time apart, misunderstandings can snowball.

It’s important to have diffusers in the group, reminding people to chill out.

Watch out for “the game of telephone” – you have to be able to talk peer to peer. Not only management talking to each other.

Time zone difference can work very positively – problems solved for you while you sleep. But it’s harder to collaborate as well, and work doesn’t fit neatly into discrete 8-hour chunks. Sometimes there is a specialist – only one person can fix the problem – so if they’re unavailable in an emergency it’s a problem.

Right now our production pipeline has som many restrictions based on how the elements plug in, and you can’t go back up the chain very easily. This inhibits non-linear collaboration.

There is a tension between the creative process and the factory line method for producing work. But the assembly line is efficient when you have a lot of work to do. Efficiency drives the process.

I don’t like the word pipeline; because it doesn’t just flow one way, particularly early in the show. Late in the show things tend to settle down, you don’t need as much collaboration because everyone knows what they’re doing. It’s a piece of the communication issue.

We have groups where everyone is in one place and one person is not, because they’re the right person for the job. When we do that it’s because their creative skill outweighs the difficulty of working with them across distance.

Q. I’m searching for this holy grail online collaboration tool – where does all your stuff live, how do you coordinate all this stuff?

A. As a small factory, communication seems to work OK, asset management is harder.
A. As a large shop, we have entirely custom asset management and it works well; communication is now the problem.
A. When you work with multiple clients, it’s different every time – for every type of collaboration we build a custom tool set. important to document the process.

Especially when focusing on creative work, you have to balance the rules with how the artist wants to work, so you don’t overconstrain that artist – it’s a balancing act. The age-old problem: how do I schedule creativity?

We can’t separate disciplines. Lighting department does rendering, compositing, colour correction etc. Handoff is difficult. You need to keep it alive.

Q. Does distance collaboration make our workplaces friendlier to women and ethnic minorities?

A. Don’t know if it’s better or worse, but it’s an open door.

It’s good for the film industry if films are not all made in California – a wider variety of kinds of people will bring richness to the work. People who choose to live elsewhere can still be part of the process.

In our studio in Singapore, we have more than forty countries represented.

Distant collaboration forces us into greater cultural sensitivity.

So far this is just beginning, but perhaps it is a precursor to a new, more thoroughly distributed future.

At Lucasfilm, we double up on supervisors – make sure there is one at each location so artists can get immediate feedback.

With a properly shared vision, I can give more people the power to make approvals.

Dreamworks: trusted luitenants are important. If something is approved, but then goes up the chain to the Director and back down to be redone, that’s no good. This is part of trust.

Q. Is there a minimum cell size? How independent can cells be?

A. The minimum size is a function of the artist. With some artists, the minimum size is 1. But not everyone works best alone.

A. I’m not going to build a light farm in a location with two guys.

A. But maybe data transfer speed increases will change that.

A. How invested people are in the goals of the company has a big effect – people can be more independent if they’re more invested.

In architecture school, I learned how to give and accept review feedback. We need to figure out how to learn these same skills for remote collaboration.

Cultural differences will always exist – and you want them, they bring different approaches to problems. Everyone needs to feel that it’s “our project”. It’s important to fully uderstand cultural differences.

In some countries “I’m not sure” means “hell no”. Managers need to have some maturity and worldliness.

Universities must make cross-disciplinary groups, and must continue with the dreaded group grade, because that’s the world we all live in. The product succeeds or fails as a whole.

Sometimes you need to have times when you’re all in a room with no time limit, to work until something is creatively resolved. It’s hard (though not impossible) to do that at a distance.

Q. Donna Cox, NCSA: Can you describe any novel proprietary collaborative software you’ve created? Do you collaborate with scientists ever?

Tommy Burnette: We have a close relationship with Stanford, we have students from there who are pretty much on staff.

Tim Fields: In the games business we often hire PhD physicists

Gibbs: We do, but have never worked with people studying this kind of problem – collaboration and communication – scientifically.

Parrish: We have developed software to allow us to hande large feature film projects without requiring a large coordination staff. It allows our supervisors to give feedback to animators, for example, over the web. Animators, modelers and riggers. It tracks our financials, tracks every detail of every project; producers use it to keep on top of budget and make sure artists aren’t working crazy hours.

Gibbs: It is crucial first to have a system that helps you be very clear about what you’re looking at. [sync]

Q: Parrish: Is collaboration making what we do better, or is it just a necessity?

A. Burnette: if it gives us access to talent we wouldn’t otherwise, it makes it better

A: McLaughlin: Being in Texas, it’s made a massive difference. Gives us access to talent, encourages our team to make better tools.

Gibbs: We always want to do more, this is access to more

Fields: We couldn’t do the scale of work we do – 400-man teams – without this.

Q from Blizzard: We have the problem of timing asset deliveries with approvals, and the difficulty of moving back up the chain. How do you deal with this? Is it harder across sites?

A: Burnette: All of our work comes in on time and no-one ever changes their mind.

[laughter]

At some point it becomes more efficient to pay people to keep track of all the information. In smaller teams there isn’t enough overhead to require it, but once you are that big, it’s vital, to manage the information flow across locations and timezones.

The artist who should have been home half an hour ago but stays an extra hour because somethings not good enough yet, is dealing with the same issue as a production manager who wants to make something better but just doesn’t have the money. Better information flows make for better decisions.

As an artist, you always want to make it better – the producer has to make the call to say stop.

[end]

For those of us interested in how humans interact with our machines, how socio-technical systems are made and how they make us, this is a precious moment. Tomorrow the probability waveforms collapse, Schroedinger’s cat will be let out of the bag; Apple will reveal the form of their long-rumoured slate.

In the realisation of a new kind of computing device many decisions are made. Not many companies are equipped to make and execute those decisions well. At the moment I can think of two: Apple and Palm. And as Palm is otherwise engaged, it falls to Apple to bring us the first fully-realised such device.

That’s why today, the moment before the Apple tablet is unveiled, is precious. From tomorrow it will be impossible to imagine a slate without reference to Apple’s design, just as now all smartphones are compared to the iPhone.

So before it’s too late, here’s what I’ve learned from six months living with iSlate prototypes.

The iPhone is Alan Kay’s Dynabook in the deck-of-cards form factor. Now Apple are ready to make the full-sized version; the paperback form factor. Kay put it at 8 by 5 inches – 10 inches diagonally. The size of a small paperback book, or a sheet of A4 folded in half (which is to say, A5). We know that this form factor is important. But it’s been hard to stop thinking of computers as anything other than television-typewriters.

The TV-typewriter form factor has found its epitome in the laptop as we know it: hinged keyboard-and-screen, trackpad and desktop metaphor. Shall I enumerate its faults? Too big and fragile to carry in an ordinary bag, we must instead heft special laptop bags with padded compartments and room for little else. Too energy-hungry for sustained use away from power. A screen that doesn’t work in sunlight. Too heavy to use while carrying. When equipped with processors fast enough to run a modern desktop operating system and 3D graphics, it’s too hot to actually use on a lap. The laptop is wonderfully portable, but not truly mobile.

The A5 dynabook won’t be better for all tasks. However it will afford different activities to those afforded by the laptop, desktop workstation and handheld device, and thereby expand the range of human creativity. That makes it exciting.

However Apple’s version will have its own affordances and constraints. It will be optimised for certain tasks; there’s more than one way to skin a universal computing device. From tomorrow we are in danger of forgetting that.

I’m conducting PhD research into creativity support tools, through a process of building and testing prototypes. Some of the prototypes have been slates, of various shapes and sizes. Form factors made from foamcore, weighted with aluminium; interactivity simulated with my (jailbroken) iPhone. I’ve been carrying them around and imagining how they fit into various scenarios. Here are some things I’ve learnt:

Form Factor, Size & Weight, and thus Connectivity

Bezels suck

A black bezel is a terrible idea for e-reading; you want a white margin around a page of text. Ideally the margin is interactive and can hold annotations anyway, so you just want all screen. If there has to be a bezel, make it as seamless as possible (and possibly accepting touch input). If there’s a rim, it should be white or silver.

7-inch tablets beat 10-inch for portability.

You can fit one in a coat pocket or a purse. But they’re small enough that you can actually lose one. You’ll want a GPS tracker in that thing.

The lighter you go, the more you can do

And the more likely you are to carry it around. Say a 10-inch tablet is 500g, splitting the difference of four iPhones or iPods Touch. That’s the weight of a medium-sized book, and much less than a laptop. For example a Macbook Air is 1360 g and a 15inch Macbook Pro is 2490 g. That weight is great for portability and reading, but for one-handed interactivity you actually want lighter. No hard drive. A Flash drive is light enough. For longevity put a memory card slot in the thing; then as memory prices plummet it’ll keep getting more useful. When costing this thing out for engineering, the primary cost you care about isn’t dollars, it’s weight.

To lose weight, put things outside the tablet

Files, and also processing. Voice recognition, 3D, video editing: control them from the tablet, but do the processing somewhere else. You should be able to control a desktop workstation from this, and also invisibly use all kinds of services that run on remote servers. This implies full-time connectivity; not just wifi but 3G and whatever comes next. It doesn’t matter how you get the connectivity there: people will still carry phones so tethering is an option.

Keyboard & Other Inputs

A 10-inch tablet can fit a full-size on-screen keyboard in landscape orientation. Typing on a full-size keyboard with one hand is a pain. Set it down on a surface and it works. Or turn the slate to portrait, and you have a keyboard that you can use with one hand while you hold the slate with the other. If you’re writing long documents, a bluetooth keyboard is fantastic.

Other input devices are great too. Game controllers, motion sensors, everything. Artists want a stylus for pressure-sensitivity. More inputs multiply possibilities. But ensure that you can use the stock configuration with multitouch alone, because every input is also a dependency.

Collaboration

A slate in a meeting room is nothing like a laptop in a meeting room. A laptop naturally faces the user; a slate naturally goes face-up on the desk. If you flip it up for privacy then it’s obvious that’s what you’re doing. So:

Slates afford information sharing in meetings. Sketch, pass it around the table. And here we find an implication for design.

Design problem

In meetings sketching is a common form of expression and communication for creative ideation. When teleconferencing, remote participants are either left out of this sketching process, or it stops.

Opportunity

Tablets can help, because digital sketches can be transmitted between locations. Draw on a tablet here, have it appear on a tablet there. The remote participants can then add to or annotate the sketch; communication is enhanced and the flow of collaborative ideation is unbroken. Win!

Extension

People also like to sketch with concrete tools (pencil and paper, arrangements of sticky notes, whiteboards, butcher’s paper etc). Tablets could have a camera or even a full-surface scanner, and thereby capture those analog sketches in digital format. This affords greater sharing with remote participants, review and archiving of materials, and with OCR and metadata, search & retrieval.

Capture & annotation

This leads me to an affordance I haven’t fully understood yet: slates afford rich annotation. You need a responsive screen: e-ink won’t do, but a Pixel-Qi transflective or something similar is ideal. But if you have that, then you can capture images, either by drawing or photographing. And then you can start to annotate the images, directly manipulating them on the device.

Finally: Infinity in the z-axis

One last aspect to note: the infinite depth afforded by zoom means there is plenty of room. Every image can be a rabbithole; a portal to an information space without inherent limit. 10 inches in the diagonal, and infinity in the z-axis. Jeff Raskin, Ken Perlin, Ivan Sutherland and the other Zooming UI inventors have a great deal to teach us here.

Right. I have more, but it’s unformed. Let’s hope that after 5 AM Sydney time tomorrow I can still think clearly about imaginary tablets as well as the real.

Augmented reality, that is an option as well!
– Erik Champion, January 28, 2010 at 7:37 am

The next time you're about to use the word “affordance”, please stop and check if the word “cue” would work instead.Because if it would, then:

1. you are using an obscure technical term for something that already has a perfectly good plain English word, and

2. you are using that technical term incorrectly.

Yes, I know that languages are living entities. None other than the eminent Don Norman, despairing in his attempts to correct the misuse of “affordances”, has cited this as a reason to abandon the term to its abusers. Because after all, words can change their meanings. Generally, I celebrate this fact. But not this time.

Technical terms are different. People can start calling air “Oxygen”, but that does not mean that scientists should change the periodic table. And this is an important technical term. It describes something very specific for which there is no other word. When J.J. Gibson went to the trouble of making up the word “affordances” in 1977 he thought long and hard about it first, and he coined it carefully so that it  makes sense. An affordance is “something that is afforded”.

“The affordances of the environment are what it offers to the animal, what it provides or furnishes, either for good or for ill” [1]

Affordances are not the cues in the environment that let you know that something is afforded to you. Affordances can be hidden, or perceived. Designers can make them easier to perceive by adding cues. Cues can lie, but affordances cannot: they are what actually is possible. Importantly they are also relational – they exist in the relationship between an environment and its user. A matchbox affords support to an ant but not to me.

I've attached an image of a green child's sippy cup to this article. To a toddler the handles on either side afford gripping with both hands at once, making it easy to tilt the cup to their face. It affords drinking. It also has a screwtop which the toddler cannot operate – no affordance for them there. But to me it does afford opening, so that I can refill the cup. This is great design, using the relational quality of affordances to promote some behaviours and constrain others, depending on the capabilities and aims of the different users.

People who are thinking and talking about design need a word for what the environment actually makes possible for a particular user in the context being described, and affordances is that word.

Finally – there is no reason to misuse the term “affordances” when you really mean “cues”. If you hear someone else misusing it (and I don’t care if it is Jared Spool), then call them out. We need this word back, and if language is indeed made by its users, well that is us. We can make the choice to use this word well, or to waste it.

Thank you for your attention.

V.

[1] Gibson, J.J., 1977, “The Theory of Affordances” in Perceiving, acting, and knowing: toward an ecological psychology
eds. R. Shaw, J. Bransford, Lawrence Erlbaum Associates, Hillsdale, NJ.

Jared M. Spool says:
November 21, 2009 at 9:54 am

Hi Viveka,
I believe that the rules of lexicography still allow a single word to have multiple definitions.

Affordances in the Gibsonian sense are very important.

However, interface design seems to want a word that isn’t cue (or clue, which is the layman alternative I tend to use) to be specific to the use intention. A cue is from the designer side, whereas an affordance is from the user side — if the user doesn’t recognize the cue, it’s not a good affordance.
So, I’m not sure what all the fuss is about. I think there’s still room in the dictionary to allow both definitions to co-exist.

Even if I am Jared Spool.
:)
Jared

viveka says:
November 21, 2009 at 3:35 pm

Hi Jared, thanks joining in :)

I agree that there's no point appealing to rules of lexicography. English lexicographers (unlike French ones) are descriptive, not prescriptive. They no longer have any special claim to authority over the use of language; English is in the hands of its users. That’s fine with me. I’m not a lexicographer, I’m a user of language.

I’m an interaction design practitioner and researcher, and I use and need “affordance” often. The difficulty is that when I try to use this very specific, obscure technical term my meaning is further obscured by the fact that some of my colleagues are not clear on what it means; or if they are they don’t know if I am clear, so I have to define it every single time I use it. This is tiresome. Fields of endeavour have jargon and specialised terms in order to speed things up when we talk among ourselves. Jargon can also be used to bamboozle outsiders, but I consider that a misuse.

Technical terms survive if they are useful. To have two separate meanings for a neologism that was invented fairly recently for a specific purpose makes that neologism less useful within its community. It makes it fabulously more useful for bamboozling, but again I don’t think that’s a valid argument.

So, let's try the experiment I suggested at the top of my post – replacing every use of affordance with “cue”:

You wrote: “if the user doesn’t recognize the cue, it’s not a good affordance”
Let’s try: “if the user doesn’t recognize the cue, it’s not a good cue”.
Yes! That makes perfect sense, it doesn’t require you to explain a new word to laypeople or to argue to experts that you don’t mean “Gibsonian” affordances when you use the word that Gibson made up, first used in an article by Gibson called “The Theory of Affordances” which contains the phrase “I made it up”.

Interface design wants a word that isn’t cue, or clue? Why not? Because we want to seem important? You can say “a cue for an affordance” if you must; sometimes it’s fun to point out that we know cool words that our clients don’t. But I have never yet found a sentence where “cue” wasn’t sufficiently clear.

So to be perfectly clear where I’m coming from here: get off my damn lawn ;)

Jared M. Spool says:
November 22, 2009 at 3:40 am

You seem to be having an issue I don’t have.

I’ve heard this argument since Norman started using the term. And yet, I’ve never once fallen victim to the confusion you describe.

So, feel free to use affordances in a different way than I use them. And, when we meet, you can feel free to correct my misuse.

After all, it’s not the first word that Australians use differently than Americans.

:)
Jared

viveka says:
November 23, 2009 at 12:20 pm

I think I get where you're coming from. If you talk mostly to lay people or novices who have never heard the word before, and you (mis)define the word for them at the start of the conversation, then I can see how you'd be sanguine about this. I run into the problem because I work with designers (who have picked up the "cue" misdefinition) and cognitive psychologists (who stick with Gibson's original meaning).

In the 2005 article I link to above though, you imply that cognitive psychologists say that “Affordances are clues”. They really don’t.

As well as HCI practice I am doing cross-disciplinary research. “Affordances” is used by people in perceptual psychology, cognitive psychology, environmental psychology, industrial design, human/computer interaction (HCI), interaction design and artificial intelligence. I have to talk to people in all these fields about my research. All of them use the word as defined by Gibson in 1977. Only a subset of HCI practitioners and early career researchers use it in the way that you prefer.

The difference is stark and yet it’s subtle. Affordances are real and underlying; cues are the surface indicators of those affordances. It’s easy to get essence and surface confused, but if you care about good design then it’s vital to separate them.

By using a word that refers to something essential as though it referred to the surface indication, HCI practitioners reveal ourselves to be shallow, concerned only with appearances and not with how things actually work. This is not something to be meekly accepted; it is something to fight.

Jared M. Spool says:
November 24, 2009 at 12:50 am

I’ve gone ahead and updated my Drag N Drop post to acknowledge your objection.

I still don’t care enough to change my usage of the word, but appreciate that you’re out there fighting the good fight for, um, something…

Jared

Clifton says:
November 25, 2009 at 7:58 pm

Love it. Completely agree, “affordance” is generally overused when “cue” is what we’re trying to say.

If a user doesn’t recognize a cue, the affordance remains. I recently discovered that the vent on my dad’s air conditioner can be lifted up to shift the angle of the vent. In the 20 years he’s owned the unit, he never knew that was possible. The affordance–the ability to be moved and manipulated–has been there for 20 years. But there wasn’t a sufficient cue for my dad to recognize the affordance. By explaining it to him, I provided another cue, in the form of words and a demonstration.

I love how many UI stories involve a product owned by the author’s parent.

viveka says:
November 25, 2009 at 9:33 pm

Thank you Clifton. And thanks to Jared for being so gracious under fire.

One nice thing about affordances in interaction design is that sometimes their function is apparent, so all you have to do is point them out or reveal them. The examples given in Norman’s Psychology of Everyday Things [alternative kickback-free link] are like this: slots, handles. If you can see the slot, then you can perceive that it affords insertion of something slot-sized. If you can see the handle you can perceive that it affords grasping, especially if it has a grip texture. On the other hand to put a grip texture on an on-screen scroll thumb is quite different: the designer is taking advantage of the cultural convention that arose from our perception of the true affordance in the real world.

My own favourite example is the sippy cup, showing how the same physical property of an object can be an affordance or a constraint (the very opposite) depending on who is interacting with it.

David Hamill says:
November 25, 2009 at 9:44 pm

I can’t see what the fuss is about. Unless use of the word is leading to miscommunication of a point then there isn’t a problem. Language is what we use to communicate with each other. If using the word ‘affordance’ has had this effect then job done. I don’t care if you start using the word ‘strawberry’ in its place, as long as people understand.

A ‘clanger’ is a mistake, but to UK adults over a certain age it’s also a small pink mouse-like creature that feeds off blue string pudding.

viveka says:
November 26, 2009 at 11:11 pm

I agree that words can mean whatever we want them to mean. There is no objective truth about what a word should mean. That doesn’t invalidate all value judgements though: we can judge the worth of a definition on whether it makes it easier or harder to communicate.

In this case, the existence of the extra similar-but-crucially-different use is indeed leading to miscommunication. For example, I am studying the effect of place on collaborative creativity. In my research I have found good evidence that the surface appearance of a place is not as important for some kinds of creative work as the affordances of that place. I want to use the term “affordances” because it is far quicker than saying “the things that the environment provides or makes possible for each participant, for good or ill, and dependent on the capabilities of that participant and their relationship with the environment”. However if people think that “affordances” means “surface appearance”, then my point makes no sense to them.

Elizabeth Buie says:
November 27, 2009 at 1:19 am

In talking about the flexibility of meanings, we need to consider whether we are talking about language in general or terminology in particular. Degrading a technical term is much more harmful to clear communication than is the ordinary evolution that happens to everyday language.

In 2002 I co-wrote an article for <interactions> called “What’s in a Word: The Semantics of Usability”, in which we argued exactly this. Here’s a quote from our article:

========

Word meanings can change over time; that’s all well and good. “Awful” used to mean “full of awe”, and seven centuries ago “nice” meant “foolish” (some might say it still does!). The rich garden of English vocabulary has grown from the endless planting of new words from foreign sources, jostling for their place in our prose and poetry. A living language is always on the move – good thing for us.

There’s a bit of a problem, though, when we want a word to conjure up something more concrete than a poetic image in the reader’s mind. When we want to use a word as terminology. Terminology is to vocabulary as bulldog is to kennel: It demands a certain kind of care. We want a term to hold its value. It has to say the same thing to everyone who needs to read or hear it.

========

Michael says:
November 25, 2009 at 9:59 pm

viveka,

I was brought here by tweet from Jared. I think he should be commended for bringing this discussion to over 9000 people.

I had no idea there was such a history behind this one word but have always felt “cue” was an underutilized word when describing, well “cues”. I’ve had many discussions when people who say affordance and needed to say cue. Now I have a better grasp for the argument.

Thanks for the lesson.

Andrew Ingram says:
November 26, 2009 at 12:51 am

It’s a common occurrence in any environment for people to adopt words without using their correct meaning, and it’s incredibly frustrating – to the point that you have to clarify definitions at the start of each meeting to avoid confusion.

I think it’s very important to only introduce technical terminology when everyday language is insufficient, and to use technical terminology consistently.

Your cue/affordance example is very good, for one thing it made the sentence much easier to understand.

Bob Kerns says:
November 26, 2009 at 3:40 am

Viveka,

I hope this doesn’t spread the focus of the debate too thinly, but here goes:

So tell me, if you substitute the word “feature” for “affordance”, what happens? Except there are also “misfeatures”. Which would correspond to your “constraint”, I think, except the original definition of “affordance” would appear to encompass both. Clearly, I’m using the term “feature” here in the sense it has come to have in the software world.

I’m just wondering if the drift in meaning isn’t a result of a lack of a strong need for the original meaning, to most people.

The distinction may be more useful in its original context, of things in the physical environment. In the software world, we are more concerned with people getting specified tasks done. We’re less likely to use a wrench as a hammer because somebody borrowed the hammer and didn’t put it back — even though a heavy wrench affords a hammer-like action.

There’s also the phrase “better affords”. Does this mean, it does it better, or it better communicates? I do NOT like the term cue as a substitute for the (mis)use of affordance. It has the serious problem of blurring the boundary between “telling the user what to do”, and “communicating the existence of an affordance (your usage)”. Both are things that are more commonly in need of discussion than your meaning of affordance — especially the communication aspect.

That’s because, if there’s no communication, the affordance is undiscovered, theoretical, futile. A lot of design activity is around communicating or efficiently presenting the affordances that are there. The affordances are often a given; the design activity neither adds nor removes them.

I think the need to discuss affordances largely centers around FAILED affordances. A lot of Norman’s book is on that, as I recall. The affordance is there, waiting to be used — but nobody uses it. Why?

A truly successful affordance often becomes synonymous with the name of the object. A hammer “hammers” things. Or the object is named for its primary affordance.

My intent here isn’t to argue for one side or the other, actually. I’m not a French lexicographer! I just think these are reasons you’re fighting an uphill battle.

BTW, I like your approach the the kickback-link issue. It clearly provides and communicates the non-kickback affordance, without needing a separate disclaimer, explanation, etc. I’m going to steal the idea. Not that I’ve ever made a dime from such links, but I can dream…

:)

Disclaimer: I used to work with Jared decades ago.

viveka says:
November 26, 2009 at 11:40 pm

Hi Bob,

Thanks for the thoughtful input, I’m enjoying this discussion immensely.

>There’s also the phrase “better affords”. Does this mean, it does it better,
>or it better communicates?

I think the beauty of the term “affordance” is that it gives us a way to talk about what’s really there: whether it’s helpful or unhelpful, visible or invisible. Accordingly I would use “better affords gripping” for a handle that has better grips in actual use. I would say “better communicates that it affords gripping” for a handle that has more visible grips.

Yes, communication of function is important – an invisible affordance is useless. However I argue that keeping our terms for communication separate from our terms for underlying affordance helps us think more clearly about exactly what is going on. Is the affordance broken, or is it poorly communicated? Is it the wrong affordance? Do we want to highlight one affordance rather than another?

On “feature” for “affordance” – I think they’re different as well. A feature connotes something useful that you want to highlight for users. Something you want to “feature” in the marketing materials. Some affordances may be features. Some constraints (e.g. password protection or parental controls) might also be features. A bad affordance can be a misfeature, as can a bad constraint.

James Landay says:
November 26, 2009 at 9:52 pm

I appreciate your effort at correcting our misuse of this term. But, “Only a subset of HCI practitioners and first-year researchers use it in the way that you prefer” is a bit strong… no, make that very strong. I have been a researcher in this field for almost 20 years, teaching introductory and advanced graduate courses for the last 13, and I have been defining it incorrectly all this time! I got my definition from a professor I assisted in teaching HCI at Carnegie Mellon. So, this means there are quite a few people running around using it incorrectly… Uh, sorry?

viveka says:
November 26, 2009 at 11:21 pm

The apology should be mine. Since I wrote this I’ve found a number of long-standing HCI practitioners who have been using “affordances” to mean “cues” or (close to correct but still not quite right) “perceived affordances”.
I still think it’s a misuse for the reasons I’ve outlined, but I can certainly understand how it came about. I’ve struck through the overstatement in my above comment accordingly.

I’m singularly impressed with people such as James who are happy to reconsider a long-held belief when presented with a reasonable counter-argument. I hope that I am as gracious when, inevitably, the same happens to me.

Gerard says:November 27, 2009 at 2:47 am

This is certainly a long-standing debate. For an argument of why the Gibsonian definition of affordance is still important, and why changing the definition is unfortunate for our profession, check out:

Torenvliet, G. L. (2003). We can’t afford it: The devaluation of a usability term. interactions, 10(4 (July/August 2003), 12-17.

Email gerard /at / torenvliet / dot / ca for a PDF.

In a few hours I’m off to Berkeley for Creativity and Cognition 2009 to participate in the Graduate Symposium. Mihaly Csikszentmihalyi, Ben Shneiderman, Jane Prophet and my advisor Ernest Edmonds will be among the speakers. I’m quite excited about the whole thing… updates to follow, or see my twitter feed in the meantime.