Burton Group Identity Blog

Facebook privacy revisited: Privacy Mirror version 2

Blogger: Ian Glazer

Facebook's recent changes to its privacy system has been garnering a lot of attention and not a lot of it is good. Both the EFF and Kaliya Hamlin (via ReadWriteWeb) have written up their takes on the matter and, all in all, I think they are decent assessments.

With all the supposed changes in Facebook's privacy system, I decided to revisit my work with Privacy Mirror (you can catch the backstory: here and then here). Having retested PM with both friends and strangers, here's what I've learned: Plus ça change, plus c'est la même chose.

The more things change, the more they stay the same.

Facebook's inconsistent treatment of privacy still remains. In a nutshell, what a 3rd party developer can see in your profile, having been granted access to you via your friends, directly depends on whether you have the same application they do. If you and your friends use the same Facebook app, then the 3rd party developer will see your profile (and photos and posts, etc.) as if that developer was your friend. If you do not use the same Facebook app that your friend does, then the 3rd party application is subject to a different set of constraints.

I question whether the recent changes Facebook has instituted have even remotely satisfied Commissioner Stoddart's concerns with Facebook, specifically 3rd party access to user information. Although users can control the scope of disclosure of their posts a bit better, defaulting settings to "Everyone" access as well as potentially making user's social graphs public undermines any attempt to cast Facebook in a pro-user control light.

There's also a nit I'd like to pick with the privacy settings system in Facebook - inconsistent save behavior. In some cases, Facebook automatically saves changed to privacy settings. In some cases, you have to press Save. This is a small point but it points to a larger issue. If service providers do not provide their users with meaningful, usable choices when it comes to controlling privacy and disclosure controls, but instead heap more controls in hard to find places, then these service providers have not aided their customers in the least. More user choices only equals more user control if those choices are clear, consumable, and centralized.

If you want to conduct some of your own testing of Facebook's privacy system, feel free to play with Privacy Mirror. The following are new features I've added:

• PM tests to see if the person your are pointing the Mirror at is a Privacy Mirror user. If they are you'll get results based on their privacy settings with respect to you as a person. If they aren't you'll get results based on their privacy settings with respect to Privacy Mirror being a 3rd party application. This behavior is core Facebook Platform behavior which I feel is inconsistent and puts people at a disadvantage.
• PM tries to find some photo albums that the person may have added
• PM tried to find some photos that are tagged with the person in question
• Added the ability to point the Mirror at a specific person better using their username

Why seeing your social activities again seems so uncomfortable?

Blogger: Ian Glazer

Continuing Burton Group’s work of social networking and social media, I've been having various forms of this conversation over the last few weeks. First, I was at TechAmerica talking about social networks, privacy, and data breaches. Although the audio isn't great, you can get the gist from this video. Then I was talking to the guys from InfoChimps ahead of their debut of some huge Twitter datasets. (The potential for data they have is pretty breath-taking.)  Meanwhile, I am prep'ing a more formalized version of this talk for an upcoming OWASP event. With all this activity I thought I'd share a part of it.

On the whole, people have no problem using social networking tools. Whether for personal or for work reasons more and more people are using a variety of tools to share and connect. And in this regard, we can think of social tools as engines for disclosure. Although people are relatively comfortable making disclosures such as "had a great meal in Ottawa" or "have to burn the midnight oil to get this blog post done," people feel uncomfortable when these disclosures appear in other places. This feeling is akin to reaching into your computer bag and finding a long lost banana: a little foreign, a little gross, and a little strange. People often want to keep their social structures separates and, using a highly technical word, people feel oogy when they discover that something they have disclosed (an activity, a group they may have joined, a relationship they formed, a trip they have taken, etc) is known by other people in other networks.

There are three axes to this problem:
* Audience
* Content
* Time

Oogy factor #1 - Audience - People often underestimate the size of the audience to whom their are disclosing information. What they think they are sharing with their team at work, is in fact shared with the enterprise. Furthermore, there are cases where the true size of the audience is not known because linkages between different social networking sites and the social graphs defined therein.

Oogy factor #2 -Content - Some disclosures are not obviously under people's control. It's obvious when I update my status in Yammer. It isn't so obvious when I join a group and that fact appears in my work activity stream.  This is unsettling as information is being disclosed about me and yet I didn't actively disclose that information. (I fell prey to this one... ask me sometime - funny story.)

Oogy factor #3 - Time - Closely tied to Content, people don't necessarily have control of when things are disclosed about them. Where social tools are reporting on activity, it isn't entirely obvious how a person controls such disclosures and when they happen.

People build mental models for their believed behavior of social tools along these three axis. If any one axis is shifted and the tool behave in a manner contrary to those mental models people feel uncomfortable. Although people are just establishing a comfort level with social tools from a consumer perspective, the enterprise is just taking its first teetering steps with social tools. There is definitely enterprise-grade ooginess ahead as enterprise grapples with the data breach and privacy implications of these tools. To that end, social tools have to provide meaningful ways for people, in the consumer setting, to adjust tool-behavior to meet their own mental models, and enterprises to accommodate wider regulatory and data protection concerns.
 
I'm going to be giving a longer version of this as a presentation to an OWASP and Tivoli users group meeting in December. If you are in the Hartford area, join us. You can register here

Gartner Gets Privacy Dead Wrong

Blogger: Bob Blakley

Andrea DiMaio of Gartner recently posted a blog entry entitled "Forget Privacy: It Is Just An Illusion".

DiMaio's lament rephrases Scott McNealy's famous quote ("You have zero privacy anyway. Get over it.")

McNealy was wrong then and DiMaio is wrong now; they're both dead wrong, and it's important.

Here's DiMaio's key sentence:

I have come to realize that, does not matter how careful we are, we are going to lose control of our privacy.

But Andrea DiMaio never had control of his privacy. And nothing - including technology - was ever going to give him control. DiMaio and McNealy assume without saying it that privacy means "keeping personal information secret". And by that definition privacy is an illusion. But "keeping personal information secret" is the wrong definition of privacy. As long as your personal information is secret, you don't even have a privacy problem. It's only when somebody else knows your personal information that you have a privacy problem.

Privacy is the problem you have after you share sensitive information.

When you discover that you might have a socially awkward medical condition and you go to the doctor, you don't keep the condition secret from him - you tell him about it so that you can get treated. And when you leave the office, you don't control your doctor; you trust him with your secret. You trust him with your private information because he has taken an oath to behave sociably and to use your personal information only in ways which benefit you.

That's how privacy works; it's not about secrecy, and it's not about control: it's about sociability. Privacy is a social good which we give to one another, not a social order in which we control one another.

Technologists hate this; social phenomena aren't deterministic and programmers can't write code to make them come out right. When technologists are faced with a social problem, they often respond by redefining the problem as a technical problem they think they can solve.

In rhetoric, we call this redefinition of the problem "framing".

The privacy framing that's going on in the technology industry today is this:

Social Frame: Privacy is a social problem; the solution is to ensure that people use sensitive personal information only in ways that are beneficial to the subject of the information.

BUT as technologists we can't (as DiMaio observes) control peoples' behavior, so we can't solve this problem. So instead let's work on a problem that sounds similar:

Technology Frame: Privacy is a technology problem; since we can't make people use sensitive personal information sociably, the solution is to ensure that people never see others' sensitive personal information.

We technologists have tried to solve the privacy problem in this technology frame for about a decade now, and, not surprisingly (information wants to be free!) we have failed. DiMaio now wants to give up. But he's forgotten the reframing; he's assuming that the technology frame is the problem, and therefore if the problem can't be solved in the technology frame it can't be solved.

The technology frame isn't the problem.  Privacy is the problem. Society can and routinely does solve the privacy problem in the social frame, by getting the vast majority of people to behave sociably. Privacy isn't a new problem. It's existed in all human societies for as long as there have been human societies. Lawyers have solved it. Doctors have solved it. Priests have solved it. Friends have solved it. They've solved it by creating social structures which discourage monstrous behavior. We even have words for people who violate the often unwritten and unspoken rules governing the handling of delicate personal information; in the old days we called a man who was careless with others' secrets a "cad". Nowadays we use another word (a word which also has an anatomical denotation, if you're wondering).

Technology can't solve privacy problems, because they're not technology problems. But technology can make privacy problems worse, by making it easy to do antisocial things, or by making it hard to recognize the sensitivity of personal information and lowering our awareness that we're in a social situation and need to behave sociably; online spaces like Facebook, whose rules for handling private information are often opaque to users, create unnecessary privacy hazards in this way (see Ian Glazer's "Privacy Mirror" experiment for an example of how opaque privacy settings can undermine the sociability of an online space).

If we accept the technology frame and let technologists define privacy as control over dissemination of information, we ARE going to have less privacy. Resisting the technology frame is critical; if we don't resist it, lots of bad things happen. For example, if we accept the "privacy is defined as control of secrecy" frame, then we will start to believe (perhaps as a society, and perhaps even as a matter of law) that as soon as someone learns a piece of information about us, that information is no longer private, and we lose subsequent protections.

We don't have to accept the technology frame. 

The assumption that led technologists to create the technology frame - that the social problem of getting people to behave sociably cannot be solved - amounts to an assumption that we will all be monsters.

This assumption is neither true nor acceptable. We've got to fight the technologists on this one.

Worldviews have consequences. A worldview that says "privacy is an illusion" can create a world in which there is no privacy, at least online.

My generation makes a distinction between the online world and "the real world". My kids' generation does not. The social world they live in will BE the online world - woven inextricably with what I grew up calling "the real world". I'm not willing to stand idly by and watch the sociability of that world destroyed by technologists who have given up because they can't see beyond their coding pads.

DiMaio concludes his post this way:

The problem for us, all of us, is that somebody will be watching all the time. We’d better behave.

The implied subtext is "because whoever's watching will be a monster, and turn us in to the authorities, and we'll be punished".

DiMaio is deeply irresponsible to encourage the view that just because the cryptographers can't give us a cloak of invisibility online it's OK to be a monster.

But he's right that we'd better behave. When we see someone else's private information, we'd better avert our gaze. We'd better not gossip about it. We'd better be sociable. Because otherwise we won't need the telescreen - we'll already have each other. And we'll get the society we deserve.

We are our brothers' keepers. We'd better start acting like it.

Technologists have a critical role to play in protecting privacy - but that role isn't building walls of secrecy. It's in building sociable spaces in the electronic world.

A sociable space is one in which people's social and antisocial actions are exposed to scrutiny so that normal human social processes can work.

A space in which tagging a photograph publicizes not only the identities of the people in the photograph but also the identities of the person who took the photograph and the person who tagged the photograph is more sociable than a space in which the only identity revealed is that of the person in the photograph - because when the picture of Jimmy holding a martini washes up on the HR department's desk, Jimmy will know that Johnny took it (at a private party) and Julie tagged him - and the conversations humans have developed over tens of thousands of years to handle these situations will take place.

A space in which personal information (a health record, say) always comes with metadata indicating who collected it, for what purpose it was collected, and under what terms and conditions it may be used is more sociable than a space in which a piece of personal information may be forwarded into another organization by someone who doesn't even know the information is personal. And so on.

At Burton Group we don't think privacy is an illusion. We think it's a hard issue - very hard - but that's why we're here: to give practical advice on hard issues. Ian Glazer and I have recently updated the privacy coverage our research and analysis customers get as part of their subscription. But privacy is so important and so widely misunderstood that we've decided to release our recent paper free to the public. It's here.

We hope you'll read it. We also hope you'll get in touch. Leave us comments here on the blog, email us, or call and ask for a dialog - even if you're not a customer.

The challenge in fixing Facebook’s underlying privacy problems

Blogger: Ian Glazer

A few Facebook hacks came across my desk this week. The first set are so called "rogue" applications which do the tediously predictable grab of user information followed by the equally tediously predictable spam-a-palooza. Calling such applications "rogue" is misleading. These didn't start out okay and turn evil somewhere along the way. These apps were built to cause trouble - they are malware. Facebook has a healthy set of malware apps and the number is growing every day. You can easily spot effected Facebook users by their status messages - "Sorry for the email - my Facebook got a virus."

The second hack is of a far more interesting class. Ronen Zilberman, a security researcher, harnessed features of the Facebook platform to unwittingly perform a man-in-the-middle attack on itself. Zilberman documents how the attack works in very clear language. You can even see a video of the attack in action. Why is this a more interesting class of attack on Facebook? First, it doesn't require an application to be added to the victim's Facebook profile. Second and more importantly, this attack fundamentally turns Facebook's goals against itself.

Facebook's mission is to "give people the power to share and make the world more open and connected." Its business is to accomplish this mission before someone else does. This requires that Facebook provide a means to connect as many people, websites and services as possible and as fast as possible. And in the course of this social networking land-grab, it is not surprising that we have seen both Facebook malware and the Facebook's platform being used to support anti-social behavior. The Facebook platform is optimized to provide frictionless connections and sharing of information. But as exploits for ill-purposes increase, Facebook has to act and act in a manner counter to their mission.

Facebook is currently trying to tackle some of its privacy issues with new privacy settings. The changes to the Privacy Settings are in beta, expected to rollout system-wide shortly. I sincerely hope that Facebook simplifies the privacy settings interface while adding more granular controls - though I am not too hopeful this will happen. Furthermore, I am very curious to see if changes in privacy settings will improve the situation I discovered with Privacy Mirror - again, not too hopeful. But changes in privacy settings are just patches on the underlying problem: increased privacy controls and platform restrictiveness are antithetical to Facebook's mission. Until Facebook institutes more control within its platform, we will continue to see more malware and more "interesting" attacks.

In order to achieve its mission, Facebook has to prove that it is a safe space in which its customers can engage in social behaviors. To accomplish this, Facebook must recognize the fact that its users have relationships with each other and that Facebook itself has a relationship with each of its users. These relationships are governed by social norms and are not dictated but negotiated through countless social interactions. These relationships and the rules governing them must be respected in order for Facebook to prove that it is a safe place to make shared information public and keep private information private.

Beyond Privacy Mirror

Blogger: Ian Glazer

Over the last two weeks, I have been using my homegrown Facebook application, Privacy Mirror, as a means of experimenting with Facebook’s privacy settings. Although Facebook provides a nice interface to view your profile through your friends’ eyes, it does not do the same for applications. I built Privacy Mirror with the hopes of learning what 3rd party application developers can see of my profile by way of my friends’ use of applications. I have yet to speak with representatives of Facebook to confirm my findings, but I am confident in the following findings.

Imagine that Alice and Bob are friends in Facebook. Alice decides to add a new application, called App X, to her profile in Facebook. (For clarity's sake, by "add", I mean that she authorizes the application to see her profile. Examples of Facebook applications include Polls, Friend Wheel, Movies, etc.) At this point, App X can see information in Alice’s profile. App X can also see that Alice is friends with Bob; in fact, App X can see information in Bob’s profile. Bob can limit how much information about him is available to applications that his friends add to their profiles through the Application Privacy settings. In this case, let's imaging that Bob has only allowed 3rd party applications to see his profile picture and profile status.

After a while, Alice tells Bob about App X. He thinks it sounds cool and adds it to his profile. At this point if App X, via Alice’s profile, looks at Bob’s profile it will see not only his profile picture and status but also his education history, hometown info, activities and movies. That is significantly more than what he authorized in his Application privacy settings. What is going here?

It appears what's going on is that if Alice and Bob both have authorized the same application, that application no longer respects either user's Application Privacy settings. Instead, it respects the Profile Privacy settings of each person. In essence, App X acts (from a privacy settings point of view) as if it were a friend of Alice and Bob and not a third-party application.

Putting my privacy commissioner hat for a moment, I’d want to analyze this situation from a consent and disclosure perspective. When Bob confirms his friendship with Alice he is, in a sense, opting in to a relationship with her. This opt-in indicates that he is willing to disclose certain information to Alice. Bob can control what information is disclosed to Alice through his Profile Privacy settings and this allows him to mitigate privacy concerns he has in terms of his relationship with Alice.

What Bob isn’t consenting to (and is not opting in to) is a relationship with Alice’s applications. Bob is completely unaware of which applications Alice currently has or will have in the future. This is an asymmetry of relationship. It is entirely possible that Alice and Bob will have applications in common and once they do the amount of profile information disclosed (by both of them) to an application can radically change and change without notice to either Alice or Bob. Furthermore, it is unclear which Facebook privacy settings Bob needs to manipulate to control what Alice’s applications can learn about him.

This lack of clarity is harmful. It shouldn’t take a few hundred lines of PHP, three debuggers, and an engineering degree to figure out how privacy controls work. This lack of clarity robs Facebook users of the opportunity to make meaningful and informed choices about their privacy.

This experiment started after I read the Canadian Privacy Commissioner’s report of findings on privacy complaints brought against Facebook. This report raised significant concerns about third-party applications and their access to profile information.

As of the beginning of Catalyst (today!), Facebook has about 15 days remaining to respond to the Canadian Privacy Commissioner’s office, I hope that this issue about third party applications and privacy controls is meaningfully addressed in Facebook's response.

Rounding out the privacy agenda at Catalyst: US-VISIT

Blogger: Gerry Gebel

We are very pleased to have Bob Mocny, director of DHS’s US-VISIT program speaking during the privacy track next Friday, July 31. Bob’s abstract, listed below, fills out an incredible privacy agenda and ensures that you will be handsomely rewarded if you stay for the full Catalyst conference. Privacy debates abound in many circles and you’ll hear several next week as speakers from Covisint, Future Identity, UCLA, Georgetown University, and Google share their issues, concerns, and opinions of privacy.

Mr’s Mocny’s session info:
A Dual Mission: Identity Management and Privacy Protection in the Federal Government

Advances in identity management technology are providing significant security benefits to the U.S. government. At the same time, federal privacy statutes require the protection of personally identifiable information collected, stored and accessed by government officials. Learn how one federal program is successfully integrating the use of biometric technology and privacy protection requirements to enhance the nation’s security. Robert Mocny, director of the U.S. Department of Homeland Security’s USVISIT program—one of the world’s largest biometric identity management systems—will discuss how the program’s privacy strategies earn public trust and contribute to the overall success of the program. The practices employed by USVISIT are applicable in both public and private sectors and important for everyone involved in developing and implementing identity management systems.

Laplace’s Demon, Santa Claus and TSA’s Secure Flight

Blogger: Ian Glazer

No doubt you frequent fliers out there have received emails from your airline of choice talking about TSA’s Secure Flight. As you make air travel reservations in the future, your airline will communicate with TSA to get, essentially, a fly/no-fly decision from the Secure Flight system. As the TSA explains in the “How it works” section of their website dedicated to Secure Flight:

Secure Flight matches the name, date of birth and gender information for each passenger against government watch lists to:

• Identify known and suspected terrorists

•  Prevent individuals on the No Fly List from boarding an aircraft

• Identify individuals on the Selectee List for enhanced screening

• Facilitate passenger air travel

• Protect individuals' privacy

After matching passenger information against government watch lists, Secure Flight transmits the matching results back to aircraft operators.

Did you notice the extreme use of irony there? Secure Flight is used to “facilitate passenger air travel” and yet Secure Flight’s sole purpose is to keep people off of planes. (I think someone at the TSA doesn’t know what facilitate means.) Irony aside, Secure Flight is ignorant of (or at least tone-deaf to) the US’ strong social and legal tradition of freedom of movement.  Secure Flight can act as a preemptive refusal of air travel in the absence of due process, which contravenes citizens’ freedom of movement.

Let’s talk about Pierre-Simon Laplace for a second. The French mathematician and astronomer described “an intellect” so vast that it knew the location and momentum of every atom in the universe. With this knowledge, this intellect (latter dubbed Laplace’s Demon by biographers) could know the future. As he wrote:

We may regard the present state of the universe as the effect of its past and the cause of its future. An intellect which at a certain moment would know all forces that set nature in motion, and all positions of all items of which nature is composed, if this intellect were also vast enough to submit these data to analysis, it would embrace in a single formula the movements of the greatest bodies of the universe and those of the tiniest atom; for such an intellect nothing would be uncertain and the future just like the past would be present before its eyes.

So with two attributes, location and momentum, Laplace’s Demon can know the future. With gender and birthday (along with name), Secure Flight can know bad guys from good guys and keep them off a plane.

The problem is that neither work. Laplace’s Demon has been slain multiple times. Secure Flight and the No-Fly List have and will continue to suffer the same fate.

Nations (UK, Israel, the US) and Agencies (DHS), like malevolent Santa Clauses, are building multiple mother-of-all naughty-and-nice lists. Their slavish devotion to Laplace’s Demon fuels their hopes that by knowing everyone’s name (and other attributes) they somehow can predict (and in some cases, prevent) the future. Hubris and ignorance are the twin attributes of such plans and neither ought to have a place in the programs of civil societies.

Personal Privacy Impact Assessments for Facebook

Blogger: Ian Glazer

I’m reading Canada’s Assistant Privacy Commissioner Elizabeth Denham’s recently released findings into complaints levied against Facebook. (Report of Findings into the Complaint Filed by the Canadian Internet Policy and Public Interest Clinic (CIPPIC)against Facebook Inc. Under the Personal Information Protection and Electronic Documents Act.) My first reaction to this is, frankly, one of jealousy. I wish we had a similar commissioner/czar/wonk here in the US. I suppose elements of the FTC work in this regard but without the same charter, which is too bad.

Section 4 of the report is, for me, where the action is at. Section 4 is concerned with 3rd party application in Facebook and use of personal data by those applications. As the Facebook platform grows with new additions like Facebook Connect, issues of third-party access to user information will continue to be a concern to those who pay attention to such things. There’s a challenge here as the ways in which 3rd party applications use user information is hard to decipher, as it is, from an end-user perspective, a fairly black-box operation.

I wonder if Facebook could build a personal privacy impact assessment (PPIA) app. The PPIA would analyze the action you are about to take on Facebook, your privacy settings, the 3rd party apps you’ve allows access to your profile, and the privacy settings you have set for those apps. The PPIA could give you a quick read on which applications would be privy to the action you are about to do. It could indicate which groups of friends (based on your privacy settings) would see what you are about to do. Essentially, it would let you see across how much of your social graph a certain action (like posting a link or photo) will travel.

We all have PPIAs built in - one that is cultivated through social interactions schooled by social norms. When it comes to dealing with large systems, like Facebook, big business, or the government for that matter, we all can use a little help.  I wonder if someone can get a PPIA prototype up ahead of Catalyst to at least give me a warning about potentially embarrassing photos being posted somewhere…

I Can See Clearly Now...

Blogger: Kevin Kampman

The Clear registered traveler program is responding to subscriber questions and concerns, and providing us some very relevant considerations for identity services. In correspondence that came out Friday afternoon, Clear indicated that:

•    The service provided are no longer available at airports
•    Privacy information has been secured in accordance with “Transportation Security Administration's Security, Privacy and Compliance Standards” (which don’t by the way, identify what happens in the case of a company failure)
•    Clear, TSA, and Lockheed Martin (identified as the lead systems integrator for Verified Identity Pass, Inc, the company behind Clear) are working on an orderly program shutdown
•    Clear computers and disks assigned to airport kiosks and to Clear employees are being “triple wiped” to destroy all data and software
•    The identity information collected by Clear could be transferred to another service provider in accordance with TSA’s Registered Traveler Program polices, but no such transfer was identified. It is more likely that the information will be destroyed.
•    Clear is working with TSA, airports, partners and subcontractors to keep subscriber information secure.
•    There will be no refunds, support, or other consideration for subscribers. 

The bottom line is that the service that subscribers bought into and the data collected is history. I for one am happy that I didn’t respond to their recent special offers, for example, for Father’s Day (“Reminder: There's still time - Dad deserves 5 star service (and a new tie)”).  Here’s a sad joke: What do a Clear smartcard and a necktie have in common? Answer: They’re both useless.

Watching Clear’s demise, I see some disturbing parallels. One example is electronic patient records. I would be very careful, based on this experience, to ask:
•    What regulations protect the information
•    Who owns the information
•    Who holds the information, and how can it be archived or transferred
•    How is it secured, and how can it be corrected, modified, and deleted
•    Who can see the information, and under what circumstances
•    How are breaches detected, how will they be remediated,
•    Who is liable for lapses, omissions, or damages?

In the case of Clear, it is likely that the only lasting damages will be to registered travelers’ wallets. However, the questions that emerged from Clear’s failure should be a bellwether for future identity-related private-public initiatives.

The role of design in protecting cyberspace: thoughts from CFP 2009

Blogger: Ian Glazer

Among the sessions in this year’s Computers Freedom and Privacy conference was a panel on the recently released National review of cyber-security. Ed Felten presented three related areas that he believes have to be improved in equal measure to improve overall cyber-security:

1. Product development
2. System administration
3. User behavior

But, to me, there was something missing from the list – product design.

Too often I have seen products whose user interface, in fact its entire user experience, was constructed after the fact.   First the special sauce gets codified, then the chrome is put on and product gets a face.  It is easy to recognize products that have been built in this way as they tend to expose their internal data models to users, forcing users to adopt the metaphors of the engineers that built the product in the first place.  These types of products make problems internal to the product problems for the end-user and this can lead to very bad things.  See Three Mile Island as an example.  Poor user experience design leads to so-called “user error,” but is it really user error if the end-user is confronted with meaningless alarms, confusing error messages, and misleading feedback?

At CFP, I talked to Bruce Schneier his research that went into Beyond Fear to get a better understanding of the psychology of fear and its relation to security.  As you probably know, humans (and other animals too) are fantastically bad about evaluating risk. Optimism bias and other factors cause us to either over or under-estimate risks. Combine this with the fact that how choices are presented directly influences how choices are made and you realize the crucial need to build better user experiences for security (frankly, all) products.

“Is everything okay with the mother ship and should we blow up Russia?” This is the question presented Buckaroo Bonzai and I think I’ve seen a form of it as a dialogue box in Windows.  Would it be considered user error if an end-user pressed the “Yes” button and nuked Moscow? Bad design is at the least confusing and at the worst dangerous.

I did talk to Ed afterwards and he acknowledged the role of design in product development. As he said, if we only attempt to improve one of the three areas product devolvement or system administration or user behavior we won’t improve cyber-security; we have to improve all three.  User experience design as a part of an improved product development processes can directly lead to better more informed user behavior. Okay you product managers and designers make your voices heard – better safer products through better design!