Snowden and the Toxicity of the Internet

I think we are still operating with very partial information here.  Glenn Greenwald has promised that he has thousands of documents from Snowden still, dozens of which are newsworthy.  It's still very unclear how Snowden came to have access to the documents he seems to have, who he really is, whether his testimony is 100% accurate, or where he has gone now.  I assume a lot more will come out in coming days and weeks.

With that caveat, here are a few thoughts: this case illustrates some long-standing concerns I have about the direction of society.

I speculate that it is going to turn out that Snowden was an electronic intruder on the government payroll.  Profiles describe him as secretive, fascinated with computers, and with knowledge of things like Tor (a peer-to-peer network for maintaining anonymity for computer communications).  His last job was working at an NSA network threat detection center, suggesting knowledge of computer security.  He had previously worked for the CIA, including overseas, suggesting a cyber-offense role.  And if it's true that he was making $200k (or even $122k) as a 29 year old IT guy with no formal qualifications in anything, I struggle to think of any explanation other than that he was technically very adept.

This would also explain how he was able to access a wide range of documents that were supposedly compartmented.  He's described as having been concerned about abuses of power before 2008, and then hoping things would change under Obama, only to realize that the Obama administration is if anything even more secretive and weaker in its commitments to civil-rights than its predecessors.  So Snowden was perhaps gathering his cache of documents for years before finally deciding to act.  I speculate that he turned his offensive cyber-attack skills on the intelligence community's own networks to gather these documents.  He may have had a lot of access - it's very common for people working in computer threat detection to have access to platforms that see everything going on in the networks in order to look for potential threats.  I've personally had access on a number of occasions to all the network traffic of some very large organizations, for example, so I can imagine that someone working for the NSA doing threat detection would have a lot of access to intelligence networks.

It would also explain his comment about being able to disable the surveillance system "in an afternoon".  That's the kind of stuff that computer security folks would tend to know how to do, that the average employee wouldn't.

I think this situation illustrates some fundamental downsides of the Internet.

It's always the case that important new technologies have major downsides that aren't evident at first.  To take just one example, the automobile in the early years promised freedom, speed, and mobility (as well as much less horse manure in the streets).  It only gradually became clear that the automobile also meant suburban sprawl, lengthy commutes in heavy traffic, major contributions to obesity, climate change, and dependence of the world economy on unstable Islamic countries in the Middle East (and thus Islamic terrorism).

Similarly, the Internet on first acquaintance promised instant access to all the world's information, opinion, and indeed people.  Wikipedia is much better than the Encyclopedia Brittanica.  Google is much better than a public library catalogue.  Amazon has a much larger selection of books than any brick-and-mortar store.  Facebook is a vastly better way to keep in touch with friends and family than occasional letters with duplicate photos.  What's not to love?

Part of the problem lies with the very seductiveness of the Internet.  Pretty much the entire planetary middle class is now on the Internet regularly, and the poor are not too far behind with mobile devices.  We have been tempted to put an enormous amount of data on there; this has proven incredibly attractive for advertisers, to the point where commercial entities can know basically everything we do if they are determined enough.  Still, the worst that advertisers can do to us is send us targeted ads and direct mail, which can be creepy but is basically harmless since we never have to follow up on the ads unless we choose to.

The government is another matter altogether.  Since the government maintains the monopoly on the use of force, it always has the ability to destroy any of our lives totally, and therefore the checks on its power are extremely important.  We must be assured that it carries out its duties with reasonable integrity and fairness.  This is why the Bill of Rights really matters, and why the increasing encroachment on it is alarming.

The immediate genesis of these encroachments is the threat of terrorism - primarily of the Islamic flavor.    However, I don't think that's the deepest issue long term.

One of the deep issues is that the Internet is built out of software, and software is inherently fragile.  A software program is a complex piece of logic which takes some inputs, performs some algorithm, and then creates some outputs.  It's created by fallible human beings, usually working under time pressure.  They invariably fail to think of all the possible inputs, and since the computer itself implements the program instructions mechanically, anything that the programmer forgets to explicitly handle is apt to cause a catastrophic failure of the program under the right conditions (which can be created by malicious input).  Frequently (not always) such failures include the possibility of of taking over the program and causing it to do whatever the supplier of input wants it to do - ie exploiting it.  So all software has numerous vulnerabilities in.  It's possible, by spending a lot more money and engineering time, to ship software with somewhat fewer vulnerabilities in.  But no software is vulnerability free - even Microsoft, which has lots of resources and has been making a reasonable effort for at least a decade, is still constantly having to fix newly discovered vulnerabilities in mature applications.

And yet, we now have a situation where all large organizations are critically dependent on their networks and software.  Whenever I've sat down with the information security staff of a large organization and asked the question "Could we take out your organization with network attacks?" The answer is invariably yes.  Often, they've known for years how to do it.  Other times, a half hour of brainstorming is enough to make it clear.  And invariably, large networks are compromised.  I've seen endless surveys of malware infections on corporate networks, and there's always some.  It might be random criminals, it might be foreign intelligence, but it's never the case that a large network is 100% under the control of it's legal owners.  Somebody else always has a foothold in there, and thus at least the potential to cause great harm.

In particular, Bradley Manning and Edward Snowden have both demonstrated that it's possible for a single individual with access to classified networks to cause enormous and lasting embarrassment to the US government.  Neither of them were trying to destroy it, but both were able to access and release an enormous amount of classified material.  One now has to assume that foreign intelligence agencies have been able to do as much.

This illustrates a larger dynamic of modern society - as a highly-optimized, just-in-time, economy that is run on software riddled with vulnerabilities, it's inherently rather fragile; much more so than it looks to a casual glance.  This gives individuals and small groups a great deal of power to cause disruption, potentially.  Not to mention that it gives a mutually assured destruction character to any future conflicts between sufficiently advanced nation states.

And, as the trend of automation and globalization continue, as we move toward the singularity, we will create both more vulnerability, and more people marginal to society, and thereby alienated from it.  This is a recipe for more damaging incidents.

So, one partial solution to this, inevitably, will involve more and more surveillance.  We will need to keep a closer and closer eye on everybody, to make sure no-one is getting out of line.  We never know who will be the next Boston bomber, the next leaker, the next foreign agent sending reams of zip files back to China.

And technology and algorithms will enable this.  As algorithms get smarter and smarter, they will inevitably be turned on us to make sure we aren't getting too malcontented and upsetting the system.  Bruce Schneier had an excellent essay a while back pointing out that the Internet is a surveillance state.  It's only going to get worse - flying drones, cameras everywhere on phones, content all in the cloud where the government can get at it.

We aren't going to go back - we will no more give up the Internet than we've given up the car in order to undo urban sprawl and obesity (though that is changing a little).

But the downsides are going to be very severe, and they are only just starting to come into view.