Tuesday, June 11, 2013

Snowden and the Toxicity of the Internet

I think we are still operating with very partial information here.  Glenn Greenwald has promised that he has thousands of documents from Snowden still, dozens of which are newsworthy.  It's still very unclear how Snowden came to have access to the documents he seems to have, who he really is, whether his testimony is 100% accurate, or where he has gone now.  I assume a lot more will come out in coming days and weeks.

With that caveat, here are a few thoughts: this case illustrates some long-standing concerns I have about the direction of society.

I speculate that it is going to turn out that Snowden was an electronic intruder on the government payroll.  Profiles describe him as secretive, fascinated with computers, and with knowledge of things like Tor (a peer-to-peer network for maintaining anonymity for computer communications).  His last job was working at an NSA network threat detection center, suggesting knowledge of computer security.  He had previously worked for the CIA, including overseas, suggesting a cyber-offense role.  And if it's true that he was making $200k (or even $122k) as a 29 year old IT guy with no formal qualifications in anything, I struggle to think of any explanation other than that he was technically very adept.

This would also explain how he was able to access a wide range of documents that were supposedly compartmented.  He's described as having been concerned about abuses of power before 2008, and then hoping things would change under Obama, only to realize that the Obama administration is if anything even more secretive and weaker in its commitments to civil-rights than its predecessors.  So Snowden was perhaps gathering his cache of documents for years before finally deciding to act.  I speculate that he turned his offensive cyber-attack skills on the intelligence community's own networks to gather these documents.  He may have had a lot of access - it's very common for people working in computer threat detection to have access to platforms that see everything going on in the networks in order to look for potential threats.  I've personally had access on a number of occasions to all the network traffic of some very large organizations, for example, so I can imagine that someone working for the NSA doing threat detection would have a lot of access to intelligence networks.

It would also explain his comment about being able to disable the surveillance system "in an afternoon".  That's the kind of stuff that computer security folks would tend to know how to do, that the average employee wouldn't.

I think this situation illustrates some fundamental downsides of the Internet.

It's always the case that important new technologies have major downsides that aren't evident at first.  To take just one example, the automobile in the early years promised freedom, speed, and mobility (as well as much less horse manure in the streets).  It only gradually became clear that the automobile also meant suburban sprawl, lengthy commutes in heavy traffic, major contributions to obesity, climate change, and dependence of the world economy on unstable Islamic countries in the Middle East (and thus Islamic terrorism).

Similarly, the Internet on first acquaintance promised instant access to all the world's information, opinion, and indeed people.  Wikipedia is much better than the Encyclopedia Brittanica.  Google is much better than a public library catalogue.  Amazon has a much larger selection of books than any brick-and-mortar store.  Facebook is a vastly better way to keep in touch with friends and family than occasional letters with duplicate photos.  What's not to love?

Part of the problem lies with the very seductiveness of the Internet.  Pretty much the entire planetary middle class is now on the Internet regularly, and the poor are not too far behind with mobile devices.  We have been tempted to put an enormous amount of data on there; this has proven incredibly attractive for advertisers, to the point where commercial entities can know basically everything we do if they are determined enough.  Still, the worst that advertisers can do to us is send us targeted ads and direct mail, which can be creepy but is basically harmless since we never have to follow up on the ads unless we choose to.

The government is another matter altogether.  Since the government maintains the monopoly on the use of force, it always has the ability to destroy any of our lives totally, and therefore the checks on its power are extremely important.  We must be assured that it carries out its duties with reasonable integrity and fairness.  This is why the Bill of Rights really matters, and why the increasing encroachment on it is alarming.

The immediate genesis of these encroachments is the threat of terrorism - primarily of the Islamic flavor.    However, I don't think that's the deepest issue long term.

One of the deep issues is that the Internet is built out of software, and software is inherently fragile.  A software program is a complex piece of logic which takes some inputs, performs some algorithm, and then creates some outputs.  It's created by fallible human beings, usually working under time pressure.  They invariably fail to think of all the possible inputs, and since the computer itself implements the program instructions mechanically, anything that the programmer forgets to explicitly handle is apt to cause a catastrophic failure of the program under the right conditions (which can be created by malicious input).  Frequently (not always) such failures include the possibility of of taking over the program and causing it to do whatever the supplier of input wants it to do - ie exploiting it.  So all software has numerous vulnerabilities in.  It's possible, by spending a lot more money and engineering time, to ship software with somewhat fewer vulnerabilities in.  But no software is vulnerability free - even Microsoft, which has lots of resources and has been making a reasonable effort for at least a decade, is still constantly having to fix newly discovered vulnerabilities in mature applications.

And yet, we now have a situation where all large organizations are critically dependent on their networks and software.  Whenever I've sat down with the information security staff of a large organization and asked the question "Could we take out your organization with network attacks?" The answer is invariably yes.  Often, they've known for years how to do it.  Other times, a half hour of brainstorming is enough to make it clear.  And invariably, large networks are compromised.  I've seen endless surveys of malware infections on corporate networks, and there's always some.  It might be random criminals, it might be foreign intelligence, but it's never the case that a large network is 100% under the control of it's legal owners.  Somebody else always has a foothold in there, and thus at least the potential to cause great harm.

In particular, Bradley Manning and Edward Snowden have both demonstrated that it's possible for a single individual with access to classified networks to cause enormous and lasting embarrassment to the US government.  Neither of them were trying to destroy it, but both were able to access and release an enormous amount of classified material.  One now has to assume that foreign intelligence agencies have been able to do as much.

This illustrates a larger dynamic of modern society - as a highly-optimized, just-in-time, economy that is run on software riddled with vulnerabilities, it's inherently rather fragile; much more so than it looks to a casual glance.  This gives individuals and small groups a great deal of power to cause disruption, potentially.  Not to mention that it gives a mutually assured destruction character to any future conflicts between sufficiently advanced nation states.

And, as the trend of automation and globalization continue, as we move toward the singularity, we will create both more vulnerability, and more people marginal to society, and thereby alienated from it.  This is a recipe for more damaging incidents.

So, one partial solution to this, inevitably, will involve more and more surveillance.  We will need to keep a closer and closer eye on everybody, to make sure no-one is getting out of line.  We never know who will be the next Boston bomber, the next leaker, the next foreign agent sending reams of zip files back to China.

And technology and algorithms will enable this.  As algorithms get smarter and smarter, they will inevitably be turned on us to make sure we aren't getting too malcontented and upsetting the system.  Bruce Schneier had an excellent essay a while back pointing out that the Internet is a surveillance state.  It's only going to get worse - flying drones, cameras everywhere on phones, content all in the cloud where the government can get at it.

We aren't going to go back - we will no more give up the Internet than we've given up the car in order to undo urban sprawl and obesity (though that is changing a little).

But the downsides are going to be very severe, and they are only just starting to come into view.


Chris Reynolds said...

Excellent. One of the frequent reasons I rate your blog.

sunbeam said...

There is some old saying that if you don't know who the patsy is when you are playing Poker, then it is you.

I guess we can change things for the internet a little, if you don't know who is wearing the tinfoil hat, then it is you?

But anyway, if I could get God or Gortek the all knowing Alien to come down from the heavens and confirm things one way or the other, this is what I'd ask him:

Does the dirctor of the CIA/NSA have a little icon on his computer that launches an app that pulls up a search box you can put names into? Then you can pick one from the list that pops up, or narrow the search.

If you pick a name, it gives you all the usual data. Then something where you can get a list of recent phone conversations. A recording if it is a cell phone, who he called, how long the conversation lasted, where the cellphone is at this current moment.

Another feature is google searches. Another is recent sightings of the face in question from various cameras (this is the weakest portion, but it is coming along nicely).

If the individual stores files online with cloud storage you can get that content if you want. Emails of course. Let your mind wander with the possibilities.

We are shaped with by the experiences we go through in life. And mine have not left me high minded. I think it is done trivially, whenever they feel like it, and for whatever reason.

When I was in college, I had a fellow student who was in an intelligence unit in the military reserver. Nothing special about it really. But he once told me a story about how they had to classify paper plates as "Secret," so they could throw them in the documents to be shredded bag.

And no one ever has, or ever will get in trouble for it unless some other branch of the government makes it an issue for some reason.

And I think Google and Facebook pretty much let the government do what they want. The less they they about it the better, for a number of reasons.

But I'd wager a guess that a number of guys at the NSA can log into google or Facebook with admin privileges as powerful as they get.

Of course I think every country in the world pretty much runs their IT services the same way.

Stuki said...

Man, aren't you pessimistic.....

The Internet is not inherently a surveillance state. There are also technological trends working in the opposite direction. As you mentioned, Tor for example, makes spying on people substantially more expensive. As does Bitcoin, at least when combined with distributed mixing like Zerocoin overlaid on top of it.

Much of the ability of advertisers to see everything we do, is because we are so darned predictable. Coming in from the same, linkable to physical address IP address, carrying the same browser cookies, and paying for things with funds linkable to offline persons. Onion routing, if used pervasively, gets rid of the first. Stripping identifying information out of browser requests is trivial. And provably mixed, anonymous currencies gets around the last of the three.

As more and more people realize that they are being spied on (and they will; since whatever the NSA sees today, the IRS, DHS trial lawyers and blackmailers/extortionists will see tomorrow), they will have an incentive to utilize available counter spy technologies. Even if that does involve some inevitable inconvenience up front.

No doubt those who see themselves as justified in spying on others, will fight tooth and nail, with threats of jailing Tor edge server owners and Bitcoin operators for aiding and abetting childporn peddlers, drug dealers and terrorists; but one would hope that even the most successfully well indoctrinated sheeple would eventually figure out that the real threats they face in their daily lives come from those insisting on spying on everything they do, not from all the imaginary boogeymen those guys invoke to justify their spying.

sunbeam said...

You need to come over to the darkside young Jedi.

Just do a quick wiki search on the history of Tor. Look who developed it.

Now ask yourself who will out of the goodness of their heart run a Tor server? How do they pay for the bandwidth for one thing?

I could be wrong, but I'd sure ike for someone to explain to me how even the limited Tor network finances itself.

Now ask yourself this question: There is a bunch of data out there. A lot. How do you know what to look at anyway?

Hey I know, let's build this thing that lets people do things anonymously. Stuff they don't want other people or governments to know about. It's a selected audience after all, people trying to conceal something, or just be discreet on general principle.

Most of it is going to be junk. People getting around sporting event blackouts and the like.

But some of it will be useful. Plus you get a list of IP numbers of people who a) can figure out how to use a proxy server, or install a Tor app, and b) know enough to be concerned.

Useful people to know about if nothing else.

I also think if I were a spook, I'd start up this service, one like adultfriendfinder, or ashleymadison. Never hurts to know about that kind of thing.

Build it, and they will come.

Wanna bet about whether they have tracked every bit torrent that existed for the past 5 or 6 years? I'd say it is a safe bet.

I also wouldn't put too much credence into how well cookie cutters and the like work. Browsers are pretty distinctive even if you turn off tracking.

Do some googling and read all about it.

sunbeam said...

Something else I wanted to add. It used to be you had to have jailers if you had the jailed.

If you had a totalitarian state, you had to have a lot of people to keep an eye on everyone.

Things have changed. Computers have changed everything obviously, but they have changed this dynamic as well.

I imagine a world, if it doesn't exist already, where mentioning the word "allah" or "revolution," or "tea party" can alert someone that something interesting may be going on. Whatever trigger you want really.

And if you store everything, and can always get at it, you can just look at someone's complete history.

This would have been clunky and unworkable in the old Soviet Union or East Germany.

Now it's eminently doable.

Mr. Sunshine said...

If you know about TOR, you're about 6 Sigmas removed from the average net user. If you use it, double that. If you use PGP, you're already on the list, along with the people who you send and receive mail and files with. Write about "Leon Ferber", an old friend of mine who invented voice and speech recognition back in the 60's and you'll be able to watch your computer /doc directories upload to a navy.mil site at 1:11am daily, if you have a packet sniffer :)

Good piece, Dr. S. I was waiting for your considered response to this deal.

dr2chase said...

Stuki, we're predictable because we're lazy creatures of habit, and we should have the simultaneously exercisable rights of both laziness and privacy.

James said...

Great, Stuart. I was curious what you've been thinking about all this. Thanks for putting it down on paper.

Aaron said...

I don't completely agree with your analysis. Although surely security breaches would not be possible without software vulnerabilities (as a necessary cause) it isn't sufficient by itself. I can easily secure any network with vulnerable nodes. The problems is that to do so would require extremely strict access control lists - which would undermine the primary purpose of the network which is communication. The problem, as I see it, with security is the tension or balance between permissive and restricted communication. The same is true in society at large in striking a balance of trust and distrust with our fellow citizens. Most of the time I can trust most of the people - which allows me to conduct useful transactions and have meaningful relationships. But sometimes I can't trust some small portion of people - so I have to invest some time and energy in defensive strategies (firearms, martial arts, locks on my doors, etc). I could have complete security by holing up in my basement with a shotgun - or by having highly restricted ACLs at the edge of the network. But I'd have a pretty meager lifestyle and a pretty meager network.

Anyhoo, as a IT security analyst I had a question regarding the Snowden leaks. My current theory is the NSA is mirroring the servers of Google, Yahoo, et al and issuing FISA-approved searches when needed - which gives the NSA what they want and allows the companies a way to save face. However, I've been contemplating a far darker possibility - that the NSA has comprised some of the root CAs. If this is the case I can't imagine what the consequences would be if it ever came to light. How likely/possible do you think this would be?

For sunbeam, only an exit relay can see the traffic on a Tor circuit but it cannot determine the origin - that's the whole point of onion routing. There is some speculation that a confederacy of Tor exit relays could begin to profile an origin point through traffic flow analysis - but it would be tricky and probably not worth anyone's time. Tor provides anonymity - not secrecy.

I predict tools like Tor and PGP will come into greater use in response to the NSA.

Aaron said...

I also wanted to point out that this Orwellian state we fear so much is probably rapidly drawing to a close (at least in regards to electronic communication) IPv6 is just around the corner and will re-establish end-to-end connectivity as originally envisioned. IPsec is required for IPv6 http://en.wikipedia.org/wiki/Ipsec - it's no longer optional. Additionally, with self-assigned nearly unique-local addresses:
- two parties could communicate in tunnel mode and nothing could be surveilled except that encrypted traffic had passed between two ISPs. Additional layers of anonymity could be provided by onion routing. IPv6 means secure end-to-end communication that is potentially anonymous. And it's already here.

Chris Reynolds said...


"- two parties could communicate in tunnel mode and nothing could be surveilled except that encrypted traffic had passed between two ISPs"


They're on a watch list...

sunbeam said...

Aaron said:

"For sunbeam, only an exit relay can see the traffic on a Tor circuit but it cannot determine the origin - that's the whole point of onion routing. There is some speculation that a confederacy of Tor exit relays could begin to profile an origin point through traffic flow analysis - but it would be tricky and probably not worth anyone's time. Tor provides anonymity - not secrecy."

Yeah, but what if it is a colossal scam? Do you know of anyone who helps run any of this? I mean someone involved with actually implementing it?

Because this troubles me:


Originally sponsored by the U.S. Naval Research Laboratory[8] (which had been instrumental in the early development of onion routing under the aegis of DARPA), Tor was financially supported by the Electronic Frontier Foundation from 2004 to 2005.[10] Tor software is now developed by the Tor Project, which has been a 501(c)(3) research-education nonprofit organization [11] based in the United States of America [1] since December 2006. It has a diverse base of financial support;[10] the U.S. State Department, the Broadcasting Board of Governors, and the National Science Foundation are major contributors.[12] As of 2012, 80% of the Tor Project's $2M annual budget comes from the United States government, with the Swedish government and other organizations providing the rest,[13] including NGOs and thousands of individual sponsors.[14]"

A couple things:

1) Even as small as TOR is presently, 2 million seems a little light to run even a small thing.

2) Why exactly are these organizations involved?

3) If TOR doesn't do exactly what you think it does, can you tell from the outside?

I'll defer to your knowledge if you say otherwise. But do you know anyone running this?

Personally, if I have things I want secret I'm not using something that has the words "US Navy, DARPA, State Department, US Government, and National Science Foundation" so prominently.

So I'll ask again: If this thing is totally compromised, and was cynically designed from the beginning to ensnare the would-be-clever, how could you tell?

Brandon Thomson said...

(sunbeam) "Yeah, but what if it is a colossal scam? Do you know of anyone who helps run any of this? I mean someone involved with actually implementing it?"

I have watched some presentations by lead developers on youtube. They seem like decent guys. I also monitored tor-relays for a while but never ended up starting a relay (mailing list located at https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays/). I do run a "entry point" node which is a less legally risky way to contribute to the network.

The development discussion is on https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev/ (that's where you would meet the people who "implement" the software).

Given the number of developers who have looked at Tor over the years it seems pretty unlikely that it is a colossal scam. However, let's assume you don't trust these people. Unfortunately, in order to trust software without trusting the people who wrote it or reviewed it, your only option is to reverse engineer it yourself or to pay someone you trust to do that. This is obviously highly impractical. Unfortunately this is a fundamental problem in computing and affects everyone who uses software they didn't write.

Even if Tor works exactly as advertised there are still certain major problems with it. The most obvious one is that use of Tor can easily be detected if someone is sniffing the wire, and such use probably makes you a target of interest for intelligence agencies.

Overall I appreciate your skepticism about Tor and certainly wouldn't ask anyone to trust it without looking into it deeply.

(sunbeam) "Even as small as TOR is presently, 2 million seems a little light to run even a small thing."

Here it's important to point out that the Tor Project, Inc as an organization is not the sole entity responsible for Tor. The code is open source and most probably many people contribute, not just employees of the Tor Project. Also, note that the Tor Project does not actually run the network; that's mostly done by volunteers who contribute the bandwidth and run the servers. These costs (the bandwidth, the CPU time, and the sysadmin time to maintain the servers) are quite significant and are not captured in that 2 million number.

Sorry I don't have time for a more detailed reply at the moment, but hopefully this info is useful for some readers.

Aaron said...


Your caution is a healthy perspective. But...

Tor is open-source - that means anyone can look at the code and determine if it's doing something other than what it claims to do. I'm not qualified to assess personally - I work in IT but it's been a long time since I wrote any code. But personally I do trust those who do look at the code and analyze it - if it were malicious someone would notice and speak up.

If you don't want to use something created by DARPA, don't use the internet! :) Ultimately, Tor is a protocol - an agreed upon way of doing something. There is any number of possible softwares that could successfully implement the protocol (the same is true of the two protocols TCP and IP invented by DARPA and used by the internet) - but as it stands there is one standard software in wide use which is available for download at the tor project website. In general, you should trust open-source software (code that can be analyzed to determine its actual behavior) and be wary of closed-source or proprietary software which cannot be analyzed.

And I don't know anyone working on the project or really know much about it except a basic understanding of how the protocol functions. So I don't know the history of the project or why any particular organization is involved. But I don't find $2million to be a small number. Most people who work on developing open-source software are doing it in their free time as volunteers - they aren't paid. Which is why open-source software is generally underdeveloped relative to proprietary closed-source software.

I recommend checking out Linux Tails - at least reading the documentation and explanation. I think it's a pretty good intro written in layman's terms on the how and why of encryption and anonymity. And it's a useful tool that anyone can use.


Here they address your concerns: https://tails.boum.org/doc/about/trust/

We all have a choice - will you end up on a "watch list" for using anonymizing or encrypting tools? I have no idea. But I do know we have a choice - we can go to protests or we can stay home because we know that the police will have a camera filming everyone participating (at least that's been my experience). We can wear a t-shirt with Arabic script when we board a plane or not (haven't tried this one myself since I'm not fond of cavity searches - lol). We can dissent or we can lose our rights through passive acquiescence. I'm reminded of the movie Spartacus when all the slaves stand up and claim to be Spartacus. If we all are on a watch-list, the watch-list becomes meaningless. Our choices mean the difference between freedom and fear.

Just my opinion, but I consider using privacy tools to be something that every responsible citizen needs to do. Just like voting or owning a firearm, if you do not exercise your rights, you will lose them. History has shown that our species has a nasty habit of taking away reasonable human rights whenever the opportunity presents itself.

But before you decide to participate in a protest, own a firearm, or use digital privacy tools - read the opinion of someone who's "been there, done that":


Unknown said...

Working for Blackwater in Japan ( not a dangerous place mind you ) as basically a security guard, my brother was earning almost 100k. Top Secret Clearance is a valuable union card.

Also, the willingness to move where ever you are in demand is very valuable too. If you are willing to move just anywhere in the US, and work as a programming contractor you'll find it difficult not to break 100k even with modest skill level. I've seen people doing what I was doing for 60k sitting ten feet from me earn 250k merely because they moved there to do it filling the demand for additional labor.

There are many many big dumb companies with billions that spend this kind of money for those willing to move ( in the case of programming which can be done from Timbuktu, this is silly ). Of course the government is the biggest dumbest corporation out there.

Still Snowden probably could command that kind of money because he had clearance and had been able to sell his being in Hawaii as having moved there to satisfy the NSA's need.