Snooping on Tor from Your Load Balancer

A couple of years ago I was assigned a simulation project for a new technology that mapped and categorized outbound end-user traffic. I thought, “Hmmm, where can I get live user traffic?” The idea of simulating end-user traffic via scripts, or (shudder) manual testing, seemed like serious drudgery—like weeding a garden, polishing silver, or proofreading my own writing. Im better than that!

An epiphany struck me: My test network in my basement could get real live end-user traffic, without simulation, by running a Tor (The Onion Router) exit node.

The Tor network is fascinating. Fascinating, people! It is the original, decentralized anonymization service that the United States government simultaneously funds and loves to hate.

DARPA—the Defense Advanced Research Projects Agency—invented the Tor network, and still partially funds it, as a way for overseas spies to get data home without blowing their cover via IP address trails. Yet, Tor annoys law enforcement (e.g., the FBI), which can’t track hackers, thieves, and drug dealers who use it. Today, the Tor network is mostly run by true (privacy) believers at the Tor Project.¹ These volunteers are constantly trying to improve the network, releasing Tor Browser updates and closing the periodic de-anonymization vulnerabilities that pop up.

It was suspected that the U.S. National Security Agency (NSA) ran up to 50 percent of the Tor nodes in the aftermath of the Heartbleed incident. But because the Tor network is actually pretty good at anonymization, the NSA still couldn’t de-anonymize users, at least according to the Snowden papers.

The Tor network is composed of two types of nodes: relay nodes and exit nodes.

Tor relay nodes are simple and safe—they’re just bridges from one node to the next in the network. They have no exposure to volatile Tor traffic passing through them because it’s encrypted.

My Decision to Run a Tor Exit Node

Tor exit nodes, on the other hand, are anything but simple and safe. A Tor exit node will appear to the Internet as if it were the actual user on the other side of Tor. The user could be a researcher. Or a journalist. Or a spy. Or a child pornographer. Or a digital pirate. Or a hacker. The only thing you know about someone who is using Tor is that, well, they don’t want to be known. They want to be anonymous.

So, I thought long and hard about whether to run a Tor exit node. I didn’t want to be wrongly fingered as a child abuser, as happened to this Seattle couple,² or as a terrorism supporter, which happened to this professor.³ On the other hand, I do believe in free speech as a general human right. And if running a Tor exit node could enable a journalist or whistle-blower in an oppressive regime the freedom to publish their findings, it seemed worth the risk. And, of course, I needed that live traffic for testing.

The Setup

I bought a mid-range PC from my local pawnshop, wiped the hard drive, and installed my favorite Linux distribution, Ubuntu Server. I followed the Tor Project’s instructions on installing and configuring a Tor exit node. (I was about to link to those instructions, but here’s a much, much better guide to running a Tor exit node.⁴) Why not run the Tor exit node from one of my main servers? Honestly, I was nervous about the node containing a zero-day break of some kind. If there’s any software in the world that a nation-state actor would like to get a back door into, it would be Tor. And with the amount of hacking traffic exiting a Tor node, it seemed like a sensible idea to isolate the exit node as much as possible.

So, I put the node on its own subnet with a single interface connected directly to a dedicated interface at my load balancer. I then put a firewall rule on the load balancer blocking all non-routable traffic coming from the node. Traffic spilling out of the exit node could only be forwarded back out to the Internet and couldn’t probe my internal network.

I defined a single virtual server on the inbound side of the load balancer to forward incoming Tor traffic (in-tor2) directly to the exit node on port 9001, and then another virtual server to forward the “exit” traffic back to the Internet (reversi).

Next, it was time to actually fire up the Tor exit node and add its startup script to my sys5 config. I had a few missteps, and eventually figured out that I needed to look at /var/log/tor/log. The Tor node isn’t really active until it can convince itself that it’s properly receiving traffic from the Tor network.

One of the missteps I found was that, by default, the Tor node would accept and relay BitTorrent traffic. My American ISP detected the BitTorrent traffic exiting my node and started sending me emails, and, I suspect, interfering with my network traffic (though I didn’t prove that beyond a suspicion).

Fortunately, the Tor Project gives instructions on how to configure your Tor exit node to filter out torrent and other troublesome traffic.⁵

I also didn’t want Tor hogging all my bandwidth, so I configured it with what I thought were sane defaults for maximum input and output network volume (100 Mbps in and out).

Initial Tor traffic was sparse and spotty. Perhaps the Tor network doesn’t automatically assume that brand new nodes have good bandwidth, so it slowly ramps up traffic it sends through the node until that node has proven itself. Eventually, after a week or so, I started seeing a steady clip of Tor traffic passing through my exit node—a couple of gigabytes per day.

What Exactly Do I Mean by Snooping?

As connections leave the Tor exit node, they are destined for actual Internet services, usually on well-known ports like HTTP, HTTPS, Email, FTP, IRC, etc. The Tor network is acting like a SNAT, masking all the true IP addresses behind it.

I configured another outbound virtual server on my load balancer to specifically deliver HTTP. Traffic destined for port 80 started flowing through it—it was actually about half the traffic exiting the Tor node.

So, the system was all set up to fully snoop on HTTP connections exiting my Tor node. Things I could have done to that traffic include:

• Record every packet
• Capture usernames and passwords
• Steal session cookies
• Poison web pages
• Inject JavaScript tracking code
• Return malware
• Clickjack websites
• Replace all advertising

I had, in effect, created a forward proxy that could have done a huge number of terrible things to the users.

But as curious as I was, my personal code would not allow me to do any of those things. I always play lawful, good characters in RPGs, which, to me, is the prime indicator of one’s personal moral compass. I’ve surveyed three dozen InfoSec professionals⁶ about what it would take to move to the dark side, and most, including me, would not turn for any amount of personal gain.

What I did do, though, was insert a little script into the proxy to count the number of unprotected login pages passing through it. I did not record, or even look for, user credentials.

On a typical day, I saw about 2,000 unprotected login pages passing through my Tor exit node.

I found that number extremely high, and worrisome. If a person is concerned enough about privacy to use the Tor network, you’d think they’d also be using HTTPS to protect their network traffic, especially around login pages.

The End of the Experiment

My Tor exit node adventure came to an end one day a few weeks later. With the node humming along in the basement, I was upstairs on my MacBook trying to log into my banking website, but the bank wasn’t accepting my connections. I assumed it was some kind of maintenance issue at the bank, so I waited a couple of days and tried again. Still no luck. And then other sites started denying my connections. One of them was Starbucks, which prevented me from reloading my Starbucks card.

I realized that, duh, my home had been globally marked as a Tor exit node, and sites were (correctly) blocking my traffic. I usually recommend to customers I talk to that they do the same. While, yes, newspapers should accept Tor traffic because they are dealing in a trade where the free flow of news is perhaps a human right, there is probably no good reason for a bank or insurance company to accept Tor traffic. And many reasons not to. An enterprise can block a whole threat surface of nasty hacking traffic just by blocking the thousand or so Tor exit nodes on the Internet.

At first it seemed I could get around the blocking problem by VPNing into work and accessing my online banking page from there. But that would have created a bridge between my Tor exit node and the corporate network, which seemed like a really bad idea. Like the kind of idea that gets you fired. But I couldn’t live without online banking, or many of the other services that were blocking me.

So, reluctantly, I disabled the inbound Tor virtual server and let all the connections bleed off. Then I shut down the pawn shop computer and renewed my ISP lease to get a new IP address.

Was It Worth It?

Ultimately, there was a decent amount of satisfaction to be gained from running the Tor exit node. The project I’d been assigned that ultimately required the user traffic got postponed, which deprioritized the need for the live traffic. But, I saved all the configurations so I can fire it up again at any time and start passing positive traffic on behalf of all the freedom fighters and oppressed journalists in the world.