Peer-to-Peer Communication Across Network Address Translators

http://www.bford.info/pub/net/p2pnat/

Peer-to-Peer Communication Across Network Address Translators

Bryan Ford

Massachusetts Institute of Technology

baford (at) mit.edu
Pyda Srisuresh

Caymas Systems, Inc.

srisuresh (at) yahoo.com
Dan Kegel

dank (at) kegel.com

J'fais des trous, des petits trous


toujours des petits trous

- S. Gainsbourg

Abstract:

Network Address Translation (NAT)causes well-known difficultiesfor peer-to-peer (P2P) communication,since the peers involved may not be reachableat any globally valid IP address.Several NAT traversal techniques are known,but
their documentation is slim,and data about their robustness or relative merits is slimmer.This paper documents and analyzesone of the simplest but most robust and practicalNAT traversal techniques,commonly known as “hole punching.”Hole punching is moderately
well-understood for UDP communication,but we show how it can be reliably used to set up peer-to-peer TCP streams as well.After gathering data on the reliability of this techniqueon a wide variety of deployed NATs,we find that about 82% of the NATs tested support
hole punching for UDP,and about 64% support hole punching for TCP streams.As NAT vendors become increasingly conscious ofthe needs of important P2P applicationssuch as Voice over IP and online gaming protocols,support for hole punching is likely to increase
in the future.

1 Introduction

The combined pressures of tremendous growthand massive security challengeshave forced the Internet to evolvein ways that make life difficult for many applications.The Internet's original uniform address architecture,in which
every node has a globally unique IP addressand can communicate directly with every other node,has been replaced with a new
de facto Internet address architecture,consisting of a global address realmand many private address realmsinterconnected by Network Address Translators (NAT).In this new address architecture,illustrated in Figure 1,only
nodes in the “main,” global address realmcan be easily contacted from anywhere in the network,because only they have unique, globally routable IP addresses.Nodes on private networkscan connect to other nodes on the same private network,and they can usually
open TCP or UDP connectionsto “well-known” nodes in the global address realm.NATs on the pathallocate temporary public endpoints for outgoing connections,and translate the addresses and port numbersin packets comprising those sessions,while generally blocking
all incoming trafficunless otherwise specifically configured.

Figure 1:Public and private IP address domains

The Internet's new de facto address architectureis suitable for client/server communicationin the typical case when the client is on a private networkand the server is in the global address realm.The architecture makes
it difficult for two nodeson different private networks to contact each other directly,however,which is often importantto the “peer-to-peer” communication protocolsused in applications such as teleconferencing and online gaming.We clearly need a way
to make such protocolsfunction smoothly in the presence of NAT.

One of the most effective methodsof establishing peer-to-peer communicationbetween hosts on different private networksis known as “hole punching.”This techniqueis widely used already in UDP-based applications,but essentially
the same technique also works for TCP.Contrary to what its name may suggest,hole punching does not compromise the security of a private network.Instead,hole punching enables applications to functionwithin the the default security policy of most NATs,effectively
signaling to NATs on the paththat peer-to-peer communication sessionsare “solicited”and thus should be accepted.This paper documents hole punching for both UDP and TCP,and details the crucial aspects of both application and NAT behaviorthat make hole punching
work.

Unfortunately,no traversal technique works with all existing NATs,because NAT behavior is not standardized.This paper presents some experimental resultsevaluating hole punching support in current NATs.Our data is derived from
results submitted by users throughout the Internetby running our “NAT Check” toolover a wide variety of NATs by different vendors.While the data points were gathered from a “self-selecting” user communityand may not be representativeof the true distribution
of NAT implementations deployed on the Internet,the results are nevertheless generally encouraging.

While evaluating basic hole punching,we also point out variations that can make hole punching workon a wider variety of existing NATsat the cost of greater complexity.Our primary focus, however,is on developing the
simplest hole punching techniquethat works cleanly and robustlyin the presence of “well-behaved” NATsin any reasonable network topology.We deliberately avoid excessively clever tricksthat may increase compatibilitywith some existing “broken” NATs in
the short term,but which only work some of the timeand may cause additional unpredictability and network brittlenessin the long term.

Although the larger address space of IPv6 [3]may eventually reduce the need for NAT,in the short term IPv6is
increasing the demand for NAT,because NAT itself provides the easiest way to achieve interoperabilitybetween IPv4 and IPv6 address domains [24].Further,the anonymity and inaccessibility
of hosts on private networkshas widely perceived security and privacy benefits.Firewalls are unlikely to go away even when there are enough IP addresses:IPv6 firewallswill still commonly block unsolicited incoming traffic by default,making hole punching useful
even to IPv6 applications.

The rest of this paper is organized as follows.Section 2introduces basic terminology and NAT traversal concepts.Section 3details
hole punching for UDP,and Section 4introduces hole punching for TCP.Section 5summarizes important properties
a NAT must havein order to enable hole punching.Section 6presents our experimental resultson hole punching support in popular NATs,Section 7
discusses related work,and Section 8 concludes.

2 General Concepts

This section introduces basic NAT terminologyused throughout the paper,and then outlines general NAT traversal techniquesthat apply equally to TCP and UDP.

2.1 NAT Terminology

This paper adopts the NAT terminology and taxonomydefined in RFC 2663 [21],as well as additional terms defined more recently in RFC 3489 [19].

Of particular importance is the notion of session.A
session endpoint for TCP or UDPis an (IP address, port number) pair,and a particular
session is uniquely identified byits two session endpoints.From the perspective of one of the hosts involved,a session is effectively identified by the 4-tuple(local IP, local port, remote IP, remote port).The
direction of a sessionis normally the flow direction of the packetthat initiates the session:the initial SYN packet for TCP,or the first user datagram for UDP.

Of the various flavors of NAT,the most common type is
traditional or outbound NAT,which provides an asymmetric bridgebetween a private network and a public network.Outbound NAT by default allows only outbound sessions to traverse the NAT:incoming packets are dropped unless the NAT identifies
themas being part of an existing sessioninitiated from within the private network.Outbound NAT conflicts with peer-to-peer protocolsbecause when both peers desiring to communicate are “behind”(on the private network side of) two different NATs,whichever peer
tries to initiate a session,the other peer's NAT rejects it.NAT traversal entailsmaking P2P sessions look like “outbound” sessionsto
both NATs.

Outbound NAT has two sub-varieties:Basic NAT, which only translates IP addresses,and
Network Address/Port Translation (NAPT),which translates entire session endpoints.NAPT, the more general variety,has also become the most commonbecause it enables the hosts on a private networkto share the use of a
single public IP address.Throughout this paper we assume NAPT,though the principles and techniques we discussapply equally well (if sometimes trivially) to Basic NAT.

2.2 Relaying

The most reliable--but least efficient--methodof P2P communication across NATis simply to make the communicationlook to the network like standard client/server communication,through relaying.Suppose two clienthosts


and


have each initiated TCP or UDP connections to a well-known server


,at


's global IP address 18.181.0.31 and port number 1234.As shown in Figure 2,the
clients reside on separate private networks, andtheir respective NATs prevent either client from directlyinitiating a connection to the other.Instead of attempting a direct connection, the two clients can simplyuse the server


to relay messages between them. For example, tosend a message to client


, client


simply sends the message toserver


along its already-established client/server connection, andserver


forwards the message on to client


using its existingclient/server connection with


.

Figure 2:NAT Traversal by Relaying

Relaying always worksas long as both clients can connect to the server.Its disadvantages are thatit consumes the server's processing power and network bandwidth,and communication latency between the peering clientsis likely increased
even if the server is well-connected.Nevertheless,since there is no more efficient techniquethat works reliably on all existing NATs,relaying is a useful fall-back strategyif maximum robustness is desired.The TURN protocol [18]defines
a method of implementing relayingin a relatively secure fashion.

2.3 Connection Reversal

Some P2P applications usea straightforward but limited technique,known as
connection reversal,to enable communicationwhen both hosts have connections to a well-known rendezvous server


and only one of the peers is behind a NAT,as shown in Figure 3.If


wants to initiate a connection to


,then a direct connection attempt works automatically,because


is not behind a NATand


's NAT interprets the connection as an outgoing session.If


wants to initiate a connection to


, however,any direct connection attempt to


is blocked by


's NAT.

can instead relay
a connection request to

through a well-known server


,asking


to attempt a “reverse” connection back to


.Despite the obvious limitations of this technique,the central idea of using a well-known rendezvous serveras an intermediary to help set up
direct peer-to-peer connectionsis fundamental to the more general hole punching techniquesdescribed next.

Figure 3:NAT Traversal by Connection Reversal

3 UDP Hole Punching

UDP hole punching enables two clients to set upa direct peer-to-peer UDP sessionwith the help of a well-known rendezvous server,even if the clients are both behind NATs.This technique was mentionedin section 5.1 of RFC 3027 [10],documented
more thoroughly elsewhere on the Web [13],and used in recentexperimental Internet protocols [17,11].Various
proprietary protocols,such as those for on-line gaming,also use UDP hole punching.

3.1 The Rendezvous Server

Hole punching assumes that the two clients,

and


,already have active UDP sessions with a rendezvous server


.When a client registers with


,the server records
two endpoints for that client:the (IP address, UDP port) pairthat the client
believes itself to be using to talk with

,and the (IP address, UDP port) pairthat the server
observes the client to be using to talk with it.We refer to the first pair as the client's
private endpointand the second as the client's public endpoint.The server mightobtain the client's private endpoint from the client itselfin a field in the body of the client's registration message,and obtain the client's public endpointfrom
the source IP address and source UDP port fieldsin the IP and UDP headers of that registration message.If the client is
not behind a NAT,then its private and public endpoints should be identical.

A few poorly behaved NATs are known to scan the body of UDP datagramsfor 4-byte fields that look like IP addresses,and translate them as they would the IP address fieldsin the IP header.To be robust against such behavior,applications
may wish to obfuscate IP addresses in messages bodies slightly,for example by transmitting the one's complement of the IP addressinstead of the IP address itself.Of course, if the application is encrypting its messages,then this behavior is not likely to be
a problem.

3.2 Establishing Peer-to-Peer Sessions

Suppose client

wants to establish a UDPsession directly with client


.Hole punching proceeds as follows:


initially does not know how to reach


,so


asks


for help establishing a UDP session with


.


replies to


with a messagecontaining


's public
and private endpoints.At the same time,

uses its UDP session with


to send


a connection request messagecontaining


's public and private endpoints.Once these messages are received,


and

know each other's public and private endpoints.
When

receives


's public and private endpoints from


,

starts sending UDP packets
toboth of these endpoints,and subsequently “locks in” whichever endpointfirst elicits a valid response from


.Similarly, when


receives


's public and private endpointsin the forwarded connection request,


starts sending UDP packets to

at each of


's known endpoints,locking in the first endpoint that works.The order and timing of these messages are not criticalas long as they are asynchronous.

We now considerhow UDP hole punching handleseach of three specific network scenarios.In the first situation,representing the “easy” case,the two clients actually reside behind the same NAT,on one private network.In the second,most
common case,the clients reside behind different NATs.In the third scenario,the clients each reside behind
two levels of NAT:a common “first-level” NATdeployed by an ISP for example,and distinct “second-level” NATssuch as consumer NAT routers for home networks.

It is in general difficult or impossible for the application itselfto determine the exact physical layout of the network,and thus which of these scenarios (or the many other possible ones)actually applies at a given time.Protocols
such as STUN [19]can provide some information about the NATs present on a communication path,but this information may not always be complete or reliable,especially when multiple levels
of NAT are involved.Nevertheless,hole punching works automatically in all of these scenarioswithout the application having to knowthe specific network organization,as long as the NATs involved behave in a reasonable fashion.(“Reasonable” behavior
for NATs will be describedlater in Section 5.)

3.3 Peers Behind a Common NAT

First consider the simple scenario in which the two clients(probably unknowingly) happen to reside behind the same NAT,and are therefore located in the same private IP address realm,as shown in Figure 4.
Client

has established a UDP session with server


, to which thecommon NAT has assigned its own public port number 62000.Client


has similarly established a session with


,to which the NAT has assigned public port number 62005.

Figure 4:UDP Hole Punching, Peers Behind a Common NAT

Suppose that client

uses the hole punching technique outlined aboveto establish a UDP session with


,using server


as an introducer.Client


sends


a message requesting a connection to


.

responds to


with


's public and private endpoints,and also forwards


's public and private endpoints to


.Both clients then attempt to send UDP datagrams to each otherdirectly at each of these endpoints.The messages directed to the public endpointsmay
or may not reach their destination,depending on whether or not the NAT supports hairpin translationas described below in Section 3.5.The messages directed at the private endpointsdo
reach their destinations, however,and since this direct route through the private networkis likely to be faster than an indirect route through the NAT anyway,the clients are most likely to select the private endpointsfor subsequent regular communication.

By assuming that NATs support hairpin translation,the application might dispense with the complexityof trying private as well as public endpoints,at the cost of making local communication behind a common NATunnecessarily pass
through the NAT.As our results in Section 6 show, however,hairpin translation is still much less common among existing NATsthan are other “P2P-friendly” NAT behaviors.For now, therefore,applications
may benefit substantiallyby using both public and private endpoints.

3.4 Peers Behind Different NATs

Suppose clients

and


have private IP addressesbehind different NATs,as shown in Figure 5.


and

have each initiated UDP communicationsessions from their local port 4321to port 1234 on server


.In handling these outbound sessions,NAT


has assigned port 62000 at its own public IP address, 155.99.25.11,for the use of


's session with


,and NAT


has assigned port 31000 at its IP address, 138.76.29.7,to


's session with


.

Figure 5:UDP Hole Punching, Peers Behind Different NATs

In

's registration message to


,

reports its private endpoint
to

as 10.0.0.1:4321,where 10.0.0.1 is


's IP address on its own private network.


records

's reported private endpoint,along with


's public endpoint as observed by


itself.

's public endpoint
in this case is 155.99.25.11:62000,the temporary endpoint assigned to the session by the NAT.Similarly, when client


registers,

records


's private endpoint as 10.1.1.3:4321and


's public endpoint as 138.76.29.7:31000.

Now client

follows the hole punching procedure described aboveto establish a UDP communication session directly
with

.First,


sends a request message to


asking for help connecting with


.In response,


sends


's public and private endpoints to


,and sends


's public and private endpoints to


.

and


each start trying to send UDP datagramsdirectly to each of these endpoints.

Since

and


are on different private networksand their respective private IP addresses are not globally routable,the messages sent to these endpointswill
reach either the wrong host or no host at all.Because many NATs also act as DHCP servers,handing out IP addresses in a fairly deterministic wayfrom a private address pool usually determined by the NAT vendor by default,it is quite likely in practicethat


's messages directed at


's private endpointwill reach
some (incorrect) host on

's private networkthat happens to have the same private IP address as


does.Applications must therefore authenticate all messagesin some way to filter out such stray traffic robustly.The messages might include application-specific
names or cryptographic tokens,for example,or at least a random nonce pre-arranged through


.

Now consider

's first message sent to


's public endpoint,as shown in Figure 5.As this outbound message
passes through

's NAT,this NAT notices that this is the first UDP packet in a new outgoing session.The new session's source endpoint (10.0.0.1:4321)is
the same as that of the existing session between

and


,but its destination endpoint is different.If NAT


is well-behaved,it preserves the identity of


's private endpoint,consistently translatingall outbound sessionsfrom private source endpoint 10.0.0.1:4321to the corresponding public
source endpoint 155.99.25.11:62000.

's first outgoing message to


's public endpointthus, in effect,“punches a hole” in


's NATfor a new UDP sessionidentified by the endpoints(10.0.0.1:4321, 138.76.29.7:31000) on


's private network,and by the endpoints(155.99.25.11:62000, 138.76.29.7:31000) on the main Internet.

If

's message to


's public endpointreaches


's NAT before


's first message to


has crossed


's own NAT,then


's NAT may interpret


's inbound messageas unsolicited incoming traffic and drop it.

's
first message to

's public address, however,similarly opens a hole in


's NAT,for a new UDP sessionidentified by the endpoints(10.1.1.3:4321, 155.99.25.11:62000) on


's private network,and by the endpoints(138.76.29.7:31000, 155.99.25.11:62000) on the Internet.Once the first messages from


and


have crossed their respective NATs,holes are open in each directionand UDP communication can proceed normally.Once the clients have verified
that the public endpoints work,they can stop sending messages to the alternative private endpoints.

3.5 Peers Behind Multiple Levels of NAT

In some topologies involving multiple NAT devices,two clients cannot establish an “optimal” P2P route betweenthem without specific knowledge of the topology.Consider a final scenario,depicted in Figure 6.Suppose
NAT

is a large industrial NAT deployed by an internetservice provider (ISP) to multiplex many customers onto a few publicIP addresses, and
NATs

and


are small consumer NAT routersdeployed independently by two of the ISP's customers to multiplextheir private home networks onto their respective
ISP-provided IPaddresses. Only server

and NAT


have globally routable IPaddresses; the “public” IP addresses used by NAT


and NAT


areactually private to the ISP's address realm, while client


's and

's addresses in turn
are private to the addressing realms of NAT

and NAT


, respectively.Each client initiates an outgoing connection to server


as before,causing NATs


and


each to create a singlepublic/private translation,and causing NAT


to establish a public/private translationfor each session.

Figure 6:UDP Hole Punching, Peers Behind Multiple Levels of NAT

Now suppose

and


attempt to establisha direct peer-to-peer UDP connectionvia hole punching.The optimal routing strategy would be for client


tosend messages to client


's “semi-public” endpoint at NAT


,10.0.1.2:55000 in the ISP's addressing realm,and for client


to send messagesto


's “semi-public” endpoint at NAT


,namely 10.0.1.1:45000.Unfortunately,


and


have no way to learn these addresses,because server


only sees the truly global public endpoints of the clients,155.99.25.11:62000 and 155.99.25.11:62005 respectively.Even if


and


had some way to learn these addresses,there is still no guarantee that they would be usable,because the address assignments in the ISP's private
address realmmight conflict with unrelated address assignmentsin the clients' private realms.(NAT


's IP address in NAT


's realmmight just as easily have been 10.1.1.3, for example,the same as client


's private address in NAT


's realm.)

The clientstherefore have no choicebut to use their global public addresses asseen by


for their P2P communication,and rely on NAT


providing
hairpin or loopback translation.When

sends a UDP datagram to


's global endpoint,155.99.25.11:62005,NAT


first translates the datagram's source endpointfrom 10.0.0.1:4321 to 10.0.1.1:45000.The datagram now reaches NAT


,which recognizes that the datagram's destination addressis one of NAT


's own translated
public endpoints.If NAT

is well-behaved,it then translates
boththe source and destination addresses in the datagramand “loops” the datagram back onto the private network,now with a source endpoint of 155.99.25.11:62000and a destination endpoint of 10.0.1.2:55000.NAT


finally translates the datagram's destination addressas the datagram enters


's private network,and the datagram reaches


.The path back to


works similarly.Many NATs do not yet support hairpin translation,but it is becoming more commonas NAT vendors become aware of this issue.

3.6 UDP Idle Timeouts

Since the UDP transport protocol provides NATswith no reliable, application-independent wayto determine the lifetime of a session crossing the NAT,most NATs simply associate an idle timer with UDP translations,closing the hole
if no traffic has used it for some time period.There is unfortunately no standard value for this timer:some NATs have timeouts as short as 20 seconds.If the application needs to keep an idle UDP session activeafter establishing the session via hole punching,the
application must send periodic keep-alive packetsto ensure that the relevant translation state in the NATsdoes not disappear.

Unfortunately,many NATs associate UDP idle timers with individual UDP sessionsdefined by a particular pair of endpoints,so sending keep-alives on one sessionwill not keep other sessions activeeven if all the sessions originate
fromthe same private endpoint.Instead of sending keep-alives on many different P2P sessions,applications can avoid excessive keep-alive trafficby detecting when a UDP session no longer works,and re-running the original hole punching procedure again “on demand.”

4 TCP Hole Punching

Establishing peer-to-peer TCP connectionsbetween hosts behind NATsis slightly more complex than for UDP,but TCP hole punching is remarkably similarat the protocol level.Since it is not as well-understood,it is currently supported
by fewer existing NATs.When the NATs involved do support it, however,TCP hole punching is just as fast and reliable as UDP hole punching.Peer-to-peer TCP communication across well-behaved NATsmay in fact be
more robust than UDP communication,because unlike UDP,the TCP protocol's state machinegives NATs on the path a standard way to determinethe precise lifetime of a particular TCP session.

4.1 Sockets and TCP Port Reuse

The main practical challengeto applications wishing to implement TCP hole punchingis not a protocol issuebut an application programming interface (API) issue.Because the standard Berkeley sockets APIwas designed around the client/server
paradigm,the API allowsa TCP stream socketto be used to initiate an outgoing connection via
connect(),or to listen for incoming connections via listen() and
accept(),but not both.Further,TCP sockets usually have a one-to-one correspondenceto TCP port numbers on the local host:after the application binds one socket to a particular local TCP port,attempts to bind a second socket to the same TCP
port fail.

For TCP hole punching to work, however,we need to use a single local TCP portto listen for incoming TCP connectionsand to initiate multiple outgoing TCP connections concurrently.Fortunately, all major operating systemssupport
a special TCP socket option,commonly named

SO_REUSEADDR

,which allows the application to bind multiple socketsto the same local endpointas long as this option is set on all of the sockets involved.BSD systems have introduced a

SO_REUSEPORT

optionthat controls port reuse separately from address reuse;on such systems
both of these options must be set.

4.2 Opening Peer-to-Peer TCP Streams

Suppose that client

wishes to set up a TCP connection with client


.We assume as usualthat both


and


already have active TCP connectionswith a well-known rendezvous server


.The server recordseach registered client's public and private endpoints,just as for UDP.At the protocol level,TCP hole punching works almost
exactly as for UDP:

Client

uses its active TCP session with


to ask


for help connecting to


.


replies to


with


's public and private TCP endpoints,and at the same time sends


's public and private endpoints to


.
From the same local TCP portsthat

and


used to register with


,

and


each asynchronously make outgoing connection attemptsto the other's public and private endpointsas reported by


,while simultaneously listening for incoming connectionson their respective local TCP ports.


and


wait for outgoing connection attempts to succeed,and/or for incoming connections to appear.If one of the outgoing connection attempts failsdue
to a network errorsuch as “connection reset” or “host unreachable,”the host simply re-tries that connection attemptafter a short delay (e.g., one second),up to an application-defind maximum timeout period.
When a TCP connection is made,the hosts authenticate each otherto verify that they connected to the intended host.If authentication fails,the clients close that connectionand continue waiting for others to succeed.The clientsuse
the first successfully authenticated TCP streamresulting from this process.

Unlike with UDP,where each client only needs one socketto communicate with both


and any number of peers simultaneously,with TCP each client application must manage several socketsbound to a single local TCP port on that
client node,as shown in Figure 7.Each client needs a stream socket representing its connection to


,a listen socket on which to accept incoming connections from peers,and at least two additional stream socketswith which to initiate outgoing
connectionsto the other peer's public and private TCP endpoints.

Figure 7:Sockets versus Ports for TCP Hole Punching

Consider the common-case scenarioin which the clients


and


are behind different NATs,as shown in Figure 5,and assume that
the port numbers shown in the figureare now for TCP rather than UDP ports.The outgoing connection attempts


and


maketo each other's private endpointseither fail or connect to the wrong host.As with UDP,it is important that TCP applications authenticate
their peer-to-peer sessions,due of the likelihood of mistakenly connecting toa random host on the local networkthat happens to have the same private IP addressas the desired host on a remote private network.

The clients' outgoing connection attemptsto each other's
public endpoints,however, cause the respective NATs to open up new “holes”enabling direct TCP communication between


and


.If the NATs are well-behaved,then a new peer-to-peer TCP streamautomatically forms between them.If


's first SYN packet to


reaches


's NATbefore


's first SYN packet to


reaches


's NAT, for example,then


's NAT may interpret


's SYNas an unsolicited incoming connection attemptand drop it.

's
first SYN packet to

should subsequently get through,however,because


's NAT sees this SYNas being part of the outbound session to


that


's first SYN had already initiated.

4.3 Behavior Observed by the Application

What the client applications observe to happen with their socketsduring TCP hole punchingdepends on the timing and the TCP implementations involved.Suppose that


's first outbound SYN packet to


's public endpointis dropped by NAT


,but


's first subsequent SYN packet to


's public endpointgets through to


before


's TCP retransmits its SYN.Depending on the operating system involved,one of two things may happen:


's TCP implementation notices thatthe session endpoints for the incoming SYNmatch those of an outbound session


was attempting to initiate.

's
TCP stack therefore associates this new sessionwith the socket that the local application on


was using to
connect() to

's public endpoint.The application's asynchronous
connect() call succeeds,and nothing happens with the application's listen socket.
Since the received SYN packet did not include an ACKfor


's previous outbound SYN,

's
TCP replies to

's public endpointwith a SYN-ACK packet,the SYN part being merely a replayof


's original outbound SYN,using the same sequence number.Once


's TCP receives


's SYN-ACK,it responds with its own ACK for


's SYN,and the TCP session enters the connected state on both ends.

Alternatively,

's TCP implementation might instead noticethat


has an active listen socket on that portwaiting for incoming connection attempts.Since


's SYN looks like an incoming connection attempt,

's
TCP creates a new stream socketwith which to associate the new TCP session,and hands this new socket to the applicationvia the application's next
accept() callon its listen socket.

's TCP then responds to


with a SYN-ACK as above,and TCP connection setup proceeds as usualfor client/server-style connections.
Since

's prior outbound
connect() attempt to

used a combination of source and destination endpointsthat is now in use by another socket,namely the one just
returned to the application via accept(),

's asynchronous
connect() attemptmust fail at some point,typically with an “address in use” error.The application nevertheless has the working peer-to-peerstream socket it needs to communicate with


,so it ignores this failure.

The first behavior above appears to be usual for BSD-based operating systems,whereas the second behavior appears more common under Linux and Windows.

4.4 Simultaneous TCP Open

Suppose that the timing of the various connection attemptsduring the hole punching processworks out so thatthe initial outgoing SYN packets from
both clientstraverse their respective local NATs,opening new outbound TCP sessions in each NAT,before reaching the remote NAT.In this “lucky” case,the NATs do not reject either of the initial SYN packets,and the SYNs cross on the wire between the two
NATs.In this case,the clients observe an eventknown as a simultaneous TCP open:each peer's TCP receives a “raw” SYNwhile waiting for a SYN-ACK.Each peer's TCP responds with a SYN-ACK,whose SYN part essentially “replays” the peer's previous outgoing
SYN,and whose ACK part acknowledges the SYN received from the other peer.

What the respective applications observe in this caseagain depends on the behavior of the TCP implementations involved,as described in the previous section.If
both clients implement the second behavior above,it may be thatall of the asynchronous
connect() callsmade by the application ultimately fail,but the application running on each clientnevertheless receives a new, working peer-to-peer TCP stream socketvia
accept()--as if this TCP stream had magically “created itself” on the wireand was merely passively accepted at the endpoints!As long as the application does not carewhether it ultimately receives its peer-to-peer TCP socketsvia
connect() or accept(),the process results in a working streamon any TCP implementationthat properly implements the standard TCP state machinespecified in RFC 793 [23].

Each of the alternative network organization scenariosdiscussed in Section 3 for UDPworks in exactly the same way for TCP.For example,TCP hole punching
works in multi-level NAT scenariossuch as the one in Figure 6as long as the NATs involved are well-behaved.

4.5 Sequential Hole Punching

In a variant of the above TCP hole punching procedureimplemented by the NatTrav library [4],the clients attempt connections to each othersequentially
rather than in parallel.For example:(1)

informs


via


of its desire to communicate,without simultaneously listening on its local port;(2)


makes a
connect() attempt to

,which opens a hole in


's NATbut then fails due to a timeoutor RST from


's NAT or a RST from


itself;(3)


closes its connection to


and does a
listen() on its local port;(4)

in turn closes its connection with


,signaling


to attempt a
connect() directly to

.

This sequential procedure may be particularly usefulon Windows hosts prior to XP Service Pack 2,which did not correctly implement simultaneous TCP open,or on sockets APIsthat do not support the

SO_REUSEADDR

functionality.The sequential procedure is more timing-dependent,however,and may be slower in the common caseand less robust in unusual situations.In step (2), for example,


must allow its “doomed-to-fail” connect() attemptenough time to ensure that at least one SYN packettraverses all NATs on its side of the network.Too little delay risksa lost SYN derailing the process,whereas too much delay increases the total timerequired
for hole punching.The sequential hole punching procedure also effectively “consumes”both clients' connections to the server


,requiring the clients to open fresh connections to


for each new P2P connection to be forged.The parallel hole punching procedure,in contrast,typically completes as soon asboth clients make their
outgoing connect() attempts,and allows each client to retain and re-usea single connection to


indefinitely.

5 Properties of P2P-Friendly NATs

This section describes the key behavioral properties NATs must havein order for the hole punching techniques described above to work properly.Not all current NAT implementations satisfy these properties,but many do,and NATs are
gradually becoming more “P2P-friendly”as NAT vendors recognize the demand for peer-to-peer protocolssuch as voice over IP and on-line gaming.

This section is not meant to bea complete or definitive specification for how NATs “should” behave;we provide it merely for informationabout the most commonly observed behaviorsthat enable or break P2P hole punching.The IETF
has started a new working group, BEHAVE,to define official “best current practices” for NAT behavior.The BEHAVE group's initial draftsinclude the considerations outlined in this section and others;NAT vendors should of coursefollow the IETF working group directlyas
official behavioral standards are formulated.

5.1 Consistent Endpoint Translation

The hole punching techniques described hereonly work automaticallyif the NAT consistently maps a given TCP or UDP source endpointon the private networkto a
single corresponding public endpointcontrolled by the NAT.A NAT that behaves in this wayis referred to as a
cone NATin RFC 3489 [19] and elsewhere,because the NAT “focuses” all sessions originating from a single private endpointthrough the same public endpoint on the NAT.

Consider again the scenario in Figure 5, for example.When client


initially contacted the well-known server


,NAT


chose to use port 62000at its own public IP address, 155.99.25.11,as a temporary public endpointto representing


's private endpoint 10.0.0.1:4321.When


later attempts to establish a peer-to-peer session with


by sending a message from the same local private endpointto


's public endpoint,

depends
on NAT

preserving the identity of this private endpoint,and re-using the existing public endpoint of 155.99.25.11:62000,because that is the
public endpoint for

to which


will be sending its corresponding messages.

A NAT that is only designed to support client/server protocolswill not necessarily preserve the identities of private endpoints in this way.Such a NAT is a
symmetric NAT in RFC 3489 terminology.For example,after the NAT assigns the public endpoint 155.99.25.11:62000to client


's session with server


,the NAT might assign a different public endpoint,such as 155.99.25.11:62001,to the P2P session that


tries to initiate with


.In this case,the hole punching process fails to provide connectivity,because the subsequent incoming messages from


reach NAT


at the wrong port number.

Many symmetric NATs allocate port numbers for successive sessionsin a fairly predictable way.Exploiting this fact,variants of hole punching algorithms [9,1]can
be made to work “much of the time” even over symmetric NATsby first probing the NAT's behaviorusing a protocol such as STUN [19],and using the resulting informationto “predict” the
public port number the NATwill assign to a new session.Such prediction techniques amount to chasing a moving target, however,and many things can go wrong along the way.The predicted port number might already be in usecausing the NAT to jump to another port
number,for example,or another client behind the same NATmight initiate an unrelated session at the wrong timeso as to allocate the predicted port number.While port number prediction can be a useful trickfor achieving maximum compatibility with badly-behaved
existing NATs,it does not represent a robust long-term solution.Since symmetric NAT provides no greater securitythan a cone NAT with per-session traffic filtering,symmetric NAT is becoming less commonas NAT vendors adapt their algorithms to support P2P protocols.

5.2 Handling Unsolicited TCP Connections

When a NAT receives a SYN packet on its public sidefor what appears to be an unsolicited incoming connection attempt,it is important that the NAT just silently drop the SYN packet.Some NATs instead actively reject such incoming
connectionsby sending back a TCP RST packet or even an ICMP error report,which interferes with the TCP hole punching process.Such behavior is not necessarily fatal,as long as the applications re-try outgoing connection attemptsas specified in step 4 of the
process described in Section 4.2,but the resulting transient errorscan make hole punching take longer.

5.3 Leaving Payloads Alone

A few existing NATsare known to scan “blindly” through packet payloadsfor 4-byte values that look like IP addresses,and translate them as they would the IP address in the packet header,without knowing anything about the application
protocol in use.This bad behavior fortunately appears to be uncommon,and applications can easily protect themselves against itby obfuscating IP addresses they send in messages,for example by sending the bitwise complement of the desired IP address.

5.4 Hairpin Translation

Some multi-level NAT situationsrequire hairpin translation supportin order for either TCP or UDP hole punching to work,as described in Section 3.5.The
scenario shown in Figure 6, for example,depends on NAT


providing hairpin translation.Support for hairpin translationis unfortunately rare in current NATs,but fortunately so are the network scenarios
that require it.Multi-level NAT is becoming more commonas IPv4 address space depletion continues,however,so support for hairpin translationis important in future NAT implementations.

6 Evaluation of Existing NATs

To evaluate the robustness of the TCP and UDP hole punching techniquesdescribed in this paperon a variety of existing NATs,we implemented and distributed a test program calledNAT Check [16],and
solicited data from Internet users about their NATs.

NAT Check's primary purposeis to test NATsfor the two behavioral properties most crucialto reliable UDP and TCP hole punching:namely,consistent identity-preserving endpoint translation(Section 5.1),and
silently dropping unsolicited incoming TCP SYNsinstead of rejecting them with RSTs or ICMP errors(Section 5.2).In addition,NAT Check separately testswhether the NAT supports hairpin
translation(Section 5.4),and whether the NAT filters unsolicited incoming traffic at all.This last property does not affect hole punching,but provides a useful indication
the NAT's firewall policy.

NAT Check makes no attemptto test every relevant facet of NAT behavior individually:a wide variety of subtle behavioral differences are known,some of which are difficultto test reliably [12].Instead,
NAT Check merely attempts to answer the question,“how commonly can the proposed hole punching techniquesbe expected to work on deployed NATs,under typical network conditions?”

6.1 Test Method

NAT Check consists of a client programto be run on a machine behind the NAT to be tested,and three well-known servers at different global IP addresses.The client cooperates with the three serversto check the NAT behaviorrelevant
to both TCP and UDP hole punching.The client program is small and relatively portable,currently running on Windows, Linux, BSD, and Mac OS X.The machines hosting the well-known servers all run FreeBSD.

6.1.1 UDP Test

To test the NAT's behavior for UDP,the client opens a socket and binds it to a local UDP port,then successively sends “ping”-like requests to servers 1 and 2,as shown in Figure 8.These
servers each respond to the client's pingswith a reply that includes the client's public UDP endpoint:the client's own IP address and UDP port numberas observed by the server.If the two servers report the same public endpoint for the client,NAT Check assumes
that the NAT properly preservesthe identity of the client's private endpoint,satisfying the primary precondition for reliable UDP hole punching.

Figure 8:NAT Check Test Method for UDP

When server 2 receives a UDP request from the client,besides replying directly to the clientit also forwards the request to server 3,which in turn replies to the clientfrom its own IP address.If the NAT's firewall properly filters“unsolicited”
incoming traffic on a per-session basis,then the client never sees these replies from server 3,even though they are directed at the same public portas the replies from servers 1 and 2.

To test the NAT for hairpin translation support,the client simply opens a second UDP socket at a different local portand uses it to send messages to the
public endpointrepresenting the client's first UDP socket,as reported by server 2.If these messages reach the client's first private endpoint,then the NAT supports hairpin translation.

6.1.2 TCP Test

The TCP test follows a similar pattern as for UDP.The client uses a single local TCP portto initiate outbound sessions to servers 1 and 2,and checks whether the public endpointsreported by servers 1 and 2 are the same,the first
preconditionfor reliable TCP hole punching.

The NAT's response to unsolicited incoming connection attemptsalso impacts the speed and reliability of TCP hole punching, however,so NAT Check also tests this behavior.When server 2 receives the client's request,instead of immediately
replying to the client,it forwards a request to server 3and waits for server 3 to respond with a “go-ahead” signal.When server 3 receives this forwarded request,it attempts to initiate an inbound connectionto the client's public TCP endpoint.Server 3 waits
up to five seconds for this connection to succeed or fail,and if the connection attempt is still “in progress” after five seconds,server 3 responds to server 2 with the “go-ahead” signaland continues waiting for up to 20 seconds.Once the client finally receives
server 2's reply(which server 2 delayed waiting for server 3's “go-ahead” signal),the client attempts an outbound connection to server 3,effectively causing a simultaneous TCP open with server 3.

What happens during this testdepends on the NAT's behavior as follows.If the NAT properly just drops server 3's “unsolicited” incoming SYN packets,then nothing happens on the client's listen socketduring the five second periodbefore
server 2 replies to the client.When the client finally initiates its own connection to server 3,opening a hole through the NAT,the attempt succeeds immediately.If on the other handthe NAT does
not drop server 3's unsolicited incoming SYNsbut allows them through(which is fine for hole punching but not ideal for security),then the client receives an incoming TCP connection on its listen socketbefore receiving server 2's reply.Finally, if the
NAT actively rejects server 3's unsolicited incoming SYNsby sending back TCP RST packets,then server 3 gives upand the client's subsequent attempt to connect to server 3 fails.

To test hairpin translation for TCP,the client simply uses a secondary local TCP portto attempt a connection to the public endpointcorresponding to its primary TCP port,in the same way as for UDP.

6.2 Test Results

The NAT Check data we gatheredconsists of 380 reported data pointscovering a variety of NAT router hardware from 68 vendors,as well as the NAT functionality builtinto different versions of eight popular operating systems.Only
335 of the total data points include results for UDP hairpin translation,and only 286 data points include results for TCP,because we implemented these features in later versions of NAT Checkafter we had already started gathering results.The data is summarized
by NAT vendor in Table 1;the table only individually lists vendorsfor which at least five data points were available.The variations in the test results for a given vendorcan be accounted
for by a variety of factors,such as different NAT devices or product lines sold by the same vendor,different software or firmware versions of the same NAT implementation,different configurations,and probably occasional NAT Check testing or reporting errors.

Table 1:User Reports of NAT Support for UDP and TCP Hole Punching

UDP TCP Hole Hole Punching Hairpin Punching Hairpin NAT Hardware Linksys 45/46 (98%) 5/42 (12%) 33/38 (87%) 3/38 (8%) Netgear 31/37 (84%) 3/35 (9%) 19/30 (63%) 0/30 (0%) D-Link 16/21 (76%) 11/21 (52%) 9/19 (47%) 2/19 (11%) Draytek 2/17 (12%) 3/12 (25%) 2/7 (29%) 0/7 (0%) Belkin 14/14 (100%) 1/14 (7%) 11/11 (100%) 0/11 (0%) Cisco 12/12 (100%) 3/9 (33%) 6/7 (86%) 2/7 (29%) SMC 12/12 (100%) 3/10 (30%) 8/9 (89%) 2/9 (22%) ZyXEL 7/9 (78%) 1/8 (13%) 0/7 (0%) 0/7 (0%) 3Com 7/7 (100%) 1/7 (14%) 5/6 (83%) 0/6 (0%) OS-based NAT Windows 31/33 (94%) 11/32 (34%) 16/31 (52%) 28/31 (90%) Linux 26/32 (81%) 3/25 (12%) 16/24 (67%) 2/24 (8%) FreeBSD 7/9 (78%) 3/6 (50%) 2/3 (67%) 1/1 (100%) All Vendors 310/380 (82%) 80/335 (24%) 184/286 (64%) 37/286 (13%)

Out of the 380 reported data points for UDP,in 310 cases (82%)the NAT consistently translated the client's private endpoint,indicating basic compatibility with UDP hole punching.Support for hairpin translation is much less common,
however:of the 335 data points that include UDP hairpin translation results,only 80 (24%) show hairpin translation support.

Out of the 286 data points for TCP,184 (64%) show compatibility with TCP hole punching:the NAT consistently translates the client's private TCP endpoint,and does not send back RST packetsin response to unsolicited incoming connection
attempts.Hairpin translation support is again much less common:only 37 (13%) of the reportsshowed hairpin support for TCP.

Since these reports were generated by a “self-selecting” community of volunteers,they do not constitute a random sampleand thus do not necessarily representthe true distribution of the NATs in common use.The results are nevertheless
encouraging:it appears that the majority of commonly-deployed NATsalready support UDP and TCP hole punchingat least in single-level NAT scenarios.

6.3 Testing Limitations

There are a few limitations in NAT Check's current testing protocolthat may cause misleading results in some cases.First, we only learned recentlythat a few NAT implementations blindlytranslate IP addresses they find in unknown
application payloads,and the NAT Check protocol currently does not protect itself from this behaviorby obfuscating the IP addresses it transmits.

Second,NAT Check's current hairpin translation checkingmay yield unnecessarily pessimistic resultsbecause it does not use the full, two-way hole punching procedurefor this test.NAT Check currently assumes that a NAT supporting
hairpin translationdoes not filter “incoming” hairpin connectionsarriving from the private networkin the way it would filter incoming connectionsarriving at the public side of the NAT,because such filtering is unnecessary for security.We later realized, however,that
a NAT might simplisticallytreat any traffic directed at the NAT's public portsas “untrusted” regardless of its origin.We do not yet know which behavior is more common.

Finally,NAT implementations existthat consistently translate the client's private endpointas long as
only one client behind the NATis using a particular private port number,but switch to symmetric NAT or even worse behaviorsif two or more clients with different IP addresses on the private networktry to communicate through the NAT from the same private
port number.NAT Check could only detect this behaviorby requiring the user to run it on two or more client hostsbehind the NAT at the same time.Doing so would make NAT Check much more difficult to use, however,and impossible for users who only have one usable
machine behind the NAT.Nevertheless, we plan to implement this testing functionalityas an option in a future version of NAT Check.

6.4 Corroboration of Results

Despite testing difficulties such as those above,our results are generally corroboratedby those of a large ISP,who recently found that of the top three consumer NAT router vendors,representing 86% of the NATs observed on their
network,all three vendors currently produce NATs compatiblewith UDP hole punching [25].Additional independent resultsrecently obtainedusing the UDP-oriented STUN protocol [12],and
STUNT, a TCP-enabled extension [8,9],also appear consistent withour results.These latter studies
provide more information on each NATby testing a wider variety of behaviors individually,instead of just testing for basic hole punching compatibilityas NAT Check does.Since these more extensive testsrequire multiple cooperating clients behind the NATand thus
are more difficult to run,however,these results are so far availableon a more limited variety of NATs.

7 Related Work

UDP hole punchingwas first explored and publicly documented by Dan Kegel [13],and is by now well-known in peer-to-peer application communities.Important
aspects of UDP hole punching have also been indirectly documentedin the specifications of several experimental protocols,such as STUN [19],ICE [17],and
Teredo [11].We know of no existing published workthat thoroughly analyzes hole punching,however,or that points out the hairpin translation issuefor multi-level NAT (Section 3.5).

We also know of no prior workthat develops TCP hole punchingin the symmetric fashion described here.Even the existence of the crucial

SO_REUSEADDR

/

SO_REUSEPORT

optionsin the Berkeley sockets APIappears
to be little-known among P2P application developers.NatTrav [4]implements a similar but asymmetric TCP hole punching procedureoutlined earlier in Section 4.5.NUTSS [9]and
NATBLASTER [1]implement more complex TCP hole punching tricksthat can work around some of the bad NAT behaviorsmentioned in Section 5,but
they require the rendezvous server to spoof source IP addresses,and they also require the client applications to have access to “raw” sockets,usually available only at root or administrator privilege levels.

Protocols such as SOCKS [14],UPnP [26],and MIDCOM [22]allow
applications to traverse a NATthrough explicit cooperation with the NAT.These protocols are not widely or consistently supportedby NAT vendors or applications, however,and do not appear to addressthe increasingly important multi-level NAT scenarios.Explicit
control of a NAT furtherrequires the application to locate the NATand perhaps authenticate itself,which typically involves explicit user configuration.When hole punching works,in contrast,it works with no user intervention.

Recent proposalssuch as HIP [15]and FARA [2]extend the Internet's
basic architectureby decoupling a host's identity from its location [20].IPNL [7],UIP [5,6],and
DOA [27]propose schemes for routing across NATsin such an architecture.While such extensionsare probably needed in the long term,hole punching enables applicationsto work
over the existing network infrastructure immediatelywith no protocol stack upgrades,and leaves the notion of “host identity” for applications to define.

8 Conclusion

Hole punching is a general-purpose techniquefor establishing peer-to-peer connections in the presence of NAT.As long as the NATs involved meet certain behavioral requirements,hole punching works consistently and robustlyfor both
TCP and UDP communication,and can be implemented by ordinary applicationswith no special privilegesor specific network topology information.Hole punching fully preserves the transparencythat is one of the most important hallmarks and attractions of NAT,and
works even with multiple levels of NAT--though certain corner case situations require hairpin translation,a NAT feature not yet widely implemented.

Acknowledgments

The authors wish to thank Dave Andersen for his crucial supportin gathering the results presented in Section 6.We also wish to thank Henrik Nordstrom,
Christian Huitema,Justin Uberti, Mema Roussopoulos,and the anonymous USENIX reviewersfor valuable feedbackon early drafts of this paper.Finally, we wish to thank the many volunteerswho took the time to run NAT Check on their systemsand submit the results.

Bibliography

1Andrew Biggadike, Daniel Ferullo, Geoffrey Wilson, and Adrian Perrig.

NATBLASTER: Establishing TCP connections between hosts behind NATs.

In ACM SIGCOMM Asia Workshop, Beijing, China, April 2005.2David Clark, Robert Braden, Aaron Falk, and Venkata Pingali.

FARA: Reorganizing the addressing architecture.

In ACM SIGCOMM FDNA Workshop, August 2003.3S. Deering and R. Hinden.

Internet protocol, version 6 (IPv6) specification, December 1998.

RFC 2460.4Jeffrey L. Eppinger.

TCP connections for P2P apps: A software approach to solving the NAT problem.

Technical Report CMU-ISRI-05-104, Carnegie Mellon University, January 2005.5Bryan Ford.

Scalable Internet routing on topology-independent node identities.

Technical Report MIT-LCS-TR-926, MIT Laboratory for Computer Science, October 2003.6Bryan Ford.

Unmanaged internet protocol: Taming the edge network management crisis.

In Second Workshop on Hot Topics in Networks, Cambridge, MA, November 2003.7Paul Francis and Ramakrishna Gummadi.

IPNL: A NAT-extended Internet architecture.

In ACM SIGCOMM, August 2002.8Saikat Guha and Paul Francis.

Simple traversal of UDP through NATs and TCP too (STUNT).
http://nutss.gforge.cis.cornell.edu/.9Saikat Guha, Yutaka Takeday, and Paul Francis.

NUTSS: A SIP-based approach to UDP and TCP network connectivity.

In SIGCOMM 2004 Workshops, August 2004.10M. Holdrege and P. Srisuresh.

Protocol complications with the IP network address translator, January 2001.

RFC 3027.11C. Huitema.

Teredo: Tunneling IPv6 over UDP through NATs, March 2004.

Internet-Draft (Work in Progress).12C. Jennings.

NAT classification results using STUN, October 2004.

Internet-Draft (Work in Progress).13Dan Kegel.

NAT and peer-to-peer networking, July 1999.
http://www.alumni.caltech.edu/~dank/peer-nat.html.14M. Leech et al.

SOCKS protocol, March 1996.

RFC 1928.15R. Moskowitz and P. Nikander.

Host identity protocol architecture, April 2003.

Internet-Draft (Work in Progress).16NAT check.
http://midcom-p2p.sourceforge.net/.17J. Rosenberg.

Interactive connectivity establishment (ICE), October 2003.

Internet-Draft (Work in Progress).18J. Rosenberg, C. Huitema, and R. Mahy.

Traversal using relay NAT (TURN), October 2003.

Internet-Draft (Work in Progress).19J. Rosenberg, J. Weinberger, C. Huitema, and R. Mahy.

STUN - simple traversal of user datagram protocol (UDP) through network address translators (NATs), March 2003.

RFC 3489.20J. Saltzer.

On the naming and binding of network destinations.

In P. Ravasio et al., editor, Local Computer Networks, pages 311-317. North-Holland, Amsterdam, 1982.

RFC 1498.21P. Srisuresh and M. Holdrege.

IP network address translator (NAT) terminology and considerations, August 1999.

RFC 2663.22P. Srisuresh, J. Kuthan, J. Rosenberg, A. Molitor, and A. Rayhan.

Middlebox communication architecture and framework, August 2002.

RFC 3303.23Transmission control protocol, September 1981.

RFC 793.24G. Tsirtsis and P. Srisuresh.

Network address translation - protocol translation (NAT-PT), February 2000.

RFC 2766.25Justin Uberti.

E-mail on IETF MIDCOM mailing list, February 2004.

Message-ID: <402CEB11.1060906@aol.com>.26UPnP Forum.

Internet gateway device (IGD) standardized device control protocol, November 2001.

http://www.upnp.org/

.27Michael Walfish, Jeremy Stribling, Maxwell Krohn, Hari Balakrishnan, Robert Morris, and Scott Shenker.

Middleboxes no longer considered harmful.

In USENIX Symposium on Operating Systems Design and Implementation, San Francisco, CA, December 2004.

Bryan Ford2005-02-17