The tunnel is up. Both ends report Phase 1 established. The monitoring dashboard is green. And yet nothing is reaching the other side. If you have built a single hybrid cloud VPN, you have probably hit this scenario. It is one of the more frustrating problems in cloud networking because every surface-level signal says everything is fine. This post walks through the real reasons VPN tunnels report up while traffic fails, with the specific checks for an AWS hybrid environment.
The short version. A tunnel reporting “up” only confirms that IKE Phase 1 negotiated successfully. It does not confirm Phase 2, route propagation, security group acceptance, or that the MTU survives end to end. The five things to check, in order: Phase 2 SA, routes both directions, security groups and NACLs, an actual end-to-end test with ping and iperf3, and finally MTU and MSS clamping. Skip a layer and you will be back at step 1 in two hours.
Tunnel up is not the same as traffic flowing
IPsec runs in two phases. Phase 1 (IKE) establishes a secure control channel between the two VPN endpoints. It authenticates peers (pre-shared key or certificate), negotiates encryption parameters for the control channel, and produces the ISAKMP security association. Most monitoring tools and dashboards consider the tunnel “up” as soon as Phase 1 completes.
Phase 2 (IPsec SA) is what actually carries user traffic. It negotiates the encryption domain (also called proxy IDs or traffic selectors), agrees on encryption and integrity algorithms for the data plane, and produces a pair of IPsec security associations (one per direction). If Phase 2 fails or is mismatched, the tunnel still looks up but no user traffic ever crosses it.
The most common Phase 2 failure on AWS site-to-site VPNs is a proxy ID mismatch. AWS accepts 0.0.0.0/0 on both sides by default with BGP, but if you have configured a static route policy-based VPN, the encryption domain on the on-prem side must exactly match the CIDR pair AWS expects. On a Cisco ASA or VyOS device, this means the crypto ACL or peer subnet definitions must mirror what is configured in the AWS Site-to-Site VPN connection.
Run these on the on-prem side to verify Phase 2 actually came up:
# Cisco IOS / IOS-XE show crypto isakmp sa show crypto ipsec sa peer <aws-public-ip> # VyOS / Vyatta show vpn ipsec sa show vpn ipsec status # strongSwan ipsec statusall swanctl --list-sas
Look at the byte counter on each SA. If encrypt increments but decrypt does not, traffic is leaving but nothing is coming back. That points at a return-path problem (routing, NAT, or security group on the AWS side), not at the tunnel itself.
NAT traversal silently breaks things
If the on-prem VPN endpoint sits behind a NAT device, IKE detects this during Phase 1 and switches to NAT-Traversal (NAT-T), which encapsulates the ESP packets inside UDP port 4500. Standard IKE uses UDP port 500, ESP itself is IP protocol 50 (no port). When NAT is in play, both ends must support NAT-T and the upstream NAT must permit UDP 4500 outbound and the corresponding return traffic.
Common symptoms. Phase 1 completes (UDP 500 is open). Phase 2 fails or comes up briefly then dies. show crypto ipsec sa shows the encapsulation as “UDP-Encaps”. Packet captures on the outside interface show UDP 4500 going out but no return. The NAT box upstream may be aggressively timing out the UDP session because IKE keepalives are not getting through.
The fix is usually a combination of opening UDP 500 and 4500 outbound on every device in the path, ensuring DPD or IKE keepalives are short enough (10 to 30 seconds typical) to keep the NAT session alive, and confirming both peers actually have NAT-T enabled. Modern protocols like WireGuard avoid most of this entirely because they only use a single UDP port and tolerate NAT more gracefully, but most enterprise hybrid clouds are still on IPsec.
Route propagation is the silent killer on AWS
A tunnel can be perfectly healthy and still pass zero traffic if the routes are not in place on both ends. On AWS site-to-site VPN with a Virtual Private Gateway (VGW) or Transit Gateway (TGW), there are three places routes have to be correct.
The on-prem route table. Your AWS CIDR (10.0.0.0/16 in the lab) must have a next hop pointing at the VPN endpoint. On a VyOS router that is typically a route via the tunnel interface or via the AWS public IP if policy-based.
The VPC route table. The on-prem CIDR (10.10.0.0/16) must have a route pointing at the VGW or TGW attachment. Without this, return traffic from the EC2 instance has no idea how to get back. The most common version of this mistake: someone enables BGP on the VPN connection but forgets to enable route propagation on the VPC route table. The routes are learned by the VGW but never installed in the subnet route table.
Check it from the AWS CLI:
# Verify VGW has learned the on-prem prefix via BGP aws ec2 describe-vpn-connections \ --vpn-connection-ids vpn-0123456789abcdef0 \ --query 'VpnConnections[*].VgwTelemetry' # Verify the route is propagated into the subnet route table aws ec2 describe-route-tables \ --route-table-ids rtb-0123456789abcdef0 \ --query 'RouteTables[*].PropagatingVgws'
The Transit Gateway route table (if you are using TGW). TGW does not automatically share routes across attachments. The VPC attachment and the VPN attachment must both be associated with a route table that has the right propagations enabled. Static routes work too but get forgotten when new VPCs are added.
Overlapping CIDRs. If your on-prem network and any AWS VPC use overlapping ranges (common with 10.0.0.0/16 everywhere), one side will simply prefer its local route and never send traffic across the VPN. The fix is re-IPing one side, which nobody wants to do mid-project.
Security groups and NACLs are quiet
AWS security groups silently drop traffic that does not match an inbound rule. Unlike on-prem firewalls, they generate no syslog by default. A common scenario: the tunnel is up, routes are propagated, the on-prem host can reach the EC2 ENI, but the EC2 security group only allows SSH from the corporate office public IP, not from the on-prem RFC1918 range that now arrives via VPN.
Check both directions. Inbound on the EC2 security group must allow traffic from the on-prem CIDR. Outbound rules on most security groups are wide open by default, but custom ones can block return traffic. The NACL on the subnet is stateless, so it needs an explicit allow in both directions including the ephemeral port range (1024 to 65535) for the return packets.
On the on-prem side, make sure the local firewall is not stripping or blocking the unencrypted inner traffic after it has been decrypted by the VPN endpoint. This is especially common when the VPN terminates on a router separate from the perimeter firewall.
Test end-to-end with iperf3, not just ping
Ping is a great first test because it confirms reachability and prints round-trip latency. It is a terrible final test because most VPNs let ICMP through cleanly even when TCP is broken by MTU or MSS issues. Always finish with a real TCP throughput test.
# On the AWS EC2 instance (server) iperf3 -s # On the on-prem host (client) iperf3 -c 10.0.1.213 -t 30 -P 4 # To check MTU and fragmentation issues ping -M do -s 1372 10.0.1.213 # 1372 + 28 (ICMP/IP) = 1400 byte packet mtr -n 10.0.1.213 # path latency per hop
If ping works but iperf3 stalls after a few hundred KB, you have an MTU problem. The VPN encapsulation adds overhead (around 73 bytes for IPsec ESP with AES-GCM), so a 1500-byte packet from the host becomes a 1573-byte packet on the wire. If anything in the path silently drops oversized packets without ICMP, TCP stalls. The fix is MSS clamping on both VPN endpoints, typically to 1379 or 1380 for AWS site-to-site VPN.
Troubleshooting in order
When the tunnel is up but no traffic flows, walk the layers in order. Skipping ahead wastes hours.
Step 1. Confirm both Phase 1 AND Phase 2 SA exist with non-zero byte counters in both directions. If the decrypt counter is zero, the problem is on the return path.
Step 2. Verify routes both directions. Walk the on-prem route table for the AWS CIDR, the VPC route table for the on-prem CIDR (including BGP propagation), and TGW route tables if used.
Step 3. Check security groups and NACLs on the AWS side, plus any inner firewall on the on-prem side.
Step 4. Run an actual end-to-end test. Ping first, then iperf3 for throughput, then a real application connection.
Step 5. If small packets work and large packets fail, clamp MSS on both VPN endpoints.
Step 6. Capture on both sides if still stuck. tcpdump -i eth0 esp on the outside interface confirms encrypted traffic is flowing. tcpdump -i <inside> host <target> on both ends shows where the decrypted packet stops.
Key takeaways
- A tunnel reporting “up” only confirms Phase 1. Always verify Phase 2 SA before suspecting anything else.
- NAT-T problems present as flaky tunnels with frequent re-keying. Open UDP 4500 and tune keepalives.
- The single most common AWS VPN problem is missing route propagation on the VPC route table.
- Security groups drop silently. Both directions need explicit allow rules for the on-prem CIDR.
- Ping is not a sufficient end-to-end test. Use iperf3 to catch MTU issues that ICMP slips through.
- Walk the layers in order. Skipping ahead wastes more time than the methodical approach takes.
FAQ
Why is my AWS VPN tunnel up but I cannot ping my EC2 instance?
Three likely causes in order of frequency: missing route propagation on the VPC route table, the EC2 security group does not allow ICMP from your on-prem CIDR, or the NACL on the subnet is blocking the return traffic. Verify all three before suspecting the VPN itself.
Should I use BGP or static routes on AWS site-to-site VPN?
BGP. Static is fine for tiny deployments but fails open during failover and does not scale. BGP enables automatic failover between the two tunnels AWS gives you, supports route propagation into the VPC route table, and works correctly with Transit Gateway. The only reason to choose static is if the on-prem device cannot do BGP.
What MSS should I clamp to for AWS site-to-site VPN?
1379 bytes is the AWS-recommended value, accounting for the IPsec ESP overhead with AES-GCM. Set this on both the on-prem VPN endpoint and ideally on the EC2 instances themselves (or via TGW MSS clamping if supported). Test with iperf3 after clamping to confirm sustained throughput.
Is WireGuard a viable alternative to IPsec for hybrid cloud?
For point-to-point connectivity between a single on-prem site and a cloud VPC, yes. WireGuard’s simpler design and single UDP port make NAT and firewall traversal much cleaner. The catch is that AWS site-to-site VPN does not natively support WireGuard. You would terminate WireGuard on an EC2 instance, which means losing AWS-managed redundancy and BGP integration. For most enterprise hybrid deployments, IPsec via VGW or TGW remains the practical choice.
Related posts
Designing or troubleshooting hybrid cloud connectivity
Reliable hybrid cloud connectivity needs more than a green tunnel light. It needs proper Phase 2 verification, route propagation discipline, security group hygiene, and end-to-end testing as part of every change. Our cloud networking practice has designed and operated hybrid VPN and Direct Connect deployments across AWS, Azure, and OCI for organizations across Western Canada. If you are stuck on a hybrid VPN that should work and does not, we will help you find it.
Last verified May 2026 by the aaanetworkx cloud networking practice. Originally drafted by Edberg Hammond.
AWS VPN tunnel Phase 2 down means the IPsec security associations cannot establish, and the AWS console only shows you a fraction of the story.
You configured an AWS Site-to-Site VPN, both Phase 1 came up, but Phase 2 is showing down on the AWS console. Traffic is not flowing between the on-premises network and the VPC. This post walks through the four real causes of AWS VPN Phase 2 down ranked by frequency, the diagnostic order, and the verified fix.
The short version. Phase 2 establishes the actual IPsec SAs that encrypt traffic between AWS and your on-premises gateway. Phase 2 down with Phase 1 up means the two sides agreed on identity and key exchange but cannot agree on what traffic to encrypt or what encryption parameters to use. About 40 percent of cases are encryption proposal mismatch. Another 30 percent are traffic selector mismatch (the on-premises side is configured for a different network than AWS expects). The remaining 30 percent split across PFS group mismatch and BGP issues that block route advertisement, where the asymmetry typically lives.
What Phase 2 down means
IPsec is a two-phase protocol. Phase 1 (IKE_SA_INIT and IKE_AUTH in IKEv2) authenticates the two endpoints and establishes a secure channel. Phase 2 (CREATE_CHILD_SA in IKEv2) negotiates the IPsec SA that actually encrypts data traffic, including which networks are protected and what encryption algorithms are used. Phase 1 up but Phase 2 down means the auth worked, but the SA negotiation for data traffic failed.
You will see this in the AWS console under VPC → Site-to-Site VPN Connections → Tunnel Details, where the Status shows Phase 2 DOWN. The on-premises VPN gateway logs typically show the rejection reason explicitly (proposal mismatch, no proposal chosen, traffic selector mismatch).
Verified against current AWS Site-to-Site VPN documentation, accessed April 2026.
The four causes, ranked
Cause one, encryption proposal mismatch, around 40 percent
AWS supports a specific set of Phase 2 encryption proposals (AES-128, AES-256, with various integrity algorithms and DH groups). If the on-premises side is configured for an algorithm AWS does not accept, Phase 2 fails immediately.
Verify by comparing the on-premises Phase 2 proposal to the AWS-supported list. AWS publishes the supported algorithms in their VPN documentation. Use one that is on both lists. AES-256 + SHA-256 + DH group 14 is a safe modern default that AWS supports.
Cause two, traffic selector mismatch, around 30 percent
AWS in route-based mode uses a single traffic selector of 0.0.0.0/0 to 0.0.0.0/0. If the on-premises side is configured with policy-based VPN that specifies specific source/destination networks, the negotiation fails because the selectors do not match.
Verify by checking the on-premises VPN config. For AWS, use route-based VPN (with virtual tunnel interfaces) and configure the on-premises side similarly. If you must use policy-based VPN on-premises, configure the policy to match 0.0.0.0/0 to 0.0.0.0/0 for the tunnel.
Cause three, PFS (Perfect Forward Secrecy) group mismatch, around 15 percent
AWS requires PFS for Phase 2 by default. If the on-premises side has PFS disabled or is using a DH group AWS does not accept, Phase 2 fails.
Verify the on-premises PFS configuration. Enable PFS, use DH group 14 or higher to match AWS expectations.
Cause four, BGP not advertising routes, around 15 percent
Phase 2 may show as up briefly then go down, or AWS may report it down because no useful BGP routes are flowing. Check the BGP session under the VPN connection. If BGP is down, Phase 2 status can show degraded even though the IPsec layer technically established.
Verify with show bgp summary on the on-premises side and confirm the BGP session to AWS is established. Routes should be exchanged. Fix the BGP issue (ASN mismatch, missing route advertisement, etc.) and Phase 2 should stabilize.
What the official documentation does not mention
AWS provides a configuration sample download for many vendor gateways but rarely emphasizes that the sample is a starting point, not a finished config. Check the actual AWS-supported algorithms in the current VPN documentation before deploying, since AWS occasionally deprecates older algorithms. The sample config may include settings that work today but will be removed in a future update.
Also, AWS VPN tunnels have two tunnels per VPN connection. If only one is up, the connection is functional but lacks redundancy. Both should be configured and both should be up under normal operations.
The architectural fix
AWS VPNs that rarely fail share four practices. They use route-based VPN end-to-end, not policy-based. They use AWS-recommended encryption algorithms (AES-256, SHA-256, DH 14 or higher, PFS enabled). They use BGP for route exchange rather than static routes, so route changes do not require manual updates. They monitor both tunnels of the VPN connection and alert if either drops. Skip any of these and VPN reliability degrades over time as AWS deprecates older settings or as on-premises networks evolve.
FAQ
Should I switch to AWS Direct Connect to avoid this?
Direct Connect is a different product (private connectivity), not a fix for VPN issues. Pick based on bandwidth, latency, and cost requirements, not specifically to avoid Phase 2 troubleshooting.
Is IKEv2 better than IKEv1 for AWS?
Yes, AWS supports both but IKEv2 is the modern standard with better re-keying behavior. Use IKEv2 unless your on-premises gateway only supports IKEv1.
Will the tunnel re-establish on its own?
If the cause is transient (brief network issue), yes. If the cause is configuration mismatch, no. The tunnel will keep failing until the underlying mismatch is fixed.
Related posts
VPN issues during cloud migration
If you are setting up an AWS VPN as part of a cloud migration and the tunnel will not come up, our cloud networking team can review the on-premises and AWS configuration in parallel and isolate the mismatch quickly. Tell us your on-premises gateway and we will help you align.
Last verified April 2026 by the aaanetworkx cloud networking practice.
AWS Direct Connect BGP not establishing almost always means a configuration asymmetry between the customer router and the AWS DX virtual interface, not a problem on the AWS side.
You provisioned an AWS Direct Connect virtual interface, the cross-connect light is up, the AWS console shows the VIF as available, but the BGP peering will not come up. This post walks through the five real causes of AWS Direct Connect BGP not establishing ranked by frequency, the diagnostic order, and the verified fix.
The short version. About 35 percent of cases are VLAN tag mismatch on the customer sub-interface. Another 25 percent are BGP MD5 authentication key mismatch. Another 20 percent are the customer using the wrong IP on the /30 peer link. The remaining 20 percent split across BGP ASN mismatch and MTU or MSS issues that drop the BGP open message, which is where the BGP session typically stalls.
What BGP not establishing means
An AWS Direct Connect VIF expects a BGP session over an 802.1Q tagged sub-interface on a /30 point-to-point peer link. The cross-connect light coming up means the optics, fiber, and Layer 1 are fine. BGP not establishing means the session is stuck in Idle, Connect, Active, or OpenSent on the customer side, and AWS reports the BGP peering as down on the VIF.
You will see this in the AWS console under Direct Connect → Virtual Interfaces → BGP Status. On the customer router, show bgp summary on Cisco or show bgp neighbor on Juniper will show the peer state. The state itself is a strong hint. Idle or Active usually means the peer is unreachable, which is a Layer 2 or Layer 3 problem. OpenSent that never advances usually means MD5 or ASN mismatch.
Verified against current AWS Direct Connect documentation, accessed May 2026.
The five causes, ranked
Cause one, VLAN tag mismatch on the sub-interface, around 35 percent
AWS assigns a specific VLAN ID to each VIF (the customer can request one or accept what AWS suggests). The customer router must tag the BGP traffic with that exact VLAN on the sub-interface facing the cross-connect. A common failure mode is the carrier (Megaport, Allstream, Bell, Equinix Fabric) tagging the VLAN differently than the customer configured, which leaves the BGP packets either untagged at the AWS end or double-tagged.
Verify with show interfaces <sub-if> on the customer router and confirm the VLAN matches what is shown in the AWS VIF config. If a carrier sits between, ask them to confirm whether they are presenting the VLAN tagged or untagged at your handoff. Q-in-Q tunneling is the most common surprise.
Cause two, BGP MD5 password mismatch, around 25 percent
Each DX VIF has an optional BGP authentication key. If you set one in the AWS console it must match exactly on the customer router. Three things break this in practice. Copy-paste introduces a leading or trailing space. The password is longer than the platform supports (Cisco IOS truncates silently above some lengths). Hidden Unicode characters from a password manager autofill that look like ASCII but are not.
Verify by re-typing the key by hand on both sides, not pasting. Keep the key under 80 ASCII characters. If you suspect MD5 is the cause, temporarily clear the key on both sides (BGP without auth) and confirm the session establishes. If it does, the failure is MD5 specifically.
Cause three, wrong customer-side IP on the /30 peer link, around 20 percent
AWS allocates a /30 to the BGP peer link. AWS holds .2 of the /30, the customer must configure .1. Engineers used to /29 or /24 peer links sometimes pick .3 or .5, or guess the prefix length. The result is the customer router cannot ARP the AWS BGP endpoint and the session stays in Active or Idle forever.
Verify by reading the exact peer IPs from the AWS VIF config, not the LOA-CFA. Configure the customer side as the .1 of the /30, with /30 mask. Ping the AWS-side .2 from the customer router. If ping fails but the cross-connect is up, the issue is at this layer.
Cause four, BGP ASN mismatch, around 10 percent
When you create the VIF in AWS, you specify your customer ASN. Private VIFs default to 64512 if you do not specify, public VIFs require a real ASN you control. If the ASN configured on the customer router does not match what is recorded in the VIF, BGP gets to OpenSent and AWS rejects the OPEN message with a notification.
Verify by reading the ASN from the AWS VIF detail panel. Match it on the customer router under router bgp <asn>. The AWS-side ASN is also shown there (typically 64512 for private VIFs, the AWS public ASN for public VIFs).
Cause five, MTU or MSS dropping the BGP open message, around 10 percent
Direct Connect supports jumbo frames on private VIFs at 9001 bytes if both ends agree, otherwise the path defaults to 1500. If the customer router has the sub-interface at 9001 but the carrier handoff is 1500, large BGP open messages with many capabilities advertised can fragment and get dropped, which leaves BGP in Active or OpenSent depending on the platform.
Verify by setting both sides to 1500 explicitly, then confirm BGP establishes. If it does, you can reintroduce jumbo frames intentionally once you confirm the carrier supports them end to end.
What the AWS sample config does not mention
AWS provides a downloadable sample config when you create a VIF. The sample is a starting point, not a finished config. It assumes a single sub-interface dedicated to the VIF, no upstream policy filtering, and the carrier presenting the VLAN tagged at your handoff. If any of those assumptions are wrong, the sample alone will not bring BGP up. Always confirm the carrier handoff and any inline policy on your edge router before troubleshooting AWS-side.
Also, a Direct Connect VIF has a single BGP session. Resilience comes from a second VIF on a second physical Direct Connect at a different AWS location, not from BGP redundancy on a single VIF. If you only have one DX, the BGP session being stable matters more than for a redundant VPN.
The architectural fix
AWS Direct Connect deployments that rarely fail share four practices. They use a dedicated sub-interface per VIF, never sharing with other traffic. They confirm the carrier handoff VLAN tagging in writing before BGP turn-up, not after. They keep the BGP MD5 key under 80 ASCII characters and store it in a secrets vault, not a chat thread. They monitor BGP session state with the AWS CloudWatch metric and alert on any drop, since a single VIF has no automatic failover.
FAQ
Will the BGP session establish on its own once the cross-connect is up?
Only if the customer side is configured correctly. The cross-connect being up is Layer 1. BGP is Layer 4 and depends on VLAN, IP, MD5, and ASN all matching. If any one is wrong, the session will keep retrying but never establish.
Should I use a public VIF or a private VIF?
Private VIF is for reaching VPCs through a virtual private gateway or transit gateway, which is what most customers want. Public VIF is for reaching AWS public services (S3, DynamoDB) over Direct Connect rather than the public internet. Public VIFs require a real ASN and AWS LOA verification of the prefixes you advertise.
Does AWS Direct Connect work for ca-central-1 from Edmonton?
Yes. The AWS Direct Connect locations for ca-central-1 are in Calgary and Toronto. From Edmonton, most carriers terminate the cross-connect in the Calgary AWS DX location with a backhaul over their core, which adds 4 to 6 ms of latency to ca-central-1.
Related posts
BGP not coming up on a new Direct Connect
If you are turning up a new AWS Direct Connect and the BGP session will not establish, our cloud networking team can review the carrier handoff, the customer-side sub-interface, and the AWS VIF config in parallel and isolate the asymmetry quickly. Tell us your carrier and edge platform and we will help you align.
Last verified May 2026 by the aaanetworkx cloud networking practice.