Layer 7 allowed traffic policy
You think it would be obvious to say that what's not the in blocking policy, is what's allowed.
Unfortunately it's more complicated than that as there are several factors that affect what's gets allowed where. Let's start the route types.
Remote gateways
PremLINK direct ISP connection only uses a dynamic IPv4 address. There is native IPv6, but like most providers offering a single IPv4 address, a single IPv6 /64 block is assigned. Thus, in order to gain stability and flexibility, our other gateways are connected by virtual tunnels.
It's a one-side-initiated connection too, so if the link is tore down and given the dynamic IP nature of the initiating side, the server has no way of getting back to the client. It's a nice passive, non-intended security feature. There are many more like these we've adopted, most of which ironically were thought as caveats, problems, or whatever at the time. Now they're embraced.
Our IPv6 blocks are provided by Hurricane Electric, both are tunnels, one of them is a tunnel within a tunnel. DreamCompute, a division of DreamHost, provides the host and networking for out main gateway and — this is where the problems begin — the outbound route for email which already has so many requirements and yet a bigger, if not the biggest headache, hardest thing to manage in the network: DNS routing.
We'll leave the name of our ISP for now, it's not a secret but why make it easier, right?
Speed
As mentioned, with connections stacked on top of other connections such as is the case of tunneling, that mean all connections together can only be as fast as the one used for transport. That's never good.
Fortunately, that main connection run on the fastest basic (i.e; non-Enterprise multi-gig, or active) fiber possible, it's only a single gigabit, but our ISP under-provisions and back when fiber wasn't available 1) it actually assigns you more bandwidth that contracted to account for the overhead that would normally result in a lower speed, that way when people run speedtests, they actually get the speed promised and don't make a big fuss on phone support. It's a good ISP so it doesn't have to deal with support. It's hard to say if that's good or bad; but fact is, it's a rock-solid, fast and non-data-capped service. Realible, cheap (about the equivalent of USD50/mo for the 1Gb/s tier). And it that's not incentive enough, they throw in six (6) landlines with unlimited worldwide calling, all independent of each other and lot of cloud freebies backed by a partnership with Azure, I believe. Which of course… Landlines, Azure? The Internet access link is enough.
That the speed is on out side it doesn't mean it's at our gateway's. But in all fairness, out of many cloud infrastructure providers, none has surpassed the 200Mb/s transfer speed; and we've tried the biggest names and the unknowns (that have become or were becoming big). Vultr, Alibaba Cloud, AWS*, Azure*, Linode, OVH, Scaleway, Namecheap*, A2 Hosting, Ionos among a few others. Also, most of these are metered. DreamCompute has many downsides; it's not the fastest and definitely not the easiest to deploy a custom virtual firewall appliance, it's also ridden by spamming users, which has gotten the whole ASN 2) in predatory blacklists, but we're more then willing to overlook all of that because of a few good things it has; no data caps, for starters, and the fact that spam is heavy on its networks must mean DreamCompute doesn't meddle in its customers' businesses for better or for worse and then there's its open source contributions, really big name projects have been tied, or were downright born at DreamHost: like a little behemoth virtualization platform that even got VMware into the action, OpenStack and a seriously badass distributed filesystem now associated with the Red Hat name, Ceph. It's like the ZFS of distributed filesystems.
So yes, the tunnel is slow. Therefore not everything gets routed through there, more on what and why later. But the IPv6 addresses are all static, and the direct connection where I get a supposedly dynamic IPv4 address, is so solid that the only way it goes down it because it's made to when firewalls are updated or the power goes out for an extender period of time. These addresses can easily remain assign for months and often their recovered on reconnection. As a user, you should not notice any difference, it happens seamlessly aided by lots of DNS planning, testing, and several platforms working together.
Exit region
The last important decider of what traffic get through is the politic system in the US that has allowed for copyright laws that can harm people for profit, far for the intellectual-anything they claim. Thus, a lot of traffic that freely flows where our servers are located is quickly red flagged if it's accidentally routed through the US; an earlier rule preventing that was removed. There was no wait to make contact with the customer when traffic was limited in such a way that it didn't behave like that, but like a failing NIC or something like that difficult to diagnose. By limiting traffic without waiting for response they also decimated all chances because email access is always routed remotely because of its requirements.
Locally, the restrictions are none, except those in Block traffic of course.
Loose routing breakdown
Exiting from the US gateway
- Traffic for some VOD services
- APNS traffic
- Traffic from Skiptube
- Traffic from Antipostal.com3): Exchange Web Services, Exchange ActiveSync, MAPI
- Traffic from email gateways: SMTP
- Apple's caching servers
- Skype for Business Server service traffic
Exiting from the local gateway
- Traffic for some VOD services
- File sharing/syncing protocols
- Traffic for some known repository-like servers4)
- Domains for Let's Encrypt API
- Domains for the Clouflare API
- Domains for the Clouflare API