This post touches upon various approaches of designing an IP addressing plan for an enterprise network. It will also deal with segmentation in general, because IP addressing and segmentation are dependent on each other.

IP Addressing

A typical enterprise network consists of multiple sites of varying sizes that might span a wide geographical area - even international and/or inter-continental. A such we have different areas of network types that needs IP addresses. At least LAN, WAN, and probably also DC are the areas or building blocks we see in our networks.

IP addresses are hierarchical in nature. We have a network part and a hosts part of an IP address. The network part, aka. the prefix, is what we see in our routing tables. The more hosts needed on a particular network, the smaller the network part of the IP address can be. We simply take bits from the network portion and assign for hosts. This is called subnetting and it’s essential to master when building an IP addressing plan. There are various ways to come up with a scheme for IPs. But before we can start assigning subnets to our sites, we must have a segmentation plan first.

Segmentation

The reason why we can’t just start assigning IP subnets to sites is because we first must discuss the need for segmentation. This is where we decide upon the security we need in our network and why we need it the way we do. For example, we probably do not want guests to be able to access the same resources as our internal employees. And even our internal employees might need segmentation from each other - perhaps both in regards to IP subnets and security policy. Segmentation is no longer needed to limit the scope of our layer 2 domains. We use segmentation to enforce a security policy which limits the possibility to communicate between hosts. The reason is not so much to optimize network performance anymore but to contain and limit the spread of malware, typically. Technically speaking we’re talking about creating VNs (virtual networks) or VRFs. But we could also acheive this segregation through intra-VN security policies using Cisco TrustSec. So which way should you go?

Using VNs

This is our proven way of segmenting at layer 3. As usual there are pros and cons to everything. With VNs the benefit is that we build fences using separate routing tables that make up our VNs. It is not possible to communicate between VNs unless we either implement route leaking or have a device such as a firewall inter-connect the VNs.

The downside with VNs is that it requires more IP subnets. A VN must have a unique subnet to be able to correctly route our traffic. That is unless we want to implement NAT which is technically valid, but comes with all sorts of challenges. Care should be taken when going down that path. Now, it might not seem like a big deal to add a few subnets to our IP plan, but think about a company that has 5 VNs in 25 locations. It can quickly become a nightmare to keep track, expand, and implement the segmentation needed using VNs. How would you keep segmentation end-to-end, for example?

So, why use VNs for segmentation? A VN gives us the flexibility and ability to route traffic any way we want. If we had just a single VN, like GRT, all traffic is routed the same way. Say you want to route guests to a dedicated firewall. This would not be plausible if guests reside in the same routing table as everything else. Yes, we could use PBR and use all sort of tricks, but really? And what if we need a deeper inspection of some traffic? A firewall will do just that for us, but we need a VN to route traffic through a firewall. At least in our campus network as it is today.

Also, when using VNs the segmentation is built-in. It is not something we add on top of our VN. It’s there, natively. We can combine VNs with intra-VN segmentation. Here, the VNs will serve as a safety net compared to using just intra-VN segmentation. If our intra-VN policy was to fail either due to mis-configuration (human factor) or a software bug, we still have our coarse segmentation in place with VNs. This means that the impact of such occurances might not be as deveasting as relying solely on using intra-VN segmetation techniques.

Using VNs is something most vendors offer and it is not proprietary in any way. It is just a routing paradigm.

Using Intra-VN

Segmentation using an intra-VN approach like with Cisco TrustSec also has its benefits and disadvantages. These are pretty much opposite of using VNs. Using only SGTs (Security Group Tags) and not VNs, we can save on the number of IP subnets required for our network. But we put all our traffic in the same bucket which might not be what we need. And as described in the previous section, we have no safety net if the technology fails or we mis-configure it.

What about a criteria for having a vendor agnostic environment or at least the possibility to change the vendor with something else?

The Plan - An Example

Let’s have a look at a practical example of taking into account the various aspects of allocating ip addresses and segments to an enterprise network. Below is a topology that could be a typical enterprise network consisting of a HQ site along with branch sites and a DC site. Connectivity to the Internet and cloud is also provided, but our focus is on creating an IP addressing plan along with segments (VNs). We’ve already touched upon why you would need segmentation and the methods of how to acheive this.

Our network design above uses software-defined (SD) technologies for WAN, LAN, and DC.

SD-LAN

The SD-LAN which could be Cisco’s Software-Defined Access (SD-Access) is segmented into three VNs:

• Users
• IoT
• Guests

Remember that VNs give us the ability to route traffic as we want to. Here, Guests are routed towards the edge FW connected to the Internet. There is no need for guests to have access to the internal network. This is a perfect use case for assigning guests to their own VN.

IoT contains “things” or devices that typically do not have a human interacting with them. They are considered headless devices and might be controlled from a management server either on-prem or in the cloud. This VN could contain devices such as HVAC systems, alarm systems, access systems, video surveillance, digital signage, and so on. Often we do not trust these devices to be secure and maintained to the same degree as our corporately managed devices (Windows PC’s, for example). This is yet another use case for assignning these types of devices to their own VN.

Note, that within each VN, an intra-VN security implementation should be in place. There are two reasons for this. The first is to be able to prevent east-west traffic which mitigates against malware spreading to other computers within the same VN. Secondly, it provides us with a handle to segregate different types of devices (or users) from each other. We could have a policy within the IoT VN that allows devices of a HVAC system to communicate with each other on certain ports while still preventing other types of devices to communicate with any other device within the IoT VN.

Here a HVAC device can communicate with other HVAC devices, but the other segments within the IoT VN cannot communicate with each other.

SD-WAN

SD-WAN is the transport agnostic network that inter-connects our sites. To maintain end-to-end segmentation, the VNs must be carried over to SD-WAN. Remember our policy to send guests our the Edge FW at the HQ site. This handoff is typically done using IP routing with BGP per VN. This is also known as Inter-AS Option A with MPLS VPNs or VRF-lite back-to-back peerings. Regardless, the deal is the same. We create VLANs to carry our traffic between the SD-LAN and SD-WAN for end-to-end segmentation. Even our intra-VN segments can be carried over in the data plane. That is, our SGT values. The point is that we must address, literally, these VLANs or linknets. And this is where it tends to get crazy. At least when you get above 5-10 VNs. We’re talking a subnet per VN per link per site. That is a lot! Of course this can be automated, but the data foundation (our IP plan) still needs to be build and maintained. Either way it is a lot of data to deal with. Especially when it comes to implementation and troubleshooting.

SD-DC

Our SD-DC also has segments for servers and a firewall to protect these and provide a secure inter-connect with other part of the network (cloud, campus, internet). Here other techniques exists for segmentation. I’ll not dive into those in this post.

Building the IP Plan

Now that we have a segmentation policy, it is time to allocate subnets for our IP plan. It is a good idea to design for future growth. Also, with software-defined networks we’re not burdended by having to create small broadcast domains as we have a control plane protocol for scaling the number of endpoints while ensuring broadcasts are not flooded everywhere. The protocols used for this are BGP EVPN and LISP with VXLAN as data plane to hold our segmentation information and tunnel the traffic in the same segment between switches. Beautiful.

So, to determine the prefix size needed we can use some math. To calculate the number of bits required for a specific number of endpoints:

log(n)/log(2)

n = number of endpoints in the VN

If we have 841 endpoints at Branch site A:

log(841)/log(2) ≈ 9.7159

If we “ceil” (round up) this number, we have 10 bits needed for endpoints on a subnet. Since, there are 32 bits in total for an IP address and the hosts portion is always assigned from the end:

32 - 10 = 22

This gives us a subnet of /22 and the number of endpoints in a /22 is:

2^10 = 1024-2 = 1022

Now we’d need some human thought about this number. Is it enough for future growth? If yes, then go with a /22 for this VN at this site.

Aggregate!

When assigning subnets be sure to think about aggregation to avoid creating more state in the network than necessary. Say you do this:

Site	Subnet
A	10.0.0.0/16
B	10.1.0.0/16

… and for Site A:

VN (A)	Subnet
User	10.0.0.0/22
IoT	10.0.4.0/22

Now, you’ll not be able to aggregate 10.0.0.0/16 anywhere, because you split it up in to smaller chunks. Instead subnet based on the VNs:

VN	Subnet
Users	10.0.0.0/16
IoT	10.1.0.0/16
Guests	10.2.0.0/16

With this approach you can aggregate all prefixes within the Users VN in a single routing entry (10.0.0.0/16) when routing towards the firewall, for example.

Which IP Addresses?

We all know of the RFC1918 IP addresses. It is perfectly fine to use these within your organization. These have existed since 1996 where they were allocated by IANA.

Several special-purpose IP addresses exist. You can find there here

Nobody says we must use the RFC1918 range. So, what are the use cases in which we might deviate from the norm? Let’s look at some use cases…

IPv4 Link Local Addresses - 169.254.0.0/16

Aka. APIPA. These addresses are non-routable, meaning we can assign them on links that just need an IP for communication, but we do not communicate with that IP or need reachability to that IP.

Firewalls typically implement HA using dedicated links for synchronizing their state and negotiating their HA role. This is a perfect example of when we can use IPv4 link local addressing:

And as long as you do not share the same network (VLAN/broadcast domain) between different fw HA deployments, you can even reuse the same addresses which makes templating/scripting such deployments easier.

Another use case is with Cisco DNA Center:

Cisco has chosen to use IPv4 link local addresses for DNAC. In particular on the cluster interface and also within the cluster for its container resources.

RFC6598 - 100.64.0.0/10

Aka. CGN addresses. These are non-routable on the Internet and dedicated for carrier grade NAT to be used by SPs for their private customers. Also heavily used for celluar communication with mobile phones.

Being a human, I tend to visualize things in order to understand them and to be able to differentiate them. Let’s say I have a software-defined network that implements the concept of underlay and overlays. The underlay provides transport between our edge nodes. The overlay is where our user traffic resides and it is encapsulated within a protocol for transport via the underlay for end-to-end communication within the SD network. We all favor using RFC1918 internally for our endpoints. For these reasons, I see no reason why you couldn’t use RFC6598 addresses for the underlay.

Here you use the RFC6598 addresses for infrastructure addressing. Each link between our switches gets an IP address in the 100.64.0.0/10 range for underlay communication. Endpoints in the overlay still get an IP in the RFC1918 range. This makes it very easy to differentiate between what is underlay and what is overlay.

Public IP Addresses

Public addresses can be routed on the Internet. Some decades ago companies used public IP addresses for their internal endpoints. Just like we would with IPv6 today. Technically speaking there is nothing wrong with using public addresses internally in your organization. It is kinda bad because of the scarceness of IPv4 addresses in the world today. But if you hold a good portion of public addresses and have an organization that grows by aquireing other companies (MA strategy), you might have a good use case for going down that road. Let’s have a look at an example:

Let’s say you own 1.0.0.0/21

Subnet	Usage
1.0.0.0/23	Public services
1.0.2.0/23	Unallocated/Free
1.0.4.0/22	Shared services

The shared services that must be used by all companies we aquire will be assigned a public IP from the range 10.0.4.0/22. This makes these services unique within our entire organization. There is no need to advertise these addresses on the Internet either. We have the first half of the 1.0.0.0/21 space for this purpose (1.0.0.0/22).

Of course we could have used RFC6598 addresses for our shared services which might be a good idea to conserve the number of IPv4 addresses. I just wanted to make a point that using public addresses within an organization might be OK depending on the type of company you have and your policies.

MPLS VPN

If you buy MPLS VPN services, you are responsible for allocating addresses for use by the SP. Remember it is your network (meaning your routing table) between your router and the SP’s PE router.

Here we have our RFC1918 for endpoint addressing behind our edge router towards the SP. Between our edge router the the SP’s CPE router we use RFC6598 and we make the SP use IPv4 link local between their CPE and PE. Why?

The CPE/PE links we have no reason to reach. This link is delivered and maintained by the SP. They use it to provide you with a service. It is their responsibility to ensure it works. It is still in the same VRF or routing table as our other addresses which is why we can use IPv4 link local instead of wasteing addresses from the other ranges. We have no need to be able to reach them. In fact, we should filter them from the updates received from the SP. Again this will help visualize which part of the network we’re dealing with, too.

Future of Segmentation

One might think about what the future might bring for networks. When I studied for CCNA back in 2007, I spotted a flaw with layer 2. There is no TTL which can be devastating for a network as traffic can loop infinitely. Later TRILL and FabricPath saw the light of the day. I wonder if we might see something like service insertion via a VXLAN header field or something along those lines.

Also, a I see a need for VXLAN inter-working. We have VXLAN in our Campus and in our DC. It should be possible to inter-connect the two domains for seamless end-to-end segmentation policies. It sure would be nice if the SGT inserted into the VXLAN header by an SD-Access edge node could travel all the way to the firewall in the DC for policy inspection based on that SGT value.

MTU Considerations

Whenever we encapsulate a packet, we risk fragmentation. You could ask whether the default MTU value of 1500 byes should be changed for networking equipment - not endpoints. That would really pave the way for creating more overlays which are very beneficial for all companies. Most equipment nowadays allows jumbo MTU configuration.

Conclusion

In this post we looked at IP addresses, their hierarchy, and how to both calculate subnets and apply them in various designs. Different special address types were discussed, too. Namely RFC1918, RFC6598, and IPv4 link local addresses.

Segmentation plays an important role in today’s networks. We talked about ways to segment a network with the purpose of creating secure networks using both VNs and SGTs.

Finally, some thoughts of the future of segmentation was touched. I do believe that some changes in the way we forward our traffic is coming to our networks in the future. Especially the ability to perform service insertion in the campus network based on header information. And interworking for VXLAN (or whatever the protocol of the future might be) would also be very useful.

I hope you enjoyed reading this rather long post on IP addressing and segmentation for enterprise networks. Thanks!