Flashcards in Test 3 SDN Firewall Deck (102)
SDN Firewall: 2 defining characteristics
SDN separates the control plane from the data plane
An SDN consolidates the control plane, so that a single software control program controls multiple data-plane elements.
Decides how to handle traffic
Forwards traffic according to decisions that the control plane makes
A programming interface (or network API) that exposes resources (e.g. processing, storage, and packet queues) on individual network nodes.
Supports construction of custom functionality to apply to a subset of packets.
OpenFlow Switch Rule Table includes:
A bit pattern (including wildcards, for matching header fields such as MAC and IP addresses, protocol, TCP/UDP port numbers, physical input port, etc)
A priority: to break ties between overlapping patterns.
A list of actions: for example, forward out a port, flood, drop, send to controller, assign a new value to a header field, etc
Optional hard and soft timeouts to evict stale rules
Byte and packet counters that collect information about how much traffic is flowing through each rule
Pyretic sequential composition
Treats the output of one policy as the input of another
Pyretic parallel composition
Applies two policy functions on the same packet and combines the results
3 Stages in History of SDN
Active Networks (1990s-2000s): introduced programmable functions in the network to enable greater innovation
Control and Data Plane Separation (2001-2007): developed open interfaces between the control and data planes
OpenFlow API and network operating systems (from 2007-2010): first instance of widespread adoption of open interface, developed ways to make control-data pane separation scalable and practical
Active Networking Programming Models
Capsule Model: code to execute at the
nodes was carried in-band in data packet
programmable router/switch model, where the code
to execute at the nodes was established by out-of-band
Technology Pushes encouraging active networking
reduction in the cost of computing
advances in programming languages such as Java offered platform portability and some code executing safety,
virtual machine technology that protected the host machine
Technology Use Pulls for active networking
Motivation similar to motivation for SDN today
Issues include network service provider frustration with the timescales necessary to develop and deploy new network services (network ossification)
this-party interest in value -added, fine-grained control to dynamically meet the needs of particular applications or network conditions
researcher desire for a platform that would support experimentation at scale
unified control over middleboxes, including firewalls, proxies, and transcoders
Active Networks Intellectual Contributions
Programmable functions in the network to lower the
barrier to innovation.
Network virtualization, and the ability to demultiplex
to software programs based on packet headers.
The vision of a unified architecture for middlebox orchestration.
Motivation for Separating Control and Data Planes
In the early 2000s, increasing traffic volumes and a
greater emphasis on on network reliability, predictability,
and performance led network operators to seek better approaches to certain network-management functions such as the control over the paths used to deliver traffic (a practice commonly known as traffic engineering)
Separate Control and Data Planes Intellectual Contributions
Logically centralized control using an open interface to the data plane.
Distributed state management
An OpenFlow switch has a table of packet-handling
rules, where each rule has:
a pattern (that matches on bits in the packet header),
a list of actions (e.g., drop, flood, forward out a particular interface, modify a header field, or send the packet to the controller),
a set of counters (to
track the number of bytes and packets), and a priority (to disambiguate between rules with overlapping patterns).
Upon receiving a packet, an OpenFlow switch identifies the highest-priority matching rule, performs the associated
actions, and increments the counters.
Intellectual Contributions of OpenFlow
Generalizing network devices and functions.
-- Still, OpenFlow does not offer data-plane support for deep packet inspection or connection reassembly; as such, OpenFlow alone cannot efficiently enable sophisticated
The vision of a network operating system.
• Distributed state management techniques
Network Operating Systems
More generally, the emergence of a network operating system
offered a conceptual decomposition of network operation into three layers :
(1) a data plane with an open interface;
(2) a state management layer that is responsible
for maintaining a consistent view of network state;
(3) control logic that performs various operations depending
on its view of network state.
presents the abstraction of a network that is decoupled
from the underlying physical equipment. Network virtualization
allows multiple virtual networks to run over a
shared infrastructure, and each virtual network can have a
much simpler (more abstract) topology than the underlying
a small set of upgraded
nodes use tunnels to form their own topology on
top of a legacy network.
In an overlay network, the upgraded nodes run their own control-plane protocol, and direct data traffic (and control-plane messages) to each other by encapsulating packets, sending them through the
legacy network, and decapsulating them at the other end.
The Mbone (for multicast) , the 6bone (for IPv6) ,
and the X-Bone  were prominent early examples.
The OpenFlow protocol defines how the controller and switches
interact. The controller maintains a connection to each switch over which OpenFlow messages are sent.
The controller uses
these OpenFlow messages to (un)install rules, query the traffic
counters, learn the network topology, and receive packets when
the switch applies the “send to controller” action.
Advantages of Pyretic
it helps support modular programming
Pyretic integrates monitoring into the policy function and supports
a high level query API. The programmer can easily combine monitoring and forwarding using parallel composition
provides facilities for creating a dynamic policy whose behavior will change over time, as specified by the programmer.
Pyretic offers a rich topology-abstraction facility that allow programmers to apply policy functions to an abstract view of the underlying network.
we use predicate policies (including match and conjunction)
to disambiguate between packets based on their location in the
network as well as their contents;
we use modification policies (such as fwd) to change the header content or location of packets;
and we use composition operators (such as +, parallel composition
and >>, sequential composition) to put together policy
Sequential composition (>>) treats the output of one policy as the
input to another
Parallel composition (+) applies two policy functions on the same
packet and combines the results
Difficulties of Network Configuration
difficult to define what we mean by correct behavior in the first place
the interactions between multiple routing protocols can lead to unpredictability. Furthermore, each autonomous system on the internet is independently configured. And the interaction between the policies of these autonomous systems can lead to unintended, or unwanted behaviour.
he third reason that configuration is hard, is that operators simply make mistakes. Configuration is difficult, and network policies are very complex. Furthermore, Network configuration has historically been distributed across hundreds, or more, network devices. Across the network where each device is configured with vendor-specific low-level configuration.
SDN provides three things:
The first, is network-wide views of both topology and traffic.
The second, is the ability to satisfy network level objectives such as those that we talked about before including load balance, security, and other high level goals.
The third thing, that software defined networking provides that network operators need, is direct control. In particular, rather than requiring network operators to configure each device individually with indirect configuration, SDN allows an operator to write a control program that directly affects the data plane. So rather than having to configure each device individually and guess or infer what might happen, software-defined networking allows a network operator to express network level objectives and direct control from A logically centralized controller.
2 SDN defining characteristics
1. In an SDN, the Control Plane runs in a logically centralized controller.
Additionally, the controller typically controls multiple routers across the network and often, the control program exerts control over all the routers in the network, thus facilitating network-wide control.
2. The separation of data and control allows a network operator to build a network with commodity devices, where the control, resides in a separate control program.
This re-factoring allows us to move from a network where devices are vertically integrated making it very tough to innovate to a network where the devices have open interfaces that can be controlled by software. Thus, allowing for much more rapid innovation.
Advantages of SDN
easier to coordinate behavior among a network of devices.
behavior of the network is easier to evolve, and it's also easier to reason about.
History of SDN
Previous to 2004, configuration was distributed
---buggy and unpredictable behavior.
Around 2004, logically centralized controller focused on the border gateway protocol, and was called the routing control platform, or RCP.
In 2005, different planes:
---The decision plane which computed the forwarding state for devices in the network,
---the Data Plane, which forwarded traffic based on decisions made by the decision plane.
---dissemination and discovery planes, which provide the decision plane the information that it needs to compute the forwarding state, which ultimately gets pushed to the data plane.
Around 2008, OpenFlow.
---emergence of cheap switches that were build based on open chip sets that could be controlled from software.
--- allowed us to decouple the control plane and the data plane in commodity switching hardware.
the Control Plane is typically a software program written in a high level language, such as Python or C,
the Data Plane is typically programmable hardware that's controlled by the control plane.
The controller effects the forwarding state that's in the switch using control commands.
--Open flow is one standard that defines a set of control commands by which the controller can control the behavior of one or more switches.
SDN has many applications including data centers, wide area backbone networks, enterprise networks, internet exchange points or IXPs, and home networks. Later modules in this course will explore how software defined networks.
Control Plane Operations
The job of the Control Plane is to compute state that ultimately ends up in the data plane.
computing a forwarding path that satisfies a high-level policy
computing shortest path routing trees.
making decisions about whether or not a user's device should be allowed to send traffic or not based on that device's MAC address
Data Plane Operations
(decisions are being made at forwarding time, not by a centralized high-level program.)
Differences in Control Plane and Data Plane
control plane is the logic that controls forwarding behavior.
---Examples of control plane functions include routing protocols as well as logic for configuring network middle boxes.
---routing protocol functions that compute the paths
Data plane: The forwarding table themselves and specifically the actions associated with forwarding traffic according to the Control plane logic
---examples of data plane function include forwarding packets at the IP layer, and doing things like switching at layer two
---the act of actually taking a packet, on an input port, and forwarding it, to an output port
why is separating the data and control planes a good idea?
independent evolution and development. Thus, software control of the network, can involve independently of the network hardware.
the opportunity to control the network behavior from a high-level software program. Controlling the network from a high-level program in theory allows network operators to debug and check network behavior more easily. Then in the status quo, where network behavior is determined by the distributed low level configuration across hundreds of switches and routers.
provides opportunities for better network management and data centers by facilitating such network tasks as virtual machine migration to adapt to fluctuating network demands.
In Routing, the separation of data and control provides more control over decision logic.
In Enterprise networks, SDN provides the ability to write security applications such as applications that manage network access control.
In Research networks, the separation of data and control. Effectively allows to virtualize the network, so that, research networks and experimental protocols can co-exist with production networks on the same, underlying network hardware.
Reasons for Separating Data and Control
Separating the data and control plane can allow for independent evolution of the data and control plane.
Separating vendor hardware from the logic that controls the behavior of the network, and the potential to more easily reason about network behavior since the behavior is now controlled from a single logically-centralized control program.
(The separation of data and control plain does not inherently make the network more fault tolerant or more scalable. )
Example SDN: Data Centers
A data center, typically consists of many racks of servers. And any particular cluster might have, as many as 20,000 servers. Assuming that each one of these servers can run about 200 virtual machines. That's 400,000 virtual machines in a cluster.
A significant problem is provisioning or migrating these virtual machines in response to varying traffic loads. SDN solves this problem by programming the switch state from a central database
If we need to provision additional virtual machines. Or migrate a virtual machine from one server to another in the data center, the state in these switches must be updated. Updating the state in this fashion is much easier to do, from a central controller or a central database, facilitating.
This type of migration is also made easier by the fact that the servers are addressed with Layer two Addressing. And the entire data center Looks like a flat layer two topology. What this means, is that a server can be migrated from one portion of the data center to another without requiring the virtual machine to obtain new addresses. All that needs to happen for forwarding to work ,is the state of these switches. Needs to be updated. The task of updating switch date in this fashion is very easy to do, when the control and data plans are separate..
how does the control data plan separation make managing data centers easier?
control data plane separation can make it easier to manage the data center by monitoring and controlling routes from a central point and allowing virtual machines to be migrated without renumbering host addresses.
The control data plane separation does not inherently make it possible to build a data center with few switches nor does it automatically balance load.
2 SDN Challenges
scalability & consistency
security or robustness
SDN and security of internet backbones
Where filtering attack traffic is a regular network management task. Suppose that an attacker is sending lots of traffic towards a victim.
In this case a measurement system might detect the attack, identify the entry point, and a controller such as the RCP might install what is called a null route to ensure that no more traffic reaches the victim from the attacker.
SDN Challenge: scalability and consistency.
In an SDN a single control element might be responsible for many forwarding elements.
So control element might be responsible for hundreds to thousands of switches.
Of course, for redundancy and reliability, typically we want to replicate the controller.
So while the controller is logically centralized, physically there may be many replicas.
And, in such a deployment scenario we need to ensure that different controller replicas see the same view of the network so that they make consistent decisions when they're installing state in the data plane.
SDN Challenge: security or robustness
we want to make sure that the network continues to function correctly in the event that a controller replica fails or is compromised.
Coping With Scalabilty
Eliminating redundant data structures can help save memory in the control program running at the controller.
Only performing ,a fixed number of network management operations such as routing, can insure that the controller doesn't have to do too much, thereby improving scalability.
Caching forwarding decisions, that the control plain has already made ,in the switches, can ensure ,that not to much traffic, is redirected to the controller, and running multiple controllers can distribute the load of the control plane across multiple replicas, sending all traffic to the controller only increases the controller load, and would not help with scale ability.
Options for SDN controllers
NOX, Ryu, Floodlight, Pyretic, Frenetic, Procera, Route Flow, Trema,
NOX vs POX
NOX provides good performance but requires you to understand and be comfortable with the facilities and semantics of low level open flow commands.
NOX also requires the programmer to write the control application in C++, which can be slow for development and debugging.
Because Ryu is implemented in Python, it still does not perform as well other SDN controllers, such as nox.
To address the shortcomings that are associated with development in C++, Pox was developed.
--Pox is widely used, maintained, and supported.
-- It's also easy to use, and easy to read and write the control programs.
--Of course, as might come with implementing a controller in python, the performance of Pox is not as good as the performance of Nox.
You might use Pox in a class project, or in a university research project where there's a need to quickly prototype, and evaluate a brand new control application.
Pox is less applicable in a large internet data center, because it does not perform as well as other controllers.
Floodlight is maintained by big switch networks.
---integration with the rest API, and
Unfortunately it also has a fairly steep learning curve. So you should use floodlight is you already know Java, if you need production level performance and support.
Customizing control with Pox
When the controller starts. It adds a listener that listens for a connection up, which is a connection from a switch.
When switch connects, it simply sense and open-flow, flow modification back to the switch it says flood all packets out every output port.
The first function here involved creates the open-flow massage.
And the second sends that message back to the switch.
Customizing control with learning switches
In contrast to Pox, a learning switch maintains a switch table that's initially empty.
But when a packet arrives on input port such that whenever a subsequent packet is destined for destination A, the switch knows to forward the packet via output port 1.
As before, when the first packet arrives at the switch, it is diverted to the controller, at this point, the controller maintains a hash table that maps the address to the out put port, when it sees that first packet From Post A, it updates the address and port table.
If the packet's a multicast packet,the controller makes a decision to flood that packet on all output ports.
Likewise, if there's no table entry for the destination for that packet, the controller also instructs the switch to forward the packet on all output ports.
If the source and destination address are the same, The controller instructs the switch to drop the packet.
Otherwise, the controller installs the flow table entry corresponding to that destination address and output port.
Installing that flow table entry in the switch prevents future packets for that flow from being redirected to the controller.
Rather, all subsequent packets on that flow can be handled directly by the switch, since it now knows which output port to send a packet for that particular destination.
Is IS-IS control plane or data plane?
control – this is used to calculate routes that allow routers to later forward data packets, but
does not carry data for any application
control plane or data plane?
data – the actual IP packets that are forwarded by routers are the packets that contain
control plane or data plane?
data – similar to b), these UDP packets contain application data
control plane or data plane?
control – this is used to automatically assign IP addresses to end hosts (and sometimes subnet
and DNS server locations as well), which is required for that end host to then be able to send
and receive data packets, but DHCP messages do not contain any application data themselves
Is 802.11 (Wi-Fi)
control plane or data plane
data – this is a link layer protocol that carries data for applications or higher level protocols
(which would be considered “data” by the link layer, even if they are not data at the application
Name some situations/scenarios in which using SDN provides a benefit.
One scenario in which SDN is helpful is when something breaks in the network (at the software/configuration level). Since the control plane is separate and policies are centralized in
the SDN controller, it is easier to see the “big picture” of what your network configuration is
actually doing and you can find and fix problems more easily.
Another scenario is when you want to update your network. Instead of buying all new hardware to get the latest control
plane features, you simply update your software in the SDN controller.
policies is easier as you just update the configuration expressed by the DNS controller software,
and you don't have to go around to each network device and update its individual piece of the
global policy separately (and hope that you didn't miss one or accidentally misconfigure one inthe process!).
Finally, SDN is useful in research or testbed network. Because SDN is flexible, you
can create new control techniques or try different policies to experiment with them, without
having to build a new piece of hardware that implements the new behavior. This is useful not
only because experiments to test new ideas are easier and less expensive, but also because it
allows rapid iteration on those ideas if it becomes apparent some further refinement on the
idea is needed.
Name some places where network virtualization is useful, and some places where it is not
normally especially useful.
Network virtualization is useful in multi tenant data centers (or “the cloud”) in order to provide
each tenant with the illusion that they have a private network connecting their servers/Vms,
and possibly to allow them some ability to configure their virtual network without affecting
other tenants. It is also useful in R&D environments (e.g., universities or anyplace else research
is done) in order to isolate networking experiments from the rest of the general-purpose traffic
in their enterprise. Thus experimental techniques that be tried without causing problems for
the rest of the network. Finally, it is useful in computer networking classes. Similar to the
research scenario, we want to try doing some different things that may not be a good idea on
the production network in order to learn more about how networking works. Virtualization
allows us to try things without breaking the network for everyone else. Other answers may be
possible, but these are three major use cases.
Network virtualization is generally bad in situations where you can consider it overkill, or where
the costs outweigh the benefits. For example your home network connecting to your ISP or the
corporate network you use at work are poor candidates for network virtualization.
Networks that are highly sensitive to latency are not good candidates. For example systems
with system critical cyber-physical devices would likely not be able to trade a layer of
virtualization to obtain the flexibility provided. For example, a network of hosts and physical
devices used to launch manned space vehicles, conduct air traffic control, control a nuclear
reactor, etc. are bad candidates for network virtualization.
Why use the Pyretic programming API when the hardware itself exposes the OpenFlow
The Pyretic API provides a high-level abstraction for SDN programmers. The OpenFlow API
exposed by devices supporting it is a low level API, on the level of assembly language. It is
inordinately difficult to develop sophisticated SDN applications with the OpenFlow API.
Additionally, the Pyretic runtime provides an efficient runtime that automatically installs
generated low level rules on hardware devices throughout the network.
How does a network policy implemented in python and executed on a Pyretic con-troller
result in policies on OpenFlow switches?
First, using the Pyretic API, the programmer specifies a high level network policy. The Pyretic
runtime connects via sockets to OpenFlow clients on the network. The Pyretic runtime
interprets packets traversing these network clients against the policy, and using its socket
connection installs OpenFlow rules to implement policy behavior
Additionally, these connections allow the Pyretic runtime to perform other actions, like
proactively installing rules to reduce network latency, reading counters, etc.
Describe the function of the following pyretic network policy functions:
Returns one packet per local port on the network spanning tree
Describe the function of the following pyretic network policy functions
match(dstip=‘192.168.1.15’) & match(srcip=‘192.168.1.120’)
Two separate match predicates are composed, the result matches any packet that has
destination IP = 192.168.1.15 and source IP – 192.168.1.120
Describe the function of the following pyretic network policy functions
match(dstip=‘10.0.0.8’) >> fwd(12)
A single match predicate sequentially composed with another, the result of which matches
packets any packet bound for IP 10.0.08 and forwards it along port 12. This effectively “filters
out” all traffic not bound for IP 10.0.0.8.
Describe the function of the following pyretic network policy functions:
match(dstip= ‘10.0.0.1’) >> ( match(srcip=‘10.0.0.15’) >> drop() +
match(srcip= ‘10.0.0.25’) >> modify(dstip=‘10.0.0.30’) )
This policy implements a complex policy. First, all traffic not bound for IP 10.0.0.1 is filtered.
Any packets bound for 10.0.0.1 is then subject to parallel composition. If the packet is from IP
10.0.0.15, it is dropped. If the packet is from 10.0.0.25, it is returned, with the destination IP
rewritten to 10.0.0.30.
Updates in Software Defined Networks: 2 Consistency problems in SDN
Packet Level Consistency Problem:
The open flow API however does not provide specific guarantees about the Level of consistency that packets along an end-to-end path can experience. So for example, updates to multiple switches along a path in a network that occur at different times may result in problems such as forwarding loops.
flow level consistency problem:
Additionally, if updates to the switches along an end-to-end path occur in the middle of a flow, Packets from the same flow may be subjected to different network states.
3 Steps of Programming SDN
The first is that the controller needs to read, or monitor network state, as well as various events that may be occurring in the network. These events may include failures, topology changes, security events, and so forth.
The second step is to compute the policy based on the state that the controller sees from the network. This is effectively what we talked about last time, is the role of the decision plane. In deciding what the forwarding behavior of the network should be, in response to various states that it reads from the network switches.
The third step is to write policy back to the switches by installing the appropriate flow table state into the switches.
Consistency problems can arise in two steps
First, the controller may read state from the network switches at different times, resulting in an inconsistent view of the network-wide state,
second, the controller may be writing policy as traffic is actively flowing through the network. Which can disrupt packets along an n to n path, or packets that should be treated consistently because they're part of the same flow. Both reading and writing networks state can be challenging because open flow rules are simple match action predicates, so it can be very difficult to express complex logic with these rules. If we want to read state that requires multiple rules, expressing a policy that allows us to read such a state can be complicated without more sophisticated predicates.
SDN difficult to express complex logic with these rules. Give example.
If we want to read state that requires multiple rules, expressing a policy that allows us to read such a state can be complicated without more sophisticated predicates. For example, let's suppose that when we are reading state, we'd like to see all web serving traffic except for source one, two, three, four.
Simple match action rules do not allow us to express such exceptions.
As a solution to this problem, we need a language primitive that allows us to express predicates. Here is a simple statement that has several predicates; such as, and and not. A runtime system can then translate these predicates into low-level open flow rules, ensuring that they are installed atomically and in the right order.
switches only have limited space for rules. Give example.
It's simply not possible to install all possible rule patterns for every set of flows that we'd like to monitor. For example, if we'd like to count the number of bytes for every source IP address, and generate a histogram with the resulting traffic, we would potentially need a flow table entry for every possible source IP address. It's simply not possible to install all of these possible rules. The solution is to have the run time system dynamically unfold rules as traffic arrives. A programmer would specify something like a group by source IP address, and the run time system would dynamically add open flow rules to the switch as traffic arrives. Thereby guaranteeing that there are only rules in the switch that correspond to active traffic.
Reading Network State
Another problem that arises when reading state, is that ,extra unexpected events may introduce inconsistencies. A common programming idiom, is that, the first packet goes to the controller and once the controller figures out what policy to apply for that flow, the controller then installs rules in the switches, in the network, corresponding to that flow. What if more packets should arrive ,at the switch before the controller has a chance to install rules for that flow? At this point, multiple packets may reach the controller, but the application It is running on top of the controller, may not need or want to see these additional packets.
So, the solution, is to have the programmer specify by a high level language a limit of one, indicating that the application should only see the first packet of the flow and that the subsequent packet should be suppressed. The runtime system then hides the extra events. So to remind you where we are, we talked about problems with consistency when reading state from the network, and we talked about ,three approaches to, helping guarantee consistency when reading state. Predicates, rule unfolding and suppression. And let's now talk about primitives that can help maintain consistency, when writing State.
reasons that a controller might want to write policy
to change the state and the network switches, including maintenance, unexpected failure, and traffic engineering.
Writing Network Policy: maintain the following in variance
there shouldn't be any forwarding loops and there shouldn't be any black holes whereby a router or switch receives a packet and doesn't know what to do with it.
There also shouldn't be cases where traffic is going where it shouldn't be allowed to go because of the network being in an inconsistent state.
example of what might happen when policies are written to the network, if they're written in an inconsistent fashion
Let's consider a case where we have a network that is performing shortest routing to some destination. And the link weights are as at shown here in the figure. Traffic in the network would flow along a path shown in green. Let's suppose now that an operator wants to change the network state to shift traffic off of this link. He could do so by updating the link weight. In doing so, the new shortest path from this top router would be as follows. But, what if the state in the top switch occurred before the state in the bottom switch could be updated? In this case, we would have a potential forwarding loop. Traffic would proceed to the bottom switch. But the bottom switch would still have the old network state, and would continue to forward traffic to the top switch, resulting in a forwarding loop. If rules are installed along a path out of order, packets may reach a switch before the new rules do. So, in this type of model we would have to think about all possible packet and event orderings to ensure that consistent behavior resulted.
Solution to inconsistent updates to policies
So we need atomic updates of the entire configuration. The solution to this problem is to use a two phase commit so that packets are either subjected to the old configuration on all switches, or to the new configuration on all switches. But packets aren't subjected to the new policy on some switches and the old policy on others. The idea is to tag the packet on ingress so that the switches maintain copies of both P1 and P2 for some time. When all switches have received rules corresponding to the new policy, then incoming packets can be tagged with P2. After some time, when we're sure that no more packets with P1 are being forwarded through the network, we can only then remove the rules corresponding to policy P1. Now, the naive version of two-phase commit, requires doing this on all switches at once. Which essentially doubles the rule space requirements, since we have to store the rules for both P1 and P2. We can limit the scope of the two phase commit by only applying this mechanism on switches that involve the affected portions of the traffic, or the affected portions of the topology.
What types of problems can arise from inconsistent applications of writing policy?
Inconsistent writes can result in forwarding loops or security policy violations where traffic ends up going to parts of the network where it shouldn't go as a result of inconsistent switch state.
The ability to respond to failures is orthogonal to consistency.
A flood of traffic at the controller technically involves problems with reading state in a consistent fashion.
But since there also involves a step where the controller writes state to the switches, while packets are still arriving at the controller, I would consider that answer to be correct as well.
What are some approaches to coping with inconsistency?
In this case, there is only one correct answer, which is keeping the old and new state on the routers and switches. This is the two-phase commit approach that we talked about.
Running different controllers for different switches could obviously result in an inconsistent state, since each of those controllers maybe making independent decisions.
Keeping a hot spare replica does no good if the replica also writes state inconsistently to the network.
And resolving conflicts on the routers also doesn't work because no router has a complete view of the network state.
an application of software defined networking
network virtualization is simply an abstraction of the physical network. Where multiple logical networks can be run on the same underlying shared physical substrate.
So in a virtual machine environment, we have virtual machines where a hypervisor, arbitrates access to the underlying physical resources. Providing to each virtual machine the illusion that it's operating on its own dedicated version of the hardware. Similarly, with virtual networking, a network hypervisor of sorts arbitrates access to the underlying physical network, to multiple virtual networks. Providing the illusion that each virtual network actually has its own dedicated physical network.
Why Use Network Virtualization
One of the main motivations for the rise of virtual networking was the" ossification" of the internet architecture. In particular because the internet protocol was so pervasive, it made it very difficult to make fundamental changes to the way the underlying internet architecture operated. There was a lot of work on overlay networks in the 2000's but one size fits all network architectures were very difficult to deploy. So rather than try to replace existing network architectures, network virtualization was intended to allow for easier evolution.
In other words, network virtualization enables evolution because we didn't have to pick a winner for a replacement for IP. We could instead let multiple architectures exist in parallel.
In practice, network virtualization has really taken off in multi -tenant data centers where there may be multiple tenants or applications running on a shared cluster of servers.
Well known examples of this include Amazon's EC2, Rack Space, and things like Google App Engine. Large service providers such as Google, Yahoo and so forth also use network virtualization to adjust the resources that are devoted to any particular service at a given time.
what are the motivations for network virtualization or virtual networks
virtual networks can facilitate research in evolution by allowing experimental networks to coexist with production networks.
Because the networks are virtual, they can be scaled up and down adjusting the resources that are devoted to any one particular service as demands change. We discuss this in the context of production networks, such as Google and Yahoo.
Virtual networks are not inherently easier to troubleshoot, nor do they necessarily provide better forwarding performance. In fact, forwarding performance may be worse, due to the additional level of indirection that has been added.
Network Virtualization distinguished from SDN
Virtualization is arguably one of the first killer applications for SDN. And in some sense, SDN is a tool for implementing Network virtualization.
the defining tenant of SDN is the separation of the data and control plant, whereas the defining tenant of Network virtualization Is to separate the underlying physical network from the logical networks that lie on top of it.
So SDN can be used to simplify many aspects of Network virtualization. But it does not inherently obstruct the details of the underlying physical network.
benefits of Network Virtualization
Some of the promised benefits of Network Virtualization are more rapid innovation since innovation can proceed at the rate at which software evolves. Rather on hardware development cycles, allowing for new forms of network control, and potentially simplifying programming.
characteristics of network virtualization, that are not necessarily characteristics of SDN
Network virtualization can allow multiple tenants to share the underlying physical infrastructure. And it also separates logical and physical networks.
characteristics of software defined networking, but not of network virtualization:
--Controlling behavior from a logically centralized controller.
--Or separating data and control planes.
Design Goals for Network Virtualization
able to support different topologies, routing and forwarding architectures and independent configurations
manageable, in other words, they should separate the policy that a network operator is trying to specify from the mechanisms of how those policies are implemented
scalable, maximizing the number or coexisting virtual networks
secure by isolating the different logical networks from one another
heterogenious in the sense that they should support different technologies
virtual networks have two components,
Virtual network nodes
physical nodes themselves must be virtualized.
One possible way virtualizing a node is a virtual machine. A more lightweight way of virtualizing a node is using a virtual environment such as a VServer or a Jail.
The hypervisor or whatever technology is enabling the virtual environment can effectively slice the underlying physical hardware to provide the allusion of multiple guest nodes or multiple virtual nodes
Examples of node virtualization include, virtual machine environment such as Xen or VMware or what's called OS level virtualization or virtual environments, such as Linux Vservers
Virtual Network Edges
connect virtual machines.
Each virtual machine or virtual environment has its own view of the network stack.
provide the appearance that these nodes are connected to one another over a Layer two topology, even if they are in fact separated by multiple IP hops.
--encapsulate the Ethernet packet as it leaves the VM on the left in an IP packet. The IP packet can then be destined for the IP address of the machine on the right, and when the packet arrives at this machine, the host can decapsulate the packet and pass the original Ethernet packet to the VM or the virtual environment that's residing on that physical node.
Virtual network virtual switch
Each of one these physical hosts, may in fact, host multiple virtual machines or virtual environments, which creates the need for a virtual switch that resides on a physical host.
This virtual switch provides the function of networking virtual machines together over a virtual layer two topology.
The Linux bridge is an example of a software switch that can perform this type of function.
Open Vswitch is another example of software that performs this type of glue function.
Example of Virtualization
OS Level Virtualization:
--upon start of mininet, each host in the virtual network is a bash process with it's own network name space
--A network name space is like a virtual machine except more lightweight
each one of the virtual nodes has its own view of the network stack
-- But shared filesystem
--not running it's own independent virtual machine
root namespace manages the communication between these distinct virtual nodes
switch that connects these nodes in the topology that you set up
Virtual ethernet pairs are assigned two name spaces
open flow switch effectively performs forwarding between the interfaces in the root name space
-- because the interfaces are paired, we get the illusion of sending traffic, between h2 and h3
Modifications via controller done in root name space
Summary of virtual networks
virtual networks facilitate:
flexible, agile deployment by enabling rapid innovation at the pace of software
SDN Programming Difficulty
very low level of abstraction in the form of match action rules
The controller only sees events that switches don't know how to handle
there can be race conditions if switch level rules are not installed properly
Solution to SDN Programming Difficulties
SDN Programming Interface: northbound API
applications or orchestration systems need to perform more sophisticated tasks, such as path computation, loop avoidance, and so forth
Goal: express what should happen in terms of higher-level behaviors without regard to such things as whether or not the rules are being installed in a consistent and correct fashion
benefits of such a northbound API are vendor independence, as well as the ability to quickly modify or customize control through various popular programming languages
Examples of such applications include the implementation of a large virtual switch abstraction, security applications, and services that may need to integrate traffic processing with middle boxes
--a SQL-Like query language
programmer counts the number of bytes, grouped by destination Mac address, and report the updates to these counters every 60 seconds
group by statement allows a grouping of counts by the destination mac address.
Where allows restrictions to only count trafic coming from a web server coming in on a particular port and every specifies that the results of this query should only be returned every 60 seconds
Issue with SDN Programming Interface
Overlapping Network Policies
an operator might write multiple modules, each of which affects the same traffic
For example, an operator might write an application that monitors traffic. Another one that specifies how routing should take place, another that involves the specification of firewall rules And yet another that balances traffic load across the links in the network. Ultimately, all of these applications, or modules, must be combined into a single set of open flow rules that together achieve the network operator's overall goal
Solution: composition operators, or ways that specify How these individual modules should be combined or composed into a single coherent application
Ways to avoid overlapping network policies
1. perform both operations simultaneously. For example, one might want to forward traffic but also count how much traffic is being forwarded. Both of those operations can be performed in parallel
2. Sequential composition performs one operation then the next.
Sequential composition performs one operation then the next.
--allows each module to partially specify functionality without having to write the policy for the entire network
-- flexibility so that one module can implement a small bit of the network function, leaving some functions for other modules
--allows for module re-use, since a module need not be tied to a particular network setting
For example, we might want to implement a firewall. And whatever traffic makes it though the firewall might then be subjected to the switching policy. One example of sequential composition, might be a load balancer. In this example, a policy might take some traffic coming from, half of the source IP addresses, and rewrite that to one server replica and take the other half of the traffic and rewrite it to the other replica. After the load balancer rewrites the destination IP address, we need a routing module to forward the traffic out the appropriate port on the switch. In this case, we've used sequential composition to first apply a load balance policy that rewrites the destination IP address based on the source IP address where the traffic is coming from and sequentially apply a routing policy that forwards the traffic out the appropriate port. Depending on the resulting destination IP address after that rewrite has taken place.
use predicates to specify which traffic traverses which modules. Those predicates can apply specific actions based on things like the input port and the packet header fields.
sits on top of an SDN controller and provides and exposes higher level abstractions that allows the operator or programmer to write policies without regard to how open flow rules eventually get installed
Summary of two composition operators.
These specify how individual simpler policies can be composed to implement more complex network applications, thus allowing different SDN control programs to independently perform tasks on the same traffic
Example SDN Language
- a way of expressing high level policies
run time provides the function of compiling these policies
key abstraction in Pyretic is the notion of located packets installed on the switches
--apply a policy based on a packet and it's location, in a network. Such as the switch, at which that packet is located or the port on which that packet arrives
Pyretic key features
1. the ability to take as input a packet, and then return packets at different locations in the network.
--allows the implementation of network policy as a function that simply takes packets and returns other packets at different locations
2. Boolean predicates.
--Unlike open flow rules, which do not permit the expression of simple conjunctions such as and, and or, or negations like not
--Pyredic allows the expressions of policies in terms of these predicates
3. virtual package header fields
--allows the programmer to refer to packet locations and also to tag packets so that specific functions can be applied at different portions of the program
4. composition operators
--such as parallel and sequential composition
5. network policy as a function
--policies are functions that map packets to other packets
- identify function, which returns the original packet
-none or drop, which returns the empty set
-match which returns the identity if the field f matches the value v and returns none or drop otherwise.
-Mod, which returns the same packet with the field f set to v.
-Forward, which is simply syntactic sugar on mod. To say that, the output port field in the packet should be modified to the parameter specified and flood which returns one packet for each port on the network spanning tree.
6. match function outputs either the packet or nothing, depending on whether the predicate is satisfied.
--For example, we could apply a match statement that says match destination IP equals ten zero zero three. And this function would take packets as input and only return packets that satisfy this particular predicate.
7. virtual packet header fields
--a unified way of representing packet metadata.
--In Pyretic, the packet is nothing more than a dictionary that maps a field name such as the destination IP address to a value. Now, these field names could correspond to fields in an actual packet header.
--But they can also be virtual. For example, we could provide a match statement based on a switch, indicating that we only want to return packets that are located at a particular switch or on the input port, indicating that we only want c packets whose attributes match a particular input port. The match function matches on this packet meta-data and the mod function can modify this meta-data
Pyretic compared to OpenFlow
--open flow rules do not permit the expression of simple conjunctions such as and, and or, or negations like not
--Pyretic allows the expressions of policies in terms of these predicates
-OpenFlow, policies are bit patterns. (match statements for which matching packets are subject to a particular action), difficult to reason about
--Pyretic, policies are functions that map packets to other packets
--In open flow, packets either match on a rule, or they simply fall through to the next rule. So, or, not, Ect can be tough to reason about
--Pyretic match function outputs either the packet or nothing, depending on whether the predicate is satisfied
Pyretic enables the notion of both sequential and parallel composition
sequential composition: The double greater than sign
-example is match all packets for a particular destination IP address and send them or forward them out a particular output port
Parallel composition: plus operator
-- Apply a different set of policies that match on a different source IP address. And output the packets on a different output port
--Pyretic allows an operator, to construct queries, which allow the program to see packet streams.
--For example, the packets query allows the operator to see packets, arriving at a particular switch with a particular source MAC address. The one parameter here indicates that we only want to see the first packet that arrives with a unique source MAC address and switch. We can then register callbacks for these packet streams. That are invoked to handle each new packet that arrives for that query.
Dynamic Policies in Pyretic
Dynamic policies are policies who's forwarding behavior can change
represented as a time series of static policies
self dot policy: current value of the policy at any time
In pyretic every first packet with a new source mac address, at the switch is read by a query. The policy is updated with a new predicate, everytime a new mapping between a MAC address and an output port is learnt.
In the assignment we created a dynamic firewall policy, register a callback to check the rules and sequentially compose your firewall policy with a learning switch, thus provided as part of the pyretic distribution.