Understanding Routing and Switch Technology, Configuration, Troubleshooting, Setup and Design.
Work in progress….
vEdge Status:
OMP peerings are formed between the system-IP of the two devices, and the protocol is responsible for the advertising service side prefixes and associated VPNs, data plane security parameters, overlay routing policy and transport network location mappings. As you can see from this list OMP does a lot more than your traditional routing protocol. Link
SITE-C_ID500_INET(config-interface-ge0/1)# commit
Aborted: ‘system is-vmanaged’: This device is being managed by the vManage. Configuration through the CLI is not allowed.
On the below document, It shows that we can still configure the vEdges even if it is attached to vManage configuration template.
The document provide two method which will be explain below.
1. Changing mode on the vManage to CLI mode in order to release the configuration from vManage.
– Go to Cisco vManage Dashboard —> Devices—> Select your device —-> Change Mode —>CLI MODE
Let configure the LAN interface…
Now.. What will happen if we enable the mode to template based ?
Final Verdict:
After switching to CLI mode the functionality of the device remains stable. All configuration are in place and most especially the control/data connection are stable. The issue is after attaching the device to it’s previous template the LAN or VPN 10 that we have configure is missing or gone since it not included to the template.
So I think if in production if we are using Template based it is best to add the parameters directly to it’s template or just create a custom template. The risk of leaving the device in “CLI mode” is if someone put the device to template based.
2. Second Way-Use of viptela_internal with the password
To get around this you’ll need to call support because no one will give you the password needed. But here are the commands that will let you over ride the vManage attached template.
https://www.thenetworkdna.com/2020/06/cisco-viptela-sdwan-configure-vedge-via.html
Template offers some significant advantages, templates general and reusable they can simplify the configuration of a number of devices.
Device templates define a devices complete operation configuration.
How to deploy template – Via cli and Feature templates.
– Cli templates – Basically a configuration for device presented as a template and pushed to the device.
– Within CLI we can also create variable this is useful when we need to apply same templates to multiple devices but there are some parameters which will be unique across the devices such as hostname, system IP etc.
Let’s start creating CLI Template….
I. CLI TEMPLATE
Assigning template to vEdge devices
Pass on the values for each device if there are hundreds of devices this could also become quite cumbersome so with that we can use CSV upload.
Next, Upload the CSV file.
II. FEATURE TEMPLATE
Issue:
Final Verdict:
Feature template is more like fill in the blanks, You need to specify the feature you want to enable or use. CLI template is faster than the feature based when it comes to setting up the basic configuration. We need to consider also the global, device specific and default option when setting up using Feature-based while for CLI we may need to review all the setup of configuration, line-by-line to make sure that we specify the correct config and also we create a variable for unique parameters. Before the actual deployment both methods required to update the parameters which are unique on each device, we accomplished this by uploading the CSV file although we need to ensure that there is no duplicate information which can cause errors. One thing I noticed is that if multiple devices are involved we need confirmation before deploying the template for a single device no notification to confirm this.
Reference(s):
Check the following:
show control connections-history
show control local-properties
show certificate installed
Check the reachability
for some reason vEdge having issue installing the certificate, Able to see that control connection is up towards vBond and vmanage but DTLS connection failed when connecting to vSmart. When I issue the command “show control local-properties” able to see that cert is not fully installer which we can validate also with this command ” show certificate installed”.
I’m able to resolve the issue by repeating the certificate installation.
On this Lab I’m going to add another vEdge so that I can start working on TLOC Extension.
Let’s start….
I. INItial Configuration
vpn 0
int G0/1
no shut
ip add 60.1.3.1/24
tunnel-interface
ncapsulation ipsec
color mpls restrict
ip route 0.0.0.0/0 60.1.3.254
system
host-name vEdge-MPLS-SA200
system-ip 1.1.200.2
site-id 200
organization-name “xxxxxx”
vbond 200.1.1.3
vEdge-MPLS-SA200# show control connections-history
vEdge-MPLS-SA200#
A: Could be due to invalid device in which the new vEdge that we enable is not in the allowed “WAN edge List”
II. Upload the ROOTCA.pem from vManage
Basically the TLOC-extension feature is designed to overcome this problem by extending the WAN transports to both SD-WAN routers without requiring direct attachment to both service provider clouds.
Type of Tloc-Extension? L2 and L3 Tloc-extension.
https://www.networkacademy.io/ccie-enterprise/sdwan/tloc-extension
https://ccieme.wordpress.com/category/networking/sd-wan/tloc-extensions/
CONFIGURATION:
a. SITE-C_ID500_INET
vpn 0
int ge0/2
ip address 192.168.20.1/30
no shutdown
tloc-extension ge0/0
int ge0/3
ip address 192.168.20.5/30
no shutdown
tunnel-interface
encapsulation ipsec
color mpls restrict
no control-connections
exit
ip route 0.0.0.0/0 192.168.20.6
b. SITE-C_ID500_MPLS
vpn 0
int ge0/3
ip address 192.168.20.6/30
no shutdown
tloc-extension ge0/0
int ge0/2
ip address 192.168.20.2/30
no shutdown
tunnel-interface
encapsulation ipsec
color biz-internet restrict
no control-connections
exit
ip route 0.0.0.0/0 192.168.20.1
VERIFICATION:
BFD sessions are down… In this instance we need to utilize the Network address Translation (NAT).
Direct Traffic to Exit to the Internet Based Only on IP Prefix
SITE-C_ID500_INET# conf t
Entering configuration mode terminal
SITE-C_ID500_INET(config)# vpn 0
SITE-C_ID500_INET(config-vpn-0)# int ge0/0
SITE-C_ID500_INET(config-interface-ge0/0)# nat
SITE-C_ID500_INET(config-nat)# commit
2. Adding route on MPLS network
Now let verify the traffic flow..
by default each color can freely connect to different color (e.g. mpls to biz-internet vise versa).
Solution: Policy related to Color Category and Carrier Information.
Verifying the status on GUI.
Configuration:
vEdge-INET-SA200# conf t
Entering configuration mode terminal
vEdge-INET-SA200(config)# vpn 0
vEdge-INET-SA200(config-vpn-0)# interface ge0/0
vEdge-INET-SA200(config-interface-ge0/0)# tunnel-interface
vEdge-INET-SA200(config-tunnel-interface)# color biz-internet restrict
vEdge-INET-SA200(config-vpn-0)# interface ge0/1
vEdge-INET-SA200(config-interface-ge0/1)# tunnel-interface
vEdge-INET-SA200(config-tunnel-interface)# color mpls restrict
vEdge-INET-SA200(config-tunnel-interface)# commit
Result:
OMP graceful restart allows OMP peers to continue operating if one of the peers becomes unavailable for some reason. If a vSmart controller becomes unavailable, its peer vEdge router continues to forward traffic, using the last-known good routing information received from the vSmart controller. Similarly, if a vEdge router becomes unavailable, its peer vSmart controller continues to use the last-known good routing information that it received from that vEdge router.
OMP graceful restart is enabled by default on vSmart controllers and vEdge routers. The default graceful restart time is 43,200 seconds (12 hours).
The graceful restart timer is set up independently on each OMP peer; that is, it is set up separately on each vEdge router and vSmart controller. To illustrate what this means, let’s consider a vSmart controller that uses a graceful restart time of 300 seconds, or 5 minutes, and a vEdge router that is configured with a timer of 600 seconds (10 minutes). Here, the vSmart controller retains the OMP routes learned from that router for 10 minutes—the graceful restart timer value that is configured on the router and that the router has sent to the vSmart controller during the setup of the OMP session. The vEdge router retains the routes it learns from the vSmart controller for 5 minutes, which is the default graceful restart time value that is used on the vSmart controller and that the controller sent to the router, also during the setup of the OMP session.
vEdge-INET-SA200# show omp peers 1.1.1.2 detail
peer 1.1.1.2
type vsmart
domain-id 1
site-id 10
overlay-id 1
state up
version 1
legit yes
control-up yes
staging no
upcount 2
downcount 1
last-uptime 2021-07-11T15:31:43+00:00
last-downtime 2021-07-11T15:27:25+00:00
uptime 0:00:06:21
hold-time 60
graceful-restart supported
graceful-restart-interval 43200
refresh supported
hello-sent 35
hello-received 29
handshake-sent 2
handshake-received 2
alert-sent 1
alert-received 0
inform-sent 10
inform-received 10
update-sent 16
update-received 8
policy-sent
policy-received
total-packets-sent 64
total-packets-received 50
routes-received 2
routes-installed 2
routes-sent 2
routes-received-ipv6 0
routes-installed-ipv6 0
routes-sent-ipv6 0
tlocs-received 2
tlocs-installed 2
tlocs-sent 2
services-received 0
services-installed 0
services-sent 2
services-received-ipv6 0
services-installed-ipv6 0
services-sent-ipv6 0
mcast-routes-received 0
mcast-routes-installed 0
mcast-routes-sent 0
vEdge-INET-SA200# show omp peers
R -> routes received
I -> routes installed
S -> routes sent
DOMAIN OVERLAY SITE
1.1.1.2 vsmart 1 1 10 up 0:00:08:42 2/2/2
sh omp route 10.0.10.0/24
sh ip route 10.0.10.0/24
sh omp route 10.0.20.0/24
sh ip route 10.0.20.0/24
vEdge-INET-SA200# show ip routes 10.0.10.0/24 detail
Codes Proto-sub-type:
IA -> ospf-intra-area, IE -> ospf-inter-area,
E1 -> ospf-external1, E2 -> ospf-external2,
N1 -> ospf-nssa-external1, N2 -> ospf-nssa-external2,
e -> bgp-external, i -> bgp-internal
Codes Status flags:
F -> fib, S -> selected, I -> inactive,
B -> blackhole, R -> recursive
“”——————————————–
proto connected
distance 0
metric 0
uptime 0:00:07:14
nexthop-ifname ge0/2
status F,S
vEdge-INET-SA200# sh omp route 10.0.10.0/24
RECEIVED FROM:
peer 0.0.0.0
path-id 66
label 1004
status C,Red,R
loss-reason not set
lost-to-peer not set
lost-to-path-id not set
Attributes:
originator 1.1.200.1
type installed
tloc 1.1.200.1, mpls, ipsec
ultimate-tloc not set
domain-id not set
overlay-id 1
site-id 200
preference not set
tag not set
origin-proto connected
origin-metric 0
as-path not set
unknown-attr-len not set
RECEIVED FROM:
peer 0.0.0.0
path-id 68
label 1004
status C,Red,R
loss-reason not set
lost-to-peer not set
lost-to-path-id not set
Attributes:
originator 1.1.200.1
type installed
tloc 1.1.200.1, biz-internet, ipsec
ultimate-tloc not set
domain-id not set
overlay-id 1
site-id 200
preference not set
tag not set
origin-proto connected
origin-metric 0
as-path not set
unknown-attr-len not set
vEdge-INET-SA200# sh ip route 10.0.10.0/24.
———————————^
syntax error: unknown element
vEdge-INET-SA200# sh ip route 10.0.10.0/24
Codes Proto-sub-type:
IA -> ospf-intra-area, IE -> ospf-inter-area,
E1 -> ospf-external1, E2 -> ospf-external2,
N1 -> ospf-nssa-external1, N2 -> ospf-nssa-external2,
e -> bgp-external, i -> bgp-internal
Codes Status flags:
F -> fib, S -> selected, I -> inactive,
B -> blackhole, R -> recursive
PROTOCOL NEXTHOP NEXTHOP NEXTHOP
1 10.0.10.0/24 connected – ge0/2 – – – – – F,S
vEdge-INET-SA200# show omp peers 1.1.1.2
R -> routes received
I -> routes installed
S -> routes sent
DOMAIN OVERLAY SITE
1.1.1.2 vsmart 1 1 10 init-in-gr 2/2/0
vEdge-INET-SA200# sh omp route 10.0.10.0/24
RECEIVED FROM:
peer 0.0.0.0
path-id 66
label 1004
status C,Red,R
loss-reason not set
lost-to-peer not set
lost-to-path-id not set
Attributes:
originator 1.1.200.1
type installed
tloc 1.1.200.1, mpls, ipsec
ultimate-tloc not set
domain-id not set
overlay-id 1
site-id 200
preference not set
tag not set
origin-proto connected
origin-metric 0
as-path not set
unknown-attr-len not set
RECEIVED FROM:
peer 0.0.0.0
path-id 68
label 1004
status C,Red,R
loss-reason not set
lost-to-peer not set
lost-to-path-id not set
Attributes:
originator 1.1.200.1
type installed
tloc 1.1.200.1, biz-internet, ipsec
ultimate-tloc not set
domain-id not set
overlay-id 1
site-id 200
preference not set
tag not set
origin-proto connected
origin-metric 0
as-path not set
unknown-attr-len not set
vEdge-INET-SA200#
vEdge-INET-SA200# sh ip route 10.0.10.0/24
Codes Proto-sub-type:
IA -> ospf-intra-area, IE -> ospf-inter-area,
E1 -> ospf-external1, E2 -> ospf-external2,
N1 -> ospf-nssa-external1, N2 -> ospf-nssa-external2,
e -> bgp-external, i -> bgp-internal
Codes Status flags:
F -> fib, S -> selected, I -> inactive,
B -> blackhole, R -> recursive
PROTOCOL NEXTHOP NEXTHOP NEXTHOP
1 10.0.10.0/24 connected – ge0/2 – – – – – F,S
vEdge-INET-SA200# show bfd sessions
SOURCE TLOC REMOTE TLOC DST PUBLIC DST PUBLIC DETECT TX
1.1.201.1 300 up mpls mpls 60.1.1.1 60.1.2.1 12386 ipsec 7 1000 0:00:09:19 0
1.1.201.1 300 up mpls biz-internet 60.1.1.1 200.1.20.1 12386 ipsec 7 1000 0:00:09:19 0
1.1.201.1 300 up biz-internet mpls 200.1.10.1 60.1.2.1 12386 ipsec 7 1000 0:00:11:39 1
1.1.201.1 300 up biz-internet biz-internet 200.1.10.1 200.1.20.1 12386 ipsec 7 1000 0:00:11:39 1
vEdge-INET-SA200# show ipsec outbound-connections
SOURCE SOURCE DEST DEST REMOTE REMOTE AUTHENTICATION NEGOTIATED
60.1.1.1 12366 60.1.2.1 12386 258 1442 1.1.201.1 mpls AH_SHA1_HMAC b876 AES-GCM-256 8
60.1.1.1 12366 200.1.20.1 12386 259 1442 1.1.201.1 biz-internet AH_SHA1_HMAC eb3f AES-GCM-256 8
60.1.1.1 12386 60.1.2.1 12386 258 1441 1.1.201.1 mpls AH_SHA1_HMAC b876 AES-GCM-256 8
60.1.1.1 12386 200.1.20.1 12386 259 1441 1.1.201.1 biz-internet AH_SHA1_HMAC eb3f AES-GCM-256 8
200.1.10.1 12346 60.1.2.1 12386 258 1441 1.1.201.1 mpls AH_SHA1_HMAC b876 AES-GCM-256 8
200.1.10.1 12346 200.1.20.1 12386 259 1441 1.1.201.1 biz-internet AH_SHA1_HMAC eb3f AES-GCM-256 8
vEdge-INET-SA200# ping vpn 1 10.0.20.250 rapid
Defaulting count to 5
Ping in VPN 1
!!!!!
— 10.0.20.250 statistics —
5 packets transmitted, 5 received, 0% packet loss
** Result: vEdge can still reach the remote vedge. BFD/IPSEC is up and routes still present on vEdge RIB.
vEdge-INET-SA200(config)# omp
vEdge-INET-SA200(config-omp)# timers graceful-restart-timer 1
vEdge-INET-SA200(config-timers)# commit
Commit complete.
omp
no shutdown
graceful-restart
timers
holdtime 3
graceful-restart-timer 1
eor-timer 1
exit
vEdge-INET-SA200# show omp peers 1.1.1.2 detail <—- Peer to vsmart
peer 1.1.1.2
type vsmart
domain-id 1
site-id 10
overlay-id 1
state init-in-gr
version 1
legit yes
control-up no
staging no
upcount 1
downcount 1
last-uptime 2021-07-11T16:04:07+00:00
last-downtime 2021-07-11T16:10:07+00:00
downtime 0:00:02:28
hold-time 3
graceful-restart in-progress
graceful-restart-interval 43200
refresh supported
hello-sent 21
hello-received 16
handshake-sent 1
handshake-received 1
alert-sent 1
alert-received 0
inform-sent 7
inform-received 7
update-sent 6
update-received 5
policy-sent
policy-received
total-packets-sent 36
total-packets-received 29
routes-received 2
routes-installed 2
routes-sent 0
routes-received-ipv6 0
routes-installed-ipv6 0
routes-sent-ipv6 0
tlocs-received 2
tlocs-installed 2
tlocs-sent 0
services-received 0
services-installed 0
services-sent 0
services-received-ipv6 0
services-installed-ipv6 0
services-sent-ipv6 0
mcast-routes-received 0
mcast-routes-installed 0
mcast-routes-sent 0
vsmart# show omp peers 1.1.200.1 detail <—- Peer to vedge
peer 1.1.200.1
type vedge
domain-id 1
site-id 200
overlay-id 1
state up
version 1
legit yes
control-up yes
staging no
upcount 7
downcount 6
last-uptime 2021-07-11T16:04:09+00:00
last-downtime 2021-07-11T15:50:47+00:00
uptime 0:00:03:08
hold-time 60
graceful-restart supported
graceful-restart-interval 1 <—–
refresh supported
hello-sent 3227
hello-received 3222
handshake-sent 7
handshake-received 7
alert-sent 5
alert-received 1
inform-sent 29
inform-received 33
update-sent 39
update-received 59
policy-sent
policy-received
total-packets-sent 3308
total-packets-received 3322
routes-received 2
routes-installed 0
routes-sent 2
routes-received-ipv6 0
routes-installed-ipv6 0
routes-sent-ipv6 0
tlocs-received 2
tlocs-installed 2
tlocs-sent 2
services-received 2
services-installed 2
services-sent 0
services-received-ipv6 0
services-installed-ipv6 0
services-sent-ipv6 0
mcast-routes-received 0
mcast-routes-installed 0
mcast-routes-sent 0
vsmart# sh omp route 10.0.10.0/24 <– Routes from vEdge vEdge-INET-SA200
% No entries found.
vsmart# sh omp route 10.0.20.0/24
RECEIVED FROM:
peer 1.1.201.1
path-id 66
label 1002
status C,R,S
loss-reason not set
lost-to-peer not set
lost-to-path-id not set
Attributes:
originator 1.1.201.1
type installed
tloc 1.1.201.1, mpls, ipsec
ultimate-tloc not set
domain-id not set
overlay-id 1
site-id 300
preference not set
tag not set
origin-proto connected
origin-metric 0
as-path not set
unknown-attr-len not set
RECEIVED FROM:
peer 1.1.201.1
path-id 68
label 1002
status C,R,S
loss-reason not set
lost-to-peer not set
lost-to-path-id not set
Attributes:
originator 1.1.201.1
type installed
tloc 1.1.201.1, biz-internet, ipsec
ultimate-tloc not set
domain-id not set
overlay-id 1
site-id 300
preference not set
tag not set
origin-proto connected
origin-metric 0
as-path not set
unknown-attr-len not set
vEdge-INET-SA200# sh omp route | t
Code:
C -> chosen
I -> installed
Red -> redistributed
Rej -> rejected
L -> looped
R -> resolved
S -> stale
Ext -> extranet
Inv -> invalid
Stg -> staged
U -> TLOC unresolved
PATH ATTRIBUTE
1 10.0.10.0/24 0.0.0.0 66 1002 C,Red,R installed 1.1.200.1 mpls ipsec –
0.0.0.0 68 1002 C,Red,R installed 1.1.200.1 biz-internet ipsec –
1 10.0.20.0/24 1.1.1.2 15 1002 C,I,R,S installed 1.1.201.1 biz-internet ipsec –
1.1.1.2 16 1002 C,I,R,S installed 1.1.201.1 mpls ipsec –
:: WHAT IF WE CHANGE THE OMP TIMERS ON VSMART
vsmart(config)# omp
vsmart(config-omp)# timers graceful-restart-timer 1
vsmart(config-timers)# commit
vEdge-INET-SA200# show omp peers 1.1.1.2 detail
peer 1.1.1.2
type vsmart
domain-id 1
site-id 10
overlay-id 1
state up
version 1
legit yes
control-up yes
staging no
upcount 2
downcount 1
last-uptime 2021-07-11T16:19:00+00:00
last-downtime 2021-07-11T16:10:07+00:00
uptime 0:00:00:21
hold-time 60
graceful-restart supported
graceful-restart-interval 1
refresh supported
hello-sent 24
hello-received 18
handshake-sent 2
handshake-received 2
alert-sent 1
alert-received 0
inform-sent 14
inform-received 10
update-sent 12
update-received 9
policy-sent
policy-received
total-packets-sent 53
total-packets-received 39
routes-received 2
routes-installed 2
routes-sent 2
routes-received-ipv6 0
routes-installed-ipv6 0
routes-sent-ipv6 0
tlocs-received 2
tlocs-installed 2
tlocs-sent 2
services-received 0
services-installed 0
services-sent 2
services-received-ipv6 0
services-installed-ipv6 0
services-sent-ipv6 0
mcast-routes-received 0
mcast-routes-installed 0
mcast-routes-sent 0
vEdge-INET-SA200# ping vpn 1 10.0.20.250 count 10000 rapid
Ping in VPN 1
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!……………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………..^C
— 10.0.20.250 statistics —
1580 packets transmitted, 177 received, 89% packet loss
vEdge-INET-SA200# show omp routes | t
Code:
C -> chosen
I -> installed
Red -> redistributed
Rej -> rejected
L -> looped
R -> resolved
S -> stale
Ext -> extranet
Inv -> invalid
Stg -> staged
U -> TLOC unresolved
PATH ATTRIBUTE
1 10.0.10.0/24 0.0.0.0 66 1002 C,Red,R installed 1.1.200.1 mpls ipsec –
0.0.0.0 68 1002 C,Red,R installed 1.1.200.1 biz-internet ipsec –
** RESULT: After we detected that the vSmart peer went down the timers kicks and after it reaches the graceful-restart value (1sec) omp cache expires.
Two things in SDWAN template are Device and Feature template.
Create a base template under Configuration > Templates
A TLOC is a Transport Locator, which is analogous to a LISP RLOC, or Routing Locator. Both abstract the router itself in favor of location-based mapping. RLOCs represent the routing location and not an endpoint. This is different than traditional routing, where a router would be identified by its physical address on each interface. From “https://carpe-dmvpn.com/2019/12/14/tlocs-cisco-sd-wan/”
Scaling tunnels is accomplished easily because the solution takes care of tunnel establishment as a by-product of policy and WAN Edge router TLOC advertisements.
it means that TLOCs are the next-hops that matter, and not the particular data plane tunnel used to get there. By default, as long as data plane tunnels are up and equally weighted, each will be used and load balanced on a per-flow basis. Implementing more granular control over particular paths and tunnels requires a centralized policy.
IPsec Inbound Connection:
IPsec Outbound Connection:
Let examine more…
Determine the actual next-hop of OMP Routers/TLOC.
More About TLOCs…..
Now, Let learn more about color.
A L L A B O U T N E T W O R K 