This popped up today when I needed to add an NM-4T to our Cisco 3620, which is set to the highest OSPF priority so that it normally becomes the DR, and our 2611 becomes the BDR. Power cycling to add the NM-4T made this swap, of course, so right now the 2611 is the DR and the 3620 is the BDR. Normal enough...
If I remember right it doesn't work like that. When you have a DR and a BDR, and you reboot the DR, yes the BDR becomes the DR. But then when the "old" DR comes back up it does not become BDR. You have to reboot your other box (the original BDR , now DR) for this too happen. In other words, the cluenote I remember is "2 reboots are necessary for a re-election of OSPF DR/BDR". This may be Ciscocentric, I don't remember.
However, both the ARCs on the subnet suddenly stopped advertising their IP pools, and stopped listening to advertisements from the other routers... basically quit exchange OSPF routes entirely. Naturally, our dialup customers weren't real happy about this. Rebooting the ARCs got things back into shape - a bit drastic of a solution.
I wonder if somehow the ospf database got fscked. I think OSPF does a "flood" every 30 minutes or so, and rebooting the 3com box would have forced at least an initial flood.
I may have narrowed this down a bit more this time though.
Cranking out a listing of OSPF neighbors indicates that maybe the ARC is having trouble talking to the new DR at all.
Here is the current DR (206.240.130.3):
fra2>show ip ospf nei
Neighbor ID Pri State Dead Time Address Interface fra-ts1.dcr.net 0 FULL/DROTHER 00:00:35 206.240.130.12 Ethernet0/0 fra1-e0-0.dcr.n 50 FULL/BDR 00:00:31 206.240.130.1 Ethernet0/0 dusty.dcr.net 0 FULL/DROTHER 00:00:32 206.240.130.7 Ethernet0/0 fra3-e0.dcr.net 20 FULL/DROTHER 00:00:33 206.240.130.4 Ethernet0/0 fra-ts2.dcr.net 0 EXSTART/DROTHER 00:00:32 206.240.130.14 Ethernet0/0 andrew.dcr.net 0 FULL/DROTHER 00:00:35 206.240.130.2 Ethernet0/0
Yes, it would appear that fra-ts2 is not establishing adjacency with the DR.......
The BDR (206.240.130.1):
fra1>show ip ospf nei
Neighbor ID Pri State Dead Time Address Interface fra-ts1.dcr.net 0 FULL/DROTHER 00:00:33 206.240.130.12 Ethernet0/0 fra2-e0-0.dcr.n 40 FULL/DR 00:00:38 206.240.130.3 Ethernet0/0 fra3-e0.dcr.net 20 FULL/DROTHER 00:00:32 206.240.130.4 Ethernet0/0 fra-ts2.dcr.net 0 FULL/DROTHER 00:00:30 206.240.130.14 Ethernet0/0 andrew.dcr.net 0 FULL/DROTHER 00:00:34 206.240.130.2 Ethernet0/0 dusty.dcr.net 0 FULL/DROTHER 00:00:30 206.240.130.7 Ethernet0/0 office1.dcr.net 15 FULL/ - 00:00:32 199.77.100.18 Serial1/0 law1-e0.dcr.net 20 FULL/ - 00:00:39 199.77.100.26 Serial1/1
fra-ts1 (206.240.130.12) is an ARC that was sick and was rebooted to fix the problem:
fra-ts1> list ospf nei OSPF NEIGHBORS IfIpAddr/IfInd RouterId Area Prior. State Event QLen Status 206.240.130.1 206.240.130.1 TRANSIT 50 Full/BDR 6 0 dynamic 206.240.130.2 128.167.1.69 TRANSIT 0 TwoWay 2 0 dynamic 206.240.130.3 206.240.130.3 TRANSIT 40 Full/DR 6 0 dynamic 206.240.130.4 206.240.130.4 TRANSIT 20 TwoWay 4 0 dynamic 206.240.130.7 206.240.130.7 TRANSIT 0 TwoWay 2 0 dynamic 206.240.130.14 206.240.130.14 TRANSIT 0 TwoWay 4 0 dynamic
fra-ts2 (206.240.130.14) is an ARC that was and still is sick right now (it won't advertise its IP pool, and its routing table has only default, itself, loopback, and the LAN). Fortunately it has no live modems at the moment, so I can leave it this way for testing:
fra-ts2> list ospf nei OSPF NEIGHBORS IfIpAddr/IfInd RouterId Area Prior. State Event QLen Status 206.240.130.1 206.240.130.1 TRANSIT 50 Full/BDR 6 0 dynamic 206.240.130.2 128.167.1.69 TRANSIT 0 TwoWay 2 0 dynamic 206.240.130.3 206.240.130.3 TRANSIT 40 ExStart 3 0 dynamic 206.240.130.4 206.240.130.4 TRANSIT 20 TwoWay 2 0 dynamic 206.240.130.7 206.240.130.7 TRANSIT 0 TwoWay 2 0 dynamic 206.240.130.12 206.240.130.12 TRANSIT 0 TwoWay 2 0 dynamic
Basically the newly appointed DR is stuck in "exstart" state with the sick ARC. Any pointers as to which debug flags I should turn on to figure out why? I'm still not quite to the OSPF Guru state yet.
You can debug ospf adjacency information on the Cisco. The last time I had adjacency problems it was a bad PtP t1 circuit. check for any wierd ping/packet loss between fra-ts2 and the DR.
Mike Andrews (MA12) * mandrews@dcr.net * http://www.bit0.com/ VP, sysadmin, & network guy, Digital Crescent Inc, Frankfort KY Internet services for Frankfort, Lawrenceburg, Owenton, & Shelbyville "With sufficient thrust, pigs fly just fine." -- RFC 1925
- To unsubscribe to usr-tc, send an email to "majordomo@xmission.com" with "unsubscribe usr-tc" in the body of the message. For information on digests or retrieving files and old messages send "help" to the same address. Do not use quotes in your message.
----------------------------------------------------- Brian Feeny (BF304) signal@shreve.net 318-222-2638 x 109 http://www.shreve.net/~signal Network Administrator ShreveNet Inc. (ASN 11881) - To unsubscribe to usr-tc, send an email to "majordomo@xmission.com" with "unsubscribe usr-tc" in the body of the message. For information on digests or retrieving files and old messages send "help" to the same address. Do not use quotes in your message.