This is a OSPF over IPSec (route-based) over BGP design.
The 2 SRX345 build chassis cluster as HQ device and use local interface ge-0/0/3 and ge-5/0/3 to connect with the SRX300 HUB. BGP is running between the HQ and SRX300 hub to learn the loopback interface routes of these spoke devices.
The HQ configures the spoke device loopback address as the IPSEC remote gateway and choose lo0 interface as IKE gateway external interface.
All the spoke devices SRX300 also connect to this SRX300 HUB device and use static route reaching to the HQ Loopback.
root@SRX345-HQ-node0# run show version node0: -------------------------------------------------------------------------- Hostname: SRX345-HQ-node0 Model: srx345 Family: junos-es Junos: 24.4R2-S3.5 root@SRX345-HQ-node0# run show configuration | display set set version 24.4R2-S3.5 set groups node0 system host-name SRX345-HQ-node0 set groups node0 system backup-router 10.219.33.129 set groups node0 system backup-router destination 0.0.0.0/0 set groups node0 interfaces fxp0 unit 0 family inet address 10.219.33.174/26 set groups node0 routing-options static route 0.0.0.0/0 next-hop 10.219.33.129 set groups node1 system host-name SRX345-HQ-node1 set groups node1 system backup-router 10.219.90.1 set groups node1 system backup-router destination 0.0.0.0/0 set groups node1 interfaces fxp0 unit 0 family inet address 10.219.90.13/26 set groups node1 routing-options static route 0.0.0.0/0 next-hop 10.219.90.1 set apply-groups "${node}" set system root-authentication encrypted-password "$6$d1xSgkhg$Br94DIKKHSDvMHGsnD8j.R6//hXIunt/4HR3VQqNWexCFI1CjBMOXKTNPM/vk5.29nQYt.DIOb/a3Ejzq77fe." set system services ssh root-login allow set system services ssh sftp-server set chassis cluster reth-count 10 set chassis cluster redundancy-group 0 node 0 priority 200 set chassis cluster redundancy-group 0 node 1 priority 100 set chassis cluster redundancy-group 1 node 0 priority 200 set chassis cluster redundancy-group 1 node 1 priority 100 set security ike proposal ike_proposal authentication-method pre-shared-keys set security ike proposal ike_proposal dh-group group14 set security ike proposal ike_proposal authentication-algorithm sha-256 set security ike proposal ike_proposal encryption-algorithm aes-256-cbc set security ike proposal ike_proposal lifetime-seconds 28800 set security ike policy ike_policy mode main set security ike policy ike_policy proposals ike_proposal set security ike policy ike_policy pre-shared-key ascii-text "$9$O6HIIhyM87s2alK2aZU.mO1R" set security ike gateway gw_to_spoke3 ike-policy ike_policy set security ike gateway gw_to_spoke3 address 10.1.1.3 set security ike gateway gw_to_spoke3 dead-peer-detection interval 10 set security ike gateway gw_to_spoke3 dead-peer-detection threshold 5 set security ike gateway gw_to_spoke3 local-identity inet 10.99.103.254 set security ike gateway gw_to_spoke3 remote-identity inet 10.1.1.3 set security ike gateway gw_to_spoke3 external-interface lo0 set security ike gateway gw_to_spoke4 ike-policy ike_policy set security ike gateway gw_to_spoke4 address 10.1.1.4 set security ike gateway gw_to_spoke4 dead-peer-detection interval 10 set security ike gateway gw_to_spoke4 dead-peer-detection threshold 5 set security ike gateway gw_to_spoke4 local-identity inet 10.99.103.254 set security ike gateway gw_to_spoke4 remote-identity inet 10.1.1.4 set security ike gateway gw_to_spoke4 external-interface lo0 set security ike gateway gw_to_spoke5 ike-policy ike_policy set security ike gateway gw_to_spoke5 address 10.1.1.5 set security ike gateway gw_to_spoke5 dead-peer-detection interval 10 set security ike gateway gw_to_spoke5 dead-peer-detection threshold 5 set security ike gateway gw_to_spoke5 local-identity inet 10.99.103.254 set security ike gateway gw_to_spoke5 remote-identity inet 10.1.1.5 set security ike gateway gw_to_spoke5 external-interface lo0 set security ike gateway gw_to_spoke6 ike-policy ike_policy set security ike gateway gw_to_spoke6 address 10.1.1.6 set security ike gateway gw_to_spoke6 dead-peer-detection interval 10 set security ike gateway gw_to_spoke6 dead-peer-detection threshold 5 set security ike gateway gw_to_spoke6 local-identity inet 10.99.103.254 set security ike gateway gw_to_spoke6 remote-identity inet 10.1.1.6 set security ike gateway gw_to_spoke6 external-interface lo0 set security ipsec proposal ipsec_proposal protocol esp set security ipsec proposal ipsec_proposal authentication-algorithm hmac-sha-256-128 set security ipsec proposal ipsec_proposal encryption-algorithm aes-256-cbc set security ipsec proposal ipsec_proposal lifetime-seconds 3600 set security ipsec policy ipsec_policy perfect-forward-secrecy keys group14 set security ipsec policy ipsec_policy proposals ipsec_proposal set security ipsec vpn vpn_to_spoke3 bind-interface st0.0 set security ipsec vpn vpn_to_spoke3 ike gateway gw_to_spoke3 set security ipsec vpn vpn_to_spoke3 ike no-anti-replay set security ipsec vpn vpn_to_spoke3 ike ipsec-policy ipsec_policy set security ipsec vpn vpn_to_spoke3 establish-tunnels immediately set security ipsec vpn vpn_to_spoke4 bind-interface st0.0 set security ipsec vpn vpn_to_spoke4 ike gateway gw_to_spoke4 set security ipsec vpn vpn_to_spoke4 ike no-anti-replay set security ipsec vpn vpn_to_spoke4 ike ipsec-policy ipsec_policy set security ipsec vpn vpn_to_spoke4 establish-tunnels immediately set security ipsec vpn vpn_to_spoke5 bind-interface st0.0 set security ipsec vpn vpn_to_spoke5 ike gateway gw_to_spoke5 set security ipsec vpn vpn_to_spoke5 ike no-anti-replay set security ipsec vpn vpn_to_spoke5 ike ipsec-policy ipsec_policy set security ipsec vpn vpn_to_spoke5 establish-tunnels immediately set security ipsec vpn vpn_to_spoke6 bind-interface st0.0 set security ipsec vpn vpn_to_spoke6 ike gateway gw_to_spoke6 set security ipsec vpn vpn_to_spoke6 ike no-anti-replay set security ipsec vpn vpn_to_spoke6 ike ipsec-policy ipsec_policy set security ipsec vpn vpn_to_spoke6 establish-tunnels immediately set security flow allow-reverse-ecmp set security flow no-local-favor-ecmp set security policies default-policy permit-all set security zones security-zone atm interfaces ge-0/0/3.0 host-inbound-traffic system-services all set security zones security-zone atm interfaces ge-0/0/3.0 host-inbound-traffic protocols all set security zones security-zone atm interfaces ge-5/0/3.0 host-inbound-traffic system-services all set security zones security-zone atm interfaces ge-5/0/3.0 host-inbound-traffic protocols all set security zones security-zone atm interfaces lo0.0 host-inbound-traffic system-services all set security zones security-zone atm interfaces lo0.0 host-inbound-traffic protocols all set security zones security-zone atm interfaces st0.0 host-inbound-traffic system-services all set security zones security-zone atm interfaces st0.0 host-inbound-traffic protocols all set security zones security-zone core host-inbound-traffic system-services all set security zones security-zone core host-inbound-traffic protocols all set security zones security-zone core interfaces reth0.0 set interfaces ge-0/0/0 gigether-options redundant-parent reth0 set interfaces ge-0/0/3 unit 0 family inet address 10.100.1.1/24 set interfaces ge-5/0/0 gigether-options redundant-parent reth0 set interfaces ge-5/0/3 unit 0 family inet address 10.100.2.1/24 set interfaces fab0 fabric-options member-interfaces ge-0/0/2 set interfaces fab1 fabric-options member-interfaces ge-5/0/2 set interfaces lo0 unit 0 family inet address 10.99.103.254/24 set interfaces reth0 redundant-ether-options redundancy-group 1 set interfaces reth0 unit 0 family inet address 172.16.1.1/24 set interfaces st0 unit 0 multipoint set interfaces st0 unit 0 family inet address 10.63.33.254/24 set forwarding-options hash-key family inet layer-3 set forwarding-options hash-key family inet layer-4 set policy-options policy-statement bgp-export term direct from protocol direct set policy-options policy-statement bgp-export term direct from route-filter 10.99.103.0/24 exact set policy-options policy-statement bgp-export term direct then accept set policy-options policy-statement bgp-export term reject then reject set policy-options policy-statement load-banlance then load-balance per-flow set protocols ospf area 0.0.0.1 interface st0.0 interface-type p2mp set protocols bgp group ebgp type external set protocols bgp group ebgp hold-time 3 set protocols bgp group ebgp export bgp-export set protocols bgp group ebgp peer-as 4780 set protocols bgp group ebgp multipath set protocols bgp group ebgp neighbor 10.100.1.254 set protocols bgp group ebgp neighbor 10.100.2.254 set protocols bgp preference 140 set routing-options autonomous-system 65534 set routing-options nonstop-routing set routing-options forwarding-table export load-banlance {primary:node0}[edit] root@SRX345-HQ-node0# run show chassis cluster status Monitor Failure codes: CS Cold Sync monitoring FL Fabric Connection monitoring GR GRES monitoring HW Hardware monitoring IF Interface monitoring IP IP monitoring LB Loopback monitoring MB Mbuf monitoring NH Nexthop monitoring NP NPC monitoring SP SPU monitoring SM Schedule monitoring CF Config Sync monitoring RE Relinquish monitoring IS IRQ storm Cluster ID: 1 Node Priority Status Preempt Manual Monitor-failures Redundancy group: 0 , Failover count: 3 node0 200 primary no no None node1 100 secondary no no None Redundancy group: 1 , Failover count: 3 node0 200 primary no no None node1 100 secondary no no None root@SRX345-HQ-node0# run show bgp summary Threading mode: BGP I/O Default eBGP mode: advertise - accept, receive - accept Groups: 1 Peers: 2 Down peers: 0 Table Tot Paths Act Paths Suppressed History Damp State Pending inet.0 4 4 0 0 0 0 Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped... 10.100.1.254 4780 15375 15358 0 0 3:53:33 Establ inet.0: 2/2/2/0 10.100.2.254 4780 15370 15353 0 0 3:53:29 Establ inet.0: 2/2/2/0 {primary:node0}[edit] root@SRX345-HQ-node0# run show ospf neighbor Address Interface State ID Pri Dead 10.63.33.5 st0.0 Init 10.1.1.5 128 33 10.63.33.6 st0.0 Init 10.1.1.6 128 39 10.63.33.3 st0.0 Full 10.1.1.3 128 34 10.63.33.4 st0.0 Full 10.1.1.4 128 38 root@SRX345-HQ-node0# run monitor traffic interface st0.0 verbose output suppressed, use <detail> or <extensive> for full protocol decode Address resolution is ON. Use <no-resolve> to avoid any reverse lookup delay. Address resolution timeout is 4s. Listening on st0.0, capture size 96 bytes Reverse lookup for 10.63.33.6 failed (check DNS reachability). Other reverse lookup failures will not be reported. Use <no-resolve> to avoid reverse lookups on IP addresses. 13:19:53.259523 In IP 10.63.33.6 > 224.0.0.5: OSPFv2, Hello, length 56 13:19:53.869204 In IP 10.63.33.3 > 224.0.0.5: OSPFv2, Hello, length 60 13:19:56.162868 In IP 10.63.33.4 > 224.0.0.5: OSPFv2, Hello, length 60 13:19:56.299477 In IP 10.63.33.5 > 224.0.0.5: OSPFv2, Hello, length 56 13:19:56.730160 Out IP truncated-ip - 18 bytes missing! 10.63.33.254 > 10.63.33.4: OSPFv2, Hello, length 72 13:19:59.105935 Out IP truncated-ip - 18 bytes missing! 10.63.33.254 > 10.63.33.3: OSPFv2, Hello, length 7213:19:59.769768 Out IP truncated-ip - 18 bytes missing! 10.63.33.254 > 10.63.33.5: OSPFv2, Hello, length 72 13:20:01.577029 Out IP truncated-ip - 18 bytes missing! 10.63.33.254 > 10.63.33.6: OSPFv2, Hello, length 72---- This indicates the RPD has sent out the OSPF hello packets to spoke 5 and spoke 6 13:20:01.800518 In IP 10.63.33.3 > 224.0.0.5: OSPFv2, Hello, length 60 13:20:03.928990 In IP 10.63.33.5 > 224.0.0.5: OSPFv2, Hello, length 56 13:20:05.820279 In IP 10.63.33.4 > 224.0.0.5: OSPFv2, Hello, length 60 13:20:05.977894 Out IP truncated-ip - 18 bytes missing! 10.63.33.254 > 10.63.33.4: OSPFv2, Hello, length 72 13:20:07.516612 Out IP truncated-ip - 18 bytes missing! 10.63.33.254 > 10.63.33.5: OSPFv2, Hello, length 72 13:20:08.261537 Out IP truncated-ip - 18 bytes missing! 10.63.33.254 > 10.63.33.3: OSPFv2, Hello, length 72 13:20:10.724776 In IP 10.63.33.3 > 224.0.0.5: OSPFv2, Hello, length 60 13:20:10.909086 In IP 10.63.33.6 > 224.0.0.5: OSPFv2, Hello, length 56 13:20:11.293165 Out IP truncated-ip - 18 bytes missing! 10.63.33.254 > 10.63.33.6: OSPFv2, Hello, length 72 13:20:12.089458 In IP 10.63.33.5 > 224.0.0.5: OSPFv2, Hello, length 56 13:20:13.542832 In IP 10.63.33.4 > 224.0.0.5: OSPFv2, Hello, length 60 13:20:15.646026 Out IP truncated-ip - 18 bytes missing! 10.63.33.254 > 10.63.33.4: OSPFv2, Hello, length 72 13:20:16.178190 Out IP truncated-ip - 18 bytes missing! 10.63.33.254 > 10.63.33.5: OSPFv2, Hello, length 72 13:20:17.441490 Out IP truncated-ip - 18 bytes missing! 10.63.33.254 > 10.63.33.3: OSPFv2, Hello, length 72 13:20:19.102660 In IP 10.63.33.3 > 224.0.0.5: OSPFv2, Hello, length 60 13:20:19.679625 In IP 10.63.33.6 > 224.0.0.5: OSPFv2, Hello, length 56 13:20:20.045475 Out IP truncated-ip - 18 bytes missing! 10.63.33.254 > 10.63.33.6: OSPFv2, Hello, length 72 13:20:20.589629 In IP 10.63.33.5 > 224.0.0.5: OSPFv2, Hello, length 56 {primary:node0}[edit] root@SRX345-HQ-node0# run show route protocol bgp inet.0: 18 destinations, 19 routes (18 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 10.1.1.3/32 *[BGP/140] 03:38:54, localpref 100 AS path: 4780 I, validation-state: unverified > to 10.100.1.254 via ge-0/0/3.0 10.1.1.4/32 *[BGP/140] 03:38:54, localpref 100 AS path: 4780 I, validation-state: unverified > to 10.100.1.254 via ge-0/0/3.0 10.1.1.5/32 *[BGP/140] 03:38:50, localpref 100 AS path: 4780 I, validation-state: unverified > to 10.100.2.254 via ge-5/0/3.0 10.1.1.6/32 *[BGP/140] 03:38:50, localpref 100 AS path: 4780 I, validation-state: unverified > to 10.100.2.254 via ge-5/0/3.0 root@SRX300-HUB# run show configuration | display set set version 25.4R1.12 set system host-name SRX300-HUB set system root-authentication encrypted-password "$6$DOwCi9dG$v.iCDC7aG4M3OmU2vpkTbGe.fRnv76KhugosmPbbxfeTUqNhrNUykcTIHuHVz0l2C5o.Z6e..n6oxMlWo51SN0" set system services ssh root-login allow set system services ssh sftp-server set security zones security-zone ebgp host-inbound-traffic system-services all set security zones security-zone ebgp host-inbound-traffic protocols all set security zones security-zone ebgp interfaces ge-0/0/1.0 set security zones security-zone ebgp interfaces ge-0/0/2.0 set security zones security-zone atm host-inbound-traffic system-services all set security zones security-zone atm host-inbound-traffic protocols all set security zones security-zone atm interfaces irb.0 set security policies default-policy permit-all set interfaces ge-0/0/1 unit 0 family inet address 10.100.1.254/24 set interfaces ge-0/0/2 unit 0 family inet address 10.100.2.254/24 set interfaces ge-0/0/3 unit 0 family ethernet-switching interface-mode access set interfaces ge-0/0/3 unit 0 family ethernet-switching vlan members vlan1 set interfaces ge-0/0/4 unit 0 family ethernet-switching interface-mode access set interfaces ge-0/0/4 unit 0 family ethernet-switching vlan members vlan1 set interfaces ge-0/0/5 unit 0 family ethernet-switching interface-mode access set interfaces ge-0/0/5 unit 0 family ethernet-switching vlan members vlan1 set interfaces ge-0/0/6 unit 0 family ethernet-switching interface-mode access set interfaces ge-0/0/6 unit 0 family ethernet-switching vlan members vlan1 set interfaces irb unit 0 family inet address 192.168.1.254/24 set forwarding-options hash-key family inet layer-3 set forwarding-options hash-key family inet layer-4 set policy-options policy-statement bgp-export1 term direct from protocol static set policy-options policy-statement bgp-export1 term direct from route-filter 10.1.1.3/32 exact set policy-options policy-statement bgp-export1 term direct from route-filter 10.1.1.4/32 exact set policy-options policy-statement bgp-export1 term direct then accept set policy-options policy-statement bgp-export1 term reject then reject set policy-options policy-statement bgp-export2 term direct from protocol static set policy-options policy-statement bgp-export2 term direct from route-filter 10.1.1.5/32 exact set policy-options policy-statement bgp-export2 term direct from route-filter 10.1.1.6/32 exact set policy-options policy-statement bgp-export2 term direct then accept set policy-options policy-statement bgp-export2 term reject then reject set policy-options policy-statement load-banlance then load-balance per-flow set vlans vlan1 vlan-id 1 set vlans vlan1 l3-interface irb.0 set protocols bgp group ebgp type external set protocols bgp group ebgp hold-time 3 set protocols bgp group ebgp peer-as 65534 set protocols bgp group ebgp multipath set protocols bgp group ebgp neighbor 10.100.1.1 export bgp-export1 set protocols bgp group ebgp neighbor 10.100.2.1 export bgp-export2 set protocols bgp preference 140 set routing-options autonomous-system 4780 set routing-options static route 10.1.1.3/32 next-hop 192.168.1.3 set routing-options static route 10.1.1.4/32 next-hop 192.168.1.4 set routing-options static route 10.1.1.5/32 next-hop 192.168.1.5 set routing-options static route 10.1.1.6/32 next-hop 192.168.1.6 set routing-options forwarding-table export load-banlance root@SRX300-HUB# run show route advertising-protocol bgp 10.100.1.1 extensive inet.0: 14 destinations, 15 routes (14 active, 0 holddown, 0 hidden) * 10.1.1.3/32 (1 entry, 1 announced) BGP group ebgp type External Nexthop: Self AS path: [4780] I * 10.1.1.4/32 (1 entry, 1 announced) BGP group ebgp type External Nexthop: Self AS path: [4780] I [edit] root@SRX300-HUB# run show route advertising-protocol bgp 10.100.2.1 extensive inet.0: 14 destinations, 15 routes (14 active, 0 holddown, 0 hidden) * 10.1.1.5/32 (1 entry, 1 announced) BGP group ebgp type External Nexthop: Self AS path: [4780] I * 10.1.1.6/32 (1 entry, 1 announced) BGP group ebgp type External Nexthop: Self AS path: [4780] I [edit] root@SRX300-HUB# root@SRX300-Spoke3# run show ospf neighbor Address Interface State ID Pri Dead 10.63.33.254 st0.0 Full 10.99.103.254 128 31 root@SRX300-Spoke3# run show ospf statistics Packet type Total Last 5 seconds Sent Received Sent Received Hello 520 1538 0 1 DbD 8 6 0 0 LSReq 1 2 0 0 LSUpdate 7 18 0 0 LSAck 11 1 0 0 DBDs retransmitted : 2, last 5 seconds : 0 LSAs flooded : 3, last 5 seconds : 0 LSAs flooded high-prio : 2, last 5 seconds : 0 LSAs retransmitted : 0, last 5 seconds : 0 LSAs transmitted to nbr: 4, last 5 seconds : 0 LSAs requested : 2, last 5 seconds : 0 LSAs acknowledged : 13, last 5 seconds : 0 Flood queue depth : 0 Total rexmit entries : 0 db summaries : 0 lsreq entries : 0 Receive errors: None [edit] root@SRX300-Spoke3# root@SRX300-Spoke3# run show configuration | display set set version 25.4R1.12 set system host-name SRX300-Spoke3 set system root-authentication encrypted-password "$6$W4aA.E9N$PIUkkf038rgjWOuPqBqvDwzjuTX2nUvC/GP/8F6mLXqmELx1i6YfjDqId61qF99mREXoeMeNAUdAFoe3CFQdA1" set system services ssh root-login allow set system services ssh sftp-server set security ike proposal ike_proposal authentication-method pre-shared-keys set security ike proposal ike_proposal dh-group group14 set security ike proposal ike_proposal authentication-algorithm sha-256 set security ike proposal ike_proposal encryption-algorithm aes-256-cbc set security ike proposal ike_proposal lifetime-seconds 28800 set security ike policy ike_policy mode main set security ike policy ike_policy proposals ike_proposal set security ike policy ike_policy pre-shared-key ascii-text "$9$wm2oZHqfn/tUj/tuOcSwYg" set security ike gateway gw_to_hq ike-policy ike_policy set security ike gateway gw_to_hq address 10.99.103.254 set security ike gateway gw_to_hq dead-peer-detection interval 10 set security ike gateway gw_to_hq dead-peer-detection threshold 5 set security ike gateway gw_to_hq local-identity inet 10.1.1.3 set security ike gateway gw_to_hq remote-identity inet 10.99.103.254 set security ike gateway gw_to_hq external-interface lo0.0 set security ipsec proposal ipsec_proposal protocol esp set security ipsec proposal ipsec_proposal authentication-algorithm hmac-sha-256-128 set security ipsec proposal ipsec_proposal encryption-algorithm aes-256-cbc set security ipsec proposal ipsec_proposal lifetime-seconds 3600 set security ipsec policy ipsec_policy perfect-forward-secrecy keys group14 set security ipsec policy ipsec_policy proposals ipsec_proposal set security ipsec vpn vpn_to_hq bind-interface st0.0 set security ipsec vpn vpn_to_hq ike gateway gw_to_hq set security ipsec vpn vpn_to_hq ike no-anti-replay set security ipsec vpn vpn_to_hq ike ipsec-policy ipsec_policy set security zones security-zone vpn host-inbound-traffic system-services all set security zones security-zone vpn host-inbound-traffic protocols all set security zones security-zone vpn interfaces ge-0/0/1.0 set security zones security-zone vpn interfaces lo0.0 set security zones security-zone vpn interfaces st0.0 set security policies default-policy permit-all set interfaces ge-0/0/1 unit 0 family inet address 192.168.1.3/24 set interfaces lo0 unit 0 family inet address 10.1.1.3/24 set interfaces st0 unit 0 family inet address 10.63.33.3/24 set protocols ospf area 0.0.0.1 interface st0.0 interface-type p2p set routing-options static route 10.99.103.254/32 next-hop 192.168.1.254 root@SRX300-Spoke4# run show ospf neighbor Address Interface State ID Pri Dead 10.63.33.254 st0.0 Full 10.99.103.254 128 39 root@SRX300-Spoke4# run show ospf statistics Packet type Total Last 5 seconds Sent Received Sent Received Hello 1693 1543 0 1 DbD 7 5 0 0 LSReq 2 2 0 0 LSUpdate 7 18 0 0 LSAck 10 1 0 0 DBDs retransmitted : 1, last 5 seconds : 0 LSAs flooded : 3, last 5 seconds : 0 LSAs flooded high-prio : 2, last 5 seconds : 0 LSAs retransmitted : 0, last 5 seconds : 0 LSAs transmitted to nbr: 2, last 5 seconds : 0 LSAs requested : 3, last 5 seconds : 0 LSAs acknowledged : 12, last 5 seconds : 0 Flood queue depth : 0 Total rexmit entries : 0 db summaries : 0 lsreq entries : 0 Receive errors: None root@SRX300-Spoke4# run show configuration | display set set version 21.4R3-S9.8 set system host-name SRX300-Spoke4 set system root-authentication encrypted-password "$6$v4wIEpCa$7rdBlKU9LppgfYSrZcyeB7k24o3gPzoXGpHj3ggL1atiskY63ff6ehw0Zs0g0mdS4j3tsFfcIxzP9wP7W4aCt0" set system services ssh root-login allow set system services ssh sftp-server set security ike proposal ike_proposal authentication-method pre-shared-keys set security ike proposal ike_proposal dh-group group14 set security ike proposal ike_proposal authentication-algorithm sha-256 set security ike proposal ike_proposal encryption-algorithm aes-256-cbc set security ike proposal ike_proposal lifetime-seconds 28800 set security ike policy ike_policy mode main set security ike policy ike_policy proposals ike_proposal set security ike policy ike_policy pre-shared-key ascii-text "$9$wm2oZHqfn/tUj/tuOcSwYg" set security ike gateway gw_to_hq ike-policy ike_policy set security ike gateway gw_to_hq address 10.99.103.254 set security ike gateway gw_to_hq dead-peer-detection interval 10 set security ike gateway gw_to_hq dead-peer-detection threshold 5 set security ike gateway gw_to_hq local-identity inet 10.1.1.4 set security ike gateway gw_to_hq remote-identity inet 10.99.103.254 set security ike gateway gw_to_hq external-interface lo0.0 set security ipsec proposal ipsec_proposal protocol esp set security ipsec proposal ipsec_proposal authentication-algorithm hmac-sha-256-128 set security ipsec proposal ipsec_proposal encryption-algorithm aes-256-cbc set security ipsec proposal ipsec_proposal lifetime-seconds 3600 set security ipsec policy ipsec_policy perfect-forward-secrecy keys group14 set security ipsec policy ipsec_policy proposals ipsec_proposal set security ipsec vpn vpn_to_hq bind-interface st0.0 set security ipsec vpn vpn_to_hq ike gateway gw_to_hq set security ipsec vpn vpn_to_hq ike no-anti-replay set security ipsec vpn vpn_to_hq ike ipsec-policy ipsec_policy set security policies default-policy permit-all set security zones security-zone vpn host-inbound-traffic system-services all set security zones security-zone vpn host-inbound-traffic protocols all set security zones security-zone vpn interfaces ge-0/0/1.0 set security zones security-zone vpn interfaces lo0.0 set security zones security-zone vpn interfaces st0.0 set interfaces ge-0/0/1 unit 0 family inet address 192.168.1.4/24 set interfaces lo0 unit 0 family inet address 10.1.1.4/24 set interfaces st0 unit 0 family inet address 10.63.33.4/24 set protocols ospf area 0.0.0.1 interface st0.0 interface-type p2p set routing-options static route 10.99.103.254/32 next-hop 192.168.1.254 root@SRX300-Spoke5# run show ospf neighbor [edit] root@SRX300-Spoke5# run show ospf statistics Packet type Total Last 5 seconds Sent Received Sent Received Hello 574 0 1 0 DbD 0 0 0 0 LSReq 0 0 0 0 LSUpdate 0 0 0 0 LSAck 0 0 0 0 DBDs retransmitted : 0, last 5 seconds : 0 LSAs flooded : 0, last 5 seconds : 0 LSAs flooded high-prio : 0, last 5 seconds : 0 LSAs retransmitted : 0, last 5 seconds : 0 LSAs transmitted to nbr: 0, last 5 seconds : 0 LSAs requested : 0, last 5 seconds : 0 LSAs acknowledged : 0, last 5 seconds : 0 Flood queue depth : 0 Total rexmit entries : 0 db summaries : 0 lsreq entries : 0 Receive errors: None root@SRX300-Spoke5# run show configuration | display set set version 22.4R3-S9.3 set system host-name SRX300-Spoke5 set system root-authentication encrypted-password "$6$49HuB0xH$XHz9Ut5JzIpcg1YV7XnBzrztVV3Lq/2fO63oCT3ebSTSTC8UUiIAvhpb6fbE96LZeeDyG6QgQSbDuSlqz6CLj0" set system services ssh root-login allow set system services ssh sftp-server set security ike proposal ike_proposal authentication-method pre-shared-keys set security ike proposal ike_proposal dh-group group14 set security ike proposal ike_proposal authentication-algorithm sha-256 set security ike proposal ike_proposal encryption-algorithm aes-256-cbc set security ike proposal ike_proposal lifetime-seconds 28800 set security ike policy ike_policy mode main set security ike policy ike_policy proposals ike_proposal set security ike policy ike_policy pre-shared-key ascii-text "$9$wm2oZHqfn/tUj/tuOcSwYg" set security ike gateway gw_to_hq ike-policy ike_policy set security ike gateway gw_to_hq address 10.99.103.254 set security ike gateway gw_to_hq dead-peer-detection interval 10 set security ike gateway gw_to_hq dead-peer-detection threshold 5 set security ike gateway gw_to_hq local-identity inet 10.1.1.5 set security ike gateway gw_to_hq remote-identity inet 10.99.103.254 set security ike gateway gw_to_hq external-interface lo0.0 set security ipsec proposal ipsec_proposal protocol esp set security ipsec proposal ipsec_proposal authentication-algorithm hmac-sha-256-128 set security ipsec proposal ipsec_proposal encryption-algorithm aes-256-cbc set security ipsec proposal ipsec_proposal lifetime-seconds 3600 set security ipsec policy ipsec_policy perfect-forward-secrecy keys group14 set security ipsec policy ipsec_policy proposals ipsec_proposal set security ipsec vpn vpn_to_hq bind-interface st0.0 set security ipsec vpn vpn_to_hq ike gateway gw_to_hq set security ipsec vpn vpn_to_hq ike no-anti-replay set security ipsec vpn vpn_to_hq ike ipsec-policy ipsec_policy set security policies default-policy permit-all set security zones security-zone vpn host-inbound-traffic system-services all set security zones security-zone vpn host-inbound-traffic protocols all set security zones security-zone vpn interfaces ge-0/0/1.0 set security zones security-zone vpn interfaces lo0.0 set security zones security-zone vpn interfaces st0.0 set interfaces ge-0/0/1 unit 0 family inet address 192.168.1.5/24 set interfaces lo0 unit 0 family inet address 10.1.1.5/24 set interfaces st0 unit 0 family inet address 10.63.33.5/24 set protocols ospf area 0.0.0.1 interface st0.0 interface-type p2p set routing-options static route 10.99.103.254/32 next-hop 192.168.1.254 root@SRX300-Spoke6# run show ospf neighbor [edit] root@SRX300-Spoke6# root@SRX300-Spoke6# run show ospf statistics Packet type Total Last 5 seconds Sent Received Sent Received Hello 504 0 0 0 DbD 0 0 0 0 LSReq 0 0 0 0 LSUpdate 0 0 0 0 LSAck 0 0 0 0 DBDs retransmitted : 0, last 5 seconds : 0 LSAs flooded : 0, last 5 seconds : 0 LSAs flooded high-prio : 0, last 5 seconds : 0 LSAs retransmitted : 0, last 5 seconds : 0 LSAs transmitted to nbr: 0, last 5 seconds : 0 LSAs requested : 0, last 5 seconds : 0 LSAs acknowledged : 0, last 5 seconds : 0 Flood queue depth : 0 Total rexmit entries : 0 db summaries : 0 lsreq entries : 0 Receive errors: None root@SRX300-Spoke6# run show configuration | display set set version 25.4R1-S2.3 set system host-name SRX300-Spoke6 set system root-authentication encrypted-password "$6$ptsX/C5u$cN9tfjFIvacIdZK4etHh6PxqcbXGDSd5SZfvelv3VlaKUeHWy20r5/3osam0rX2vfNQhsETDrn7gBpx9BrF3R." set system services ssh root-login allow set system services ssh sftp-server set security ike proposal ike_proposal authentication-method pre-shared-keys set security ike proposal ike_proposal dh-group group14 set security ike proposal ike_proposal authentication-algorithm sha-256 set security ike proposal ike_proposal encryption-algorithm aes-256-cbc set security ike proposal ike_proposal lifetime-seconds 28800 set security ike policy ike_policy mode main set security ike policy ike_policy proposals ike_proposal set security ike policy ike_policy pre-shared-key ascii-text "$9$wm2oZHqfn/tUj/tuOcSwYg" set security ike gateway gw_to_hq ike-policy ike_policy set security ike gateway gw_to_hq address 10.99.103.254 set security ike gateway gw_to_hq dead-peer-detection interval 10 set security ike gateway gw_to_hq dead-peer-detection threshold 5 set security ike gateway gw_to_hq local-identity inet 10.1.1.6 set security ike gateway gw_to_hq remote-identity inet 10.99.103.254 set security ike gateway gw_to_hq external-interface lo0.0 set security ipsec proposal ipsec_proposal protocol esp set security ipsec proposal ipsec_proposal authentication-algorithm hmac-sha-256-128 set security ipsec proposal ipsec_proposal encryption-algorithm aes-256-cbc set security ipsec proposal ipsec_proposal lifetime-seconds 3600 set security ipsec policy ipsec_policy perfect-forward-secrecy keys group14 set security ipsec policy ipsec_policy proposals ipsec_proposal set security ipsec vpn vpn_to_hq bind-interface st0.0 set security ipsec vpn vpn_to_hq ike gateway gw_to_hq set security ipsec vpn vpn_to_hq ike no-anti-replay set security ipsec vpn vpn_to_hq ike ipsec-policy ipsec_policy set security zones security-zone vpn host-inbound-traffic system-services all set security zones security-zone vpn host-inbound-traffic protocols all set security zones security-zone vpn interfaces ge-0/0/1.0 set security zones security-zone vpn interfaces lo0.0 set security zones security-zone vpn interfaces st0.0 set security policies default-policy permit-all set interfaces ge-0/0/1 unit 0 family inet address 192.168.1.6/24 set interfaces lo0 unit 0 family inet address 10.1.1.6/24 set interfaces st0 unit 0 family inet address 10.63.33.6/24 set protocols ospf area 0.0.0.1 interface st0.0 set routing-options static route 10.99.103.254/32 next-hop 192.168.1.254
After upgrading the Junos of HQ devices from 23.4R2-S5.5 to 24.4R2-S3.5 , the spoke 5 and spoke 6 ospf neighbor stuck in "init" state always. {primary:node0}[edit] root@SRX345-HQ-node0# run show ospf neighbor Address Interface State ID Pri Dead 10.63.33.5 st0.0 Init 10.1.1.5 128 33 10.63.33.6 st0.0 Init 10.1.1.6 128 39 10.63.33.3 st0.0 Full 10.1.1.3 128 34 10.63.33.4 st0.0 Full 10.1.1.4 128 38
{primary:node0}[edit]root@SRX345-HQ-node0# run show route protocol bgp
inet.0: 18 destinations, 19 routes (18 active, 0 holddown, 0 hidden)+ = Active Route, - = Last Active, * = Both
10.1.1.3/32 *[BGP/140] 03:38:54, localpref 100 AS path: 4780 I, validation-state: unverified > to 10.100.1.254 via ge-0/0/3.010.1.1.4/32 *[BGP/140] 03:38:54, localpref 100 AS path: 4780 I, validation-state: unverified > to 10.100.1.254 via ge-0/0/3.010.1.1.5/32 *[BGP/140] 03:38:50, localpref 100 AS path: 4780 I, validation-state: unverified > to 10.100.2.254 via ge-5/0/3.010.1.1.6/32 *[BGP/140] 03:38:50, localpref 100 AS path: 4780 I, validation-state: unverified > to 10.100.2.254 via ge-5/0/3.0
{primary:node0}[edit] root@SRX345-HQ-node0# run show chassis cluster status Monitor Failure codes: CS Cold Sync monitoring FL Fabric Connection monitoring GR GRES monitoring HW Hardware monitoring IF Interface monitoring IP IP monitoring LB Loopback monitoring MB Mbuf monitoring NH Nexthop monitoring NP NPC monitoring SP SPU monitoring SM Schedule monitoring CF Config Sync monitoring RE Relinquish monitoring IS IRQ storm Cluster ID: 1 Node Priority Status Preempt Manual Monitor-failures Redundancy group: 0 , Failover count: 3 node0 200 primary no no None node1 100 secondary no no None Redundancy group: 1 , Failover count: 1 node0 200 primary no no None node1 100 secondary no no None
{primary:node0}
root@SRX345-HQ-node0> start shell
root@SRX345-HQ-node0:~ # vty fwdd
FLOWD_OCTEON(SRX345-HQ-node1 vty)# show nhdb management fabric Stashed Nexthop table: Chas-id NHID JNH Remote-Chassis-IP-Addr -------------------------------------------------------- 0 1354 0x502b01 30.18.0.1 Fabric NH stats: NH add messages : 9 NH delete messages : 0 JNH add messages : 10 JNH delete messages : 0 Fabric stats: Receive : 9560028 Fragments Received : 0 Parsed : 0 Transmit : 7606346 Receive RTOs : 746543 Transmit RTOs : 27006 Receive Probes : 7156823 Transmit Probes : 7156821 Receive flow_fwd : 621667 Transmit flow_fwd : 422322 Receive Pkt Forwarding : 0 Receive Pkt Forwarding Error : 0 Transmit Pkt Forwarding : 0 Receive fabric fwd : 352356 Transmit fabric fwd : 197 Receive Mcast fabric fwd : 0 Transmit Mcast fabric fwd : 0 Receive Mcast fabric fwd Error: 0 Receive Invalid Types : 0 Receive Parse Error : 682639 Receive IP Checksum Error : 682639 <<<<<<The OSPF packets which sent to the spoke 5 and spoke 6 dropped here due to ip checksum error Receive IP Invalid src-ip : 0 Receive Payload Offset Error : 0 Receive TLV Length Error : 0 Receive TLV Discard : 0 No Fabric : 0 Transmit failed (ifnf) : 0 Transmit need reinject : 0 Transmit reinject successful : 0 nxt iphdr sanity error : 0 nxt ip6hdr sanity error : 0
This issue is addressed in the PR1976758: https://prsearch.juniper.net/problemreport/PR1976758
The outbound OSPF packets over IPSec in SRX cluster environment with Active/Standby mode might be encrypted & encapsulated by the node where the IPSec tunnel is not active, and get overwritten with the original Ethernet type & IP header by mistake during VLAN tag insertion for being transmitted to another node over the cluster fabric link due to two more bytes introduced into the head room by some enhancement since Junos 24.2R1. Thus on the receiving node of the same cluster, such corrupted OSPF packets are dropped because of IP checksum errors.
The fix will choose the right node where the IPSec tunnel is active to do encryption & encapsulation for the outbound OSPF packets over IPSec. In addition, the new codes also avoid overriding the original Ethernet type & IP header during VLAN tag for the cross-node traffic.
root@SRX345-HQ-node0> show version
node0:
--------------------------------------------------------------------------
Hostname: SRX345-HQ-node0
Model: srx345
Family: junos-es
Junos: 26.2I-20261021.0.1757 （This private image include the fix code）
root@SRX345-HQ-node0> show chassis cluster status
Monitor Failure codes:
CS Cold Sync monitoring FL Fabric Connection monitoring
GR GRES monitoring HW Hardware monitoring
IF Interface monitoring IP IP monitoring
LB Loopback monitoring MB Mbuf monitoring
NH Nexthop monitoring NP NPC monitoring
SP SPU monitoring SM Schedule monitoring
CF Config Sync monitoring RE Relinquish monitoring
IS IRQ storm
Cluster ID: 1
Node Priority Status Preempt Manual Monitor-failures
Redundancy group: 0 , Failover count: 3
node0 200 primary no no None
node1 100 secondary no no None
Redundancy group: 1 , Failover count: 1
root@SRX345-HQ-node0> show security ike security-associations
Index State Initiator cookie Responder cookie Mode Remote Address
5403321 UP 8e74928ba8d77714 175ecb89c9917f12 Main 10.1.1.6
5403322 UP efcb017c3f1da4c3 c764cf5e8abf8441 Main 10.1.1.5
16378718 UP 09c7824fc0cd105e 375a5288f5065ed3 Main 10.1.1.3
16378719 UP f553e048f5193a80 942b9b045b42029f Main 10.1.1.4
root@SRX345-HQ-node0> show security ipsec security-associations
Total active tunnels: 4 Total Ipsec sas: 4
ID Algorithm SPI Life:sec/kb Mon lsys Port Gateway
<131074 ESP:aes-cbc-256/sha256 15b0cf99 2660/ unlim - root 500 10.1.1.3
>131074 ESP:aes-cbc-256/sha256 a92221f9 2660/ unlim - root 500 10.1.1.3
<131075 ESP:aes-cbc-256/sha256 8f9139eb 2659/ unlim - root 500 10.1.1.4
>131075 ESP:aes-cbc-256/sha256 1f0acafd 2659/ unlim - root 500 10.1.1.4
<131073 ESP:aes-cbc-256/sha256 a1c0531e 3102/ unlim - root 500 10.1.1.5
>131073 ESP:aes-cbc-256/sha256 69a46b6c 3102/ unlim - root 500 10.1.1.5
<131076 ESP:aes-cbc-256/sha256 c4a69004 1063/ unlim - root 500 10.1.1.6
>131076 ESP:aes-cbc-256/sha256 8817b71f 1063/ unlim - root 500 10.1.1.6
root@SRX345-HQ-node0> show ospf neighbor
Address Interface State ID Pri Dead
10.63.33.5 st0.0 Full 10.1.1.5 128 36
10.63.33.3 st0.0 Full 10.1.1.3 128 39
10.63.33.4 st0.0 Full 10.1.1.4 128 32
10.63.33.6 st0.0 Full 10.1.1.6 128 36
On the fix Junos version, the IPSEC tunnel choose the right node to do the encryption and decryption at the same time, this means the ospf packets never pass through the chassis cluster FAB link with the IPSEC header.
root@SRX345-HQ-node0>
BSD platform (OCTEON processor, 552MB memory, 0KB flash)
FLOWD_OCTEON(SRX345-HQ-node0 vty)# show usp ipsec sa
Tunnel | Crypto/Hash | | |Auto|Dyna|Anti|VPN |Bind ifp/| Bytes | Bytes |Invalid|TS check|Inactive| Local | Remote |Thd |HW |
id | Algo |Proto| Status |Key | EP |Rply|Mon |Policy id| In | Out | SPI | FAIL | SA |Address | Address|ID |Off|
--------+---------------+-----+---------+----+----+----+----|---------|-------------|-------------|-------+--------|--------|--------|-------------|---|
131073 | AES256/SHA256|ESP |Active |Auto|Disa|Disa|VD |st0.0 | 0| 0|0 |0 |0 |10.99.103.254 |10.1.1.5|0||No|
|No|
131074 | AES256/SHA256|ESP |Active |Auto|Disa|Disa|VD |st0.0 | 10256| 21132|0 |0 |0 |10.99.103.254 |10.1.1.3|0||No|
131075 | AES256/SHA256|ESP |Active |Auto|Disa|Disa|VD |st0.0 | 10468| 21320|0 |0 |0 |10.99.103.254 |10.1.1.4|0||No|
131076 | AES256/SHA256|ESP |Active |Auto|Disa|Disa|VD |st0.0 | 0| 0|0 |0 |0 |10.99.103.254 |10.1.1.6|0||No|
-----------------------------------------------------------------
{secondary:node1}
root@SRX345-HQ-node1> start shell
root@SRX345-HQ-node1:~ # vty fwdd
FLOWD_OCTEON(SRX345-HQ-node1 vty)# show usp ipsec sa
131073 | AES256/SHA256|ESP |Active |Auto|Disa|Disa|VD |st0.0 | 6256| 12156|0 |0 |0 |10.99.103.254 |10.1.1.5|0||No|
131074 | AES256/SHA256|ESP |Active |Auto|Disa|Disa|VD |st0.0 | 0| 0|0 |0 |0 |10.99.103.254 |10.1.1.3|0||No|
131075 | AES256/SHA256|ESP |Active |Auto|Disa|Disa|VD |st0.0 | 0| 0|0 |0 |0 |10.99.103.254 |10.1.1.4|0||No|
131076 | AES256/SHA256|ESP |Active |Auto|Disa|Disa|VD |st0.0 | 25336| 51556|0 |0 |0 |10.99.103.254 |10.1.1.6|0||No|