Home » D-Link Manuals » Networking » D-Link DFL-260-IPS-12 » Manual Viewer

D-Link DFL-260-IPS-12 Product Manual - Page 441

Note: L2TP with Microsoft Vista, Ike_invalid_payload, Ike_invalid_cookie, Payload_Malformed

View all D-Link DFL-260-IPS-12 manuals

Add to My Manuals
Save this manual to your list of manuals

Page 441 highlights

9.7.5. Specific Error Messages Chapter 9. VPN tunnels use different pre-shared keys, you will receive an "Incorrect pre-shared key" error message. The problem is solved if we reorder the list and move VPN-3 above L2TP. The gateway office3gw will be then matched correctly and VPN-3 will be the tunnel selected by NetDefendOS. 3. Ike_invalid_payload, Ike_invalid_cookie In this case the IPsec engine in NetDefendOS receives an IPsec IKE packet but is unable to match it against an existing IKE. If a VPN tunnel is only established on one side, this can be the resulting error message when traffic arrives from a tunnel that does not exist. An example would be if, for some reason, the tunnel has only gone down from the initiator side but the terminator still sees it as up. It then tries to send packets through the tunnel but when they arrive at the initiator it will drop them since no matching tunnel can be found. Simply remove the tunnel from the side that believes it is still up to solve the immediate problem. An investigation as to why the tunnel only went down from one side is recommended. It could be that DPD and/or Keep-Alive is only used on one side. Another possible cause could be that even though it has received a DELETE packet, it has not deleted/removed the tunnel. 4. Payload_Malformed This problem is very similar to the Incorrect pre-shared key problem described above. A possible reason is that the PSK is of the wrong TYPE on either side (Passphrase or Hex key). Verify that you are using the same type on both sides of the IPsec tunnel. If one side is using Hex and the other Passphrase, this is most likely the error message that you will receive. 5. No public key found This is a very common error message when dealing with tunnels that use certificates for authentication. Troubleshooting this error message can be very difficult as the possible cause of the problem can be quite extensive. Also it is very important to keep in mind that when dealing with certificates you may need to combine the ikesnoop logs with normal logs as ikesnoop does not give that much information about certificates, while normal logs can provide important clues as to what the problem could be. A good suggestion before you start to troubleshoot certificate based tunnels is to first configure it as a PSK tunnel and then verify that it can successfully establish, then move on to using Certificates. (Unless the configuration type prohibits that). The possible causes are as follows: • The certificate on either side is not signed by the same CA server. • The certificate's validity time has expired or it has not yet started to be valid. The latter can happen if the clock is set incorrectly on either the CA server or the NetDefend Firewall or they are in different time zones. • The NetDefend Firewall is unable to reach the Certificate Revocation List (CRL) on the CA server in order to verify if the certificate is valid or not. Double-check that the CRL path is valid in the certificate's properties. (Using the CRL feature could be turned off.) Make sure also that there is a DNS client configured in NetDefendOS in order for it to be able to correctly resolve the path to the CRL. Note: L2TP with Microsoft Vista With L2TP, Microsoft Vista tries by default to contact and download the CRL list, while Microsoft XP does not. This option can be turned off in Vista. 441

Section	Page
User Manual	3
Table of Contents	4
Preface	14
Chapter 1. NetDefendOS Overview	16
1.1. Features	16
1.2. NetDefendOS Architecture	19
1.2.1. State-based Architecture	19
1.2.2. NetDefendOS Building Blocks	19
1.2.3. Basic Packet Flow	20
1.3. NetDefendOS State Engine Packet Flow	23
Chapter 2. Management and Maintenance	28
2.1. Managing NetDefendOS	28
2.1.1. Overview	28
2.1.2. The Default Administrator Account	29
2.1.3. The Web Interface	29
2.1.4. The CLI	33
2.1.5. CLI Scripts	41
2.1.6. Secure Copy	45
2.1.7. The Console Boot Menu	47
2.1.8. Management Advanced Settings	48
2.1.9. Working with Configurations	49
2.2. Events and Logging	55
2.2.1. Overview	55
2.2.2. Log Messages	55
2.2.3. Creating Log Receivers	56
2.2.4. Logging to MemoryLogReceiver	56
2.2.5. Logging to Syslog Hosts	56
2.2.6. SNMP Traps	58
2.2.7. Advanced Log Settings	59
2.3. RADIUS Accounting	60
2.3.1. Overview	60
2.3.2. RADIUS Accounting Messages	60
2.3.3. Interim Accounting Messages	62
2.3.4. Activating RADIUS Accounting	62
2.3.5. RADIUS Accounting Security	62
2.3.6. RADIUS Accounting and High Availability	62
2.3.7. Handling Unresponsive Servers	63
2.3.8. Accounting and System Shutdowns	63
2.3.9. Limitations with NAT	63
2.3.10. RADIUS Advanced Settings	63
2.4. Hardware Monitoring	65
2.5. SNMP Monitoring	67
2.5.1. SNMP Advanced Settings	68
2.6. The pcapdump Command	70
2.7. Maintenance	73
2.7.1. Auto-Update Mechanism	73
2.7.2. Backing Up Configurations	73
2.7.3. Restore to Factory Defaults	74
Chapter 3. Fundamentals	77
3.1. The Address Book	77
3.1.1. Overview	77
3.1.2. IP Addresses	77
3.1.3. Ethernet Addresses	79
3.1.4. Address Groups	80
3.1.5. Auto-Generated Address Objects	81
3.1.6. Address Book Folders	81
3.2. Services	82
3.2.1. Overview	82
3.2.2. Creating Custom Services	83
3.2.3. ICMP Services	86
3.2.4. Custom IP Protocol Services	88
3.2.5. Service Groups	88
3.2.6. Custom Service Timeouts	89
3.3. Interfaces	90
3.3.1. Overview	90
3.3.2. Ethernet Interfaces	92
3.3.2.1. Useful CLI Commands for Ethernet Interfaces	95
3.3.3. VLAN	97
3.3.4. PPPoE	101
3.3.5. GRE Tunnels	103
3.3.6. Interface Groups	107
3.4. ARP	108
3.4.1. Overview	108
3.4.2. The NetDefendOS ARP Cache	108
3.4.3. Creating ARP Objects	110
3.4.4. Using ARP Advanced Settings	112
3.4.5. ARP Advanced Settings Summary	113
3.5. IP Rule Sets	116
3.5.1. Security Policies	116
3.5.2. IP Rule Evaluation	118
3.5.3. IP Rule Actions	119
3.5.4. Editing IP rule set Entries	120
3.5.5. IP Rule Set Folders	121
3.5.6. Configuration Object Groups	122
3.6. Schedules	126
3.7. Certificates	128
3.7.1. Overview	128
3.7.2. Certificates in NetDefendOS	129
3.7.3. CA Certificate Requests	130
3.8. Date and Time	132
3.8.1. Overview	132
3.8.2. Setting Date and Time	132
3.8.3. Time Servers	133
3.8.4. Settings Summary for Date and Time	136
3.9. DNS	139
Chapter 4. Routing	142
4.1. Overview	142
4.2. Static Routing	143
4.2.1. The Principles of Routing	143
4.2.2. Static Routing	147
4.2.3. Route Failover	151
4.2.4. Host Monitoring for Route Failover	154
4.2.5. Advanced Settings for Route Failover	156
4.2.6. Proxy ARP	157
4.3. Policy-based Routing	160
4.3.1. Overview	160
4.3.2. Policy-based Routing Tables	160
4.3.3. Policy-based Routing Rules	160
4.3.4. Routing Table Selection	161
4.3.5. The Ordering parameter	161
4.4. Route Load Balancing	165
4.5. OSPF	171
4.5.1. Dynamic Routing	171
4.5.2. OSPF Concepts	174
4.5.3. OSPF Components	179
4.5.3.1. OSPF Router Process	179
4.5.3.2. OSPF Area	181
4.5.3.3. OSPF Interface	182
4.5.3.4. OSPF Neighbors	184
4.5.3.5. OSPF Aggregates	184
4.5.3.6. OSPF VLinks	184
4.5.4. Dynamic Routing Rules	185
4.5.4.1. Overview	185
4.5.4.2. Dynamic Routing Rule	186
4.5.4.3. OSPF Action	187
4.5.4.4. Routing Action	187
4.5.5. Setting Up OSPF	188
4.5.6. An OSPF Example	191
4.6. Multicast Routing	194
4.6.1. Overview	194
4.6.2. Multicast Forwarding with SAT Multiplex Rules	195
4.6.2.1. Multicast Forwarding - No Address Translation	195
4.6.2.2. Multicast Forwarding - Address Translation Scenario	197
4.6.3. IGMP Configuration	199
4.6.3.1. IGMP Rules Configuration - No Address Translation	200
4.6.3.2. IGMP Rules Configuration - Address Translation	202
4.6.4. Advanced IGMP Settings	204
4.7. Transparent Mode	207
4.7.1. Overview	207
4.7.2. Enabling Internet Access	211
4.7.3. Transparent Mode Scenarios	213
4.7.4. Spanning Tree BPDU Support	217
4.7.5. Advanced Settings for Transparent Mode	218
Chapter 5. DHCP Services	223
5.1. Overview	223
5.2. DHCP Servers	224
5.2.1. Static DHCP Hosts	227
5.2.2. Custom Options	228
5.3. DHCP Relaying	230
5.3.1. DHCP Relay Advanced Settings	231
5.4. IP Pools	233
Chapter 6. Security Mechanisms	237
6.1. Access Rules	237
6.1.1. Overview	237
6.1.2. IP Spoofing	238
6.1.3. Access Rule Settings	238
6.2. ALGs	240
6.2.1. Overview	240
6.2.2. The HTTP ALG	241
6.2.3. The FTP ALG	244
6.2.4. The TFTP ALG	253
6.2.5. The SMTP ALG	254
6.2.5.1. Anti-Spam Filtering	257
6.2.6. The POP3 ALG	263
6.2.7. The PPTP ALG	264
6.2.8. The SIP ALG	265
6.2.9. The H.323 ALG	275
6.2.10. The TLS ALG	289
6.3. Web Content Filtering	292
6.3.1. Overview	292
6.3.2. Active Content Handling	292
6.3.3. Static Content Filtering	293
6.3.4. Dynamic Web Content Filtering	295
6.3.4.1. Overview	295
6.3.4.2. Setting Up WCF	296
6.3.4.3. Content Filtering Categories	300
6.3.4.4. Customizing HTML Pages	307
6.4. Anti-Virus Scanning	309
6.4.1. Overview	309
6.4.2. Implementation	309
6.4.3. Activating Anti-Virus Scanning	310
6.4.4. The Signature Database	311
6.4.5. Subscribing to the D-Link Anti-Virus Service	311
6.4.6. Anti-Virus Options	311
6.5. Intrusion Detection and Prevention	315
6.5.1. Overview	315
6.5.2. IDP Availability for D-Link Models	315
6.5.3. IDP Rules	317
6.5.4. Insertion/Evasion Attack Prevention	318
6.5.5. IDP Pattern Matching	319
6.5.6. IDP Signature Groups	320
6.5.7. IDP Actions	322
6.5.8. SMTP Log Receiver for IDP Events	322
6.6. Denial-of-Service Attack Prevention	326
6.6.1. Overview	326
6.6.2. DoS Attack Mechanisms	326
6.6.3. Ping of Death and Jolt Attacks	326
6.6.4. Fragmentation overlap attacks: Teardrop, Bonk, Boink and Nestea	327
6.6.5. The Land and LaTierra attacks	327
6.6.6. The WinNuke attack	327
6.6.7. Amplification attacks: Smurf, Papasmurf, Fraggle	328
6.6.8. TCP SYN Flood Attacks	329
6.6.9. The Jolt2 Attack	329
6.6.10. Distributed DoS Attacks	329
6.7. Blacklisting Hosts and Networks	331
Chapter 7. Address Translation	334
7.1. Overview	334
7.2. NAT	335
7.3. NAT Pools	340
7.4. SAT	343
7.4.1. Translation of a Single IP Address (1:1)	343
7.4.2. Translation of Multiple IP Addresses (M:N)	348
7.4.3. All-to-One Mappings (N:1)	350
7.4.4. Port Translation	350
7.4.5. Protocols Handled by SAT	351
7.4.6. Multiple SAT Rule Matches	351
7.4.7. SAT and FwdFast Rules	352
Chapter 8. User Authentication	355
8.1. Overview	355
8.2. Authentication Setup	357
8.2.1. Setup Summary	357
8.2.2. The Local Database	357
8.2.3. External RADIUS Servers	359
8.2.4. External LDAP Servers	359
8.2.5. Authentication Rules	366
8.2.6. Authentication Processing	368
8.2.7. A Group Usage Example	369
8.2.8. HTTP Authentication	369
8.3. Customizing HTML Pages	373
Chapter 9. VPN	377
9.1. Overview	377
9.1.1. VPN Usage	377
9.1.2. VPN Encryption	378
9.1.3. VPN Planning	378
9.1.4. Key Distribution	379
9.1.5. The TLS Alternative for VPN	379
9.2. VPN Quick Start	381
9.2.1. IPsec LAN to LAN with Pre-shared Keys	382
9.2.2. IPsec LAN to LAN with Certificates	383
9.2.3. IPsec Roaming Clients with Pre-shared Keys	384
9.2.4. IPsec Roaming Clients with Certificates	386
9.2.5. L2TP Roaming Clients with Pre-Shared Keys	387
9.2.6. L2TP Roaming Clients with Certificates	388
9.2.7. PPTP Roaming Clients	389
9.3. IPsec Components	391
9.3.1. Overview	391
9.3.2. Internet Key Exchange (IKE)	391
9.3.3. IKE Authentication	397
9.3.4. IPsec Protocols (ESP/AH)	398
9.3.5. NAT Traversal	399
9.3.6. Algorithm Proposal Lists	401
9.3.7. Pre-shared Keys	402
9.3.8. Identification Lists	403
9.4. IPsec Tunnels	406
9.4.1. Overview	406
9.4.2. LAN to LAN Tunnels with Pre-shared Keys	408
9.4.3. Roaming Clients	408
9.4.4. Fetching CRLs from an alternate LDAP server	413
9.4.5. Troubleshooting with ikesnoop	414
9.4.6. IPsec Advanced Settings	421
9.5. PPTP/L2TP	425
9.5.1. PPTP Servers	425
9.5.2. L2TP Servers	426
9.5.3. L2TP/PPTP Server advanced settings	430
9.5.4. PPTP/L2TP Clients	431
9.6. CA Server Access	434
9.7. VPN Troubleshooting	437
9.7.1. General Troubleshooting	437
9.7.2. Troubleshooting Certificates	437
9.7.3. IPsec Troubleshooting Commands	438
9.7.4. Management Interface Failure with VPN	439
9.7.5. Specific Error Messages	439
9.7.6. Specific Symptoms	442
Chapter 10. Traffic Management	444
10.1. Traffic Shaping	444
10.1.1. Overview	444
10.1.2. Traffic Shaping in NetDefendOS	445
10.1.3. Simple Bandwidth Limiting	447
10.1.4. Limiting Bandwidth in Both Directions	448
10.1.5. Creating Differentiated Limits Using Chains	449
10.1.6. Precedences	450
10.1.7. Pipe Groups	455
10.1.8. Traffic Shaping Recommendations	458
10.1.9. A Summary of Traffic Shaping	459
10.1.10. More Pipe Examples	460
10.2. IDP Traffic Shaping	465
10.2.1. Overview	465
10.2.2. Setting Up IDP Traffic Shaping	465
10.2.3. Processing Flow	466
10.2.4. The Importance of Specifying a Network	466
10.2.5. A P2P Scenario	467
10.2.6. Viewing Traffic Shaping Objects	468
10.2.7. Guaranteeing Instead of Limiting Bandwidth	469
10.2.8. Logging	469
10.3. Threshold Rules	470
10.3.1. Overview	470
10.3.2. Limiting the Connection Rate/Total Connections	470
10.3.3. Grouping	471
10.3.4. Rule Actions	471
10.3.5. Multiple Triggered Actions	471
10.3.6. Exempted Connections	471
10.3.7. Threshold Rules and ZoneDefense	471
10.3.8. Threshold Rule Blacklisting	471
10.4. Server Load Balancing	473
10.4.1. Overview	473
10.4.2. SLB Distribution Algorithms	474
10.4.3. Selecting Stickiness	475
10.4.4. SLB Algorithms and Stickiness	476
10.4.5. Server Health Monitoring	477
10.4.6. Setting Up SLB_SAT Rules	478
Chapter 11. High Availability	482
11.1. Overview	482
11.2. HA Mechanisms	484
11.3. Setting Up HA	487
11.3.1. HA Hardware Setup	487
11.3.2. NetDefendOS Manual HA Setup	488
11.3.3. Verifying the Cluster Functions	489
11.3.4. Unique Shared Mac Addresses	490
11.4. HA Issues	491
11.5. Upgrading an HA Cluster	493
11.6. HA Advanced Settings	495
Chapter 12. ZoneDefense	497
12.1. Overview	497
12.2. ZoneDefense Switches	498
12.3. ZoneDefense Operation	499
12.3.1. SNMP	499
12.3.2. Threshold Rules	499
12.3.3. Manual Blocking and Exclude Lists	499
12.3.4. ZoneDefense with Anti-Virus Scanning	501
12.3.5. Limitations	501
Chapter 13. Advanced Settings	504
13.1. IP Level Settings	504
13.2. TCP Level Settings	508
13.3. ICMP Level Settings	513
13.4. State Settings	514
13.5. Connection Timeout Settings	516
13.6. Length Limit Settings	518
13.7. Fragmentation Settings	520
13.8. Local Fragment Reassembly Settings	524
13.9. Miscellaneous Settings	525
Appendix A. Subscribing to Updates	527
Appendix B. IDP Signature Groups	529
Appendix C. Verified MIME filetypes	533
Appendix D. The OSI Framework	537

Match case Limit results 1 per page

tunnels use different pre-shared keys, you will receive an "

Incorrect pre-shared key

" error message.

The problem is solved if we reorder the list and move

VPN-3

above

L2TP

. The gateway

office3gw

will be then matched correctly and

VPN-3

will be the tunnel selected by NetDefendOS.

Ike_invalid_payload, Ike_invalid_cookie

In this case the IPsec engine in NetDefendOS receives an IPsec IKE packet but is unable to match it

against an existing IKE.

If a VPN tunnel is only established on one side, this can be the resulting error message when traffic

arrives from a tunnel that does not exist. An example would be if, for some reason, the tunnel has

only gone down from the initiator side but the terminator still sees it as up. It then tries to send

packets through the tunnel but when they arrive at the initiator it will drop them since no matching

tunnel can be found.

Simply remove the tunnel from the side that believes it is still up to solve the immediate problem.

An investigation as to why the tunnel only went down from one side is recommended. It could be

that

DPD

and/or

Keep-Alive

is only used on one side. Another possible cause could be that even

though it has received a

DELETE

packet, it has not deleted/removed the tunnel.

Payload_Malformed

This problem is very similar to the

Incorrect pre-shared key

problem described above. A possible

reason is that the PSK is of the wrong TYPE on either side (Passphrase or Hex key).

Verify that you are using the same type on both sides of the IPsec tunnel. If one side is using Hex

and the other Passphrase, this is most likely the error message that you will receive.

No public key found

This is a very common error message when dealing with tunnels that use certificates for

authentication.

Troubleshooting this error message can be very difficult as the possible cause of the problem can be

quite extensive. Also it is very important to keep in mind that when dealing with certificates you

may need to combine the

ikesnoop

logs with normal logs as

ikesnoop

does not give that much

information about certificates, while normal logs can provide important clues as to what the problem

could be. A good suggestion before you start to troubleshoot certificate based tunnels is to first

configure it as a PSK tunnel and then verify that it can successfully establish, then move on to using

Certificates. (Unless the configuration type prohibits that).

The possible causes are as follows:

•

The certificate on either side is not signed by the same CA server.

•

The certificate's validity time has expired or it has not yet started to be valid. The latter can

happen if the clock is set incorrectly on either the CA server or the NetDefend Firewall or they

are in different time zones.

•

The NetDefend Firewall is unable to reach the

Certificate Revocation List

(CRL) on the CA

server in order to verify if the certificate is valid or not. Double-check that the CRL path is valid

in the certificate's properties. (Using the CRL feature could be turned off.) Make sure also that

there is a DNS client configured in NetDefendOS in order for it to be able to correctly resolve

the path to the CRL.

Note: L2TP with Microsoft Vista

With L2TP, Microsoft Vista tries by default to contact and download the CRL list,

while Microsoft XP does not. This option can be turned off in Vista.

9.7.5. Specific Error Messages

Chapter 9. VPN

441