Frame Relay Troubleshooting

Introduction

This topic describes the troubleshooting tools that are provided with the Frame Relay software. Using them can help you save time and money. Time is saved by diagnosing problems in the field without special data communications instrumentation. Money is saved by using the traffic statistics gathering capabilities to provide information for traffic studies.

We don't expect everyone installing and administering this product to be a data communications expert. The information here may be more than you need. However, the tools are no less useful for having background information on data communications described along with them.

Troubleshooting Tools

Two kinds of information can help you determine your problem:

statistics

Statistics are cumulative counts which can be examined at any time. These are often useful during isolation of both functional and performance problems.
protocol traces

Protocol traces show you the protocol exchanged with the remote system from the local system's point of view.

The tsgstat utility provides protocol statistic information, and tsgtrace provides protocol trace information.

The utilities are stored in /sbin. To preserve security you must have root privileges to use these utilities because they allow access to all information transferred on a Frame Relay link.

Using tsgstat

tsgstat is menu-driven and is easy to use. To invoke tsgstat, as root, enter the command:

/etc/tsgstat [-n <netid>]

The optional <netid> is the one (from /etc/packetnets) that you wish to examine. If the netid is not specified, a netid of 0 will be used.

Device Level Statistics

To obtain information on the device level (Layer I) statistics, run tsgstat -b and you will see the following format:

 
Device Level Statistical Information for Network ID 0

         frames received OK     249398   bytes received OK      71294837
         frames transmitted OK  127196   bytes transmitted OK   9415440
         underrun count         0        overrun count          0
         Aborts received        0
         FCS errors             0
         frames < 2 bytes       0
         no recv buffer avail   0
         no header buffer avail 0
         no flush buffer avail  0
         CTS changes            1        DCD changes            1
         DSR changes            1        RI changes             0
         CTS                    ON       DCD                    ON
         DSR                    ON       RI                     OFF

Normally, none of the statistics values except for frame and byte counts should be incrementing rapidly. Quick changes in any other values indicates a problem of some kind.

Field

Description

Frames received OK

Frames received which do not have a CRC error associated with them. Rapid incrementing indicates an active receive data path.

# bytes received OK

Simply a count of the number of data bytes (protocol headers plus data) received. The averages size of a frame is the #bytes received OK divided by the frames received OK.

Frames transmitted ok

Number of frames the transmitter has sent.

# bytes sent to txISR / bytes transmitted OK

Total characters sent on the outbound leg, excluding flags and CRCs.

Lack ipb count

Number of incoming frames that had to be ignored because the buffer pool was depleted. If this count is being incremented frequently, it is most likely that your configuration and/or traffic levels are overloading Wanware in some way. You may wish to consult your supplier for tuning assistance.

no buffer available

The number of times frames had to be ignored because UNIX streams buffers we depleted. Running netstat -m will provide information about what sizes of buffers are low (for tuning purposes).

Aborts rx'd

Number of abort sequences (strings of 7 or more contiguous 1 bits) received. The occasional abort is OK, but there's a problem if this value is greater than 10% of good frames received. Check for a problem with the Received Data signal or with the remote transmitter.

overrun count

Number of frames tossed out because characters weren't read into memory as fast as they arrived. Usually indicates that you are running the NetcomHighway link above its maximum rated speed.

underrun count

Number of frames unsuccessfully transmitted because characters were not supplied to the serial communications interface fast enough to maintain synchronization. Significant values (10% of frames transmitted OK), indicate Netcom is overloaded, possibly due to one or both links being clocked at higher than rated speeds.

CRC errors / FCS errors

Number of frames received with bad FCS (Frame Check Sequence - Cyclic Redundancy Check bytes received don't match the values calculated from the data received). Indicates a noisy data link, or problems with the remote transmitter.

Frames < 2 bytes

Number of frames that were too short (less than 2 bytes long). Indicates a noisy data link, or problems with the remote transmitter.

CTS / DSR / DCD

Current state of the CTS (Clear To Send), DSR (Data Set Ready) and DCD (Data Carrier Detect) signals.

CTS changes

Number of times the CTS signal changed. A rapidly increasing count indicates trouble with the modem or cable.

DSR changes

Number of times the Data Set Ready pin toggles. Rapid incrementing indicates a problem with the modem or the cable between the modem and the communications controller.

DCD changes

Number of times the Data Carrier Detect pin interface) toggles. Rapid incrementing indicates a problem with the data link or the modem.

Frame Relay Statistics

Statistics are gathered on a per Network ID and on a per DLCI basis.

For Frame Relay networks, here is a sample of the per network statistical display:

 
Frame Relay Statistical Information for Network ID 1

         Received    Frames:         137894        Bytes:      58433325
                     FECN set:            9        BECN set:         36
                     DE set:         137894
                     Mgmt. DLCI:      69477
                     Unknown DLCI:        0
                     Invalid DLCI:        0
         Transmitted Frames:         103432        Bytes:       8601100
                     DE set:              0
                     Mgmt. DLCI:      69477
         Active DLCIs:                    1
         Inability to transmit:           0
         Errors:     Type:         No Error Since Reset

                     Current Time:  Wed Oct 12 14:22:14 1994
                     Error Time:    Tue Oct  4 13:22:18 1994
                     Data:                0 bytes

The counts represented in this display are for a specific Network ID. In order to obtain counts on a per DLCI basis use tsgstat -d DLCI.

Field

Description

Frames

Number of frames which have been processed since system startup or reset.

Bytes

Number of bytes which have been processed since system startup or reset.

FECN set

Number of frames which had the FECN bit set in their frame relay headers.

BECN set

Number of frames which had the BECN bit set in their frame relay headers.

DE set

Number of frames which had the DE bit set in their frame relay headers.

Mgmt. DLCI

Number of frames used by the PVC Management System.

Unknown DLCI

Number of frames received on an unknown or un-configured DLCI.

Invalid DLCI

Number of frames received on an invalid DLCI. An invalid DLCI is one which does not fall in the range of possible DLCIs. For example, DLCI 1 is invalid.

Active DLCIs

Number of currently available DLCIs on this Network ID.

Inability to Transmit

Number of problems encountered at the Hardware Device Driver when attempting to send a frame.

Errors

The first line contains a general error message equivalent to the SNMP error code. If more information is available, the second line will contain more information on the type of error encountered. The erroneous frame will be stored for as long as there are no other errors encountered and is displayed on the last line.

Using tsgtrace

The Wanware tsgtrace utility allows you to see information exchanged between the remote and local systems in real-time, as they are being transmitted. The program's options are described fully in the corresponding man pages.

You can use tsgtrace to capture information for transmission to protocol support groups - either ourselves or your network provider.

To run the trace for Frame Relay links, run:

 
tsgtrace -t | tee /tmp/WANtrace

This will put a trace in /tmp/WANtrace. If you do not need to store the information in a file, ignore the | tee ... part. You may also need to add a -n some_number to the above commands if you are tracing a network other than netid 0.

When you've captured the event that you think is causing the problem, press any key to terminate the program.

The options in the command line are described in the following table.

Option

Description

-I

Do not interpret IP protocol headers including IP addresses and upper protocol types.

-t

Time stamp each frame as it is shown.

-a

Do not interpret the information it receives (other than protocol headers) as ASCII text, using a period (".") to represent unprintable characters. Rather, display the data in hexadecimal form only.

-e

Interpret the information it receives (other than protocol headers) as EBCDIC text, using a period (".") to represent unprintable characters.

-s80

Show up to 80 bytes per frame. This is necessary if you wish to see a large amount of the data in the IP (and upper) headers or user data.

Revision 6.3.0 (April 2004)

Field	Description
`Frames received OK`	Frames received which do not have a CRC error associated with them. Rapid incrementing indicates an active receive data path.
`# bytes received OK`	Simply a count of the number of data bytes (protocol headers plus data) received. The averages size of a frame is the #bytes received OK divided by the frames received OK.
`Frames transmitted ok`	Number of frames the transmitter has sent.
`# bytes sent to txISR / bytes transmitted OK`	Total characters sent on the outbound leg, excluding flags and CRCs.
`Lack ipb count`	Number of incoming frames that had to be ignored because the buffer pool was depleted. If this count is being incremented frequently, it is most likely that your configuration and/or traffic levels are overloading Wanware in some way. You may wish to consult your supplier for tuning assistance.
`no buffer available`	The number of times frames had to be ignored because UNIX streams buffers we depleted. Running netstat -m will provide information about what sizes of buffers are low (for tuning purposes).
`Aborts rx'd`	Number of abort sequences (strings of 7 or more contiguous 1 bits) received. The occasional abort is OK, but there's a problem if this value is greater than 10% of good frames received. Check for a problem with the Received Data signal or with the remote transmitter.
`overrun count`	Number of frames tossed out because characters weren't read into memory as fast as they arrived. Usually indicates that you are running the NetcomHighway link above its maximum rated speed.
`underrun count`	Number of frames unsuccessfully transmitted because characters were not supplied to the serial communications interface fast enough to maintain synchronization. Significant values (10% of frames transmitted OK), indicate Netcom is overloaded, possibly due to one or both links being clocked at higher than rated speeds.
`CRC errors / FCS errors`	Number of frames received with bad FCS (Frame Check Sequence - Cyclic Redundancy Check bytes received don't match the values calculated from the data received). Indicates a noisy data link, or problems with the remote transmitter.
`Frames < 2 bytes`	Number of frames that were too short (less than 2 bytes long). Indicates a noisy data link, or problems with the remote transmitter.
`CTS / DSR / DCD`	Current state of the CTS (Clear To Send), DSR (Data Set Ready) and DCD (Data Carrier Detect) signals.
`CTS changes`	Number of times the CTS signal changed. A rapidly increasing count indicates trouble with the modem or cable.
`DSR changes`	Number of times the Data Set Ready pin toggles. Rapid incrementing indicates a problem with the modem or the cable between the modem and the communications controller.
`DCD changes`	Number of times the Data Carrier Detect pin interface) toggles. Rapid incrementing indicates a problem with the data link or the modem.

Field	Description
`Frames`	Number of frames which have been processed since system startup or reset.
`Bytes`	Number of bytes which have been processed since system startup or reset.
`FECN set`	Number of frames which had the FECN bit set in their frame relay headers.
`BECN set`	Number of frames which had the BECN bit set in their frame relay headers.
`DE set`	Number of frames which had the DE bit set in their frame relay headers.
`Mgmt. DLCI`	Number of frames used by the PVC Management System.
`Unknown DLCI`	Number of frames received on an unknown or un-configured DLCI.
`Invalid DLCI`	Number of frames received on an invalid DLCI. An invalid DLCI is one which does not fall in the range of possible DLCIs. For example, DLCI 1 is invalid.
`Active DLCIs`	Number of currently available DLCIs on this Network ID.
`Inability to Transmit`	Number of problems encountered at the Hardware Device Driver when attempting to send a frame.
`Errors`	The first line contains a general error message equivalent to the SNMP error code. If more information is available, the second line will contain more information on the type of error encountered. The erroneous frame will be stored for as long as there are no other errors encountered and is displayed on the last line.

Option	Description
`-I`	Do not interpret IP protocol headers including IP addresses and upper protocol types.
`-t`	Time stamp each frame as it is shown.
`-a`	Do not interpret the information it receives (other than protocol headers) as ASCII text, using a period (".") to represent unprintable characters. Rather, display the data in hexadecimal form only.
`-e`	Interpret the information it receives (other than protocol headers) as EBCDIC text, using a period (".") to represent unprintable characters.
`-s80`	Show up to 80 bytes per frame. This is necessary if you wish to see a large amount of the data in the IP (and upper) headers or user data.