|
Logging
Logging
Overview
The
Common Log File - logging so you can analyze user access
Logging
for Debugging
Other Log Files
Troubleshooting
Resources
Web
Crossing provides a number of log files and settings which control
their operation. Using Web Crossing's logging features you can
discover problems in network activity, analyze the kinds of users
who visit your site and keep track of site traffic, track the
flow of mail and NNTP messages in and out of your system and generate
useful debugging information to help isolate speed or other problems
that may occur with any large serving system
Some
of the log files are controlled by settings inside the Control
Panel and some are automatically generated as part of the standard
operation of Web Crossing. The various log file settings are not
all in one place, so we need to go through through the different
kinds of log files, what they are used for and where they are
controlled by the sysop.
Common
Log
Perhaps
the most familiar log file for web server administrators is the
so called Common Log format file used by almost all web
servers. The Common Log format is an ordinary text file and can
be found in the common.log file in your WebX directory.
Each line of common.log looks like the following sample,
taken from WebX Harbor:
210.226.166.197 - doug [22/May/2000:17:10:01
+0900] "POST /cgi-bin/harbor/admin/Logging HTTP/1.0" 200 10732
200 19251 195.173.201.1 - Smith%20James [22/May/2000:17:10:32
+0900] "GET /cgi-bin/harbor HTTP/1.1" 200 14667
195.173.201.1 - Smith%20James [22/May/2000:17:10:59 +0900] "GET
/cgi-bin/harbor/view/ShowInContext/WebX_Harbor_Institute_of_Technology_(WIT)
HTTP/1.1" 200 17649
Each
line is quite long, so we have inserted a blank line between records
in the above listing, just to make it clearer. Your own common.log
file will not have blank lines between each record.
The
information provided in each record is:
- The
requesting IP address.
- The
authenticated user name, if available.
- The
date and time of the access.
- The
action taken (form submission, URL visited, etc.)
- The
HTTP
Status code for the request (in the above example, 200 means
"OK - the request was fulfilled").
- The
number of bytes transferred during this request.
There
is also what is known as an "extended common log format"
that provides the referrer (the IP address which contained
the link to your site) and the agent (the type of browser
used for the access), in addition to the above data. You can check
a box in the Control Panel > General
Settings to include those two items in your common.log
file as shown in Figure 1.
Figure
1 - Other Common Log Settings
You
can set the common.log file name to whatever you like.
Also, the second checkbox lets you create daily common.log
files automatically. This is convenient for backup and removal
because this file can grow to be quite large.
In
order to have Web Crossing generate a common.log file you
must be using Direct Web Service.
If you are running as a CGI under
another web server, such as Apache, you must use the Common Log
features of your web server.
After
you have confirmed that you are using Direct Web Service, you
can turn on Common Log logging in the Control
Panel. The setting is located near the bottom of the Control
Panel, where all the logging controls are gathered in one
place.
|
Note:
In previous versions of Web Crossing, all the logging
controls were scattered in different places, inside different
Control Panel sub-menus. Only since the spring of 2000
have the different log settings been organized better
and placed in one location at the bottom of the main Control
Panel. If you are using an earlier version of Web Crossing
we urge you to upgrade - it is free if you're upgrading
from 3.x! If you cannot upgrade yet, you can find your
Common Log settings inside the Direct Web Service Control
Panel.
|
Figure
2
shows the Log Settings section in the Control Panel.
figure
2 - Log Settings in the Control Panel
To
turn on Common Log format logging just
- Check
the Common Log checkbox and
- Click
the Install New Log Settings button
That's
all there is to it. Web Crossing will then start logging all incoming
requests and you can view the common.log file whenever
you wish.
|
Be
careful!
The common.log file, like some other log files you can
turn on in the Control Panel, grows in size fairly quickly,
depending on how many accesses you have. If you are not
careful, your log file can easily grow to be larger than
your webx.db database file itself. So be sure to periodically
delete or move the file elsewhere.
|
Debugging
In
addition to the Common Log format, there are other logging options
you can turn on and off in the Control Panel,
as shown in figure 1, above.
Log
NNTP/HTTP requests that a log file called nntp.log
be created in the WebX directory. The contents are a log of news
message activity. This includes all NNTP transfers made between
Web Crossing and a news server,
and also between Web Crossing and user newsreaders.
|
Note:
There is still some repositioning of some log file settings
taking place. For example, you can currently specify the
NNTP log file name in the NNTP Control Panel. And the"Other
Common Log Settings" shown in Figure 1 above are currently
found in the General Settings Control Panel. Both of these
scheduled to be moved to the top-level Control Panel along
with the other log settings.
|
The
next checkbox, located next to Web Crossing Log File, is
the master switch for all the logging features that follow in
the settings box. These features are mainly used for debugging
purposes:
- Log
Requests - logs all incoming client requests and provides
much the same information as the common.log file described
above, but in a different, nonstandard format.
- Log
Post Data - logs all the data resulting from posts made
via Web Crossing forms.
- Log
User Authentication - logs user login activity.
|
Security
Caution!
If you log User Authentication the log file generated
will contain the users' passwords in unencrypted
format. These will be the actual passwords - not the "pretty
encoded" format used by SGML export of user data
elsewhere. Anybody who has access to the log files generated
with this feature on will be able to read all the passwords
of anybody who logged in while the log file was being
generated. Obviously this feature only be used for testing
purposes and the sysop should immediately remove the log
file after it is no longer needed. (This applies to the
TCP/IP data logging below as well).
|
- Log
Per-request timestamps - logs detailed date and time information
for each request, also similar to the information provided by
the common.log file.
- Log
TCP/IP sessions, Log TCP/IP data and Log Master/Slave
synchronization process - these settings generate an enormous
amount of data which track server network activity in excruciating
detail. Do not turn these on unless you need to do some debugging
for a short period of time. Web Crossing support might ask you
to generate a log file containing these data if you have an
issue that requires a detailed analysis of your network activity.
If you need to log TCP/IP data you can also specify a TCP/IP
Service Port to limit the data to a specific service, such
as SMTP or NNTP. This can be very useful in isolating problems.
To
generate a debugging log, select the log options you need, check
the master switch and click the Install New Log Settings
button to begin logging. When you are finished generating your
log, uncheck the master switch and click the Install New Log
Settings button to stop logging and flush the data to the
log file. (You can leave the options selected; when logging is
off the option settings are ignored.)
If
you look in your WebX directory you will find a new log file has
been created. The name of the file is log1, log2,
log3, etc. Web Crossing keeps track of the last log file
created and creates the next log file by appending a new number
at the end of the filename.
Other
Log Files
The
other log files you will find in your WebX directory are:
- webx.log
- This file is always generated by Web Crossing and contains
a record of server starts and stops, server errors and also,
very usefully, WCTL compilation
errors. It is worthwhile checking this file regularly to see
if a server error has occurred.
- webxroll.1,
webxroll.2, etc. - These are roll-forward logs and contain
a list of all webx.db database changes since the last
backup took place.
- logRoll
- This file contains a record of how the roll-forward logs have
been used.
- logUsage
- This file contains a daily record of overall Web Crossing
activity. The fields in the file are more or less self-explanatory,
but just for the sake of completeness they are:
(sample data from a typical day at WebX Harbor)
| DAY
year=2000 month=4 day=10 |
Date
of the log entry |
| StatTotalPageViews=394 |
Number
of total page views |
| StatHttpOut=1501 |
Number
of static pages served |
| StatHttpOutEnc=0 |
Number
of attachments served via HTTP |
| StatHttpOutEncKB=0 |
Total
size (in bytes) of attachments served via HTTP |
| StatWebxOut=393 |
Number
of dynamic (WebX) pages served |
| StatWebxOutEnc=0 |
Number
of total attachments served |
| StatWebxOutEncKB=0 |
Total
size of all attachments served |
StatWebxInEnc=0
|
Number
of all attachments received |
| StatWebxInEncKB=0 |
Total
size of all attachments received |
StatNntpOut=0
|
Number
of NNTP messages sent |
| StatNntpOutEnc=0 |
Number
of attachments served via NNTP |
| StatNntpOutEncKB=0 |
Total
size of all attachments
served via NNTP |
| StatNntpIn=8 |
Number
of NNTP messages received |
| StatNntpInEnc=0 |
Number
of NNTP attachments
received |
| StatNntpInEncKB=0 |
Total
size of all attachments received via NNTP |
| StatEmailOut=17 |
Number
of mail messages sent out |
| StatEmailOutEnc=0 |
Number
of attachments sent out via email |
| StatEmailOutEncKB=0 |
Total
size of attachments
sent out via email |
| StatEmailIn=0 |
Number
of mail messages received |
| StatEmailInEnc=0 |
Number
of attachments
received via email |
| StatEmailInEncKB=0 |
Total
size of attachments received via email |
| StatChatMessage=6 |
Number
of chat message sent |
| StatChatEcho=4
|
Number
of chat messages echoes back after being sent |
- logEmail
- This very important file contains a record of all email coming
in and out of Web Crossing, including failed attempts to check
POP3 accounts. You should definitely check this file from time
to time so see if your system is experiencing any mail routing
difficulties.
- chat
logs - You can log chat activity. These settings are explained
in the chat section.
Troubleshooting
- It's
all well and good that Web Crossing generates mountains of common.log
data but... it's not very readable is it?
- That
is an understatement. The log files can be scanned from time
to time to look for trouble, but you should use a common log
analyzing tool, such as Analog,
in order to generate nice HTML reports from your common.log
data.
Resources
Control
Panel
Recommended
Websites
|
