Académique Documents
Professionnel Documents
Culture Documents
Introduction
Since you probably use one or more of Mikrotik's routerboards and their RouterOS you,
as a network administrator, would probably be interested at one point what web sites your
network users visit. WebProxy Log can help you with that!
WebProxy Log package consist of WebProxy Log and WebProxy Log Catcher - intended
to receive, import, store Mikrotik's web-proxy log data and finally, generate reports from
those data. Both programs are unofficial (means not made by or supported by Mikrotik
company), freeware and made to use with Windows (the newer the better ;).
This module requires simple configuration and does the following: Listens on a certain IP
address/UDP port for log data which your pre-configured Mikrotik's logging environment
is sending. Once WebProxy Log Catcher 'catches' the data, it writes that data to a file on
your hard drive (to a log file).
WebProxy Log
WebProxy Log is a primary program which imports log files from the import location to
it's local database and can generate: surf logs (certain surfer's Internet traffic report
between two dates), top 10 visited servers pie chart, top 10 surfers pie chart and the 'Who
visited' report (which users visited certain server for how many times).
System Requirements:
You can see screenshots and download the package to try it out yourself.
If you wish you can discuss about this software at our discussion site.
Logging_environment
Learn to set up Logging environment
Deprecated
Updated Nov 5, 2010 by extensis
You'll need
Mikrotik configuration
First of all, if you are not using manual client-side proxy configuration, you'll need to set
up transparent proxy on your Mikrotik. You can learn here more about it.
When you're sure that clients will make traffic through proxy, you'll have to set up
logging in your Mikrotik. Either by using the console or WinBox:
• go to 'System/Logging/Actions'
• add new Action with following details:
• Name: WebProxyLog
• Type: remote
• Remote Address: enter here the IP address of the remote computer where
WebProxy Log Catcher will be running
• Remote Port: enter here UDP port of the computer where WebProxy Log
Catcher will be listening from the remote computer - usually for syslog 514
• now go to 'System/Logging/Rules'
• add new rule with following details:
• Topics: web-proxy, !debug
• Prefix: proxy
• Action: WebProxyLog (or whatever name you gave to the action)
Make sure that UDP port you specified for remote logging is open both in
Mikrotik's firewall and in remote computer's firewall!
That's it!
IP_Alias
Learn to configure IP aliases list
Deprecated
Updated Jun 25, 2010 by extensis
You'll need
• WebProxy Log
About
Since you don't want in your report to see IP addresses as users, you can open IP alias
configuration (menu 'Settings/IP aliases') and attach there an alias to a certain IP address.
Reports will then show names instead of IP addresses.
IP addresses are automatically added to this list when importing a log file, or manually
added with the '+' button. Removing an alias is easy; select a field containing an IP
address or alias and click '-' button.
You can export IP aliases list to a CSV file by clicking 'Export', and import the same
CSV file by clicking 'Import'.
'Frob DB' button searches your database for missing IP addresses that are not in the list
and adds if any is found.
Whether or not you make any changes, if you don't click 'Apply' button, when closing the
IP aliases dialog, no changes will be made. Logs
Learn to work with logs
Deprecated
Updated Jun 26, 2010 by extensis
You'll need
Importing logs
All log files (any file that has 'log' extension) from pre-configured import location will
then be read by WebProxy Log, reformatted and stored in local database. Imported log
files, since they're stored in database, can be deleted or archived (recommended).
Deleting records
When you press the 'Delete' button, you are prompted to confirm deletion of n records
found in the database.
When deleting records, you can backup database (this is done BEFORE actual deletion)
and to optimize database after deletion.
Optimization is a process which cleans your database of unneeded records (the deleted
ones) and which defragments the database (can take while on large databases).
You'll need
• WebProxy Log
• imported logs
Once you have imported logs and attached IP aliases, you are ready to generate some
report.
This report shows the list of visited websites for selected user within selected dates.
To generate this report, first select user from a user drop-down list in the main window,
select from and to dates (this can be done by clicking-and-dragging on the calendar),
select some filters (if you want), and click 'Generate' button.
Print and Save is available for 'Internet traffic' and 'Who visited...' reports only!
This report shows a list of which users visited a certain page for how many times.
To generate this report go to 'Reports/Who visited...', in the dialog enter what page are
you interested for (eg. '*google.com') and click OK.
Top 10 reports
This reports show a pie-charts that represent how many times each server has been visited
or which user has visited most pages.
For 'Top 10 servers' report you can use Site filter (eg. filter out unwanted servers in your
report).
Backup database
'Backup database' option can be activated at some points when you're uncertain of some
actions (eg. when deleting records). When 'Backup database' option is selected, a copy of
your database file will be made in WebProxy Log's configuration directory.
Optimize
Optimization is a process which cleans your database of unneeded records (the deleted
ones) and which defragments the database (can take a while on large databases).
Insecure import
'Insecure import' is actually not so insecure and can't lead to database corruption. It's only
a different approach when writing data to the database.
When you have this option turned off, WebProxy Log will wait for database to finish
writing each line of data and then send new line of data to be written. That is not such a
big deal if you have small log files (up to 2 MB daily), but on large log files (above 5 MB
daily) the import process takes a log time to finish.
Therefore you can use 'insecure import' so WebProxy Log will send all the lines of data
to the database without waiting each of them to be actually written. This 'insecure import'
drastically speeds up the import process.