| SHARE: |
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
██ ██
█▌ - LOGZILLA FORMERLY PHP-SYSLOG-NG - █▌
█▌ █▌
█ ▐▌
█ UPDATE: LogZilla is now PAID version only; offering a '30 day demo'. For ▐▌
█ this reason I am removing this article from the menu 'LOGS - MIX OSes', ▐▌
█ I'll leave it online for reference only. ▐▌
█ When I first starting using it the demo would be a one year license after ▐▌
█ which you would download a new version with another one year license or ▐▌
█ there about. ▐▌
█ ▐▌
█ ___________..___________ ▐▌
█ ▐▌
█ LogZilla is used to integrate syslog-ng messages (syslog-ng tutorial here) ▐▌
█ into a beautiful web interface - demo HERE login/pass is demo/demo. ▐▌
█ LogZilla site is HERE and the forums are HERE with excellent support from ▐▌
█ the author and friends. ▐▌
█ ▐▌
█ There are install instructions HERE that are easy to follow. ▐▌
█ ▐▌
█ I decided to write my own mewbies tutorial so that we cover all the ▐▌
█ prerequisites steps first so very little, if any, trouble shooting after. ▐▌
█ There are quite a few things to do, but how nice when you do open your ▐▌
█ LogZilla page for the first time and it all works smoothly :D . This is ▐▌
█ how I did it on my Debian. ▐▌
█ ▐▌
█ PREREQUISITES: ▐▌
█ INSTALL LOGZILLA: ▐▌
█ SPHINX: ▐▌
█ LOG REPLAY VIA CONSOLE: ▐▌
█ TROUBLE SHOOTING: ▐▌
█ MY MYSQL MISC NOTES: ▐▌
█ ▐▌
█ ___________..___________ ▐▌
█ ▐▌
█ PREREQUISITES: ▐▌
█ `````````````` ▐▌
█ Apache ▐▌
█ MySQL 5.1 or higher ▐▌
█ PHP 4.0.5 or higher with ionCube ▐▌
█ syslog-ng ▐▌
█ build-essential ▐▌
█ gcc ▐▌
█ libapache2-mod-php5 ▐▌
█ libmysqlclient15-dev ▐▌
█ libmysqlclient16 ▐▌
█ msttcorefonts ▐▌
█ php5-cli ▐▌
█ php5-gd ▐▌
█ php5-mysql ▐▌
█ perl ▐▌
█ And perl modules: ▐▌
█ Date::Calc (libdate-calc-perl) ▐▌
█ String::CRC32 (libstring-crc32-per) ▐▌
█ Text::LevenshteinXS ▐▌
█ ▐▌
█ To check if you have those needed, except for Text::LevenshteinXS and ▐▌
█ ionCube - covered after: ▐▌
█ dpkg -l | grep -E 'apache|mysql-*5*|php|libapache*-mod-php' ▐▌
█ Output to look for would be similar to: ▐▌
█ ii apache2 2.2.9-10+lenny8 Apache HTTP Server metap ▐▌
█ ii libapache2-mod-php5 5.2.6.dfsg.1-1+lenny8 server-side, HTML-embedd ▐▌
█ ii libapache2-mod-python 3.3.1-7 Python-embedding module ▐▌
█ ii libapache2-reload-perl 0.10-2 Reload Perl modules when ▐▌
█ ii mysql-client-5.1 5.1.37-1~bpo50+1 MySQL database client ▐▌
█ ii mysql-server-5.1 5.1.37-1~bpo50+1 MySQL database server ▐▌
█ ii php5 5.2.6.dfsg.1-1+lenny8 server-side, HTML-embedd ▐▌
█ ii php5-cli 5.2.6.dfsg.1-1+lenny8 command-line interpreter ▐▌
█ ii php5-gd 5.2.6.dfsg.1-1+lenny8 GD module for php5 ▐▌
█ ii php5-mysql 5.2.6.dfsg.1-1+lenny8 MySQL module for php5 ▐▌
█ And: ▐▌
█ dpkg -l build-essential gcc libdate-calc-perl libmysqlclient15-dev \ ▐▌
█ libmysqlclient16 libstring-crc32-perl msttcorefonts perl syslog-ng ▐▌
█ ii build-essential 11.4 Informational list of ▐▌
█ ii gcc 4:4.3.2-2 The GNU C compiler ▐▌
█ ii libdate-calc-perl 5.4-5+b1 Perl library for ▐▌
█ ii libmysqlclient15-dev 5.0.51a-24+lenny4 MySQL database ▐▌
█ ii libmysqlclient16 5.1.37-1~bpo50+1 MySQL database client ▐▌
█ ii libstring-crc32-perl 1.4-2+b1 Perl interface for cycli ▐▌
█ ii msttcorefonts 2.7 transitional dummy ▐▌
█ ii perl 5.10.0-19lenny2 Larry Wall's Practical ▐▌
█ ii syslog-ng 2.0.9-4.1 Next generation logging ▐▌
█ ▐▌
█ PREREQUISITES INSTALL NOTES: ▐▌
█ -For MySQL 5.1 on Debian we'll do that last, and yes you must have 5.1 or ▐▌
█ higher. ▐▌
█ ▐▌
█ -Module Text::LevenshteinXS will be installed using 'cpan' - shown how ▐▌
█ after. ▐▌
█ ▐▌
█ INSTALL: ▐▌
█ Add/change the aptitude install list with the programs you need. For ▐▌
█ example, the items I needed to install: ▐▌
█ su ▐▌
█ aptitude update ▐▌
█ aptitude install libmysqlclient-dev libmysqlclient15-dev ▐▌
█ aptitude install libdate-calc-perl libstring-crc32-perl msttcorefonts ▐▌
█ *Note: selecting "libmysqlclient15-dev" instead of the ▐▌
█ virtual package "libmysqlclient-dev" ▐▌
█ ▐▌
█ INSTALL PERL MODULE Text::LevenshteinXS: ▐▌
█ To view what perl modules you have installed: ▐▌
█ instmodsh ▐▌
█ l ▐▌
█ q ▐▌
█ Example output: ▐▌
█ Installed modules are: ▐▌
█ Crypt::ircBlowfish ▐▌
█ POE::Component::Client::FTP ▐▌
█ POE::Filter::Ls ▐▌
█ Perl ▐▌
█ ▐▌
█ To install the perl module Text::LevenshteinXS you need to use cpan (it's ▐▌
█ not on apt-get or aptitude yet)- ▐▌
█ cpan -i Text::LevenshteinXS ▐▌
█ Hit enter key to all questions to select its default, then when you come ▐▌
█ to: ▐▌
█ Select your continent (or several nearby continents) [] ▐▌
█ You need to enter a number from the list presented and a few more location ▐▌
█ selections. ▐▌
█ Once it begins to install the output will be similar to: ▐▌
█ [snip] ▐▌
█ Installing ▐▌
█ /usr/local/lib/perl/5.10.0/auto/Text/LevenshteinXS/LevenshteinXS.bs ▐▌
█ [snip] ▐▌
█ JGOLDBERG/Text-LevenshteinXS-0.03.tar.gz ▐▌
█ make install -- OK ▐▌
█ Warning (usually harmless): 'YAML' not installed, will not store ▐▌
█ persistent state ▐▌
█ ▐▌
█ Side note: Another method to view your installed modules with install ▐▌
█ dates and paths: ▐▌
█ perldoc perllocal ▐▌
█ q ▐▌
█ ▐▌
█ INSTALL IONCUBE FOR PHP: ▐▌
█ View my 'IONCUBE & ZEND ENCODED PHP FILES' tutorial under 'WEB SERVERS' ▐▌
█ HERE how to install ionCube. Scroll down to 'PREREQUISITES FOR IONCUBE ▐▌
█ ENCODED FILES:' ▐▌
█ After you have confirmed you have ionCube, return here and continue. ▐▌
█ ▐▌
█ If you are using PHP5.3 view HERE. ▐▌
█ ▐▌
█ DEBIAN MYSQL 5.0 UPGRADE TO 5.1: ▐▌
█ This is how I did it successfully and easily for my setup which installed ▐▌
█ 5.1.37-1~bpo50+1. ▐▌
█ For 5.1.48-1 and others view HERE. ▐▌
█ ▐▌
█ 1. To upgrade MySQL from 5.0.x to 5.1 on Debian successfully you'll need ▐▌
█ to comment out a line on your 'my.cnf' file - normally in your home dir or ▐▌
█ /etc/my.cnf or /etc/mysql/my.cnf ▐▌
█ I didn't have this in my home, nor that line in my /etc/mysql/my.cnf ▐▌
█ If you have this line comment it out (place a # in front of the line) or ▐▌
█ just remove the line: skip-bdb ▐▌
█ There might be other issues related to upgrading MySQL so I recommend you ▐▌
█ first read up before proceeding. ▐▌
█ ▐▌
█ 2. At the date of writing this you can not use apt-get install ▐▌
█ mysql-server to install 5.1. ▐▌
█ ▐▌
█ You'll need to add sources.list the lenny-backports: ▐▌
█ pico /etc/apt/sources.list ▐▌
█ Add this at the bottom of the file: ▐▌
█ deb http://people.debian.org/~nobse/mysql-dfsg-5.1/ lenny-backports main ▐▌
█ ▐▌
█ 3. Then proceed with the upgrade: ▐▌
█ aptitude update ▐▌
█ aptitude -P -t lenny-backports install mysql-server-5.1 ▐▌
█ Answer Yes to all questions. ▐▌
█ ▐▌
█ 4. After the upgrade is finished you need to upgrade your databases. ▐▌
█ First: ▐▌
█ chmod 666 /var/lib/mysql/mysql_upgrade_info ▐▌
█ Or the upgrade will give the error "Could not create the upgrade info ▐▌
█ file '/var/lib/mysql/mysql_upgrade_info' in the MySQL Servers datadir, ▐▌
█ errno: 13" ▐▌
█ Then run the upgrade: ▐▌
█ mysql_upgrade -u root -p --verbose --force ▐▌
█ Output might be similar to: ▐▌
█ mysql.general_log ▐▌
█ Error : You can't use locks with log tables. ▐▌
█ status : OK ▐▌
█ mysql.slow_log ▐▌
█ Error : You can't use locks with log tables. ▐▌
█ status : OK ▐▌
█ [snip] ▐▌
█ Running 'mysql_fix_privilege_tables'... ▐▌
█ OK ▐▌
█ ▐▌
█ These lock errors you can ignore; there are more of a message and ▐▌
█ supposedly will be removed in future versions, read HERE. ▐▌
█ ▐▌
█ Restart mysql: ▐▌
█ /etc/init.d/mysql restart ▐▌
█ ▐▌
█ MYSQL SETTINGS: ▐▌
█ For this version you need to increase the thread_stack, read HERE. These ▐▌
█ errors would be present in /var/log/daemon.log "mysqld: ... Thread stack ▐▌
█ overrun: 3971 bytes used of a 131072 byte stack, and 128000 bytes needed. ▐▌
█ Use 'mysqld -O thread_stack=#' to specify a bigger stack" ▐▌
█ ▐▌
█ To fix this change the setting in 'my.cnf' /etc/mysql/my.cnf or yours ▐▌
█ might be here /etc/my.cnf: ▐▌
█ pico /etc/mysql/my.cnf ▐▌
█ Has this: ▐▌
█ thread_stack = 128K ▐▌
█ Change to: ▐▌
█ thread_stack = 256K ▐▌
█ ▐▌
█ Next in my.cnf event scheduler must be set to ON ▐▌
█ Check if you have the code 'event_scheduler=' first, if not add it toward ▐▌
█ the bottom of the file (mine didn't have this directive at all): ▐▌
█ event_scheduler=ON ▐▌
█ ▐▌
█ Save your changes ▐▌
█ ▐▌
█ Then turn on even scheduler in MySQL: ▐▌
█ mysql -u root -p ▐▌
█ SELECT @@event_scheduler; ▐▌
█ ▐▌
█ My output stated off: ▐▌
█ +-------------------+ ▐▌
█ | @@event_scheduler | ▐▌
█ +-------------------+ ▐▌
█ | OFF | ▐▌
█ +-------------------+ ▐▌
█ 1 row in set (0.00 sec) ▐▌
█ ▐▌
█ To turn it on: ▐▌
█ SET GLOBAL event_scheduler = 1; ▐▌
█ ▐▌
█ Output then stated: ▐▌
█ Query OK, 0 rows affected (0.00 sec) ▐▌
█ SELECT @@event_scheduler; ▐▌
█ Output now states: ▐▌
█ +-------------------+ ▐▌
█ | @@event_scheduler | ▐▌
█ +-------------------+ ▐▌
█ | ON | ▐▌
█ +-------------------+ ▐▌
█ 1 row in set (0.00 sec) ▐▌
█ ▐▌
█ exit; ▐▌
█ ▐▌
█ Then restart MySQL: ▐▌
█ /etc/init.d/syslog-ng restart ▐▌
█ ▐▌
█ Note: If your event scheduler isn't on then during installation of ▐▌
█ LogZilla you'll receive this error: ▐▌
█ DBD::mysql::st execute failed: Cannot proceed because system tables used ▐▌
█ by Event Scheduler were found damaged at server start at ./install.pl line ▐▌
█ 411, <STDIN> line 17. ▐▌
█ ▐▌
█ SSYSLOG-NG SETTINGS: ▐▌
█ pico /etc/syslog-ng/syslog-ng.conf ▐▌
█ Search for (Ctrl+w): udp(); ▐▌
█ If it is commented out: ▐▌
█ # udp(); ▐▌
█ Remove the comment; change to: ▐▌
█ udp(); ▐▌
█ ▐▌
█ Then restart syslog-ng: ▐▌
█ /etc/init.d/syslog-ng restart ▐▌
█ ▐▌
█ PHP SETTINGS: ▐▌
█ We also need to have set for PHP a 'maximum execution time' of a minimum ▐▌
█ of 300 seconds and the 'memory limit' of at least 128MB : ▐▌
█ pico /etc/php5/apache2/php.ini ▐▌
█ Search for (Ctrl+w): max_execution_time ▐▌
█ Mine has (my execution time is already set very high for other reasons): ▐▌
█ max_execution_time = 9600 ▐▌
█ memory_limit = -1 ▐▌
█ I changed it to: ▐▌
█ max_execution_time = 9600 ▐▌
█ memory_limit = 128M ▐▌
█ ▐▌
█ Restart web server: ▐▌
█ /etc/init.d/apache2 restart ▐▌
█ exit ▐▌
█ ▐▌
█ INSTALL LOGZILLA: ▐▌
█ ````````````````` ▐▌
█ Finally, we are ready to start working on LogZilla installation. ▐▌
█ Check the site for the latest version. For this tutorial I've used 3.0.85: ▐▌
█ wget http://php-syslog-ng.googlecode.com/files/logzilla_3.0.85.tgz ▐▌
█ su ▐▌
█ cp /home/user/logzilla_3.0.85.tgz /var/www/logzilla_3.0.85.tgz ▐▌
█ cd /var/www/ ▐▌
█ tar xvzf logzilla_3.0.85.tgz ▐▌
█ ▐▌
█ My own preference when dealing with sensitive information on web servers ▐▌
█ is to rename the default install path. So for logzilla I'll rename it to ▐▌
█ for example 'lz' and I'll refer to path 'logzilla' as lz for the rest of ▐▌
█ this tutorial: ▐▌
█ mv /var/www/logzilla/ /var/www/lz/ ▐▌
█ rm logzilla_3.0.85.tgz ▐▌
█ cat /var/www/lz/README ▐▌
█ ▐▌
█ Note about license key, read HERE & HERE, "Copy will simply stop working ▐▌
█ (the web interface portion). At that point, you can either download the ▐▌
█ new version or a new license key." ▐▌
█ cat /var/www/lz/license.txt ▐▌
█ Expires: 12 Sep 2010 ▐▌
█ ▐▌
█ For LogZilla v3.0.85 only, remove the facebox call as it is not being used ▐▌
█ from this v + as stated HERE: ▐▌
█ pico /var/www/lz/html/includes/css.php ▐▌
█ Remove this section: ▐▌
█ <!-- BEGIN Facebox --> ▐▌
█ <link type="text/css" rel="stylesheet" ▐▌
█ href="includes/js/jquery/plugins/facebox/facebox.css" /> ▐▌
█ <!-- END Facebox --> ▐▌
█ ▐▌
█ PRE INSTALL NOTES: ▐▌
█ A. LogZilla comes with an installer- perl script. You'll be asked a number ▐▌
█ of questions, to select the default answer in [brackets] just hit your ▐▌
█ enter key, otherwise type in your own setting. ▐▌
█ ▐▌
█ B. Below are the questions you'll be asked and the answers I used; if I ▐▌
█ haven't typed an answer for the question it's because I hit enter key to ▐▌
█ select the answer provided in [brackets]. ▐▌
█ ▐▌
█ C. The 3rd from last question "Enter the base url for your site (include ▐▌
█ trailing slash) [/logs/]" - This path is what you will use in your apache ▐▌
█ settings as an 'alias'. This means that if your answer is /logs/ your URL ▐▌
█ to go to /var/www/logzilla/ would be http://yoursite.com/logs/ . Another ▐▌
█ example would be the answer / which then logzilla install would be reached ▐▌
█ by going to http://yoursite.com ▐▌
█ ▐▌
█ D. If you want to change the answer to one of your questions in the first ▐▌
█ section 'before the install is complete', just quit the installer and ▐▌
█ start over. ▐▌
█ If you need to change anything in the install once it is finished see ▐▌
█ under 'TROUBLE SHOOTING' - 'REINSTALL LOGZILLA FRESH'. ▐▌
█ ▐▌
█ INSTALL: ▐▌
█ OK lets do it: ▐▌
█ First make a copy of your 'syslog-ng.conf' just in case you want to ▐▌
█ install LogZilla fresh: ▐▌
█ cp /etc/syslog-ng/syslog-ng.conf /etc/syslog-ng/syslog-ng-pre_logzilla.conf▐▌
█ ▐▌
█ cd /var/www/lz/scripts/ ▐▌
█ Run the installer: ▐▌
█ ./install.pl ▐▌
█ ▐▌
█ Enter the MySQL root username [root]: ▐▌
█ Note: Mysql passwords with a ' in them may not work ' ▐▌
█ Enter the password for root [mysql]: MyRootMysqlPass ▐▌
█ Database to install to [syslog]: lz ▐▌
█ Database table to install to [logs]: ▐▌
█ Enter the name of the MySQL server [127.0.0.1]: localhost ▐▌
█ Enter the port of the MySQL server [3306]: ▐▌
█ Enter the name to create as the owner of the logs database [syslogadmin]: ▐▌
█ Enter the password for the syslogadmin user [syslogadmin]: SysLogPass ▐▌
█ Enter the name to create as the WEBSITE owner [admin]: ▐▌
█ Enter the password for admin [admin]: AdminPass ▐▌
█ Enter your email address [cdukes@cdukes.com]: mine@email.com ▐▌
█ Enter a name for your website [The home of LogZilla]: LZS ▐▌
█ Enter the base url for your site (include trailing slash) [/logs/]: ▐▌
█ Where should log files be stored? [/var/log/logzilla]: ▐▌
█ How long should I keep old logs? (in days) [30]: ▐▌
█ ▐▌
█ Path Updates: ▐▌
█ Getting ready to replace paths in all files with "/var/www/lz" ▐▌
█ Ok to continue? [y]: ▐▌
█ Updating file paths ▐▌
█ Updating log paths ▐▌
█ ▐▌
█ Database Installation: ▐▌
█ All data will be installed into the lz database ▐▌
█ Ok to continue? [y]: ▐▌
█ ▐▌
█ Config.php generation: ▐▌
█ Generating /var/www/lz/html/config/config.php ▐▌
█ Ok to continue? [y]: ▐▌
█ ▐▌
█ System files: ▐▌
█ Adding LogZilla logrotate.d file to /etc/logrotate.d ▐▌
█ Ok to continue? [y]: ▐▌
█ Where is your syslog-ng.conf file located? ▐▌
█ [/etc/syslog-ng/syslog-ng.conf]: ▐▌
█ Adding syslog-ng configuration to /etc/syslog-ng/syslog-ng.conf ▐▌
█ Ok to continue? [y]: ▐▌
█ Found 1 sources ▐▌
█ Which source definition would you like to use? [s_all]: ▐▌
█ ▐▌
█ LogZilla installation complete... ▐▌
█ ▐▌
█ POST INSTALL NOTES: ▐▌
█ During the 'Config.php generation:' step during install it stated ▐▌
█ "Generating /var/www/logzilla/html/config/config.php" ▐▌
█ This will be removed after by the installer: ▐▌
█ ls -al /var/www/ ▐▌
█ ▐▌
█ LogZilla logs are rotated daily (by this file /etc/logrotate.d/logzilla) ▐▌
█ and the logs will be 0 bytes unless debugging is enabled: ▐▌
█ ls -al /var/log/logzilla ▐▌
█ ▐▌
█ TEST MYSQL LOGZILLA USER CAN LOGIN: ▐▌
█ Test that the new user, syslogadmin, you created during install can login, ▐▌
█ change 'lz' below to the name of your database (default syslog): ▐▌
█ mysql -u syslogadmin -p lz ▐▌
█ If you logged in fine: ▐▌
█ exit; ▐▌
█ If you receive the error "ERROR 1045 (28000): Access denied for user ▐▌
█ 'syslogadmin'@'localhost' (using password: YES)" ▐▌
█ To fix this, replace PASSWORD below to password you selected for ▐▌
█ syslogadmin and name of data base (lz) IF you changed it during the ▐▌
█ install: ▐▌
█ mysql -u root -p mysql ▐▌
█ GRANT ALL PRIVILEGES ON lz.* TO syslogadmin@localhost IDENTIFIED BY "PASSWORD";
█ FLUSH PRIVILEGES; ▐▌
█ exit; ▐▌
█ ▐▌
█ Test the user login again: ▐▌
█ mysql -u syslogadmin -p lz ▐▌
█ exit; ▐▌
█ ▐▌
█ ADD LOGZILLA TO WEB SERVER: ▐▌
█ pico /etc/apache2/apache2.conf ▐▌
█ First if you don't have your 'ServerName' and your ServerRoot' directives ▐▌
█ set you need to add it, at the top of the file after all the comments (#). ▐▌
█ For example, mine already has it: ▐▌
█ [snip] ▐▌
█ # Do NOT add a slash at the end of the directory path. ▐▌
█ # ▐▌
█ ServerName "mystite.com" ▐▌
█ ServerRoot "/etc/apache2" ▐▌
█ Options -Indexes FollowSymLinks Includes ExecCGI ▐▌
█ [snip] ▐▌
█ ▐▌
█ Then you need to create a vhost file LogZilla, name file how you like, ▐▌
█ again I use lz: ▐▌
█ pico /etc/apache2/sites-available/lz ▐▌
█ Paste this in, changing '/logs' to the path you used during installation ▐▌
█ for the alias and the path to your logzilla/html/: ▐▌
# LogZilla
Alias /logs "/var/www/lz/html/"
Options Indexes MultiViews FollowSymLinks
AllowOverride All
Order allow,deny
Allow from all
█ ▐▌
█ Enable it: ▐▌
█ a2ensite lz ▐▌
█ ▐▌
█ Reload apache: ▐▌
█ /etc/init.d/apache2 reload ▐▌
█ ▐▌
█ Now finally give LogZilla a test: http://yoursite.com/logs/login.php ▐▌
█ Login using admin username/password set during install.pl ▐▌
█ ▐▌
█ IONCUBE: ▐▌
█ Just to check all is fine with our ionCube installation, go to: ▐▌
█ http://yoursite.com/logs/ioncube/ ▐▌
█ If all is fine it will state: ▐▌
█ ionCube Loader Wizard ▐▌
█ The ionCube Loader 3.3.17 is already installed [snip] ▐▌
█ ▐▌
█ If all is well then press back button on your browser, then in your shell: ▐▌
█ rm /var/www/lz/html/ioncube -rf ▐▌
█ rm /var/www/lz/html/ioncube_64 -rf ▐▌
█ ▐▌
█ DEDUPLICATION: ▐▌
█ From the User Guide: "When Deduplication is enabled, "similar" messages ▐▌
█ are rolled up into a single message. When that happens, the FO, LO and ▐▌
█ Counter columns get updated." ▐▌
█ "If deduplication is disabled, the FO and LO columns are identical." ▐▌
█ So to enable this, on your LogZilla page go to Admin's Options/Server ▐▌
█ Admin, click on the blue button left column and select 1 to enable it. ▐▌
█ ▐▌
█ SPHINX: ▐▌
█ ``````` ▐▌
█ INSTALL: ▐▌
█ From the User Guide: "Sphinx is used to provide the full text search ▐▌
█ capabilities for LogZilla, it is an order of magnitude faster than using ▐▌
█ MySQL alone (much, much faster)." ▐▌
█ ▐▌
█ Note that during the installing process of LogZilla it will have updated ▐▌
█ the /sphinx/sphinx.conf file (Modifying ../sphinx/sphinx.conf) to your ▐▌
█ correct logzilla install paths so that you don't need to edit this file. ▐▌
█ To install Sphinx: ▐▌
█ cd /var/www/lz/sphinx/src ▐▌
█ tar xzvf sphinx-0.9.9.tar.gz ▐▌
█ cd sphinx-0.9.9 ▐▌
█ Compile it: ▐▌
█ ./configure --prefix `pwd`/../.. ▐▌
█ Output: A long list of checking etc will scroll by, then it will end with: ▐▌
█ You can now run 'make' to build Sphinx binaries, ▐▌
█ and then run 'make install' to install them. ▐▌
█ ▐▌
█ First check if there are any serious errors such as 'ERROR: cannot find ▐▌
█ MySQL include files.' ▐▌
█ If so it's most likely that you don't have all the prerequisites. ▐▌
█ If you did have a serious error don't continue to the next step, fix the ▐▌
█ error/s, and run the ./configure cmd again before proceeding. ▐▌
█ View HERE for my list of negatives 'no' the ./configure output gave to me ▐▌
█ and how I fixed those. None will hinder Sphinx for LogZilla, that I know ▐▌
█ of, only for other programs you might want to use Sphinx on. ▐▌
█ ▐▌
█ After you have the output you would like on the sphinx configure, run: ▐▌
█ make && make install ▐▌
█ Output - you'll have a long pause followed by a long install log: ▐▌
█ Making all in src ▐▌
█ make[1]: Entering directory `/var/www/lz/sphinx/src/sphinx-0.9.9/src' ▐▌
█ [snip] ▐▌
█ make[1]: Leaving directory `/var/www/lz/sphinx/src/sphinx-0.9.9' ▐▌
█ ▐▌
█ CREATE YOUR FIRST INDEX: ▐▌
█ cd /var/www/lz/sphinx ▐▌
█ ./indexer.sh full ▐▌
█ Output: ▐▌
█ Starting Sphinx Indexer: 2010-07-25 12:41:47 ▐▌
█ No previous index files found ▐▌
█ Creating NEW indexes, this may take a while, so be patient... ▐▌
█ Running command: /var/www/lz/sphinx/bin/indexer --config ▐▌
█ /var/www/lz/sphinx/sphinx.conf idx_logs idx_delta_logs ▐▌
█ Sphinx 0.9.9-release (r2117) ▐▌
█ [snip] ▐▌
█ Finished Sphinx Indexer: 2010-07-25 12:41:48 ▐▌
█ ▐▌
█ NOTE: If you see the error when creating the full index: ▐▌
█ "ERROR: index 'idx_logs': sql_query_pre[0]: Column 'max_id' cannot be null ▐▌
█ (DSN=mysql://syslogadmin:***@127.0.0.1:3306/syslog)." ▐▌
█ This means the database doesn't have your log entries yet, wait a bit then ▐▌
█ try again. ▐▌
█ ▐▌
█ Test the other indexer commands that we'll use on crontab to be sure no ▐▌
█ errors: ▐▌
█ ./indexer.sh delta ▐▌
█ ./indexer.sh merge ▐▌
█ ▐▌
█ START SPHINX SEARCH DAEMON: ▐▌
█ Start sphinx search daemon 'searchd': ▐▌
█ bin/searchd ▐▌
█ Output: ▐▌
█ [snip] ▐▌
█ listening on 127.0.0.1:9312 ▐▌
█ listening on all interfaces, port=3312 ▐▌
█ ▐▌
█ ps x ▐▌
█ 19613 ? S 0:00 bin/searchd ▐▌
█ After a reboot output will be similar to: ▐▌
█ S 0:03 /var/www/lz/sphinx/bin/searchd -c /var/www/lz/sphinx/sphinx.conf ▐▌
█ ▐▌
█ SPHINX INDEXER AUTO RUN: ▐▌
█ A crontab file is included in ▐▌
█ scripts/contrib/system_configs/logzilla.crontab. Read it to explain the ▐▌
█ settings: ▐▌
█ cat /var/www/lz/scripts/contrib/system_configs/logzilla.crontab ▐▌
█ Line 1: 12:30 AM on the first of every month it will run full ▐▌
█ Line 2: Every 5 mins of every day it will run delta ▐▌
█ Line 3: 12:00 AM every day it will run merge ▐▌
█ To add these: ▐▌
█ crontab -e ▐▌
█ Paste in at the bottom:, changing to your correct paths - 2 paths per line:▐▌
30 0 1 * * /var/www/lz/sphinx/indexer.sh full >> /var/log/logzilla/sphinx_indexer.log 2>&1
*/5 * * * * /var/www/lz/sphinx/indexer.sh delta >> /var/log/logzilla/sphinx_indexer.log 2>&1
0 0 * * * /var/www/lz/sphinx/indexer.sh merge >> /var/log/logzilla/sphinx_indexer.log 2>&1
█ ▐▌
█ Note: ▐▌
█ Before crontab runs those cmds you do not have the file ▐▌
█ 'sphinx_indexer.log', after it runs the first time (within 5 mins) it will ▐▌
█ create it: ▐▌
█ ls -al /var/log/logzilla ▐▌
█ BTW this log will be also be rotated by /etc/logrotate.d/logzilla as it ▐▌
█ declares *.log. ▐▌
█ ▐▌
█ SPHINX SEARCH DAEMON AUTO RUN AFTER REBOOT: ▐▌
█ pico /etc/rc.local ▐▌
█ Paste in just before the last line 'exit 0' changing the paths to your ▐▌
█ own: ▐▌
█ /var/www/lz/sphinx/bin/searchd -c /var/www/lz/sphinx/sphinx.conf ▐▌
█ ▐▌
█ SPHINX LOG ROTATION: ▐▌
█ ls -al /var/www/lz/sphinx/log ▐▌
█ I didn't find a log rotation setup for it, nor any other logs: ▐▌
█ grep -r sphinx* /etc/* ▐▌
█ find / -name *query.log ▐▌
█ ▐▌
█ So following the information HERE, I'll set up log rotation for those by ▐▌
█ doing this: ▐▌
█ Take note of the perms, owner and group of the logs as we'll use the same ▐▌
█ for our logrotate config. ▐▌
█ Mine are 600 root root (-rw------- 1 root root) ▐▌
█ Create the log rotation configuration file: ▐▌
█ pico /etc/logrotate.d/sphinx ▐▌
█ Paste in, changing to your correct paths and preferences: ▐▌
/var/www/lz/sphinx/log/*.log {
weekly
notifempty
rotate 3
compress
delaycompress
create 600 root root
postrotate
killall -SIGUSR1 searchd
endscript
}
█ ▐▌
█ To test, but NOT actually rotate sphinx logs: ▐▌
█ logrotate --debug --force /etc/logrotate.d/sphinx ▐▌
█ ▐▌
█ To rotate the logs manually: ▐▌
█ logrotate --force -v /etc/logrotate.d/sphinx ▐▌
█ ls -al /var/www/lz/sphinx/log ▐▌
█ ▐▌
█ Done :D Have a look at the User Guide HERE. ▐▌
█ For large servers read HERE how to increase your UDP buffer size. ▐▌
█ ▐▌
█ LOG REPLAY VIA CONSOLE: ▐▌
█ ``````````````````````` ▐▌
█ This will allow you to replay a log file taken from another server. ▐▌
█ Included is a sample log (syslog.sample.gz) for you test logreplay with ▐▌
█ it: ▐▌
█ cd /var/www/lz/scripts/contrib/logreplay ▐▌
█ This will extract the 3MB sample 'syslog.sample' and delete the pack ▐▌
█ (syslog.sample.gz): ▐▌
█ gzip -d syslog.sample.gz ▐▌
█ ▐▌
█ To use this we need to compile the included spoof program so that it "will ▐▌
█ rewrite the outgoing syslog packet and insert the hostnames from the ▐▌
█ syslog.sample file so that when syslog-ng receives the messages they ▐▌
█ appear to come from that host instead of your local machine.": ▐▌
█ gcc spoof_syslog.c -o spoof_syslog ▐▌
█ There shouldn't be any output/reply after running that cmd. ▐▌
█ Doing this created the binary spoof_syslog used by logreplay.pl ▐▌
█ ls -al ▐▌
█ -rwxr-xr-x 1 root root 9357 2010-07-25 08:13 spoof_syslog ▐▌
█ Set executable perms on logreplay.pl script: ▐▌
█ chmod +x logreplay.pl ▐▌
█ Run the scripts help menu: ▐▌
█ ./logreplay.pl -h ▐▌
█ Play the sample log: ▐▌
█ ./logreplay.pl -v -f ./syslog.sample -s ./spoof_syslog ▐▌
█ Output will end with: ▐▌
█ Host limit of 50 reached, use ./logreplay.pl -l to set a higher limit ▐▌
█ Sent 123 messages out ▐▌
█ ▐▌
█ TROUBLE SHOOTING: ▐▌
█ ````````````````` ▐▌
█ 1. Best place is the LogZilla forum HERE. ▐▌
█ ▐▌
█ 2. If you aren't sure the behaviour of your LogZilla page is correct I ▐▌
█ test it by doing the same search on the Demo HERE (demo:demo). ▐▌
█ ▐▌
█ 3. Turn on debugging via LogZilla page, top link Admin /Server Admin, ▐▌
█ page 2, click on the blue marble left column. Once this is on it will show ▐▌
█ the debug information on every page and start writing to the file ▐▌
█ /var/log/logzilla/db_insert.log . Check that file for errors: ▐▌
█ tail -50 /var/log/logzilla/db_insert.log ▐▌
█ Remember to turn debugging off as this file can grow large very fast. If ▐▌
█ you have left it on, not to worry as the LogZilla's log rotation will ▐▌
█ rotate it daily (/etc/logrotate.d/logzilla). ▐▌
█ ▐▌
█ 4. Take a look at your logs for errors related: ▐▌
█ tail -20 /var/log/syslog ▐▌
█ cat /var/log/syslog | grep mysqld ▐▌
█ cat /var/log/syslog | grep ERROR ▐▌
█ cat /var/log/syslog | grep crashed ▐▌
█ ▐▌
█ tail -20 /var/log/daemon.log ▐▌
█ cat /var/log/daemon.log | grep ERROR ▐▌
█ ▐▌
█ If you see: ▐▌
█ [Note] Plugin 'FEDERATED' is disabled. ▐▌
█ Not to worry as the federated engine was disabled in v 5.1 ▐▌
█ ▐▌
█ If you see: ▐▌
█ [Warning] Event Scheduler: [root@localhost][lz.logs_del_partition] No data ▐▌
█ - zero rows fetched, selected, or processed ▐▌
█ This is just a message stating that there is nothing to delete; don't ▐▌
█ worry about it, view HERE. ▐▌
█ ▐▌
█ If you have errors similar to (replace lz with your database name): ▐▌
█ [ERROR] /usr/sbin/mysqld: Table './lz/settings' is marked as crashed and ▐▌
█ should be repaired ▐▌
█ [Warning] Checking table: './lz/settings' ▐▌
█ ▐▌
█ Then you should repair your database and/or tables: ▐▌
█ mysqlcheck -u syslogadmin -p --auto-repair -A ▐▌
█ Or as root to check all database tables: ▐▌
█ mysqlcheck -uroot -p -A -a -c -o -g --auto-repair ▐▌
█ This will analyze, check, optimize, upgrade if needed and repair. ▐▌
█ Or if you want to check all your databases: ▐▌
█ mysqlcheck -uroot -p --check --all-databases --auto-repair ▐▌
█ ▐▌
█ Once that is done, to check your tables (replace lz with your database ▐▌
█ name): ▐▌
█ mysqlcheck -uroot -p lz --check ▐▌
█ mysql -u root -p lz ▐▌
█ Show tables; ▐▌
█ CHECK TABLE banned_ips, cache, facilities, groups, help, history, hosts, logs, logs_archive, lzecs;
█ CHECK TABLE mne, programs, settings, severities, sph_counter, suppress, totd, ui_layout, users;
█ ▐▌
█ 5. If logging into your LogZilla page you receive the error: ▐▌
█ Error in perform_query function ▐▌
█ No DB link for query: SELECT name,value, type FROM settings ▐▌
█ Mysql_error: Access denied for user 'syslogadmin'@'localhost' (using ▐▌
█ password: YES) ▐▌
█ Refer to earlier step above - 'TEST MYSQL LOGZILLA USER CAN LOGIN:' ▐▌
█ ▐▌
█ 6. If doing a search you receive the error "Warning: Wrong parameter count ▐▌
█ for max() in ▐▌
█ /var/www/lz/html/includes/portlets/portlet-chart_adhoc.php on line 453" ▐▌
█ This only means that the search didn't yield results, view HERE. ▐▌
█ ▐▌
█ 7. Change the URL alias or base path: ▐▌
█ If you need to change the alias for your LogZilla, login to the MySQL ▐▌
█ database and change /logs/ below to the alias you would like: ▐▌
█ mysql -u syslogadmin -p lz ▐▌
█ update settings set value='/logs/' where name='SITE_URL'; ▐▌
█ Output will be: ▐▌
█ Query OK, 1 row affected (0.00 sec) ▐▌
█ Rows matched: 1 Changed: 1 Warnings: 0 ▐▌
█ exit; ▐▌
█ ▐▌
█ Then change your alias in apache: ▐▌
█ pico /etc/apache2/sites-available/lz ▐▌
█ /etc/init.d/apache2 reload ▐▌
█ ▐▌
█ Reopen your browser if changes aren't showing. ▐▌
█ ▐▌
█ If you need to change the path base: ▐▌
█ mysql -u syslogadmin -p lz ▐▌
█ update settings set value='/var/www/html/lz' where name='PATH_BASE'; ▐▌
█ Query OK, 1 row affected (0.01 sec) ▐▌
█ Rows matched: 1 Changed: 1 Warnings: 0 ▐▌
█ ▐▌
█ 8. REINSTALL LOGZILLA FRESH: ▐▌
█ If for some reason you want to start with a fresh LogZilla install you'll ▐▌
█ need to do a few simple steps: ▐▌
█ A. SYSLOG-NG SETTINGS: ▐▌
█ Restore syslog-ng.conf to pre-logzilla install state: ▐▌
█ IF you didn't make a copy of syslog-ng.conf prior to installing LogZilla: ▐▌
█ pico /etc/syslog-ng/syslog-ng.conf ▐▌
█ At the bottom of the file are LogZilla added settings from: ▐▌
█ ########################################################################## ▐▌
█ # Clay's LogZilla config below ▐▌
█ To: ▐▌
█ # Clay's LogZilla config above ▐▌
█ ########################################################################## ▐▌
█ Remove the entire section. ▐▌
█ ▐▌
█ IF you made a back up copy of syslog-ng.conf prior to installing LogZilla: ▐▌
█ rm /etc/syslog-ng/syslog-ng.conf ▐▌
█ mv /etc/syslog-ng/syslog-ng-pre_logzilla.conf /etc/syslog-ng/syslog-ng.conf▐▌
█ ▐▌
█ Then in both cases: ▐▌
█ /etc/init.d/syslog-ng restart ▐▌
█ ▐▌
█ B. MYSQL DATABASE AND USERS: ▐▌
█ Note that I have named my database 'lz', change below to the name of your ▐▌
█ database: ▐▌
█ mysql -u root -p ▐▌
█ DROP DATABASE lz; ▐▌
█ FLUSH PRIVILEGES; ▐▌
█ USE mysql; ▐▌
█ Then to remove the user/s added: ▐▌
█ SELECT User,Password,Host from user; ▐▌
█ DELETE FROM user WHERE user='syslogadmin'; ▐▌
█ SELECT User,Password,Host from user; ▐▌
█ show databases; ▐▌
█ FLUSH PRIVILEGES; ▐▌
█ exit; ▐▌
█ ▐▌
█ C. During the install process it states ▐▌
█ "Adding LogZilla logrotate.d file to /etc/logrotate.d" ▐▌
█ It will NOT make a double file and or entry, so you can leave this file as ▐▌
█ is: /etc/logrotate.d/logzilla ▐▌
█ ▐▌
█ Done. Restart your install with ./install.pl ▐▌
█ ▐▌
█ MY MYSQL MISC NOTES: ▐▌
█ ```````````````````` ▐▌
█ Login: ▐▌
█ mysql -u root -p ▐▌
█ Logout: ▐▌
█ exit; ▐▌
█ Select a database, for example lz: ▐▌
█ USE lz; ▐▌
█ Login selecting lz database: ▐▌
█ mysql -u root -p lz ▐▌
█ Or as the LogZilla user: ▐▌
█ mysql -u syslogadmin -p lz ▐▌
█ View LogZilla database settings; ▐▌
█ SELECT name,value, type FROM settings; ▐▌
█ View users: ▐▌
█ USE mysql; ▐▌
█ SELECT User,Password,Host from user; ▐▌
█ Change admin pass for LogZilla: ▐▌
█ update users set pwhash=md5('MYNEWPASSWORD') where username='admin'; ▐▌
█ Delete a user: ▐▌
█ DELETE FROM user WHERE user='syslogadmin'; ▐▌
█ Show databases: ▐▌
█ Show databases; ▐▌
█ View scheduler settings: ▐▌
█ SELECT @@event_scheduler; ▐▌
█ Create new database, name would be test for example: ▐▌
█ create database test; ▐▌
█ Find HERE a list of some other common mysql cmds. ▐▌
█ Find HERE and HERE a fix myisam problems. ▐▌
█ ▐▌
█ After upgrading your mysql you might notice some processes running that ▐▌
█ weren't before depending on your setup. These are normal and the ▐▌
█ daemon.error refers to the logging facility of mysql: ▐▌
█ ps x or ps ax | grep mysql or ps -ef | grep -in mysql ▐▌
█ 31097 ? S 0:00 /bin/sh /usr/bin/mysqld_safe ▐▌
█ 31212 ? Sl 52:47 /usr/sbin/mysqld --basedir=/usr ▐▌
█ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid▐▌
█ --socket=/var/run/mysqld/mysqld.sock --port=3306▐▌
█ 31214 ? S 0:00 logger -t mysqld -p daemon.error ▐▌
█ ▐▌
█ //---------------------------------------------------------------------- ▐▌
█ ▐▌
█ If you find mistakes, have suggestions, and or questions please post at ▐▌
█ mewbies forum HERE - thank you. ▐▌
█ ▐▌
█ Last update on 25 Aug '10 ▐▌
█ ▐▌
█▌ █▌
█▌ - mewbies.com - █▌
█▌ █▌
██▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄██