MEWBIES@: Facebook Twitter G+ YouTube DeviantArt Forum Wall
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
██ ██
█▌ - RESCAN & TOTAL-RESCAN FOR GLFTPD - █▌
█▌ █▌
█ ▐▌
█ This tutorial is how-to fix some issues with project-zs-ng_r2500-v1.2.0 ▐▌
█ 'rescan' and the use of 'total-rescan' via site command. ▐▌
█ ▐▌
█ PZS-NG'S RESCAN USAGE & PROBLEMS ▐▌
█ INSTALL UPDATED RESCAN ▐▌
█ TOTAL-RESCAN ▐▌
█ SCRIPTS TO RUN TOTAL-RESCAN VIA A SITE CMD ▐▌
█ TOTAL-RESCAN RUN IN --NORMAL MODE ▐▌
█ TOTAL-RESCAN SORT MP3 ▐▌
█ SCRIPT TO RUN TOTAL-RESCAN ON MULITPLE PATHS ▐▌
█ ▐▌
█ ___________..___________ ▐▌
█ ▐▌
█ PZS-NG'S RESCAN USAGE & PROBLEMS: ▐▌
█ pzs-ng's included rescan only checks the files in the directory you ▐▌
█ select, not recursive; 1 directory per command. ▐▌
█ The command line usage default is run in '--quick' mode; recheck those ▐▌
█ not marked as verified already. ▐▌
█ ▐▌
█ In shell - example how to check the files in the directory 'whatever': ▐▌
█ /jail/glftpd/bin/rescan --chroot=/jail/glftpd --dir=/site/incoming/whatever▐▌
█ /* wild cards do not work. ▐▌
█ ▐▌
█ To use normal mode- recheck all files in that dir: ▐▌
█ /jail/glftpd/bin/rescan --chroot=/jail/glftpd --normal --dir=/site/incoming/whatever
█ ▐▌
█ Or via site cmd, while in the directory to execute the cmd on: ▐▌
█ site rescan --normal ▐▌
█ ▐▌
█ If you want it to run in --normal mode by default put in your zconfig.h: ▐▌
█ #define rescan_default_to_quick FALSE ▐▌
█ Then you can run rescan in normal mode, via site cmd: site rescan ▐▌
█ ▐▌
█ Under site rescan --help it states: ▐▌
█ --dir=<PATH> - cd to (chroot'ed) PATH before beginning to rescan. ▐▌
█ Futhermore to use this cmd you need to add this directive in zconfig.h: ▐▌
█ #define rescan_chdir_flags "17ABC" ▐▌
█ Change the flags to those users you want to allow access to this cmd. ▐▌
█ This cmd allows a user to scan X directory without the need to cd to it ▐▌
█ first . Example, to rescan the directory whatever, path must start with ▐▌
█ site: ▐▌
█ site rescan --dir=/site/incoming/whatever ▐▌
█ Note that if you execute this cmd in a directory that doesn't contain ▐▌
█ files it will make a sym link to that directory stating (no-sfv). ▐▌
█ ▐▌
█ PROBLEMS: ▐▌
█ A. Zips: ▐▌
█ If you use rescan on a directory containing multiple zip files only; no ▐▌
█ file_id.diz or *.nfo, you'll need to run rescan twice for it to verify all ▐▌
█ the zip files. ▐▌
█ The reason being is it will extract the the file_id.diz, count how many ▐▌
█ should be there, create incomplete folder based on one zip, and symlink, ▐▌
█ and a symlink (no-nfo) then exit. ▐▌
█ Run it a 2nd time, and all those problems are solved except the no .nfo - ▐▌
█ as it does not extract .nfo files. ▐▌
█ ▐▌
█ To fix these issues (run twice and no nfo extraction) you'll need to use ▐▌
█ the updated rescan -see below details how to 'INSTALL UPDATED RESCAN'. ▐▌
█ If you do not want to update your rescan (why wouldn't you? :) ) then I ▐▌
█ have a simple solution so users only need to run the cmd once (which will ▐▌
█ run it twice) - see end of this tutorial 'SCRIPT TO RUN TOTAL-RESCAN TWICE ▐▌
█ THROUGH SHELL' - which could also be used to run total-rescan on multiple ▐▌
█ paths with one cmd line easily. ▐▌
█ ▐▌
█ B. Can't rescan recursively: ▐▌
█ Use total-rescan - see below details 'TOTAL-RESCAN'. ▐▌
█ ▐▌
█ ___________..___________ ▐▌
█ ▐▌
█ INSTALL UPDATED RESCAN: ▐▌
█ ``````````````````````` ▐▌
█ There is an updated rescan, with all the zip problems mentioned above ▐▌
█ fixed -2010-02-03 11:17. ▐▌
█ Read HERE, download from HERE. Link is dead; I've placed a copy HERE: ▐▌
█ ▐▌
█ cd ~ ▐▌
█ wget http://mewbies.com/glftpd_3rd_party_scripts/rescan.revised.tar.gz ▐▌
█ tar xvzf rescan.revised.tar.gz ▐▌
█ (it will be extracted to here: ~/zipscript/src/rescan.c) ▐▌
█ Backup your original rescan.c just in case: ▐▌
█ cd ~/project-zs-ng_r2500-v1.2.0/zipscript/src/ ▐▌
█ mv rescan.c rescan-orginal.c ▐▌
█ Copy the new one over: ▐▌
█ cp ~/zipscript/src/rescan.c ~/project-zs-ng_r2500-v1.2.0/zipscript/src/rescan.c
█ cd ../.. ▐▌
█ First we need to remove the directory lib (that is created by pzs-ng) so ▐▌
█ back it up to be safe AND you might have other files in there needed by ▐▌
█ other programs (we'll put those back after): ▐▌
█ ls -alt /jail/glftpd/lib/ ▐▌
█ su ▐▌
█ cd /jail/glftpd/ ▐▌
█ rar a -m5 -v15000000kb -vn lib_backup lib/ ▐▌
█ rm /jail/glftpd/lib/ -rf ▐▌
█ exit ▐▌
█ make distclean ▐▌
█ ./configure --with-install-path=/jail/glftpd ▐▌
█ Note this will not re-create/overwrite your existing zsconfig.h, only if ▐▌
█ it doesn't exist zsconfig.h will be created. ▐▌
█ make ▐▌
█ su ▐▌
█ make install ▐▌
█ cd scripts/libcopy/ ▐▌
█ ./libcopy.sh /jail/glftpd/ ▐▌
█ ▐▌
█ Test on your glFTPd: ▐▌
█ site rescan --normal ▐▌
█ If all went well, then clean up: ▐▌
█ ls -alt /jail/glftpd/lib/ ▐▌
█ Unrar the backup we created and copy any missing files back over: ▐▌
█ cd /jail/glftpd/ ▐▌
█ unrar x lib_backup.rar lib_backup/ && cd lib_backup && ls -alt ▐▌
█ Copy back over to /jail/glftpd/lib/ any missing files. ▐▌
█ After clean up: ▐▌
█ cd .. && rm lib_backup -rf ▐▌
█ You could also delete lib_backup.rar or wait to be sure. ▐▌
█ exit ▐▌
█ rm ~/project-zs-ng_r2500-v1.2.0/zipscript/src/rescan-orginal.c ▐▌
█ rm ~/zipscript -rf ▐▌
█ ▐▌
█ If after doing 'site rescan --normal' you receive the error on site: ▐▌
█ site rescan --help: ▐▌
█ [1] 200- /bin/rescan: relocation error: /lib/tls/i686/cmov/libc.so.6: ▐▌
█ symbol _dl_out_of_memory, version GLIBC_PRIVATE not defined in file ▐▌
█ ld-linux.so.2 with link time reference ▐▌
█ [1] 200 Error executing command. ▐▌
█ ▐▌
█ It is most likely because you didn't remove /lib/ directory, 'make ▐▌
█ distclean' and or didn't './libcopy.sh /jail/glftpd/'. View the same ▐▌
█ problem HERE (if link is dead then here). ▐▌
█ ▐▌
█ ___________..___________ ▐▌
█ ▐▌
█ TOTAL-RESCAN: ▐▌
█ ````````````` ▐▌
█ To recursively check site use daxxar's 'total-rescan.pl' (at the time of ▐▌
█ this tutorial latest v is 1.5rc2). This rescans a directory and all of its ▐▌
█ sub-directories that contain a .sfv and /or .zip. It can also be used as ▐▌
█ an audio file sorter. ▐▌
█ This will use the same binary 'rescan' above, but with a different set of ▐▌
█ directives, so if you aren't using the updated/fixed rescan you'll still ▐▌
█ have the same problems with zip files as stated above. ▐▌
█ This can only be run as root, not via site cmd. To use this script via a ▐▌
█ site cmd - see below 'SCRIPTS TO RUN TOTAL-RESCAN VIA A SITE CMD'. ▐▌
█ ▐▌
█ PREREQUISITES: ▐▌
█ dpkg -l perl locales libperl-dev ▐▌
█ Install those that weren't on your list: ▐▌
█ su ▐▌
█ aptitude update ▐▌
█ aptitude install perl locales libperl-dev ▐▌
█ exit ▐▌
█ ▐▌
█ Download total-rescan HERE. If that link is down I have put it HERE, or: ▐▌
█ wget http://mewbies.com/glftpd_3rd_party_scripts/total_rescan.rar ▐▌
█ rar x total_rescan.rar ▐▌
█ gzip -d total-rescan.pl.gz ▐▌
█ su ▐▌
█ cp total-rescan.pl /jail/glftpd/bin/total-rescan.pl ▐▌
█ chmod 744 /jail/glftpd/bin/total-rescan.pl ▐▌
█ chmod 755 /jail/glftpd/bin/rescan ▐▌
█ cat /jail/glftpd/bin/total-rescan.pl | less ▐▌
█ For example to rescan 'incoming': ▐▌
█ /jail/glftpd/bin/./total-rescan.pl site/incoming /jail/glftpd ▐▌
█ Or to rescan all of site: ▐▌
█ /jail/glftpd/bin/./total-rescan.pl site /jail/glftpd ▐▌
█ ▐▌
█ You could also set a cron job for it to automatically run at x times, for ▐▌
█ example (my note; ignore: I'm not using this on crontab): ▐▌
█ crontab -e ▐▌
█ Paste in, change paths and times to your needs ▐▌
█ */30 * * * * /jail/glftpd/bin/total-rescan.pl site/ZIPS >/dev/null 2>&1 ▐▌
█ 0 2 * * * /jail/glftpd/bin/total-rescan.pl site/blahblah >/dev/null 2>&1 ▐▌
█ @weekly /jail/glftpd/bin/total-rescan.pl site/M3/archive >/dev/null 2>&1 ▐▌
█ ▐▌
█ exit ▐▌
█ ▐▌
█ If you want a script to run total-rescan, in your shell, on multiple paths ▐▌
█ at the same time view here. ▐▌
█ ▐▌
█ ___________..___________ ▐▌
█ ▐▌
█ SCRIPTS TO RUN TOTAL-RESCAN VIA A SITE CMD: ▐▌
█ ``````````````````````````````````````````` ▐▌
█ To be able to run total-rescan from a site cmd, as it can only be run from ▐▌
█ root, this is the summary of what we'll do: ▐▌
█ -Add a site cmd that creates a 'signal' file and echos the user to wait ▐▌
█ one minute. ▐▌
█ -Add a cron job to check every one minute if that signal file is ▐▌
█ found; run script - that will run total-rescan, then the script will ▐▌
█ remove the signal file. If signal file isn't found, it does nothing. ▐▌
█ ▐▌
█ Easy enough, just a number or steps involved. Download the scripts from ▐▌
█ HERE or: ▐▌
cd ~
wget http://mewbies.com/glftpd_scripts_mewbies/Total.Rescan.Signal.and.Checker.Scripts.via.Site.cmd.for.glFTPd-MWB/mwbtrsg1.zip
unzip mwbtrsg1.zip -d ~/mwbtrsg1 && cd mwbrtlg1
█ 1. SIGNAL FILE: 'total_rescan_signal.sh' to create an empty file for cron ▐▌
█ to search for every one minute. ▐▌
█ ▐▌
█ Check the location of your sh: ▐▌
█ which sh ▐▌
█ My output: /bin/sh ▐▌
█ pico total_rescan_signal.sh ▐▌
█ a. If the location of your sh is different; change the top line to your ▐▌
█ path: #!/bin/sh ▐▌
█ b. Change 'touch /site/incoming/tr_signal.txt' to *any gl /site/ path(s) ▐▌
█ you would like the signal file created; it doesn't matter which path ▐▌
█ as long as it is a gl site path (we'll hide tr_signal.txt from users ▐▌
█ later). ▐▌
█ *If your site isn't in jail (chroot) then state the full path e.g.: ▐▌
█ touch /glftpd/site/incoming/tr_signal.txt ▐▌
█ c. Save changes. ▐▌
█ ▐▌
█ 2. SCRIPT TO LOOK FOR SIGNAL FILE AND RUN TOTAL-RESCAN: ▐▌
█ total_rescan_checker.sh will execute total-rescan only if the file ▐▌
█ file 'tr_signal.txt' exist, then after remove 'tr_signal.txt': ▐▌
█ ▐▌
█ In summary - change all paths to match yours: ▐▌
█ pico total_rescan_checker.sh ▐▌
█ a. If the location of your sh is different; change the top line to your ▐▌
█ path: #!/bin/sh ▐▌
█ b. Change the two paths for tr_signal.txt for if and rm to the correct ▐▌
█ full path for your tr_signal.txt (in total_rescan_signal.sh). ▐▌
█ c. Change 'site/whatever' to the path(s) you want to run total-rescan ▐▌
█ site cmd in, also change '/jail/glftpd/bin/./total-rescan.pl' and ▐▌
█ '/jail/glftpd' to your correct gl paths. ▐▌
█ Note: Only if you did not update your rescan you'll need total-rescan to ▐▌
█ run twice; just repeat the path twice that you want rescanned, e.g.: ▐▌
█ /jail/glftpd/bin/./total-rescan.pl site/whatever /jail/glftpd ▐▌
█ /jail/glftpd/bin/./total-rescan.pl site/whatever /jail/glftpd ▐▌
█ ▐▌
█ d. Save changes. ▐▌
█ ▐▌
█ su ▐▌
█ cp total_rescan_signal.sh /jail/glftpd/bin/total_rescan_signal.sh ▐▌
█ cp total_rescan_checker.sh /jail/glftpd/bin/total_rescan_checker.sh ▐▌
█ chmod 755 /jail/glftpd/bin/total_rescan_signal.sh ▐▌
█ chmod 755 /jail/glftpd/bin/total_rescan_checker.sh ▐▌
█ Test it: ▐▌
█ /jail/glftpd/bin/./total_rescan_checker.sh ▐▌
█ ▐▌
█ ▐▌
█ 3. ADD THE CMD TO YOUR GLFTPD.CONF: ▐▌
█ pico /jail/glftpd/etc/glftpd.conf ▐▌
█ Under the section: ▐▌
█ # CUSTOM SITE COMMANDS # ▐▌
█ changing flag '!8 *' to suit your needs e.g. 1 =STAFF add these lines: ▐▌
█ site_cmd RESCAN2 EXEC /bin/total_rescan_signal.sh ▐▌
█ custom-rescan2 !8 * ▐▌
█ ▐▌
█ Save changes but don't exit pico yet (Ctrl+o) (we still need to hide the ▐▌
█ file 'tr_signal.txt'). ▐▌
█ Test the site cmd; logout of FTP, login, cd to the path you designated in ▐▌
█ 'total_rescan_signal.sh' to run the cmd, then: site rescan2 ▐▌
█ You should now see a 0 byte file 'tr_signal.txt'. ▐▌
█ At this point, that is all it does - creates that file. Okay back to pico: ▐▌
█ ▐▌
█ 4. HIDE SIGNAL FILE FROM USERS ON SITE: ▐▌
█ If you want to hide tr_signal.txt from users on the site until the script ▐▌
█ removes it, then on /jail/glftpd/etc/glftpd.conf under the section: ▐▌
█ # PRIVPATHS: Directories should be uniquely named (no wildcards) # ▐▌
█ Add this: ▐▌
█ hidden_files / tr_signal.txt ▐▌
█ ▐▌
█ While you're there if you want to hide 'rmlog.sh' that total-rescan ▐▌
█ creates; add this line also: ▐▌
█ hidden_files / rmlog.sh ▐▌
█ ▐▌
█ Save changes, exit pico. ▐▌
█ ▐▌
█ 5. ADD TO CRONTAB: ▐▌
█ This will add the run 'total_rescan_checker.sh' every 1 minute. If the ▐▌
█ signal file doesn't exist- nothing. If it does exist it will execute. ▐▌
█ crontab -e (you are still root) ▐▌
█ Copy/paste this line in at the bottom: ▐▌
█ * * * * * /jail/glftpd/bin/./total_rescan_checker.sh >/dev/null 2>&1 ▐▌
█ ▐▌
█ Done: ▐▌
█ exit ▐▌
█ Test it by running the site command, site rescan2, and wait 1 minute ▐▌
█ maximum and then refresh. It does not matter what directory you are when ▐▌
█ issuing the site cmd as script already states what path to rescan. ▐▌
█ ▐▌
█ 6. ADD NEW CMD TO SITE HELP: ▐▌
█ If you want to add it to the site news: ▐▌
█ pico /jail/glftpd/ftp-data/misc/newsfile ▐▌
█ ▐▌
█ If you want to add it to site help: ▐▌
█ pico /jail/glftpd/ftp-data/help/site.help.all ▐▌
█ Paste this in (don't hit enter after the last line or there will be a ▐▌
█ gap), replacing X with the directory you stated in the script: ▐▌
█ !D!5 RESCAN2!B!5:!h!5 To rescan all the directories in X ▐▌
█ ▐▌
█ ___________..___________ ▐▌
█ ▐▌
█ TOTAL-RESCAN RUN IN --NORMAL MODE: ▐▌
█ `````````````````````````````````` ▐▌
█ By default it is run in --quick mode - only rechecks files not marked as ▐▌
█ verified already. ▐▌
█ So if there is a missing file 'after' an upload has already been checked ▐▌
█ as good/complete, when you rescan: ▐▌
█ it will not make the sym link incomplete ▐▌
█ it will not rewrite complete directory within ▐▌
█ it will though make the missing file and (no-nfo) sym link if .nfo is ▐▌
█ missing. ▐▌
█ To fix this and have it run in --normal mode always (slower): ▐▌
█ pico /jail/glftpd/bin/total-rescan.pl ▐▌
█ Has this: ▐▌
█ my $rescan = '/bin/rescan'; ▐▌
█ Change to: ▐▌
█ my $rescan = '/bin/rescan --normal'; ▐▌
█ ▐▌
█ Test it: ▐▌
█ /jail/glftpd/bin/./total-rescan.pl site/blahblah /jail/glftpd ▐▌
█ ▐▌
█ ___________..___________ ▐▌
█ ▐▌
█ TOTAL-RESCAN SORT MP3: ▐▌
█ `````````````````````` ▐▌
█ You can use total-rescan to sort MP3s and not verify the files. ▐▌
█ Read total-rescan.pl for the settings. This is how I have done it: ▐▌
█ Create a copy of it, naming it anything you like: ▐▌
█ cp /jail/glftpd/bin/total-rescan.pl /jail/glftpd/bin/total-rescan-mp3.pl ▐▌
█ Then edit to your needs: ▐▌
█ pico /jail/glftpd/bin/total-rescan-mp3.pl ▐▌
█ Changing these two settings to: ▐▌
█ my $zipscan = 0; ▐▌
█ my $onlysort = 1; ▐▌
█ ▐▌
█ Then run it from shell to test in directory for example 'whatever': ▐▌
█ /jail/glftpd/bin/./total-rescan-mp3.pl site/incoming/MP3/whatever/jail/glftpd
█ ▐▌
█ Then you could cron it to have it done automatically at x times: ▐▌
█ crontab -e ▐▌
█ Paste in for example, changing to your own needs: ▐▌
█ */30 * * * * /jail/glftpd/bin/total-rescan-mp3.pl site/M3-Today >/dev/null 2>&1
█ 30 2 * * * /jail/glftpd/bin/total-rescan-mp3.pl site/incoming/M3/whatever >/dev/null 2>&1
█ @weekly /jail/glftpd/bin/total-rescan-mp3.pl site/archive/M3/blahblah >/dev/null 2>&1
█ ▐▌
█ You could also create a script, like I did above, to have it work as a ▐▌
█ site cmd. ▐▌
█ ___________..___________ ▐▌
█ ▐▌
█ SCRIPT TO RUN TOTAL-RESCAN ON MULITPLE PATHS: ▐▌
█ ````````````````````````````````````````````` ▐▌
█ Just a silly script to run daxxar's 'total-rescan.pl' on multiple paths ▐▌
█ at the same time with one cmd. This can also be used if you aren't using ▐▌
█ the fixed/update rescan to run rescan twice with one cmd. Download here or:▐▌
cd ~
wget http://mewbies.com/glftpd_scripts_mewbies/Total.Rescan.Run.On.Multiple.Paths.Script.for.glFTPd-MWB/mwbtrmp1.zip
unzip mwbtrmp1.zip -d ~/mwbtrmp1 && cd mwbtrmp1
which sh
pico total-rescan_multi.sh
█ Change #!/bin/sh to location of your sh and paths to location you want to ▐▌
█ rescan, save changes. ▐▌
█ su ▐▌
█ cp total-rescan_multi.sh /jail/glftpd/bin/total-rescan_multi.sh ▐▌
█ chmod 755 /jail/glftpd/bin/total-rescan_multi.sh ▐▌
█ ▐▌
█ Test it: ▐▌
█ /jail/glftpd/bin/./total-rescan_multi.sh ▐▌
█ exit ▐▌
█ ▐▌
█ //---------------------------------------------------------------------- ▐▌
█ ▐▌
█ If you find mistakes, have suggestions, and or questions please post at ▐▌
█ mewbies forum HERE - thank you. ▐▌
█ ▐▌
█ Last update on 18 Jul '10 28 Mar '14 ▐▌
█ ▐▌
█▌ █▌
█▌ - mewbies.com - █▌
█▌ █▌
██▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄██