MEWBIES@:  Facebook  Twitter  G+  YouTube  DeviantArt  Forum  Wall
 SHARE:
    ▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
   ██                                                                       ██
  █▌                -   RESCAN & TOTAL-RESCAN FOR GLFTPD   -                 █▌
 █▌                                                                           █▌
 █                                                                            ▐▌
 █ This tutorial is how-to fix some issues with project-zs-ng_r2500-v1.2.0    ▐▌
 █ 'rescan' and the use of 'total-rescan' via site command.                   ▐▌
 █                                                                            ▐▌
 █ PZS-NG'S RESCAN USAGE & PROBLEMS                                           ▐▌
 █ INSTALL UPDATED RESCAN                                                     ▐▌
 █ TOTAL-RESCAN                                                               ▐▌
 █ SCRIPTS TO RUN TOTAL-RESCAN VIA A SITE CMD                                 ▐▌
 █ TOTAL-RESCAN RUN IN --NORMAL MODE                                          ▐▌
 █ TOTAL-RESCAN SORT MP3                                                      ▐▌
 █ SCRIPT TO RUN TOTAL-RESCAN ON MULITPLE PATHS                               ▐▌
 █                                                                            ▐▌
 █                          ___________..___________                          ▐▌
 █                                                                            ▐▌
 █ PZS-NG'S RESCAN USAGE & PROBLEMS:                                          ▐▌
 █ pzs-ng's included rescan only checks the files in the directory you        ▐▌
 █ select, not recursive; 1 directory per command.                            ▐▌
 █ The command line usage default is run in '--quick' mode; recheck those     ▐▌
 █ not marked as verified already.                                            ▐▌
 █                                                                            ▐▌
 █ In shell - example how to check the files in the directory 'whatever':     ▐▌
 █ /jail/glftpd/bin/rescan --chroot=/jail/glftpd --dir=/site/incoming/whatever▐▌
 █ /* wild cards do not work.                                                 ▐▌
 █                                                                            ▐▌
 █ To use normal mode- recheck all files in that dir:                         ▐▌
 █ /jail/glftpd/bin/rescan --chroot=/jail/glftpd --normal --dir=/site/incoming/whatever
 █                                                                            ▐▌
 █ Or via site cmd, while in the directory to execute the cmd on:             ▐▌
 █ site rescan --normal                                                       ▐▌
 █                                                                            ▐▌
 █ If you want it to run in --normal mode by default put in your zconfig.h:   ▐▌
 █ #define rescan_default_to_quick      FALSE                                 ▐▌
 █ Then you can run rescan in normal mode, via site cmd: site rescan          ▐▌
 █                                                                            ▐▌
 █ Under site rescan --help it states:                                        ▐▌
 █ --dir=<PATH>    - cd to (chroot'ed) PATH before beginning to rescan.       ▐▌
 █ Futhermore to use this cmd you need to add this directive in zconfig.h:    ▐▌
 █ #define rescan_chdir_flags           "17ABC"                               ▐▌
 █ Change the flags to those users you want to allow access to this cmd.      ▐▌
 █ This cmd allows a user to scan X directory without the need to cd to it    ▐▌
 █ first . Example, to rescan the directory whatever, path must start with    ▐▌
 █ site:                                                                      ▐▌
 █ site rescan --dir=/site/incoming/whatever                                  ▐▌
 █ Note that if you execute this cmd in a directory that doesn't contain      ▐▌
 █ files it will make a sym link to that directory stating (no-sfv).          ▐▌
 █                                                                            ▐▌
 █ PROBLEMS:                                                                  ▐▌
 █ A. Zips:                                                                   ▐▌
 █ If you use rescan on a directory containing multiple zip files only; no    ▐▌
 █ file_id.diz or *.nfo, you'll need to run rescan twice for it to verify all ▐▌
 █ the zip files.                                                             ▐▌
 █ The reason being is it will extract the the file_id.diz, count how many    ▐▌
 █ should be there, create incomplete folder based on one zip, and symlink,   ▐▌
 █ and a symlink (no-nfo) then exit.                                          ▐▌
 █ Run it a 2nd time, and all those problems are solved except the no .nfo -  ▐▌
 █ as it does not extract .nfo files.                                         ▐▌
 █                                                                            ▐▌
 █ To fix these issues (run twice and no nfo extraction) you'll need to use   ▐▌
 █ the updated rescan -see below details how to 'INSTALL UPDATED RESCAN'.     ▐▌
 █ If you do not want to update your rescan (why wouldn't you? :) ) then I    ▐▌
 █ have a simple solution so users only need to run the cmd once (which will  ▐▌
 █ run it twice) - see end of this tutorial 'SCRIPT TO RUN TOTAL-RESCAN TWICE ▐▌
 █ THROUGH SHELL' - which could also be used to run total-rescan on multiple  ▐▌
 █ paths with one cmd line easily.                                            ▐▌
 █                                                                            ▐▌
 █ B. Can't rescan recursively:                                               ▐▌
 █ Use total-rescan - see below details 'TOTAL-RESCAN'.                       ▐▌
 █                                                                            ▐▌
 █                          ___________..___________                          ▐▌
 █                                                                            ▐▌
 █ INSTALL UPDATED RESCAN:                                                    ▐▌
 █ ```````````````````````                                                    ▐▌
 █ There is an updated rescan, with all the zip problems mentioned above      ▐▌
 █ fixed -2010-02-03 11:17.                                                   ▐▌
 █ Read HERE, download from HERE. Link is dead; I've placed a copy HERE:      ▐▌
 █                                                                            ▐▌
 █ cd ~                                                                       ▐▌
 █ wget http://mewbies.com/glftpd_3rd_party_scripts/rescan.revised.tar.gz     ▐▌
 █ tar xvzf rescan.revised.tar.gz                                             ▐▌
 █ (it will be extracted to here: ~/zipscript/src/rescan.c)                   ▐▌
 █ Backup your original rescan.c just in case:                                ▐▌
 █ cd ~/project-zs-ng_r2500-v1.2.0/zipscript/src/                             ▐▌
 █ mv rescan.c rescan-orginal.c                                               ▐▌
 █ Copy the new one over:                                                     ▐▌
 █ cp ~/zipscript/src/rescan.c ~/project-zs-ng_r2500-v1.2.0/zipscript/src/rescan.ccd ../..                                                                   ▐▌
 █ First we need to remove the directory lib (that is created by pzs-ng) so   ▐▌
 █ back it up to be safe AND you might have other files in there needed by    ▐▌
 █ other programs (we'll put those back after):                               ▐▌
 █ ls -alt /jail/glftpd/lib/                                                  ▐▌
 █ su                                                                         ▐▌
 █ cd /jail/glftpd/                                                           ▐▌
 █ rar a -m5 -v15000000kb -vn lib_backup lib/                                 ▐▌
 █ rm /jail/glftpd/lib/ -rf                                                   ▐▌
 █ exit                                                                       ▐▌
 █ make distclean                                                             ▐▌
 █ ./configure --with-install-path=/jail/glftpd                               ▐▌
 █ Note this will not re-create/overwrite your existing zsconfig.h, only if   ▐▌
 █ it doesn't exist zsconfig.h will be created.                               ▐▌
 █ make                                                                       ▐▌
 █ su                                                                         ▐▌
 █ make install                                                               ▐▌
 █ cd scripts/libcopy/                                                        ▐▌
 █ ./libcopy.sh /jail/glftpd/                                                 ▐▌
 █                                                                            ▐▌
 █ Test on your glFTPd:                                                       ▐▌
 █ site rescan --normal                                                       ▐▌
 █ If all went well, then clean up:                                           ▐▌
 █ ls -alt /jail/glftpd/lib/                                                  ▐▌
 █ Unrar the backup we created and copy any missing files back over:          ▐▌
 █ cd /jail/glftpd/                                                           ▐▌
 █ unrar x lib_backup.rar lib_backup/ && cd lib_backup && ls -alt             ▐▌
 █ Copy back over to /jail/glftpd/lib/ any missing files.                     ▐▌
 █ After clean up:                                                            ▐▌
 █ cd .. && rm lib_backup -rf                                                 ▐▌
 █ You could also delete lib_backup.rar or wait to be sure.                   ▐▌
 █ exit                                                                       ▐▌
 █ rm ~/project-zs-ng_r2500-v1.2.0/zipscript/src/rescan-orginal.c             ▐▌
 █ rm ~/zipscript -rf                                                         ▐▌
 █                                                                            ▐▌
 █ If after doing 'site rescan --normal' you receive the error on site:       ▐▌
 █ site rescan --help:                                                        ▐▌
 █ [1] 200- /bin/rescan: relocation error: /lib/tls/i686/cmov/libc.so.6:      ▐▌
 █ symbol _dl_out_of_memory, version GLIBC_PRIVATE not defined in file        ▐▌
 █ ld-linux.so.2 with link time reference                                     ▐▌
 █ [1] 200 Error executing command.                                           ▐▌
 █                                                                            ▐▌
 █ It is most likely because you didn't remove /lib/ directory, 'make         ▐▌
 █ distclean' and or didn't './libcopy.sh /jail/glftpd/'. View the same       ▐▌
 █ problem HERE (if link is dead then here).                                  ▐▌
 █                                                                            ▐▌
 █                          ___________..___________                          ▐▌
 █                                                                            ▐▌
 █ TOTAL-RESCAN:                                                              ▐▌
 █ `````````````                                                              ▐▌
 █ To recursively check site use daxxar's 'total-rescan.pl' (at the time of   ▐▌
 █ this tutorial latest v is 1.5rc2). This rescans a directory and all of its ▐▌
 █ sub-directories that contain a .sfv and /or .zip. It can also be used as   ▐▌
 █ an audio file sorter.                                                      ▐▌
 █ This will use the same binary 'rescan' above, but with a different set of  ▐▌
 █ directives, so if you aren't using the updated/fixed rescan you'll still   ▐▌
 █ have the same problems with zip files as stated above.                     ▐▌
 █ This can only be run as root, not via site cmd. To use this script via a   ▐▌
 █ site cmd - see below 'SCRIPTS TO RUN TOTAL-RESCAN VIA A SITE CMD'.         ▐▌
 █                                                                            ▐▌
 █ PREREQUISITES:                                                             ▐▌
 █ dpkg -l perl locales libperl-dev                                           ▐▌
 █ Install those that weren't on your list:                                   ▐▌
 █ su                                                                         ▐▌
 █ aptitude update                                                            ▐▌
 █ aptitude install perl locales libperl-dev                                  ▐▌
 █ exit                                                                       ▐▌
 █                                                                            ▐▌
 █ Download total-rescan HERE. If that link is down I have put it HERE, or:   ▐▌
 █ wget http://mewbies.com/glftpd_3rd_party_scripts/total_rescan.rar          ▐▌
 █ rar x total_rescan.rar                                                     ▐▌
 █ gzip -d total-rescan.pl.gz                                                 ▐▌
 █ su                                                                         ▐▌
 █ cp total-rescan.pl /jail/glftpd/bin/total-rescan.pl                        ▐▌
 █ chmod 744 /jail/glftpd/bin/total-rescan.pl                                 ▐▌
 █ chmod 755 /jail/glftpd/bin/rescan                                          ▐▌
 █ cat /jail/glftpd/bin/total-rescan.pl | less                                ▐▌
 █ For example to rescan 'incoming':                                          ▐▌
 █ /jail/glftpd/bin/./total-rescan.pl site/incoming /jail/glftpd              ▐▌
 █ Or to rescan all of site:                                                  ▐▌
 █ /jail/glftpd/bin/./total-rescan.pl site /jail/glftpd                       ▐▌
 █                                                                            ▐▌
 █ You could also set a cron job for it to automatically run at x times, for  ▐▌
 █ example (my note; ignore: I'm not using this on crontab):                  ▐▌
 █ crontab -e                                                                 ▐▌
 █ Paste in, change paths and times to your needs                             ▐▌
 █ */30 * * * * /jail/glftpd/bin/total-rescan.pl site/ZIPS >/dev/null 2>&1    ▐▌
 █ 0 2 * * * /jail/glftpd/bin/total-rescan.pl site/blahblah >/dev/null 2>&1   ▐▌
 █ @weekly /jail/glftpd/bin/total-rescan.pl site/M3/archive >/dev/null 2>&1   ▐▌
 █                                                                            ▐▌
 █ exit                                                                       ▐▌
 █                                                                            ▐▌
 █ If you want a script to run total-rescan, in your shell, on multiple paths ▐▌
 █ at the same time view here.                                                ▐▌
 █                                                                            ▐▌
 █                          ___________..___________                          ▐▌
 █                                                                            ▐▌
 █ SCRIPTS TO RUN TOTAL-RESCAN VIA A SITE CMD:                                ▐▌
 █ ```````````````````````````````````````````                                ▐▌
 █ To be able to run total-rescan from a site cmd, as it can only be run from ▐▌
 █ root, this is the summary of what we'll do:                                ▐▌
 █   -Add a site cmd that creates a 'signal' file and echos the user to wait  ▐▌
 █    one minute.                                                             ▐▌
 █   -Add a cron job to check every one minute if that signal file is         ▐▌
 █    found; run script - that will run total-rescan, then the script will    ▐▌
 █    remove the signal file. If signal file isn't found, it does nothing.    ▐▌
 █                                                                            ▐▌
 █ Easy enough, just a number or steps involved. Download the scripts from    ▐▌
 █ HERE or:                                                                   ▐▌
cd ~
wget http://mewbies.com/glftpd_scripts_mewbies/Total.Rescan.Signal.and.Checker.Scripts.via.Site.cmd.for.glFTPd-MWB/mwbtrsg1.zip
unzip mwbtrsg1.zip -d ~/mwbtrsg1 && cd mwbrtlg1
1. SIGNAL FILE: 'total_rescan_signal.sh' to create an empty file for cron ▐▌ █ to search for every one minute. ▐▌ █ ▐▌ █ Check the location of your sh: ▐▌ █ which sh ▐▌ █ My output: /bin/sh ▐▌ █ pico total_rescan_signal.sh ▐▌ █ a. If the location of your sh is different; change the top line to your ▐▌ █ path: #!/bin/sh ▐▌ █ b. Change 'touch /site/incoming/tr_signal.txt' to *any gl /site/ path(s) ▐▌ █ you would like the signal file created; it doesn't matter which path ▐▌ █ as long as it is a gl site path (we'll hide tr_signal.txt from users ▐▌ █ later). ▐▌ █ *If your site isn't in jail (chroot) then state the full path e.g.: ▐▌ █ touch /glftpd/site/incoming/tr_signal.txt ▐▌ █ c. Save changes. ▐▌ █ ▐▌ █ 2. SCRIPT TO LOOK FOR SIGNAL FILE AND RUN TOTAL-RESCAN: ▐▌ █ total_rescan_checker.sh will execute total-rescan only if the file ▐▌ █ file 'tr_signal.txt' exist, then after remove 'tr_signal.txt': ▐▌ █ ▐▌ █ In summary - change all paths to match yours: ▐▌ █ pico total_rescan_checker.sh ▐▌ █ a. If the location of your sh is different; change the top line to your ▐▌ █ path: #!/bin/sh ▐▌ █ b. Change the two paths for tr_signal.txt for if and rm to the correct ▐▌ █ full path for your tr_signal.txt (in total_rescan_signal.sh). ▐▌ █ c. Change 'site/whatever' to the path(s) you want to run total-rescan ▐▌ █ site cmd in, also change '/jail/glftpd/bin/./total-rescan.pl' and ▐▌ █ '/jail/glftpd' to your correct gl paths. ▐▌ █ Note: Only if you did not update your rescan you'll need total-rescan to ▐▌ █ run twice; just repeat the path twice that you want rescanned, e.g.: ▐▌ █ /jail/glftpd/bin/./total-rescan.pl site/whatever /jail/glftpd ▐▌ █ /jail/glftpd/bin/./total-rescan.pl site/whatever /jail/glftpd ▐▌ █ ▐▌ █ d. Save changes. ▐▌ █ ▐▌ █ su ▐▌ █ cp total_rescan_signal.sh /jail/glftpd/bin/total_rescan_signal.sh ▐▌ █ cp total_rescan_checker.sh /jail/glftpd/bin/total_rescan_checker.sh ▐▌ █ chmod 755 /jail/glftpd/bin/total_rescan_signal.sh ▐▌ █ chmod 755 /jail/glftpd/bin/total_rescan_checker.sh ▐▌ █ Test it: ▐▌ █ /jail/glftpd/bin/./total_rescan_checker.sh ▐▌ █ ▐▌ █ ▐▌ █ 3. ADD THE CMD TO YOUR GLFTPD.CONF: ▐▌ █ pico /jail/glftpd/etc/glftpd.conf ▐▌ █ Under the section: ▐▌ █ # CUSTOM SITE COMMANDS # ▐▌ █ changing flag '!8 *' to suit your needs e.g. 1 =STAFF add these lines: ▐▌ █ site_cmd RESCAN2 EXEC /bin/total_rescan_signal.sh ▐▌ █ custom-rescan2 !8 * ▐▌ █ ▐▌ █ Save changes but don't exit pico yet (Ctrl+o) (we still need to hide the ▐▌ █ file 'tr_signal.txt'). ▐▌ █ Test the site cmd; logout of FTP, login, cd to the path you designated in ▐▌ █ 'total_rescan_signal.sh' to run the cmd, then: site rescan2 ▐▌ █ You should now see a 0 byte file 'tr_signal.txt'. ▐▌ █ At this point, that is all it does - creates that file. Okay back to pico: ▐▌ █ ▐▌ █ 4. HIDE SIGNAL FILE FROM USERS ON SITE: ▐▌ █ If you want to hide tr_signal.txt from users on the site until the script ▐▌ █ removes it, then on /jail/glftpd/etc/glftpd.conf under the section: ▐▌ █ # PRIVPATHS: Directories should be uniquely named (no wildcards) # ▐▌ █ Add this: ▐▌ █ hidden_files / tr_signal.txt ▐▌ █ ▐▌ █ While you're there if you want to hide 'rmlog.sh' that total-rescan ▐▌ █ creates; add this line also: ▐▌ █ hidden_files / rmlog.sh ▐▌ █ ▐▌ █ Save changes, exit pico. ▐▌ █ ▐▌ █ 5. ADD TO CRONTAB: ▐▌ █ This will add the run 'total_rescan_checker.sh' every 1 minute. If the ▐▌ █ signal file doesn't exist- nothing. If it does exist it will execute. ▐▌ █ crontab -e (you are still root) ▐▌ █ Copy/paste this line in at the bottom: ▐▌ █ * * * * * /jail/glftpd/bin/./total_rescan_checker.sh >/dev/null 2>&1 ▐▌ █ ▐▌ █ Done: ▐▌ █ exit ▐▌ █ Test it by running the site command, site rescan2, and wait 1 minute ▐▌ █ maximum and then refresh. It does not matter what directory you are when ▐▌ █ issuing the site cmd as script already states what path to rescan. ▐▌ █ ▐▌ █ 6. ADD NEW CMD TO SITE HELP: ▐▌ █ If you want to add it to the site news: ▐▌ █ pico /jail/glftpd/ftp-data/misc/newsfile ▐▌ █ ▐▌ █ If you want to add it to site help: ▐▌ █ pico /jail/glftpd/ftp-data/help/site.help.all ▐▌ █ Paste this in (don't hit enter after the last line or there will be a ▐▌ █ gap), replacing X with the directory you stated in the script: ▐▌ █ !D!5 RESCAN2!B!5:!h!5 To rescan all the directories in X ▐▌ █ ▐▌ █ ___________..___________ ▐▌ █ ▐▌ █ TOTAL-RESCAN RUN IN --NORMAL MODE: ▐▌ █ `````````````````````````````````` ▐▌ █ By default it is run in --quick mode - only rechecks files not marked as ▐▌ █ verified already. ▐▌ █ So if there is a missing file 'after' an upload has already been checked ▐▌ █ as good/complete, when you rescan: ▐▌ █ it will not make the sym link incomplete ▐▌ █ it will not rewrite complete directory within ▐▌ █ it will though make the missing file and (no-nfo) sym link if .nfo is ▐▌ █ missing. ▐▌ █ To fix this and have it run in --normal mode always (slower): ▐▌ █ pico /jail/glftpd/bin/total-rescan.pl ▐▌ █ Has this: ▐▌ █ my $rescan = '/bin/rescan'; ▐▌ █ Change to: ▐▌ █ my $rescan = '/bin/rescan --normal'; ▐▌ █ ▐▌ █ Test it: ▐▌ █ /jail/glftpd/bin/./total-rescan.pl site/blahblah /jail/glftpd ▐▌ █ ▐▌ █ ___________..___________ ▐▌ █ ▐▌ █ TOTAL-RESCAN SORT MP3: ▐▌ █ `````````````````````` ▐▌ █ You can use total-rescan to sort MP3s and not verify the files. ▐▌ █ Read total-rescan.pl for the settings. This is how I have done it: ▐▌ █ Create a copy of it, naming it anything you like: ▐▌ █ cp /jail/glftpd/bin/total-rescan.pl /jail/glftpd/bin/total-rescan-mp3.pl ▐▌ █ Then edit to your needs: ▐▌ █ pico /jail/glftpd/bin/total-rescan-mp3.pl ▐▌ █ Changing these two settings to: ▐▌ █ my $zipscan = 0; ▐▌ █ my $onlysort = 1; ▐▌ █ ▐▌ █ Then run it from shell to test in directory for example 'whatever': ▐▌ █ /jail/glftpd/bin/./total-rescan-mp3.pl site/incoming/MP3/whatever/jail/glftpd █ ▐▌ █ Then you could cron it to have it done automatically at x times: ▐▌ █ crontab -e ▐▌ █ Paste in for example, changing to your own needs: ▐▌ █ */30 * * * * /jail/glftpd/bin/total-rescan-mp3.pl site/M3-Today >/dev/null 2>&1 █ 30 2 * * * /jail/glftpd/bin/total-rescan-mp3.pl site/incoming/M3/whatever >/dev/null 2>&1 █ @weekly /jail/glftpd/bin/total-rescan-mp3.pl site/archive/M3/blahblah >/dev/null 2>&1 █ ▐▌ █ You could also create a script, like I did above, to have it work as a ▐▌ █ site cmd. ▐▌ █ ___________..___________ ▐▌ █ ▐▌ █ SCRIPT TO RUN TOTAL-RESCAN ON MULITPLE PATHS: ▐▌ █ ````````````````````````````````````````````` ▐▌ █ Just a silly script to run daxxar's 'total-rescan.pl' on multiple paths ▐▌ █ at the same time with one cmd. This can also be used if you aren't using ▐▌ █ the fixed/update rescan to run rescan twice with one cmd. Download here or:▐▌
cd ~
wget http://mewbies.com/glftpd_scripts_mewbies/Total.Rescan.Run.On.Multiple.Paths.Script.for.glFTPd-MWB/mwbtrmp1.zip
unzip mwbtrmp1.zip -d ~/mwbtrmp1 && cd mwbtrmp1
which sh
pico total-rescan_multi.sh
█ Change #!/bin/sh to location of your sh and paths to location you want to ▐▌ █ rescan, save changes. ▐▌ █ su ▐▌ █ cp total-rescan_multi.sh /jail/glftpd/bin/total-rescan_multi.sh ▐▌ █ chmod 755 /jail/glftpd/bin/total-rescan_multi.sh ▐▌ █ ▐▌ █ Test it: ▐▌ █ /jail/glftpd/bin/./total-rescan_multi.sh ▐▌ █ exit ▐▌ █ ▐▌ █ //---------------------------------------------------------------------- ▐▌ █ ▐▌ █ If you find mistakes, have suggestions, and or questions please post at ▐▌ █ mewbies forum HERE - thank you. ▐▌ █ ▐▌ █ Last update on 18 Jul '10 28 Mar '14 ▐▌ █ ▐▌ █▌ █▌ █▌ - mewbies.com - █▌ █▌ █▌ ██▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄██