[SOLVED] Differences in Perl Scripts

[AlSam]

I am interested in backing up recordings and am not sure which script I should use. The two that may help are AST_CRON_audio_3_ftp_FTPSSL.pl and AST_CRON_audio_4_ftp2_FTPSSL.pl but I don't know the difference in their functionality. Can anyone tell me what they are? Also, as I was reading the first few lines of each script, I found that one is a STEP-3 and the other a STEP-4 program. Must there be a STEP-3 program being run before running a STEP-4 program?

[AlSam]

..or maybe is there documentation available on them (besides the comments at the beginning of them)?

[mflorell]

Yes, a level-4 CRON-audio script can only run after the level-3 script has run, this is because of where the recordings are located. We created the level-4 scripts so that clients wanting to send recordings to 2 separate archive servers could do so in an automated way.

[AlSam]

Thanks for the response. Do these scripts move recordings or just copy them?

[williamconley]

Thanks for the response. Do these scripts move recordings or just copy them?

Vicidial Recording Management Clarified

[AlSam]

Thanks for the info, williamconley. I reviewed it and am a bit confused by the first two lines in point 4. The first bullet states that the mp3 file is moved from one folder to another but the following bullet states that it is now redundant. How is it redundant if it has been moved and not just copied?

[AlSam]

Also, one of the points references the astguiclient.conf. Is this meant to have the server directly deposit recordings to the server configured in the file? Does the perl script depend on the configurations in the astguiclient.conf file?

[williamconley]

Thanks for the info, williamconley. I reviewed it and am a bit confused by the first two lines in point 4. The first bullet states that the mp3 file is moved from one folder to another but the following bullet states that it is now redundant. How is it redundant if it has been moved and not just copied?

Because it was moved from one place to another on the Dialer. But it was first Copied to the archive server. Thus the local file is now the redundant copy. This is signified by moving it to the "you can delete me, because I'm redundant" FTP folder.

Once the upload to the archive server has completed, there is Zero need for the copy on the dialer. However: If you don't have a mirror or other copy of the audio files on the archive server, now you are left with the danger of the Archive server's HD dying the losing ALL your recordings now that they are conveniently all collected from all dialers to a single location. That single location is now prone to death of HD. You can either set up a mirror of that drive ... or use FTP2 to copy all the files AGAIN to another server. That's what FTP2 is for. If you don't use FTP2, then the files in FTP can be deleted any time you like. But if you DO use FTP2 to create your redundant backup of the archive server, then you can delete the files in FTP2 any time you like instead, but don't delete anything from FTP as those are still "in transit" and have not yet landed in the FTP2 server.

Described it like ... three times. I hope it sticks. lol

Also, one of the points references the astguiclient.conf. Is this meant to have the server directly deposit recordings to the server configured in the file? Does the perl script depend on the configurations in the astguiclient.conf file?

/usr/share/astguiclient/AST_CRON_audio_3_ftp.pl Uses credentials from /etc/astguiclient.conf, yes. AST_CRON_audio_4_ftp2.pl does not.

Updated Wiki.

[AlSam]

Thanks. That was very helpful.

[AlSam]

Tried running

Code: Select all

/usr/share/astguiclient/AST_CRON_audio_3_ftp_FTPSSL.pl --help

which resulted in

Code: Select all

Can't locate Net/FTPSSL.pm in @INC (you may need to install the Net::FTPSSL module) (@INC contains: /usr/lib/perl5/site_perl/5.18.2/x86_64-linux-thread-multi /usr/lib/perl5/site_perl/5.18.2 /usr/lib/perl5/vendor_perl/5.18.2/x86_64-linux-thread-multi /usr/lib/perl5/vendor_perl/5.18.2 /usr/lib/perl5/5.18.2/x86_64-linux-thread-multi /usr/lib/perl5/5.18.2 /usr/lib/perl5/site_perl .) at /usr/share/astguiclient/AST_CRON_audio_3_ftp_FTPSSL.pl line 392.
BEGIN failed--compilation aborted at /usr/share/astguiclient/AST_CRON_audio_3_ftp_FTPSSL.pl line 392.


Took its suggestion (you may need to install the Net::FTPSSL module) and ran

Code: Select all

perl -MCPAN -e 'shell'

then

Code: Select all

install Net::FTPSSL

which resulted in

Code: Select all

....
Writing Makefile for Net::FTPSSL
Writing MYMETA.yml and MYMETA.json
Can't exec "make": No such file or directory at /usr/lib/perl5/5.18.2/CPAN/Distribution.pm line 2084.
  CLEACH/Net-FTPSSL-0.42.tar.gz
  make -- NOT OK
'YAML' not installed, will not store persistent state
Running make test
  Can't test without successful make
Running make install
  Make had returned bad status, install seems impossible
Failed during this command:
 CLEACH/Net-FTPSSL-0.42.tar.gz                : make NO


It references the make package. Do I download it from the OpenSUSE repositories or does it come from CPAN? I see it also states YAML not installed. Again, is this something that should be downloaded from OpenSUSE repositories or CPAN? Installation of the make package via OpenSUSE repository causes the addition of the site-config package. Will this addition have a negative impact on the server's proper operation?

Thanks

[mflorell]

On OpenSuSE, if you can install it through 'yast' then that should go much faster. The error you are getting is because you don't have the 'make' package installed, so cpan can't install anything. I'd suggest just trying to install the perl module though yast first.

[AlSam]

Thanks for the quick response. I was able to install the required module. Is there a newer version (current is 180616-2248) of this script (AST_CRON_audio_3_ftp_FTPSSL.pl) which includes the --ftp-validate option referenced under the Suggestions section in the Vicidial Recording Management Clarified document? Also, is there any other reference material regarding the options available for the script? I am not clear on a few of them like debug, debugX, test, transfer-limit, list-limit, and max-files.

[mflorell]

I think we updated that script in February, so I'd recommend updating to the latest svn/trunk revision.

As for documentation, all there is is right in the script, usually in the comments at the top.

[williamconley]

Just because it's a good idea ... went ahead and moved this topic to "Support" instead of "General Discussion".

[AlSam]

How does it determine which recordings it will process each time it runs? I find that my FTP server has recordings from yesterday yet there are still recordings from over a year ago on the Vicidial server.

[mflorell]

Each script will only look in one specific folder for recordings to process. The folder it looks in depends on the script and the option flags.

[AlSam]

I may not have provided enough detail in my question. I am using the --MP3 parameter and it, in fact, is processesing recordings from said folder. However, I don't understand the order in which recordings are selected to be processed from within the MP3 folder.

[mflorell]

The script uses the perl "readdir" function which really just calls the OS directory list function. It is not sorted by the script in any way, it uses whatever operating system order that the file list is supplied to it in. This might be the last modification date of the file or the creation date usually.

[AlSam]

OK. I'll look into it. Thanks.
I noticed that the --max-files switch does nothing. I reviewed the script and found the only time it is referenced is in the help section. I am running the 200205-1716 version of the script.

[AlSam]

Thanks for all the help. I was able to setup the transfer of files successfully. I am going to be transferring recordings from a different Vicidial platform which has multiple Asterisk servers. Is it advisable to dump all recordings from the different Asterisk servers into a single directory within the storage server? What are the chances of having recordings with the same name in more than one Asterisk server? Are there any best practices that should be followed in this circumstance?

[mflorell]

Due to filesystem limitations on the destination archive server, we always recommend using the dated directories that the Vicidial FTP perl scripts use.

[AlSam]

I may not have explained myself properly. What I meant is that if it would be advisable to have the following structure or not.

Or if recordings should go into the date folders regardless of the Asterisk server from whence they came.

[mflorell]

We usually recommend putting each cluster's recordings in the same directory structure. So if your Astersk1, 2, 3... are all on the same VICIdial cluster, we would recommend sending them all to the same set of dated directories.

[AlSam]

So there's no chance there would be two different recordings with the same name on different Asterisk servers (same cluster) which might end up having one overwrite the other?

[mflorell]

I don't know how you are currently naming your recordings. If you use something like "FULLDATE_CUSTPHONE_AGENT" then yes there is no chance that recordings will have duplicate names.