Dserver


Dserver is a program which implements functions of a server for automatic uploading data to a remote host. This document presents concept, installation hints and user documentation.

Contents:

Purpose.

Dserver is a program which implements functions of a server for automatic uploading data. It supports the following model if interaction between client and server:

Client sends a e-mail message to a server and asks it for a service: to get via ftp a set of files with the specified URLs, to retrieve them to the host where dserver is running, to uncompress if necessary, to execute a specific procedure in according with data type and finally to send a e-mail message to confirm receiving the request and status of processing the request: success or error. Client and server communicate by usual e-mail and ftp services. Dserver uploads files from client's host to dserver's host automatically, it detects losing connection and restarts retrieval if it is necessary.

It is assumed that client either has an ftp server at his host computer or has a reliable and fast access to a nearby located ftp server. Client sends e-mail message to dserver with a request which specifies URL of the files to be transferred. URL http:// or ftp:// should be accessible without password, f.e. via anonymous ftp.

Dserver uses popular software product
WGET (http://sunsite.auc.dk/wget/) developed in the framework of GNU project which retrieves files via ftp regardless of temporal losing connection. For ultra reliability a wrapper utility wget_mon is provided. wget_mon launches wget as a subprocess and periodically checks whether wget is running. If it detects that wget is stalled, wget_mon relaunches wget.

Benefits in using dserver with respect to manual uploading files:

Dserver
  1. works purely automatically;
  2. works reliably at slow and unstable networks with frequent losing connection;
  3. executes a procedure which can
  4. notifies user about success or failure of the operation;
  5. provides more security: client doesn't have access to the server host machine and all operations are controlled by server. Dserver retrieves the data only from the specified hosts and directories.

Usage.

Client.

Client sends a e-mail message to dserver with arbitrary subject. Message can be either created manually or generated by utility
dclient which provides the end user interface. It is assumed that dclient is used for routine file submissions.

Body of the message can contain comments and commands. Lines starting with "#" or empty lines are considered as comments and ignored. A Command consists of two words separated by one or more delimiters: a keyword and a value. Symbols BLANK, TAB and BINARY_ZERO are considered as delimiters. Commands may follow in arbitrary order. Each keyword has ending symbol : (semi-column) which is considered as a part of the keyword. List of supported keywords:

  1. DATA_TYPE: Type of the data to be transferred. If more than one file is to be transmitted all files are assumed to have the same type. The following values are supported:

    1. ASIS -- the file is transferred as is. If it has extension ".gz" or ".gzp" then it is de-compressed by gzip. No further procedures are executed. File remains in the incoming directory after transferring.


    2. DBH -- the file has MARK3-DBH format. File name should satisfy IVS convention for database names. If the file name doesn't satisfy this convention, the request is canceled and a e-mail with error message is sent to user. If the file has extension ".gz" then it is de-compressed by gzip. A specific procedure is launched upon retrieving the file. It checks the format of the first physical record of the file and then moves to the destination directory in according with specifications of IVS Data Center. finally it checks whether the file appeared in the destination directory and sends a success report to a user.


    3. DESC -- the file contains a solution description. Filename should satisfy IVS convention for EOPS, EOPI, TRF or CRF names. If the file name doesn't satisfy this convention, the request is canceled and a e-mail with error message is sent to user. If the file has extension ".gz" then it is de-compressed by gzip. Dserver checks whether the description file appeared in the destination directory and sends a success report to the user. It also make a symbolic link of the description file to the ivsproduct/eops or ivsproduct/eopi directory


    4. DSNX -- the file contains a listing in Sinex format of the daily standalone VLBI solution. Filename should satisfy IVS convention for DSNX names. If the file name doesn't satisfy this convention, the request is canceled and a e-mail with error message is sent to the user. If the file has extension ".gz" then it is de-compressed by gzip. Finally, dserver checks whether the file appeared in the destination directory and sends a success report to the user.


    5. ISNX -- the file contains a listing in Sinex format of the daily standalone VLBI solution. Filename should satisfy IVS convention for ISNX names. If the file name doesn't satisfy this convention, the request is canceled and a e-mail with error message is sent to the user. If the file has extension ".gz" then it is de-compressed by gzip. Finally, dserver checks whether the file appeared in the destination directory and sends a success report to the user.
      There was no need to introduce ISNX datatype, since it is identical to DSNX. This is an oversight


    6. EOPS -- the file contains Earth Orientation Parameters series of 24 hours VLBI experiments and it has a free format. Filename should satisfy IVS convention for EOPS names. If the file name doesn't satisfy this convention, the request is canceled and a e-mail with error message is sent to user. If the file has extension ".gz" then it is de-compressed by gzip. Finally, dserver checks whether the EOP file appeared in the destination directory and sends a success report to a user.


    7. EOPI -- the file contains Earth Orientation Parameters series of NEOS Intensive VLBI experiments and it has a free format. Filename should satisfy IVS convention for EOPI names. If the file name doesn't satisfy this convention, the request is canceled and a e-mail with error message is sent to user. If the file has extension ".gz" then it is de-compressed by gzip. Finally, dserver checks whether the EOP file appeared in the destination directory and sends a success report to a user.


    8. LOG -- the file contains a log file. The log file name should satisfy the IVS convention: it should have the IVS session code in its name, the two letter station name and an extension .log. The file can be compressed with gzip or bzip2.


    9. MK5VLBI -- the file contains a compressed directory with full correlator output of the Mark4/Mark5 correlator including files created by fringe-fitting program Fourfit. The Filename should satisfy the IVS convention for full correlator output file names. If the file name doesn't satisfy this convention, the request is canceled and a e-mail with error message is sent to the user. Directory tree should satisfy IVS convention. The directory tree should be compressed using tar + bzip2 program and have extension ".tar.bz2". Finally, dserver checks whether the file appeared in the destination directory, whether it can be decompressed and sends a success report to the user.


    10. PING -- no data transferring is requested. Dserver sends back a message to user notifying him/her that dserver is running. If DATA_TYPE is PING then other commands are ignored.

      One and only one command with keyword DATA_TYPE: is allowed.


    11. SKED -- the file contains a sked file. The sked file name should satisfy the IVS convention: it should have the IVS session code in its name and an extension .skd. The file can be compressed with gzip or bzip2.


    12. TRF -- the file contains a listing of a global solution in Sinex format. The trf file name should satisfy the IVS convention: it should have the IVS. The file can be compressed with gzip or bzip2.


  2. URL: -- URL (universal resource locator) of the file to be sent. Prefixes ftp:// and http:// are supported. No password is sent to URL, therefore it is assumed that the URL is accessible for reading for anonymous ftp user without restrictions. More than one command with keyword URL is allowed (but not more than 64).


  3. CONFIRM_EMAIL: -- value of this keyword is empty or a blank-separated list of valid e-mail addresses. In the latter case dserver sends a e-mail to these addresses after successful syntax parsing his/her request but before execution. The message confirms acceptance of a valid request to dserver. The e-mail list can contain the blank-separated dot character. This dot character is expended to the actual e-mail address of the person who submitted request.

    Multiple CONFIRM_EMAIL lines are not allowed.


  4. SUCCESS_EMAIL: -- value of this keyword is empty or a list of valid e-mail addresses. In the latter case dserver sends a e-mail to these addresses upon successful processing the request. The e-mail list can contain the blank-separated dot character. This dot character is expended to the actual e-mail address of the person who submitted request.

    Multiple SUCCESS_EMAIL lines are not allowed.


Example 1:
DATA_TYPE:     DBH
URL:           ftp://kuestner.geod.uni-bonn.de/pub/vlbi/99FEB04XA_V004.gz
URL:           ftp://kuestner.geod.uni-bonn.de/pub/vlbi/99FEB04SA_V002.gz
CONFIRM_EMAIL: 
SUCCESS_EMAIL: .

Example 2:

DATA_TYPE:     ASIS
URL:           ftp://gemini.gsfc.nasa.gov/pub/core/core-whitepaper.ps
CONFIRM_EMAIL: petrov@miro.geod.uni-bonn.de
SUCCESS_EMAIL: petrov@miro.geod.uni-bonn.de

Example 3:

DATA_TYPE: PING

Dserver sends an error message to the address specified in the field "From:" in the letter from client in the case when it considers the request as failed. Request is considered as successful if all files were retrieved and no errors were detected in execution of a data_type specific procedure for all files. Request is considered as failed in all other files. Reasons for failure:

  1. a syntax error was detected;
  2. an URL doesn't match access list.
  3. an error in file retrieval was detected;
  4. an error in decompression was detected;
  5. an error in file format checking was detected;
  6. an error in file moving was detected;
  7. status "success" was not set up due to unexpected reason.

Server.

Start: Dserver is launched by a command start_dserver.csh
Stop: Dserver is terminated by a command stop_dserver.csh
Check: Command check_dserver.csh checks whether dserver is running on local host.

If for some reason dserver detects an error and stops, it sends a message to a system administer with a short log file attached. A procedure monitor_dserver.csh is launched by cron and checks: does the process for dserver is still running. It sends a e-mail message to a system administrator if dserver terminated without any visible reason.

Download

The latest version of the source code of is dserver-20080731.tar.bz2.

Installation.

Dserver runs on processors i386, x86_64 and PA-RISC architectures under Linux and HP-UX. It suppose to run under SunOS, although it was not tested. Optional program wget_mon will run only under Linux. Dserver refers to a library
petools. It depends on the following free software product petools, wget, unzip, gzip and bzip2

You need first to install petools library.
Full petools requires BLAS library: either ATLAS or Intel Mathematical Kernel Library., however for some applications, including dserver, limited version of petools without blas is sufficient. Refer to petools documentation to see how it can be configured without blas.

You also need to have Fortran90 and C compilers.

To compile and install dserver, one should execute three standard Unix commands: Configure has options

During installation dserver does not parse configuration file, but checks whether the file exists.

Customization.

Client.

Program
dclient provides flexible interface for a client of dserver and is recommended for everyday use.

Server.

All host dependencies are gathered in a configuration file. This file contains configuration parameters used by a program dserver.e and C-shell programs which works in the context of dserver. Configuration file contains settings for a program dserver.e and for C-shell programs. It consist of two parts: A) customization of for dserver.e and B) customization of C-Shell programs. Lines starting from ## or #! are considered as comments and ignored. Format of part A): #, blank, keyword, delimiters, value. All parameters should be specified. No defaults are supported. All keywords have to be specified. Values are of character of Integer*4 type. Delimiter is one or more symbols BLANK, TAB, BIN_ZERO.

Supported keywords:

Part B of the configuration file contains usual definitions of environment variables in according with C-Shell syntax: setenv Supported variables:
#!/bin/csh
#!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
#!                                                                         !
#!  --- dserver configuration for Leipzig (BKG)      21-SEP-2000 12:12:35  !
#!                                                                         !
#!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
##
##  --- Settings for dserver.e ---
##
# SLEEP_INTR:   10
# STALE_INTR:   3
# STALE_REQ:    432000	    
# M_LEN:        128
# M_RET:        4
# SYS_MAILBOX:  /usr/mail/dserver
# LOC_MAILBOX:  /disc1/dserver/mailbox
# MAIL_COMMAND: mailx -m
# DSERVER_ROOT: /users/dserver
# TMP_FILE:     /disc1/dserver/tmp
# MASTER_DIR:   /disc1/vlbi/ivscontrol/
##
##  --- Settings for C-shell scripts
##
setenv e_mail_command       "mailx -m"
setenv host_name            ws07
setenv e_mail_admin         vt@leipzig.ifag.de
setenv DSERVER_ROOT         /users/dserver/
setenv incoming_dir         ${DSERVER_ROOT}/incoming
setenv master_dir           /disc1/vlbi/ivscontrol/
setenv mail_action_list     ${DSERVER_ROOT}/bin/mail_action.list
#
setenv mk5vlbi_dir          /pub/ivsdata/raw_mark4
setenv fits_dir             /pub/ivsdata/raw_fits
setenv db_dir               /disc1/vlbi/ivsdata/db
setenv ivsdata_dir          /disc1/vlbi/ivsdata/ivsproducts
setenv ivsproducts_dir      /disc1/vlbi/ivsdata/ivsproducts
setenv ivsdocuments_dir     /pub/ivsdocuments
##
setenv url_incoming         file://home/dserver/incoming
setenv url_mk5vlbi_dir      ftp://leipzig.ifag.de/ivsdata/raw_mark4
setenv url_fits_dir         ftp://leipzig.ifag.de/ivsdata/raw_fits
setenv url_db_dir           ftp://leipzig.ifag.de/vlbi/ivsdata/db
setenv url_ivsdata_dir      ftp://leipzig.ifag.de/vlbi/ivsdata/ivsproducts
setenv url_ivsproducts_dir  ftp://leipzig.ifag.de/vlbi/ivsdata/ivsproducts
setenv url_ivsdocuments_dir ftp://leipzig.ifag.de/ivsdocuments
##
##  --- This settings should not be changed
##
setenv conf_name            ${DSERVER_ROOT}/bin/dserver_config.csh
setenv check_name           ${DSERVER_ROOT}/bin/check_dserver.csh
setenv database_check_name  ${DSERVER_ROOT}/bin/database_check
setenv dserver_cron         ${DSERVER_ROOT}/dserver.crn
setenv dserver_move_name    ${DSERVER_ROOT}/bin/dserver_move
setenv exec_name            ${DSERVER_ROOT}/bin/dserver
setenv init_name            ${DSERVER_ROOT}/bin/init_dserver.csh
setenv learn_year           ${DSERVER_ROOT}/bin/learn_year
setenv monitor_name         ${DSERVER_ROOT}/bin/monitor_dserver.csh
setenv terminate_name       ${DSERVER_ROOT}/bin/terminate_dserver.csh
setenv use_gzip_in_moving   yes             
#
setenv bzip2_name           bzip2
setenv gzip_name            gzip
setenv unzip_name           unzip
setenv wget_name            "wget --tries=3600 --wait=120"
setenv zip_name             zip
##
##  --- Log file names
##
setenv logall_name          ${dserver_dir}/archive.log
setenv logstart_name        ${dserver_dir}/start.log
setenv new_files_log        ${dserver_dir}/new_files.log
setenv resubmitted_db_log   ${DSERVER_ROOT}/resubmitted_db.log

 

 

#!/bin/csh
#!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
#!                                                                            !
#!  --- dserver configuration for cddis1.gsfc.nasa.gov  17-JAN-2008 09:56:23  !
#!                                                                            !
#!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
##
##  --- Settings for dserver.e ---
##
# SLEEP_INTR:   10
# STALE_INTR:   3
# STALE_REQ:    432000
# M_LEN:        128
# M_RET:        4
# SYS_MAILBOX:  /var/mail/dserver
# MAIL_COMMAND: nail
# DSERVER_ROOT: /raid3/ftp/pub/misc/dserver
# LOC_MAILBOX:  /raid3/ftp/pub/misc/dserver/mbox
# TMP_FILE:     /raid3/ftp/pub/misc/dserver/tmp.txt
# MASTER_DIR:   /vlbi/ivscontrol/
##
##  --- Settings for C-shell scripts
##
setenv e_mail_command       /home/dserver/bin/nail
setenv host_name            cddis2.gsfc.nasa.gov
setenv e_mail_admin         "lpetrov@lacerta.gsfc.nasa.gov"
setenv DSERVER_ROOT         /raid3/ftp/pub/misc/dserver
setenv incoming_dir         ${DSERVER_ROOT}/incoming
setenv master_dir           /vlbi/ivscontrol/
setenv mail_action_list     ${DSERVER_ROOT}/bin/mail_action.list
##
setenv mk5vlbi_dir          /vlbi/raw/mk4
setenv fits_dir             /vlbi/raw/fits
setenv db_dir               /vlbi/ivsdata/db
setenv ivsdata_dir          /vlbi
setenv ivsproducts_dir      /vlbi/ivsproducts
setenv ivsdocuments_dir     /vlbi/ivsdocuments
##
setenv url_incoming         ftp://cddisa.gsfc.nasa.gov/pub/misc/dserver/incoming
setenv url_mk5vlbi_dir      ftp://cddisa.gsfc.nasa.gov/vlbi/raw/mk4
setenv url_fits_dir         ftp://cddisa.gsfc.nasa.gov/vlbi/raw/fits
setenv url_db_dir           ftp://cddisa.gsfc.nasa.gov/vlbi/ivsdata/db
setenv url_ivsdata_dir      ftp://cddisa.gsfc.nasa.gov/vlbi/ivsdata
setenv url_ivsproducts_dir  ftp://cddisa.gsfc.nasa.gov/vlbi/ivsproducts
setenv url_ivsdocuments_dir ftp://cddisa.gsfc.nasa.gov/vlbi/ivsdocuments
##
## --- Don't edit below this line!
##
setenv check_name           ${DSERVER_ROOT}/bin/check_dserver.csh
setenv database_check_name  ${DSERVER_ROOT}/bin/database_check
setenv dserver_cron         ${DSERVER_ROOT}/dserver.crn
setenv dserver_move_name    ${DSERVER_ROOT}/bin/dserver_move
setenv exec_name            ${DSERVER_ROOT}/bin/dserver
setenv init_name            ${DSERVER_ROOT}/bin/init_dserver.csh
setenv learn_year           ${DSERVER_ROOT}/bin/learn_year
setenv monitor_name         ${DSERVER_ROOT}/bin/monitor_dserver.csh
setenv terminate_name       ${DSERVER_ROOT}/bin/terminate_dserver.csh
setenv use_gzip_in_moving   yes             
##
##  NB: option --persistent is supported ONLY by a custom version of wget!!
##
setenv wget_exe             "/home/dserver/bin/wget"
set wget_name_str =  ( \
                            "${DSERVER_ROOT}/bin/wget_mon" \
                            "$wget_exe" \
			    "++debug 2" \
			    "++check_time 60" \
			    "++max_idle_time 1200" \
			    "++max_run_time  172800" \
			    "++max_retries   1024" \
			    "--quiet" \
			    "--passive" \
                            "--continue" \
                            "--no-directories" \
                            "--persistent" \
                            "--tries=3600" \
                            "--waitretry=5" \
                      )
setenv wget_name "$wget_name_str"
##setenv wget_name            "/usr/bin/wget -c -nd --tries=3600 --wait=120 --waitretry=10 --quiet"
setenv gzip_name            /bin/gzip
setenv zip_name             /usr/bin/zip
setenv bzip2_name           /usr/bin/bzip2
setenv unzip_name           /usr/bin/unzip
##
##  --- Log file names
##
setenv logall_name          ${DSERVER_ROOT}/archive.log
setenv logstart_name        ${DSERVER_ROOT}/start.log
setenv new_files_log        ${DSERVER_ROOT}/new_files.log
setenv resubmitted_db_log   ${DSERVER_ROOT}/resubmitted_db.log
###
setenv WAIT_SIGCHLD NO

Running.

In order to start dserver, the configuration file should be sources (under tcsh or csh) source {configuration_file} $DSERVER_ROOT/bin/start_dserver.csh boot

In order to stop dserver, the configuration file should be sources (under tcsh or csh) source {configuration_file} $DSERVER_ROOT/bin/stop_dserver.csh

Dserver does not require root privileges, and it will refuse to run if a superuser accidentally try to launch it.

It is recommended a special user account "dserver" with normal privileges is created. Normally dserver is started in the system startup script under user dserver.

Since dserver may abnormally terminate, a cron process $DSERVER_ROOT/bin/monitor_dserver.csh should be scheduled by cron every hours. In the case if dserver is abnormally terminated, monitor_dserver.csh will send a e-mail to system administrator and restart dserver. Cron command can be found in $DSERVER_ROOT/bin/cron_command.sh

Maintenance.

  1. Requests file has restricted length: it can contain no more than 4096 requests. It should be cleaned from time to time. Log file also should be cleaned occasionally.


  2. System administrator should provide enough space in REQ_DIR directory and in the directory where incoming files are coming (ref. dserver.csh) and for incoming_dir.


  3. System administrator should allow to user dserver to use cron. Procedure dserver/cron_command.csh should be executed.


  4. System administrator should add dserver to the list of resident programs which start after re-booting the system.


  5. Dserver can be stopped for a short period of time (rather less than STALE_REQ). It will not damage process handling except a number of retries for the processes, which were executing at the moment when dserver was stopped, will be increased by one upon starting dserver anew.


  6. System administrator should set access lists in according with needs of the Data Center for maintaining security. Dserver doesn't maintain passwords and can be accessible by either user of Internet. Security is achieved by letting dserver to retrieve files with the specified URL only. The list of allowed URLs is kept in the files xxx.acl in ACCESS_DIR directory, where xxx is a data_type in letters of lower register. Format of access list: empty lines or lines starting from ## are considered as comments and ignored. Each line contains one URL where dserver is permitted get files from. URL can contain wild-card symbols * and/or ?.

    Example 1:
    ## 
    ##  Template access list for dserver for data_type ASIS
    ##
    ##  Access permitted for everybody
    *
    
    Example 2:
    ## 
    ##  Access list for dserver in GIUB (Bonn) for data_type DBH
    ##
    ##  Access permitted only from the specific directories on kuestner and picasso
    ##
    ftp://kuestner.geod.uni-bonn.de/pub/vlbi/*
    ftp://picasso.geod.uni-bonn.de/pub/vlbi/*
    
    Example 1 allows to upload files for the Data Center to everybody. Security restrictions are lifted. Everybody is able to initiate the process of database transferring to dserver in the example 2. If the directory ftp://picasso.geod.uni-bonn.de/pub/vlbi/ is not writable for everybody then actually only a person who has permission to write in this directory is able to upload new database to dserver.


Known problems.

  • Some mailer agents don't put a full Internet address of a sender in the field From in sending e-mail to a local machine. Dserver checks a presence of a symbol @ in the field From and it ignores such a mail message in the case if it doesn't find it. It may result to the situation when dserver "doesn't see" local users.


History.

1999.09.02 v 0.0 Beginning of development.
1999.09.08 v 1.0 First version was released.
1999.09.11 v 1.1 Module CHECK_DBHNAME is added which checks validity of the database name and checks whether this database name is specified in the master file.
1999.09.28 v 1.2 Module DSERVER_MODE is added. It moves a submitted database to the final destination directory. Procedures zip_dserver.csh and unzip_dserver.csh were added. Option -m which prevents appearing mime-header in e-mails issued by dserver was added. Module dserver_setup.csh for multiple installations was added.
1999.10.04 v 1.3 Support of access list is added. Check of whether the process is a subprocess of dserver is added. Procedure monitor_dserver.csh is added. Procedure cron_command.csh is added. Corrected a bug: the previous version might send more than one error message in the case of erroneous filename.
1999.10.08 v 1.31 Corrected a bug: the previous version supported only one e-mail address in fields CONFIRM_EMAIL, SUCCESS_EMAIL
2000.01.23 v 1.4 support of new environment variables in configuration file is added: use_gzip_in_moving, use_incoming2ivs, incoming2ivs_name, incoming2ivs_conf . They allow to move the file by either Perl-program ivs2incoming or by Fortran-program dserver_move.e . The latter program may move file with gzipping on the fly or without. An Error in makefile has been fixed.
2000.02.11 v 1.5 1) dserver_move was updated to remove the earlier version of the databases if they exist; 2) Another check was added in C-shell programs which processes request. They check whether the file has appeared after moving to destination directory; 3) subroutine parse_mailbox was updated: it sets status ignore to the messages from dserver itself; 4) dbh.csl was updated in order to support submitting databases from MPIfR.
2000.03.10 v 1.6 1) added a new configuration parameter: MAIL_COMMAND instead of hard-coded "mailx -m" in the previous versions since I realized that there are systems which have a command mailx which does not support a switch -m. 2) Added C-shell program for installing install_dserver.csh
2000.03.10 v 1.61 1) Corrected dserver.csh -- added unsetenv noclobber. Made cosmetic improvements in code; 2) fixed a bug made in version 1.6: mail from dserver was not processed correctly; 3) Added capacity to remove earlier versions which has been gzipped.
2000.03.23 v 1.62 Improved start_deserved.csh, stop_dserver.csh, monitor_dserver.csh and check_dserver.csh : the new version of these programs are able to work on one machine, but dserver is running on another machine.
2000.04.27 v 1.63 Corrected a bug: the previous version didn't work when local mailbox didn't exist.
2000.09.21 v 1.8 Added support of a new data type: EOPS.
2000.11.20 v 1.9 Added support of new data types: EOPI and DESC.
2002.06.11 v 1.10 Added support of a new data type: DSNX. Added support of expansion of "." in the CONFIRM_EMAIL and SUCCESS_EMAIL to the EMAIL_FROM. Added new fields in the new_files.log file: the date of submission and email_from. Changed installation dserver_setup: added new environment variable PET_UTIL. Changed desc.csh C-shell program: the new version creates the symbolic link for description in the ivsproduct/eopi or ivsproduct/eops directories.
2003.03.03 v 1.11 Added support of the postfix format of the e-mail header. The old version of dserver supported only sendmail's format: email@address . The new version supports also the format: (User Name) email@address It does not crash if the line From is not in neither sendmail or postfix format.
2003.10.09 v 1.13 Added additional check: after dserver moves compressed file to the final destination, it tests uncompressing. In addition it prints in log file size and date of last modification. Logic for moving files has been changed. dserver first gzips files into the destination directory with temporary name. After that it renames the file in the destination directory. It eliminates the possibility that another user or mirroring process will catch the file which has not been completely compressed.
2005.04.08 v 2.0 Converted to Fortran95. Now dserver is linked against petools instead of Calc/Solve. Support of Linux and SunOS is added. Support of data type MK5VLBI is added. Configuration file was simplified.
2005.07.11 v 2.1 Added two new features: a) support of new data types, LOG and SKED; b) support of sending an e-mail when data of the specified type are coming.
2006.05.19 v 2.2 Support of new data type full correlator output in FITS-IDI or in Mformat. Fixed bugs in installation procedure. Improved documentation.
2006.09.11 v 2.3 Added a wrapper-program wget_mon . wget_mon launches wget and periodically checks whether wget stalled. If it stalls, i.e. no input/output operation took place for the specified period of time, wget_mon relaunches wget.
2008.01.17 v 2.4 Added support of the new data type ISNX. There was no need in that. A caprise... In fact, the only difference between DSNX and ISNX is that they use different directories for submission.
Added check of the database creation date. Attempt to submit a database with creation date older than the database that is already in the data center is blocked. Submission of the database with creation date newer than the database that is already in the data center triggers sending the mail to the recipients in the "RSMB" mailing list.
The list of resubmitted databases is maintained in the file $resubmitted_db_log
2008.07.31 v 2.5 Added support of GNU FORTRAN compliler. Added support of the field Reply-To in the incoming mesasge. If the incoming email header has Reply-To line, the email address there will overrite the email address specified in the field From: . Starting from this version, the error messages are sent to both the email specified in the field Reply-To: (or From: if Replay-To is missing), CONFIRM_EMAIL, and SUCCESS_EMAIL.


Dserver was developed by Leonid Petrov ( http://lpetrov.net ) Last update: 2008.07.31_17:37:45