Subject: real time RUN processing conforming to reprocessing
From: LNGS US MACRO group (MacroUSA.Public@lngs.infn.it)
Date: Mon Feb 28 2000 - 08:01:47 EST
Hi all--
Yesterday and today, I gave it a try to make the handling of the
real time RUN processing more flexible, and closer to your shiftworker's
needs.
(sorry, Chris, you have to look into this new scheme; hopefully will
make your stuff better and ready for possible insertion in the reprocessing
chain -- BTW, your job is a serious cpu eater.)
(Yianni, please make any updates and/or final set ups in the BMON/TRK/CAL
part which I left as it was.)
My main motivation is in order to fix a day/RUN beyond which we'll
NOT be spinning any more DD's. Hopefully, this will be today and
the current DD (DD323)
For this reason, I would like to see STRICTLY ENFORCED the daily check
of the real time jobs processing RUNs as they are collected by MACRO.
The shiftworker should make sure:
(1) all (real time) RUNs are processed and all output files are correctly
produced
(2) the disks used by this operation are always cleaned up and with enough
free disk space. If you have to leave on vacation you have to arrange
in advance for guaranteeing that enough space is left and/or there is
somebody else overseeing the situation.
In order to make the shiftworker's life easier I made the following
arrangements:
(1) I adapted the whole tree of jobs IN THE SAME SCHEME AS THE JOBS
MANAGING the DD tapes currently being spinned.
(2) DISK$MACROSCRA3:[MACROUSA0.MONITOR] has been abandonded at the 99%
level. DISK$MACROSCRA3:[MACROUSA0.MONITOR2.COM] is the new area
where all the source command scripts live.
(3) The DEFINE_LOGICALS_DDNOW.COM defines a set of logicals/symbols
that control the vast (but not the entire) flow of jobs.
DISKs for certain tasks are FIXED there (*NO* pooling of disks):
DISK$SCRAUSA8:[MACRODATA] raw data
DISK$SCRAUSA8:[MACRODATA.REORDER] reordered raw data
DISK$SCRAUSA8:[GCDSTS] GC dsts
DISK$SCRAUSA8:[RAREDSTS] RARE/RRNW/OVER/FMT/LAMO/F2MON dsts
DISK$SCRAUSA8:[RAREDSTS.MINI] Chris' MINI dsts
DISK$SCRAUSA5:[EXTENDED_LGB.V5] area where E-LGB is first created
DISK$SCRAUSA5:[EXTENDED_LGB.NWFD] area where WFD files first created
DISK$SCRAUSA5:[MACROUSA0.MONITOR.ALOG...] log area for ALL jobs
DISK$MACROSCRA3:[MACROUSA0.WWW.PROTECTED.LGB.V5] archive/www area of
E-LGB files
DISK$MACROSCRA3:[MACROUSA0.WWW.PROTECTED.LGB.NWFD] archive/www area of
WFD files
(4) MASTER_DISKWHIZ sets the MINIMUM disk sizes required in order to
proceed in a run's processing. HAVE A LOOK AT IT. This is called by
DO_MONITOR and MR_MONITOR at the very beginning. If criteria are not
met, it sends email and waits for an hour before retrying. You are
supposed to take immediate action in cleaning the disks upon such an
occasion.
(5) In order to update the MACRO_RUNS.LIS , it waits until the file
is successfully copied from VXMACB to VAXGS. This means that you
will no longer have to edit this file and exclude runs for which
the copying failed.
I have been testing the new structure over the last 24 hours. However
IT IS IMPOSSIBLE to test ALL conditions that are supposed to be monitored
by them. For this reason, shiftworkers should be EXTREMELY careful and
observant in following up the real time processing over the next few days.
You can adapt and make any changes in the
DISK$MACROSCRA3:[MACROUSA0.MONITOR2.COM] area that you think will make
your job easier AS LONG AS YOU DON'T LOSE ANY REAL TIME RUN PROCESSING.
I believe this is the last touch I put on this task that from now on
will be among the shiftworkers duties.
--Erik
This archive was generated by hypermail 2a24 : Mon Feb 28 2000 - 08:01:49 EST