How to create a calibration DB for attico data

Bob Nolty (nolty@cithe504.cithep.caltech.edu)
Sat, 3 Aug 1996 08:55:18 -0700

Hi all --

I just created the CALMOD database for attico running from scratch at
Caltech. There have been a number of glitches and ad hoc corrections to the
constants over the past couple of years so the process was not
straightforward. I thought I should post my notes here in case anyone
is trying to pull off a similar feat in the future.

Bob
************************************************************************
I find several kumacs missing. I ftp runs 9914, 9965, 10011, 10053,
10098, 10133, 10171, 10266, 10349, 10429*, 10472, 10526, 10578* from
vaxgs::DISK$MACRO:[MACROCAL.KUMACS.NEW]. (Later I see that 10429 and
10578 were already here and I overwrote them.) I move the following
from v2.4/newkumacs to kumacs: geom6500angles.kumac, resp9815.kumac,
run7788dump.kumac, runs 8534, 8582, 8628, 8677, 8724, 8775, 8817,
8866, 8925, 8963, 8995, 9031, 9083, 9123, 9179, 9231, 9279, 9327,
9381, 9436, 9479, 9540, 9585, 9645, 9706, 9775, 9819, 9873. Now
everything worthwhile is concentrated in v2.4/kumacs. The things
remaining in v2.4/newkumacs are essentially identical to similar files
in v2.4/kumacs, or are unnecessary dumps.

After the first time I did all the following, I noticed that
run8925.kumac contained the data for run8963.kumac, so run 8925 never
got loaded and all later databases were bogus. I ftp'ed run8925.kumac
from vaxgs and redid everything. Also the first time through at least
some of the constants from run11493 never loaded.

I copy Six Month Run constants to this directory, and hand-edit
STEERING files to refer only to Six Month Run runs.

I run run6500dump.kumac, run7200dump.kumac and run7244dump.kumac.
I determine info in geom6500angles.kumac isn't in DB so I run
geom6500angles.kumac.

I add the next several runs with this shell script:

#! /bin/csh
rm -f calmod.log
nice ./calmod < kumacs/run7277.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run7309.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run7346.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run7379.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run7410.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run7466.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run7503.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run7539.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run7578.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run7605.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run7651.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run7689.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run7723.kumac |& grep -v '^CAL' >>& calmod.log

I run run7788dump.kumac. (This is necessary because of a mistake
Alice made while generating the constants. The incremental kumac
run7788.kumac was made with respect to a corrupt run7723 database, so
it cannot give the right results when run against a good run7723
database.)

I continue on with this batch file:
#! /bin/csh
rm -f calmod.log
nice ./calmod < kumacs/run7825.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run7869.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run7894.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run7928.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run7989.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run8035.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run8064.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run8094.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run8137.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run8177.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run8228.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run8279.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run8333.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run8383.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run8430.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run8487.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run8534.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run8582.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run8628.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run8677.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run8724.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run8775.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run8817.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run8866.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run8925.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run8963.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run8995.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run9031.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run9083.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run9123.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run9179.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run9231.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run9279.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run9327.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run9381.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run9436.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run9479.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run9540.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run9585.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run9645.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run9706.kumac |& grep -v '^CAL' >>& calmod.log
nice ./calmod < kumacs/run9775.kumac |& grep -v '^CAL' >>& calmod.log

I load resp9815.kumac, run9819.kumac and run9873.kumac.

Now I have to apply some corrections to timewalk constants.
#! /bin/csh
rm -f calmod.log
nice ./calmod < kumacs-cor/twcsm2.kumac >>& calmod.log
nice ./calmod < kumacs-cor/twcsm3.kumac >>& calmod.log
nice ./calmod < kumacs-cor/twcsm4.kumac >>& calmod.log
nice ./calmod < kumacs-cor/twcsm6.kumac >>& calmod.log
nice ./calmod < kumacs-cor/twcsm6s.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor8177.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor8228.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor8279.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor8333.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor8383.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor8430.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor8487.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor8534.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor8582.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor8628.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor8677.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor8724.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor8775.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor8817.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor8866.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor8925.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor8963.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor8995.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor9031.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor9083.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor9123.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor9179.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor9231.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor9279.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor9327.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor9381.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor9436.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor9479.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor9540.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor9585.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor9645.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor9706.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor9775.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor9819.kumac >>& calmod.log
nice ./calmod < kumacs-cor/cor9873.kumac >>& calmod.log

I finish up loading all the IU constants I have available now. It
fails the first time; I have to comment out bogus pedestals in 11074
and 11126. I delete all the new runs and load again.
#! /bin/csh
rm -f calmod.log
nice ./calmod < kumacs/run9914.kumac >>& calmod.log
nice ./calmod < kumacs/run9965.kumac >>& calmod.log
nice ./calmod < kumacs/run10011.kumac >>& calmod.log
nice ./calmod < kumacs/run10053.kumac >>& calmod.log
nice ./calmod < kumacs/run10098.kumac >>& calmod.log
nice ./calmod < kumacs/run10133.kumac >>& calmod.log
nice ./calmod < kumacs/run10171.kumac >>& calmod.log
nice ./calmod < kumacs/run10266.kumac >>& calmod.log
nice ./calmod < kumacs/run10349.kumac >>& calmod.log
nice ./calmod < kumacs/run10429.kumac >>& calmod.log
nice ./calmod < kumacs/run10472.kumac >>& calmod.log
nice ./calmod < kumacs/run10526.kumac >>& calmod.log
nice ./calmod < kumacs/run10578.kumac >>& calmod.log
nice ./calmod < kumacs/run10634.kumac >>& calmod.log
nice ./calmod < kumacs/run10691.kumac >>& calmod.log
nice ./calmod < kumacs/run10726.kumac >>& calmod.log
nice ./calmod < kumacs/run10763.kumac >>& calmod.log
nice ./calmod < kumacs/run10819.kumac >>& calmod.log
nice ./calmod < kumacs/run10866.kumac >>& calmod.log
nice ./calmod < kumacs/run10910.kumac >>& calmod.log
nice ./calmod < kumacs/run10952.kumac >>& calmod.log
nice ./calmod < kumacs/run11000.kumac >>& calmod.log
nice ./calmod < kumacs/run11046.kumac >>& calmod.log
nice ./calmod < kumacs/run11074.kumac >>& calmod.log
nice ./calmod < kumacs/run11126.kumac >>& calmod.log
nice ./calmod < kumacs/run11160.kumac >>& calmod.log
nice ./calmod < kumacs/run11192.kumac >>& calmod.log
nice ./calmod < kumacs/run11234.kumac >>& calmod.log
nice ./calmod < kumacs/run11295.kumac >>& calmod.log
nice ./calmod < kumacs/run11322.kumac >>& calmod.log
nice ./calmod < kumacs/run11373.kumac >>& calmod.log
nice ./calmod < kumacs/run11426.kumac >>& calmod.log
nice ./calmod < kumacs/run11456.kumac >>& calmod.log
nice ./calmod < kumacs/run11493.kumac >>& calmod.log
nice ./calmod < kumacs/run11528.kumac >>& calmod.log
nice ./calmod < kumacs/run11559.kumac >>& calmod.log
nice ./calmod < kumacs/run11591.kumac >>& calmod.log
nice ./calmod < kumacs/run11629.kumac >>& calmod.log
nice ./calmod < kumacs/run11669.kumac >>& calmod.log
nice ./calmod < kumacs/run11714.kumac >>& calmod.log
nice ./calmod < kumacs/run11773.kumac >>& calmod.log
nice ./calmod < kumacs/run11817.kumac >>& calmod.log
nice ./calmod < kumacs/run11845.kumac >>& calmod.log
nice ./calmod < kumacs/run11881.kumac >>& calmod.log

That completes an IU-style database. Now it's time to apply the Okada
corrections that improve performance for certain weeks and boxes,
especially intererp constants. I am using Okada constants I ftp'ed
from VAXGS and renamed according to the following schedule:
------------------------------------------------------------------------
These files were copied from VAXGS::DISK$MACRO:[MACROCAL.KUMACS.NEW]
on 24 Jul 96. The original files and dates were:

COLIN1.KUMAC;4 194 16-JAN-1996 21:18 -> ceocal9914.kumac
COLIN2.KUMAC;3 194 16-JAN-1996 21:18 -> ceocal9965.kumac
COLIN3.KUMAC;3 228 16-JAN-1996 21:18 -> ceocal10011and10098.kumac
COLIN4.KUMAC;3 36 16-JAN-1996 21:19 -> inter7689-8995.kumac
COLIN5.KUMAC;3 62 16-JAN-1996 21:19 -> inter9031-9965.kumac
COLIN6.KUMAC;3 67 16-JAN-1996 21:19 -> inter10011-10910.kumac
COLIN7.KUMAC;1 34 15-FEB-1996 21:15 -> southresp8817-10952.kumac
COLIN9.KUMAC;3 40 23-FEB-1996 11:10 -> inter10952-11493.kumac
COLIN10.KUMAC;2 11 4-MAR-1996 12:48 -> southresp11000-11493.kumac

I compared the contents of these files to the contents of files from
rsgs02:~okada/inter.tar.gz and renamed them. Note that
southresp8817-10952 was missing from the tar file -- Colin just forgot
to include it, though he emailed it to us in February. The last two
files (inter10952-11493 and southresp11000-11493) did not exist at the
time the tar file was made; apparently Colin came up with those later.
------------------------------------------------------------------------
I load ceocal9914, ceocal9965, ceocal10011and10098. Then I load up
the rest:
#! /bin/csh
rm -f calmod.log
nice ./calmod < kumacs-ceo/inter7689-8995.kumac |& grep -v '^CALM' >>& calmod.log
nice ./calmod < kumacs-ceo/inter9031-9965.kumac |& grep -v '^CALM' >>& calmod.log
nice ./calmod < kumacs-ceo/inter10011-10910.kumac |& grep -v '^CALM' >>& calmod.log
nice ./calmod < kumacs-ceo/inter10952-11493.kumac |& grep -v '^CALM' >>& calmod.log
nice ./calmod < kumacs-ceo/southresp8817-10952.kumac |& grep -v '^CALM' >>& calmod.log
nice ./calmod < kumacs-ceo/southresp11000-11493.kumac |& grep -v '^CALM' >>& calmod.log

That is all.

nolty 26-28Jul96
------------------------------------------------------------------------
After the second time through, I notice some of the constants from
run11493.kumac failed to load again. This time I see the problem is
because the kumac file contained two calls to the calmod executable,
and when I piped it into calmod only the first executable ran. I
comment out the EXIT and startup cards. To fix the rest of the
database, I delete all runs after 11493, then load

#! /bin/csh
rm -f calmod.log
nice ./calmod < kumacs/run11493.kumac >>& calmod.log
nice ./calmod < kumacs/run11528.kumac >>& calmod.log
nice ./calmod < kumacs/run11559.kumac >>& calmod.log
nice ./calmod < kumacs/run11591.kumac >>& calmod.log
nice ./calmod < kumacs/run11629.kumac >>& calmod.log
nice ./calmod < kumacs/run11669.kumac >>& calmod.log
nice ./calmod < kumacs/run11714.kumac >>& calmod.log
nice ./calmod < kumacs/run11773.kumac >>& calmod.log
nice ./calmod < kumacs/run11817.kumac >>& calmod.log
nice ./calmod < kumacs/run11845.kumac >>& calmod.log
nice ./calmod < kumacs/run11881.kumac >>& calmod.log
nice ./calmod < kumacs-ceo/inter10952-11493.kumac |& grep -v '^CALM' >>& calmod.log
nice ./calmod < kumacs-ceo/southresp11000-11493.kumac |& grep -v '^CALM' >>& calmod.log
------------------------------------------------------------------------
I create ntuples with my calhis program and extensively compare the
new DB with the GS DB -- they appear to be exactly identical.

Finally I copy all the PHRASE files from Kate's database -- I have no
idea how they were produced.