Present at Gran Sasso: Doug Michael (on shift), Erik Katsavounidis (on shift),
Rich Baker (sort of), James Gill (Swarthmore student), Lauren Hopkins
(Swarthmore student), Sophia Kyriazopoulou (special dispensation), Nat Longley,
Chris Orth, Paolo Saggese, Stefano Stalio
Data Acquisition
----------------
Normal data acquisition has proceeded but we have recently
been plagued by a number of run failures during the night which have cost
significant downtime. Some of the failures may be associated with "noise"
events with a lot of readout and no-Q's which may be exacerbated by the
relatively recent installation of the cross-readout of streamer tubes
on scintillator monopole triggers and vice-versa. In many of these cases,
no hardware failure has occured so an automatic restart procedure would
manage to avoid large downtime. Sandro Marini has agreed to investigate how
we can institute such a procedure. (There are subtleties about how commands
are sent to the data acquisition programs which make it trickier than just
having a command file to issue the restart commands.)
Streamer-tube/Scintillator monopole trigger cross readout
---------------------------------------------------------
As reported in a previous shift report, the cross readout of streamer
tubes on scintillator monopole triggers and of WFD's on streamer tube
triggers has been implemented by Chris Orth and Ivan Dmitri. Chris showed
a plot of the number of WFD readouts induced by "trigger 9", the OR of
all streamer-tube triggers which cause WFD readouts. The rate dropped
by about a factor of 5 over a two week period and then jumped back to
its previous level and remains high. Chris did not know the source of
this fluctuation. Ivan is at CERN so was unavailable for comment.
WFD Readout
-----------
Chris reported on his checks of WFD readout efficiency for different
triggers. To first order, he found the readout to be 100% efficient for
almost all triggers which are supposed to produce WFD readout. He still had
some software bugs that need to be fixed in the checking program. He found
one problem which was apparently real associated with SM4 CSPAM/FMT/HIPT,
apparently the result of a cable which is flipped with respect to other
supermodules due to a cable fix. (The cable is defined to be "correct"
as-is.) He needs to modify the readout software to account for this effect.
Data Acquisition Limits for Large Events in WFD's
-------------------------------------------------
The other observed anomaly in the data is for muons which have produced
very large pulses. In these cases, a positive "overshoot" exists on the
signals in the WFD's which causes several 10's of microseconds of non-zero
data to be recorded. Currently, a 10KB (word?) limitation has been imposed
on the amount of data to be read from any one WFD channel in an event. This
means that on events with very large initial pulse amplitudes, although
the initial pulse may be recorded in the WFD, since readout goes backwards
in time, the initial pulse is not readout due to the data limitation. The
limit has been imposed to avoid acquisition system problems. This is a serious
problem for monopole and nuclearite searches. Possible solutions are being
investigated.
PMT Gains
---------
Some informal surveying of PMT gains by Doug and Stefano suggest that the
time has come for another round of PMT gain re-setting. We now have a second
Tektronix 2465B oscilloscope which is the primary tool that we use for the
gain setting. This should help speed the process up somewhat. Also, we
anticipate that it will not be necessary to make very many tank-end
interventions, unlike last fall when the gain re-setting was done. We will start
preparing for this work. We plan to do this (probably) sometime in the first
half of September in order to have time for preparation and allow coordination
with threshold re-setting and other work. More information will follow once
our plans become more firm.
James has compiled the data on past PMT gain settings and changes and
projected (based on perhaps too few points) the voltages that tubes may need
to be set to in the future. The good news is that most of the tubes will
require voltages well below 1800V, the nominal "danger point" for breakdown
in the EMI dynode chain. It is now common practice to replace the PMT base
on any tube which needs more than this voltage and/or requires some tank-end
access with one of the bases which has been modified to reduce the operating
voltage by about 10%.
Stefano showed some plots from Roberto Pazzi in which a handful of tankends in
the detector appear to have suffered signficant gain changes in the PHRASE
after the day-long power outage in June. The gain differences seem to have been
stable after the step-function change. The changes are typically around 10%
for tanks 2b02, 2b06, 2b07, 2c06, 2c08, 2c13, 2e11, 6s02 and 6s05. The
following tanks suffered bigger changes: 3b01-40%, 4e09-20%, 1b14-30%. In the
case of 1b14, the single SPE gain was clearly low while in tank 3b01 the single
SPE gain was clearly OK. The problem will be investigated.
New Disks
---------
We have two new 2.1 GB disks (purchased at the amazing new Fry's Electronics
store in Burbank) which have been installed on the vaxgs cluster. These disks
belong to the MACRO USA group and have a primary function of making our
various data monitoring tasks easier. (We are in the process of getting more
disk space (9.1 GB) for the Unix machines here and also for vxmacb (4.2 GB))
These disks *will* be generally available to all MACRO USA collaborators to
make dealing with data at the Gran Sasso more manageable. However, if you use a
big fraction of the disk space, you must alert the disk manager (Erik) of your
plans or risk instant deletion of all your files if the space is needed for
something with "higher priority". We hope the simplicity of this disk
management scheme will permit MACRO USA collaborators to make effective use of
this space while keeping it from being abused.
Power Monitoring for VME (and NIM?) Crates
------------------------------------------
Lauren is working on building a prototype of a planned voltage monitoring
circuit for the ERP and WFD VME crates. (Nat has some money from NSF for this
project.) The system will work by sending a single warning signal to a CAMAC
latch if any voltage in a crate is more than 10% away from nominal. The new
cards will plug directly into a spare slot in the VME crates. They are designed
so that a complete power failure will manage to set a warning condition in the
CAMAC latch. Doug suggested that monitoring of the NIM crates would also be
attractive and challenged Nat to also figure out how to do this. CAMAC crate
voltages are already monitored via the DSP modules resident in each crate.
Data Monitoring Improvements
----------------------------
Paolo showed new histograms based on Erik's end-of-run summary files.
The histograms provide a convenient graphical display of trigger rate
information compared to reference values. The addition of the reference
information is the key component which makes it possible for the random
shiftworker (who very likely has no idea what the correct value *should* be)
to be alerted to problems as they develop. In addition, Erik has added the
possibility to include averages over series of runs in order to improve
statistics and/or look at problems during specific time periods. In addition
to the trigger-rate plots, Paolo showed new histograms for the CSPAM system,
again making use of reference values to make identification of problems easier.
Doug and Nat both lobbied for making this type of histogram available to
the shiftworker "at the touch of a button" from the scintsoft menu. We expect
this should be the case in the near future. We expect more histograms of this
variety to be developed for all systems.
Attico Slow Monopole Trigger Clock Distribution
-----------------------------------------------
The chips which distribute the internal logic clock in the attico SMT
modules tend to run quite hot and several have failed as a result. Some
jury-rigged heat sink solutions have helped but not managed to solve the
problem. Paolo will get proper heat sinks for these chips as a first step.
Since several of the chips have already died, it is likely necessary to make
some new replacement boards for these (they sit on small daughter cards)
but this is a longer term issue. We will look into the possibility of
producing these boards in Italy using a simple pc board design program.
LED fixing and replacement
--------------------------
During the SMT calibrations in February, a number of tankends were
identified in which neither of the LED's was functional (perhaps 25 tanks
in the detector). The source of these failures is being studied by Nat
and James. In some cases, a cable failure is the source of the problem but
in others, both LED's are in fact dead. Stefano has developed a technique for
replacing LED's without having to open the tank end. (This was actually an
old idea that Jeff Steele had thought about for putting a different color of
LED into the tanks.) The approch involves putting an LED into the tank through
the scintillator fill-tube. The LED is soldered to a teflon-coated cable which
is soldered to a LIMO header which is mounted in the screw-in cap for the
fill-tube. The LED is "potted" in epoxy which has been tested
in scintillator with no visible degradation for a period of 10 days. (A longer
term test will be carried out.) This permits much easier replacement of LED's
than if the tank end has to be opened. All non-functioning LED's will be
replaced soon using this technique.
N/S Face ERP LUT's
------------------
The checkered history of the N/S faces in the ERP seems to continue. Rich
Baker showed plots of LUT "bit maps" for normal tanks and for some N/S face
tanks. There are clearly very odd problems in the N/S face tanks (they
suffer low rates as well) which Rich plans to address over the next few weeks.
Given that nobody was impressed with the quality of the LUT's in these
faces thus far, a discussion ensued on how Rich plans to improve the situation.
It was decided that he should start by translating "reasonable" LUT's from
a standard vertical box as a starting point. From there, the rate can be
tuned up or down simply by shifting the threshold energy. Finally, once
enough data has been acquired to give some decent calibrations, Kate's
LUT generation program can be re-employed. (It is believed that the LUT's
are currently bad because reliable calibrations have not been available
on those faces and Kate's program needs some reasonable initial calibration.)
Variation in light output from lasers
-------------------------------------
Stefano has observed that the amount of light produced in laser calibration
runs varies by as much as 50% over just a few weeks between calibrations for
the same laser. Doug suspects that the variation is due to insufficient
warm-up time for the laser since they are usually left powered off these days.
Stefano will continue to investigate.
Labelling WFD stop master cables
--------------------------------
Stefano has now comleted the task of labelling the WFD stop master cables.
Also, complete logic diagrams (including rack locations) should now exist for
this system.
Stabilization of LeCroy HV
--------------------------
Paolo reports that the LeCroy HV supplies have now been much more stable
since isolation transformers have been installed for each HV supply and also
the voltage has been increased to 230 V from about 206 V before (at least
for SM1). He has also changed the HV trip alarms to be the same from each
mainframe.
LED Simulation of Monopoles
---------------------------
Chris Orth is working on Monopole calibrations. Ultimately, he plans for a
calibration which simulates an isotropic flux. As a start, he is at least
making sure to exercise all major combinations of faces of the detector and
boxes. The main point of these calibrations is to check trigger, readout and
timing efficiencies, including the timing information of the WFD stop managers
for a wide range of monopole velocities. These particular calibrations will
not attempt to test the limits of sensitivity of the slow monopole trigger, a
process which is tricky and time consuming due to the variability of light
output from different LED's. He performed a first test of this calibration on
SM3-4 on July 10th and plans to implement this on all SM's once the result from
the current calibration is checked.
Inter-ERP Fixed
---------------
The intererp module was malfunctioning due to a cold solder joint. This has
been fixed by Nat and Chris.
The "Where will Kate go?" Sweepstakes!!!
----------------------------------------
As some of you may know, Kate Scholberg is currently debating where she will
take a postdoc job. Now, you too can join in the drama and excitement of the
decision process! To enter, simply send me an email, predicting where Kate will
end up. You must enter before the big decision is announced (which could be any
day now). Here are some of the possibilities which Kate is currently debating:
Boston University
SLAC
LBL
UCSC
Waiting tables at Denny's
CS Northridge
The only two people not allowed to enter are Kate and Chris. The prize
will be a bottle of Amaro Gran Sasso. For multiple entries on a single
choice, the tiebreaker will be the date that Kate decides. Hence, in your
email, also send me the date on which you predict Kate will decide. The
winner will be the person who comes closest to the date Kate decides. Any
further ties will be broken by a random drawing. Only one bottle of amaro
will be awarded. Only one entry per person is allowed.