Trigger studies

In Run6 one half of PbGl was masked in ERT 4x4c trigger. An attempt to investigate the reason is made. Initial mask map was produced in pedestal scan prior to Run. Several MONDO chips were found to be very noisy and threshold for them was set to the maximum value of 63. Here's the list of such MONDO chips and map of corresponding FEMs.

 root:/emc/E1/SM02/ASIC3/MONDO5 : 4x4a threshold value = 63
 root:/emc/E1/SM02/ASIC4/MONDO5 : 4x4a threshold value = 63
 root:/emc/E1/SM03/ASIC4/MONDO0 : 2x2 threshold value = 63
 root:/emc/E1/SM06/ASIC2/MONDO1 : 4x4a threshold value = 63
 root:/emc/E1/SM06/ASIC2/MONDO3 : 4x4a threshold value = 63
 root:/emc/E1/SM07/ASIC5/MONDO5 : 4x4a threshold value = 63
 root:/emc/E1/SM10/ASIC3/MONDO0 : 4x4a threshold value = 63
 root:/emc/E1/SM10/ASIC3/MONDO1 : 4x4a threshold value = 63
 root:/emc/E1/SM10/ASIC4/MONDO0 : 4x4a threshold value = 63
 root:/emc/E1/SM10/ASIC4/MONDO1 : 4x4a threshold value = 63
 root:/emc/E1/SM15/ASIC5/MONDO5 : 4x4a threshold value = 63
 root:/emc/E1/SM18/ASIC5/MONDO0 : 2x2 threshold value = 63
 root:/emc/E1/SM23/ASIC5/MONDO5 : 4x4a threshold value = 63
 root:/emc/E1/SM28/ASIC0/MONDO1 : 2x2 threshold value = 63
 root:/emc/E0/SM02/ASIC0/MONDO4 : 4x4a threshold value = 63
 root:/emc/E0/SM07/ASIC0/MONDO0 : 4x4a threshold value = 63
 root:/emc/E0/SM07/ASIC5/MONDO5 : 4x4a threshold value = 63
 root:/emc/E0/SM09/ASIC0/MONDO0 : 4x4a threshold value = 63
 root:/emc/E0/SM12/ASIC3/MONDO3 : 2x2 threshold value = 63
 root:/emc/E0/SM12/ASIC5/MONDO0 : 2x2 threshold value = 63
 root:/emc/E0/SM14/ASIC5/MONDO5 : 4x4a threshold value = 63
 root:/emc/E0/SM15/ASIC5/MONDO5 : 4x4a threshold value = 63
 root:/emc/E0/SM19/ASIC3/MONDO3 : 2x2 threshold value = 63
 root:/emc/E0/SM19/ASIC4/MONDO5 : 2x2 threshold value = 63
 root:/emc/E0/SM21/ASIC1/MONDO5 : 4x4a threshold value = 63
 root:/emc/E0/SM22/ASIC2/MONDO0 : 2x2 threshold value = 63
 root:/emc/E0/SM23/ASIC0/MONDO2 : 4x4a threshold value = 63
 root:/emc/E0/SM23/ASIC5/MONDO5 : 4x4a threshold value = 63
 root:/emc/E0/SM27/ASIC5/MONDO2 : 4x4a threshold value = 63
 root:/emc/E0/SM30/ASIC0/MONDO2 : 4x4a threshold value = 63
 root:/emc/E0/SM31/ASIC2/MONDO1 : 2x2 threshold value = 63

Some ASICs have broken MONDO chips which always have positive trigger signal regardless thresholds, channel nasking and real signal. To check if some of such ASICs are in the detector setup threshold were set to 45 and trigger card output had been directly measured with a scope. No one such card had been found. Tests were made with HV off. Hence trigger failures happen only with HV on.

ERT pedestal scan allows to find MONDOs with presumably dead trigger circuit. Such MONDO chips do not issue 4x4 triggers signal at any pedestal value. Here's the list of such MONDOs:


Sec    FEM  ASIC(0-5) MONDO(0-5 corr. A-F) 
7 	0 	3 	3 
7 	2 	2 	2 
7 	5	0 	5 
7 	7 	2 	5 
7 	12 	0 	5 
7 	21 	0 	5       
7 	21 	2 	5       
7 	21 	3 	5       
7 	26 	4	4 
7 	30 	0 	0 
7 	30 	0 	3 
6 	1 	5 	0 
6 	6 	2 	4 
6 	20 	0 	2 
6 	20 	2 	5

During the Run whole FEMs were masked if hit rate was too high. The masing history can be found on this page. In Run6 mask were accumulated for some period, then cleared after shutdown and again were accumulated till the end of Run. 17 FEMs were masked in 4.5 weeks prior to shutdown and 30 FEMs were masked in 6.5 weeks after shutdown. Masking rate is 3.8 and 4.6 FEMs per week respectivly. Both mask maps have only 8 coinciding FEMs and this number is close to the number of concidences in randomly chosen sets of 17 and 30 FEMs. This fact point for randomly appearing and dissapearing noise as a reason of PbGl ERT malfunction.
Here's the maps of masked FEMs before and after shutdown.

.	.
.	.

Hit rates of all 4 triggers in each SM as a function of run number were analyzed based on the data produced by Kenichi. Basic conclusions are:

There are no geometrical patterns corresponding to hardware units in noisy FEMs save for a few insignificant cases
There are no time correlations in appearance and dissaperance of noisy FEMs
Increasing of hit rate is an instant process. No cases of slow rise have been found.
Hit rate falling back to normal value is also an instant process.
Simultaneous jump of hit rate of 4x4c and 4x4a (the next lowest threshold) happens in only several cases. In the most cases 4x4c rate jumps without noticeable change in 4x4a rate

Some plots:

Ratio of 4x4c hit rate to average over Run6 hit rate vs FEM number (PbGl is 72-135) and run number (without gaps).

Ratio of 4x4a hit rate to average over Run6 hit rate vs FEM number (PbGl is 72-135) and run number (without gaps).

Ratio of 4x4b hit rate to average over Run6 hit rate vs FEM number (PbGl is 72-135) and run number (without gaps).

Ratio of 2x2 hit rate to average over Run6 hit rate vs FEM number (PbGl is 72-135) and run number (without gaps).

Examples of normal FEMs. 4x4c hit rate vs run number.
Shutdown was before run 400.

Examples of noisy FEMs.

Correlation between 4x4a and 4x4c hit rates in one SM.

Normal FEM

Noisy FEM

Hit rate jumps are caused by instantly appearing noise source. This source is geometry and time independent. It is related to HV on PMTs. Also it is limited in amplitude. One of propable reasons is our HV bases noise. The best check for this is looking on the data from physics running. If a single tower is hot it direcctly points on such U-bases.
There're two solutions. First, fix these bases on-line. Second, find hot towers on-line and mask only 2x2 tiles instead of the whole FEM.

Last update:
Andrey Kazantsev