For the PHENIX HV system, we have only set alarms on the status word supplied by the LeCroy mainframe for each channel. If the status word is 0, this means the channel has been disabled and we have defined that to be a minor alarm. If the status word has any high-order bits set, so that its value exceeds 7, this means the channel has tripped, and we have defined that to be a major alarm.

In addition to the alarm severity, there is also an alarm status. The alarm status can take on many values. We have not made use of the alarm status to any extent in PHENIX, so I don't discuss it here.

The alarm handler is started by giving the start_alarm command from the phoncs account. (To execute this, or any other, EPICS-related command, be sure you have already done the source ~phoncs/epics/scripts/setup_epics command.) The alarm handler indicator button will appear:

By pressing the button, you can bring up the main alarm handler gui:

You see that the various HV subgroups for the PHENIX subsystems are listed in the frame on the left, under a single overall group called "phhv". If you press on one of those subgroup names, you get more information about the alarm status of its channels. For example, if we press on "DC_E_N" we get this:

Remember that

Alarm Acknowledgment: In the left frame, there are two columns of colored indicators to the left of the subsystem HV subgroup names. The leftmost column is the acknowledge column. When an alarm sounds, the SA1 needs to acknowledge that the alarm has occurred. To do this, you simply click on the colored indicator in the leftmost column. The button will return to gray and the handler will stop flashing and/or beeping. In our example, there is an unacknowledged major alarm (a tripped channel) in the MUID group, and an unacknowledged minor alarm (a disabled channel) in the MUTR_S group. Note that the overall alarm group "phhv" has an unacknowledged major alarm: the overall phhv group alarm indicator will show the worst alarm of any of the subsystem groups shown. You can acknowledge all the alarms in the phhv group by pressing on its acknowledgment indicator.

Alarm Severity: The other column of colored indicators is the severity column. This column shows the most severe alarm state in the group. It does not go away when you acknowledge --- it only goes away when the alarm condition has been eliminated, i.e. when the value of the status word returns to a normal state.

Active Alarms: By default, the alarm handler shows the alarm state of ALL channels. This is usually not desirable. It is better if only those channels with active alarms are shown. To do this, click on the "Setup" menu item and select "Active Alarms Only". In our example, this step has already been taken.

Beep Control: Yes, the beeping alarm handler is irritating! You can control the level of severity at which it beeps: minor, major, or invalid. To do this, click on the "Setup" menu item, follow the "Beep Severity" arrow, and select one of "Minor", "Major", or "Invalid." The beep will occur only if the alarm is at least as severe as the level you have set. I recommend that you set it at "Major".

Disabling Alarms: If a subsystem expert is training their HV system, and it is alarming a lot, and if the relevant subsystem expert is in close communication with the shift crew, then it may be desirable to temporarily disable the alarms for their subsystem alarm group. This should only be done in consultation with the shift leader and should be recorded in the log. To do this, first click on the subsystem name in the left frame, then click on the "Action" menu item, select "Modify Mask Settings", and you will see this gui:

To disable the alarms for the selected alarm group, click on "Disable" in the "Enable/Disable Alarms" row. Verify that this has the desired effect in the left frame: The alarm severity indicator will disappear for that group. To re-enable the alarms, simply select the group again, bring up this same gui and click on "Enable" in the "Enable/Disable Alarms" row. Note that in our example, a large number of groups have been disabled from alarming: BB, DC_W_N, PC_E, RICH_E_N, and PBSC_W_T.

Stale "invalid" indicators: When an IOC crashes, the alarm handler will no longer be able to get information concerning the HV channels served by that IOC, and this will create an "invalid" alarm (white alert). After rebooting the IOC, these invalid alarms should go away, and they normally do. However, it sometimes happens that they do not. In this case, it is necessary to restart the alarm handler to restore communication with the IOC.