########################################################################### # # # Brief Introduction to the PHENIX and Drift Chamber # # High Voltage Control System # # # # Johann M. Heuser, Stony Brook 5/12/00 # # latest update: 11/20/00 # ########################################################################### --------------------------------------------------------------------------- I N T R O D U C T I O N --------------------------------------------------------------------------- This tutorial provides a brief introduction to the PHENIX High Voltage Control System. Special emphasis is put on how to operate the Drift Chamber Control Panels and shell scripts. More detailed information on various aspects of the Drift Chamber High Voltage requirements, the detector setup in the experimental hall and the PHENIX High Voltage Control System can be perused in a black reference folder placed in the book case above the PHONCS5 workstation in the PHENIX counting house: "Drift Chamber High Voltage Documentation" This documentation can also be found on the web: http://www.phenix.bnl.gov/phenix/WWW/publish/heuser/hvcontrol_dc/hv.html If you need to start from scratch and have to set up the computer session, read section SETUP PROCEDURES. If the environment is ready to use, follow section STANDARD HIGH VOLTAGE PROCEDURES to learn how to work with the system. --------------------------------------------------------------------------- S E T U P P R O C E D U R E S --------------------------------------------------------------------------- 1./ How to start: account and password ======================================= When you modify voltages and switch on/off equipment, you have to be physically present in the counting house. It's PHENIX law ... Sit in front of phoncs5.phenix.bnl.gov (the HV control station). Usually, everything is "ready to use" there. You do not need to start up any program and can jump directly to step 4./ of this manual. If you have to start a new session or want to use another x-terminal in the counting room: open a terminal window, log onto phoncs0.phenix.bnl.gov account: phoncs password: ------------- and follow steps 2 and 3 of this tutorial to set up your High Voltage control session. 2./ execute HV setup-script: ============================ if required, execute the following HV setup-script: a) PHENIX standard: source /export/software/oncs/epics/R3.13.0.beta11/scripts/setup_epics b) or use Johann's enhanced version (RECOMMENDED): source /home/phoncs/heuser/official_setup_hv --> using this script, the following alias names have been defined for your convenience, in addition to all other functionality of the setup_epics procedure: alias commands to start MEDM and ALARM HANDLER: =============================================== * MEDM: - start_medm * ALARM: - start_alarm 3./ go to working directories: ============================== 3.1/ - in case you have to start up MEDM: ----------------------------------------- o use the alias names and type: (only available after having performed step 2b) ) start_medm o the full procedure to start MEDM is: cd /export/software/oncs/epics/R3.13.0.beta11/epicsB/app/hvca/medm/adl --> here you find the *.adl files for the MEDM gui a) medm & --> open --> use file "phhv.adl" or b) medm -x phhv.adl & 3.2/ - in case you have to start up the ALARM HANDLER: ------------------------------------------------------ !!! The ALARM HANDLER provides active protection for the Drift Chamber in the case of channels tripping and therefore is required to run when the Drift Chamber High Voltage is ON !!! o use the alias names and type: (only available after having performed step 2b) ) start_alarm o the full procedure to start the Alarm Handler is: cd /export/software/oncs/epics/R3.13.0.beta11/epicsB/app/hvca/medm/alh --> here you find the alarm handler configuration files *.alhConfig alh phhv.alhConfig& (alarm handler for all PHENIX) --> in file "phhv.alhDisabledChannels" channel names can be entered that should not be taken into account for alarm detection with the Alarm Handler. if you modify this file, you have to rebuild the Alarm Handler Configuration file and restart the Alarm Handler. --> See the related item for instructions. --------------------------------------------------------------------------- S T A N D A R D H I G H V O L T A G E P R O C E D U R E S --------------------------------------------------------------------------- 4./ how to switch on High Voltage / modify control parameters: =============================================================== a/ switch on HV mainframes: --------------------------- - use the MEDM main control panel: - Drift Chamber has the crates numbered: 34 DC_WEST_NORTH 50 DC_WEST_SOUTH 65 DC_EAST_NORTH 81 DC_EAST_SOUTH note: mainframe 50 is shared with the Pad Chambers. b/ set Voltages: ---------------- --> b-1) describes the interactive way, suited for minor changes of the setup or whenever "conditioning" work is to be done --> b-2) describes a more handy way using shell-scripts for "large scale" operations. This is intended for "experts" in first respect. b-1) the interactive way: ------------------------ on main control panel: - DC_EAST is the line of HV group # 4 - DC_WEST_NORTH is the line of HV group # 5 - DC_WEST_SOUTH is the line of HV group # 6 --> DC WEST control is split into a north and south branch! There are 350 HV channels for every Drift Chamber. 320 Channels are physical channels that go to the detector. The remaining 30 channels are the bulk supplies in the 1469N HV cards. Voltages can be set via the Voltage/Current control panels. There are two control panels for every group of four keystons: one panel for the "lower" keystone parts (XUV1 nets) and one panel for the "upper" parts (XUV2 nets). Click on the boxes to activate the panels. There are 32 physical channels and 3 bulk supply channels listed per panel. The channel names describe both the location of the channel and the voltage to be supplied: e.g.: HV_DC_W_S_KS00_X1__C Cathode in keystone 0 of Drift Chamber West, X1 net (mainframe located on South side) HV_DC_W_S_KS00_X1__P Potential HV_DC_W_S_KS00_X1__G Gate HV_DC_W_S_KS00_X1__B Backdrift Voltages can be set individually for every Cathode. Voltages can only be set in groups for Potential, Gate and Backdrift. The "bulk supply" channels to use are on the bottom of the panel: HV_DC_W_S_KS00-03_PBS Potential Bulk Supply HV_DC_W_S_KS00-03_GBS Gate Bulk Supply HV_DC_W_S_KS00-03_BBS Backdrift Bulk Supply Set voltages by placing the mouse pointer into the V_set field and typing in the number, e.g. "-100.0"; then hit the enter key. The voltage becomes a "demand voltage" V_dem and processes are started to set the actual hardware to produce this voltage. After a while (~20 seconds), the readback voltage is confirmed in box "V_meas". ################################################################# # Beware !!!! Currently there is no protection against # # false voltage input realised in MEDM/EPICS # # system. # # Be very careful when changing voltages !!! # ################################################################# The current for every channel can be monitored in fields I_meas and I_peak. The fields "Ena/Dis" can be used to enable/disable the channels. Enabled channels show a green box, disabled channels show a yellow box. The "Status" field reflects this and represents the status readback value: Status = 0: channel off Status = 1: channel on Status = 3: channel ramping up Status = 5 channel ramping down Status = 12345 or other unusual number in RED color: THE CHANNEL HAS TRIPPED OFF. --> see the related item for further instructions on trip recovery. Cathode channels can simply be switched on individually by clicking on the "Ena" fields. Potential, Gate and Backdrift channels require a more complex handling because of their common connection to bulk supplies. In order to switch on e.g. a Potential channel *__P that was off before, first ramp down the related bulk supply *_PBS by clicking on its "Dis" button. Then wait for rampdown completion until the Status is 0. All Potential channels that are connected to this bulk supply and that are enabled follow the voltage rampdown of its bulk supply. Then enable/disable all desired Potential channels, wait until their status is reflected on their Status fields, and enable the Potential bulk supply *_PBS again. Eventually you modified the bulk supply voltage before. During the rampup, you may see the voltage rising in all related display fields. !!! ATTENTION: take care that the triple-set of Potential, Gate and Backdrift voltages of every wire net are either in on-state or off-state together at the same time: e.g.: HV_DC_W_S_KS00_X1__P | either all on HV_DC_W_S_KS00_X1__G | or HV_DC_W_S_KS00_X1__B | all off Control parameters as trip current limits, ramp up and ramp down rates etc. can be set in similar manner by activating the panels under the Control parameter buttons on the Main MEDM Menu. Once you are happy with the setup of channels, you may save the entire setup of parameters for DC_EAST and DC_WEST from the Main Menu. Click on "Save" --> "Save all settings with time stamp" --> an ASCII file (in SNAP format) will appear under the path /export/software/oncs/epics/R3.13.0.beta11/epicsB/... ...app/hvca/medm/adl/dc_setup You may save the "enable/disable" status of all channels by clicking on: "Save: --> "Save enable/disable status of channels" This information is used when using the "Enable" button to activate all channels of either DC_EAST or DC_WEST together. To activate the entire group DC_EAST or DC_WEST, click on the group's "Enable" button on the Main MEDM Menu. To disable - i.e. switch off - the entire group DC_EAST or DC_WEST, click on the group's "Rampdown" button on the Main MEDM Menu. All channels that can ramp down are going to be disabled by this command. You will see that all Cathodes and all Bulk supplies are disbled, and thus all voltages ramp down with the rates specified in the Control paramets panels. Finally, all voltages will be zero or close to zero at the minimum possible value (10-20 V). The individual Potential, Gate and Backdrift channels will stay in "enabled" status but are off since their bulk supplies are off. This feature was implemented to avoid the "immediate-off" that occurs if the Potential, Gate and Bulk channels are disabled individually while the bulk supplies still deliver voltage. Once all channels are at zero voltage, you might disable them all entirely by starting a shell script "set_DC_E_dis.csh". However, this is not required in standard circumstances! b-2) use shell scripts to set voltages and other control parameters -------------------------------------------------------------------- ... this is intended for experts ... voltages and control parameters can also be adjusted or read out by using shell scripts that use the EPICS Backup and Restore Tool BURT. Several shell scripts have already been provided under the path: /export/software/oncs/epics/R3.13.0.beta11/epicsB/... ...app/hvca/medm/adl/dc_setup e.g.: set_DC_E_en.csh set_DC_E_0V.csh set_DC_E_10V.csh set_DC_E_100V.csh set_DC_E_500V.csh set_DC_E_dis.csh --> see documentation for further details: "How to use shell scripts from the terminal prompt to set values of EPICS records and retrieve values from EPICS records" 5/ Trips and Trip Recovery: =========================== If a Drift Chamber High Voltage channel trips during operation, it is most likely due to current exceeding the current limit. The trip status is displayed in two ways: - Voltage/control parameter panel: the status is shown in red color, status number is e.g. 12345. - The Alarm Handler detects the trip, its main icon changes to red color and most likely starts beeping. Open the Alarm Handler Main Window by clicking on the icon. If Drift Chamber East or West is causing the Alarm condition, a red color is visible next to the DC_W or DC_E group. Open the group and browse the channel list to identify the tripped channel. If a Cathode channel has tripped, usually just this individual channel is off or in its trip state, respectively. If a Potential, Gate or Backdrift channel has tripped, the Alarm Handler has automatically issued a safety-off command for the neighbouring channels of this Potential, Gate, Backdrift triplet of channels. Usually, one channel should be in tripped state, the other two channels should be in disabled state. However, it might happen that all channels of this triplet tripped at the same time or during the safety-off, so that all trip flags have been cleared. Then go to the MEDM Voltage/Control panel in which the tripped channel is handled and try to recover from this trip. - HIGHLY RECOMMENDED: follow the instructions that can be read when clicking on the HELP button at the bottom of every page!! --------------------------------------------------------------------------- Manual procedure ... for experts only: ====================================== This is for non-standard situations only and requires some expertise: 1. If a Cathode channel (_C) has tripped, simply click on its ENABLE button. 2. If Potential, Gate or Backdrift (_P, _G, _B) channels have tripped, follow a slightly more complicated procedure: a/ Disable the tripped _P, _G or _B channel. Its status will change to 0. Note that the 'partners' of the tripped channel have been shut down by the Alarm Handler already and that their status is 0 as well. b/ Then disable ALL THREE bulk supply channels _PBS, _GBS, _BBS on the bottom of the page by clicking on their DISABLE buttons. The bulk supplies ramp down (status changes to 5) and take down with them all voltages of the the other _P, _G, _B channels that are connected to the bulk supplies as well. WAIT until the bulk supply channels have completed their rampdown and their status shows 0. c/ Then enable the individual channels in the region again where the trip had occured. !!! IMPORTANT: enable all three channels of the triplet _P, _G, _B !!! Their status will change to 1 after a few seconds. d/ Last step: enable ALL THREE bulk supply channels again. Click on the enable buttons next to the three channels _PBS, _GBS, _BBS at the bottom of the page. The status bits of the bulk supplies change to 3 during ramp-up. Watch the voltage and current readings of the bulk supply channels and the channels connected during the rampup. The detector should be fine again !! --------------------------------------------------------------------------- 6/ How to reboot the crate controllers IOCONDEV3, IOCONDEV4 and IOCONDEV5 ========================================================================= Sometimes just MEDM hangs and it is enough to exit MEDM and Alarm Handler and start them again. However, occasionally the communication from IOCONDEV3, IOCONDEV4 or IOCONDEV5 to the hardware gets lost, or there is some other kind of hang-up in these VME systems. The diagnostics is then -- there are white "missing" icons on the MEDM panels -- the Alarm Handler is blinking white -- error messages or outdated (old) sequencer messages on the serial terminals on PHONCS5 You need to reboot the crate controllers then: --> reset either IOCONDEV3, IOCONDEV4 or IOCONDEV5 e.g. by physically pushing the reset pin on their front panel --> on PHONCS5, see the "reboot" messages in the serial terminal. then type "10 minutes) to bring up all the HV control system again!!! See the PHENIX High Voltage Control Manual for details. 7/ How to rebuild and reconfigure the MEDM HV control GUI, the ALARM handler ============================================================================ See the PHENIX High Voltage Control Manual for details. In brief: * cd /export/software/oncs/epics/R3.13.0.beta11/epicsB/app/hvca/db * check/modify the subsystems listed in the configuration files: iocondev3.grp iocondev4.grp iocondev5.grp gui.grp the full set of sub-systems is: iocondev3.grp ============= 6 DC_W_S 9 PC_W 12 RICH_W_S 15 PBSC_W iocondev4.grp ============= 1 MVD 2 BB 3 ZDC 5 DC_W_N 13 RICH_W_N 16 MUID iocondev5.grp ============= 4 DC_E 7 PC_E 10 TEC_E 11 RICH_E 14 PBSC_E gui.grp ======= 1 MVD 2 BB 3 ZDC 4 DC_E 5 DC_W_N 6 DC_W_S 7 PC_E 9 PC_W 10 TEC_E 11 RICH_E 12 RICH_W_S 13 RICH_W_N 14 PBSC_E 15 PBSC_W 16 MUID note: in those files, get rid of all characters (CR/LF, blanks) after the last line, otherwhise a funny detector group will be created when building the MEDM gui. * build the MEDM GUI + Alarm Handler: make -s gui * build only Alarm Handler: make -s alarm Similarly, the EPICS databases for IOCONDEV3, IOCONDEV4 and IOCONDEV5 can be build: (usually not required under standard circumstances) make -s iocondev3 make -s iocondev4 make -s iocondev5 !!! Remember to restart MEDM and Alarm Handler if you modified the setup !!!