HOWTO update EMCAL Objy DB from calibration ASCII files


Last modified: Wed Feb 4 18:25:43 CET 2004 Valid for emcDB version >= 1.5 (04-February-2004)
Previous version of this document is here

This page will teach you how to use the emcDB program to write calibration data ASCII files into the PHENIX EMCAL Objectivity databases, and to browse EMCAL databases content.

This is an interim version, where internals have been changed to allow for the introduction of PostgreSQL in the game, in the very near future. Chances exist that those internal changes broke something...

Disclaimer: the emcDB program is only a convenient tool to update/dump the EMCAL databases. It is not a bullet-proof program ! It can for sure be improved. If you have special requests, do not hesitate to contact me (aphecetc@in2p3.fr).

Important warning

In order not to pollute the official PHENIX Objectivity Federation with garbage or to corrupt it, it is strongly recommended that before using emcDB in update mode (option --update) you test it under a private Federation.

Setup of a private Objectivity Federation

  1. Get a Federation ID (FDID) for you private Federation (FD). There's a PHENIX web page for this.
  2. A shell script is there to help you create/destroy your private Federation.
> unsetenv OO_FD_BOOT (to insure you won't play with the real one...)
> ./privateFD.csh /phenix/dataXX/your_user_id/TESTDB TEST your_FDID new
  to create a new private Federation with the relevant EMCAL databases 
> ./privateFD.csh /phenix/dataXX/your_user_id/TESTDB TEST your_FDID del
  to erase your private Federation

your_FDID is the one you get from step 1 above.

If you want to know more about what's going on with Objy stuff, you may want to start with David's Objectivity tutorial (but that's not required for the following).

Reminder

So far, we have 18 EMCAL calibration databases, corresponding to 9 calibration data "flavours" (there are 2 DBs per flavour : one tag-DB and one data-DB). The database names are of the form : calib.emc.XYZ.pdb, where X is the data flavour :

Y is Tag (for Tag-DB) or Cal (for data-DB), and Z is a sequence number (0000 for the moment).

In the following, databases are referred to by the flavour name (e.g. Gains).

Using the emcDB program

Pre-run check-list

You must set the OO_FD_BOOT environment variable to the Federation boot file you'd like to access.

setenv OO_FD_BOOT yourFDBOOT (to access your private FD)
or
setenv OO_FD_BOOT ${PHENIX_FD_BOOT} (to access PHENIX FD)

Getting help on the emcDB program

You can get the syntax by doing:

> emcDB --help

Browsing what is in a given flavour database

Use the following syntax :

emcDB --dump calib.emc.[flavourName]
where flavourName is one of Gains, Pedestals, Pedestals5 (for the RunII pedestals), QAs, LCTofs, WalkTofs, TofT0s. Note that flavours are case-sensitive.

The above command is an interactive one, i.e. for each bank header, you'll be asked if you want to dump also the content of this bank (type 0 to continue, -1 to abort program, or any number n in between to see n channels). This can be quite painful for databases with a lot of banks. To overcome this difficulty, you can either use the --batch option to not be prompted for any answer, and/or restrict the set of bank you'd like to inspect, using the --(inserted|start|end)(After|Before) and/or --bankID options, e.g.

emcDB --dump calib.emc.Gains --bankID 108 --insertedAfter "2001,12,20,12,36,45"

which will show you only Gains banks for FEM 108, inserted into database after 2001, December, the 20th, at 12 h 36 m 45 s.

Putting calibration ASCII files into the Objy DB

The general syntax is :

emcDB --update directory

where directory is the path to the calibration data files. Please note that official ASCII calibration files are usually to be found somewhere under /afs/rhic/phenix/phnxemc.

The above command will read all calibration files to be found under directory and will put them into the relevant database. Special note on pedestals : depending on the number of pedestal-calibration files for one FEM (3 or 5), a different database will be filled (calib.emc.Pedestals or calib.emc.Pedestals5). It should be automatic.

31-Aug-2001 Due to a possible problem in PHTimeStamp::operator >> , please always use the --forceDate option (see below), as the timestamp specified in the ASCII files themselves is currently not properly read out.

22-Jan-2002 The above bug should be fixed. But still you might want to use the --forceDate option to bypass the date which is in the files (e.g. to insure that a set of banks has the same start-of-validity period).

The --update option will write things into the Federation, so please be sure you are talking to the Federation you expect, i.e. the official PHENIX one (if OO_FD_BOOT variable is set to PHENIX_FD_BOOT) or your private one (if OO_FD_BOOT variable is set to yourFDBOOT).

Special note on ToF-related calibrations

The ToF directory usually contains 3 calibration flavours : the walk corrections, least-count, and t0-drift ones. If you'd like to update only one of them, here's the procedure to follow :

  1. Make a fresh somewhere/ToF directory
  2. Copy the relevant flavour you'd like to put in the DB, e.g. :
    cp /yourplace/ToF/*WALK* somewhere/ToF
    
  3. Then use emcDB as usual :
    emcDB --forceDate "year,month,day,hour,minute,second" 
          --update somewhere/ToF
    

Specifying the date : the --forceDate option

Usually, start-of-validity dates for the calibration data are fetch from the ASCII files themselves. If you want to override those, use the --forceDate option of the emcDB program. It takes a date string, in a format which is the one of the PHTimeStamp constructor :

> emcDB --forceDate "2001,08,01,12,45,00" 
  --update /afs/rhic/phenix/phnxemc/EMC/m_080701

Special note on PbGl gains for Run II

This is a little bit special case.

Many Gains files are split over many directories in e.g. /phenix/data24/vikonnik/Gains/RunXXXX, each of those directories being a normal top directory for calibration flavours, i.e. :

/phenix/data24/vikonnik/Gains/Run31250/Gains
                             /Run31250/QA
                             /Run33350/Gains
                             /Run33350/QA
...

The idea is to use each RunXXXX directory as a calibration set. For this, we need both start and end of validity period for each set (start=time stamp of first run of the set, end=time stamp of first run of the next set to avoid time-gaps).

So the first step is to get those time stamps. We do have a PHENIX Run Database, so we will use it for that purpose. We could have make it transparent, but a) some run used by PbGl are not in this database and hence the relationship runnumber<->time stamp can not be found using the Run DB and b) for testing purpose we'd like to be able to read PHENIX Run DB while writing Gains into a private Federation (and currently Objy is not able to open 2 Federations in one single process...). So we use an interim file.

Jan-2004The Objectivity Run database is no longer maintained. More recent runs (and also older ones) are in principle available in Postgres db instead. So the old way of getting the runnumber to timestamp relationship has changed as well :

cd some_dir
emcDB --runtimes 90000 --dbms Pg

This will select Postgre (Pg) as the DataBase Management System from which to fetch runtimes, for runnumber greater or equal to 90000, and will produce a emc_runtimes_initfromfile.tmp.txt file, like :

36503,Sun Dec 16 15:10:53 2001,Sun Dec 16 15:11:22 2001
36504,Sun Dec 16 15:11:43 2001,Sun Dec 16 15:12:13 2001
36505,Sun Dec 16 15:12:34 2001,Sun Dec 16 15:13:02 2001
36506,Sun Dec 16 15:13:25 2001,Sun Dec 16 15:13:52 2001
36507,Sun Dec 16 15:14:14 2001,Sun Dec 16 15:14:42 2001
36508,Sun Dec 16 15:15:03 2001,Sun Dec 16 15:15:32 2001
36509,Sun Dec 16 15:15:59 2001,Sun Dec 16 15:16:26 2001

Then, using information contained in Viktor's files for the run missing in PHENIX Run DB, we completed this file to obtain a new one (emc_runtimes_initfromfile.tmp.txt), covering all the runs we need.

We then use the --updatePbGlGains option to actually read the Gains files and put them into the Gains database, into a given Federation (might be PHENIX_FD_BOOT...).

The command below has not been thoroughly re-checked to be working after the latest changes to the emcDB program. Please report any problem you may find when using it. Also note that only emcDB version >= 1.7 (23 March 2004) has this option reconnected.

cd some_dir (same as before !)
setenv OO_FD_BOOT some_federation
emcDB --updatePbGlGains /phenix/data24/vikonnik/Gains

And that's it ! Please note that it might be a little long...

Comparing Objy content and ASCII files

If you'd like to know from where a given Objy bank is coming from (and assuming you still have the directory that was used when using --update), you may do :

> emcDB --dump calib.emc.Gains --bankID 0 --insertedAfter "2002,1,1,0,0,0" --compare /afs/rhic/phenix/phnxemc/EMC/m_091001
--------------------------------------------------------------------------------
emcDB 1.5 (04 February 2004)
--------------------------------------------------------------------------------
Dumping calib.emc.Gains calibration database 
--------------------------------------------------------------------------------
BankID = 0
Insert   : Mon Jan 14 13:05:22 2002
StartVal : Mon Sep 10 10:30:26 2001
EndVal   : Mon Jan 18 22:14:07 2038
Calibration Description : 
User Name = Unknown
[# of channels] Show channels [n] Next bank  [q] Quit, [c] Compare to file(s) 
Please enter command > 

If you then hit the C key, you'll see something like :

MATCHES the one in /afs/rhic/phenix/phnxemc/EMC/m_091001

Please note that you can use several --compare options to compare the current Objy bank against several directories.

Please note also that for pedestals, you need to use the --config gdb.dat option (which is used to find the FEM absolutePosition to PinNumber relationship). The gdb.dat file you can probably get from CVS online_configuration directory.


Laurent Aphecetche

Go Home