- To be able to make some tests, we have defined our own fake calibration object (PdbTryChan) to be saved in the banks. The pdbcal framework allows us to do this in a couple of minutes, i.e. it works well for that purpose.
- About Sergei's question 'Is object selection fast?', the next generation of pdbcal headers, using Interval Skip List, should settle this point, and allow people to define time intervals like [now,ever] so you don't need the end time when you create a bank. It's of course an important point, but I would say we first need a full prototype to resolve this.
- Hisa said versioning will be added. How do we foreseen to use this versioning ?
- If I'm correct, the calibration DB has the following structure (assuming
you write PdbTryChan objects in it) :
So there's only one EMC container (emcCalContainer) in the cal. DB (or in the tag DB). This container cannot grow up to infinity (for Objy pages of 8192 bytes, max container size if 0.5 GB).
What happens when this container is full ? Should WE define a new database then ? Or will pdbcal do this for us (it does not for the moment I guess) ?
Some simple tests with pdbcal
Transaction(s)
We soon realized that we cannot do whatever we want to access the database. As you might know, the elementary unit of communication with the database is called a transaction. Objy ensure that a transaction is either a full success or a full failure (it can not be partially fullfilled). We first (naively) tried to write 1 PdbTryChan (representing 1 integer and 2 floats of calibration, for one tower) per bank, and 1000 such banks, in 1000 transactions (i.e. one each time we have a new calibration breakpoint). This is far TOO SLOW. From rlnx01, this is about 4 minutes (real time). If we do the same in ONE transaction only, it takes about 20 seconds ! May be you all know that, but I did not.So, we'd better have few huge transactions than a huge number of tiny transactions.
This simple fact is a great constraint on the write frequency we will choose (see question 1), and thus on the interfaces between Tracing process and DB on one hand, and between DB and reading process (offline reconstruction or whatever) on the other hand. Whatever the 'user interface' will be, we know that the underlying implementation must not make too tiny transactions.
Temptative timing for writing in the cal. DB
We define one transaction to be the writing of the full EMCAL (6 PbSc sectors + 2 PbGl sectors), where :- A PbSc sector is 18 banks with 144 calibration objects (PbdTryChan) in each bank
- A PbGl sector is 192 banks with 24 calibration objects in each bank.
| Number of transactions | Time (s) |
|---|---|
| 1 x (1 PbSc sector only) | 5-9 |
| 1 x (1 PbGl sector only) | 8-11 |
| 1 x Full EMCAL | 13-20 |
| 10 x Full EMCAL | 90 |
| 50 x Full EMCAL | 410 |
Working plan
Could we come to a reasonable working plan towards a realistic prototype ? May be based on a slighty modified version of what I proposed in my mail to Edward on 13-Oct-1999 (see Database discussion) :- from a data file, using some tracers, write a little piece of software to put, every time you have a breakpoint, some calibration object somewhere in a buffer object (stack or whatever the like). If this object is full, dump physically to Objy.
- from the same data file, read back the data and calibrate them using the db (this should be then used to connect with Gabor work I guess)
- simulate the fact that offline software could (and would certainly) want to update the calibrations, i.e. design an interface for that.
- try step (1) and (2) concurrently (i.e. writing in the db new calibrations for new runs and reading at the same time calibrations for older runs).
Laurent.Aphecetche@subatech.in2p3.fr
Last modified: Thu Nov 4 19:31:34 CET 1999