A PHENIX-managed repository has been created on GitHub in order to capture the original code (i.e. before adjustments were made to make it suitable for long-term preservation): https://github.com/PhenixCollaboration/emcal
Parts of this analysis use data samples kept in the storage area intended for long-term preservation. This is its location in the GPFS filesystem of BNL SDCC:
/gpfs/mnt/gpfs02/phenix/data_preservation/phnxreco/emcal
There are “DeadWarn” and “Timing” type of maps which are prerequisite of this analysis and they are considered as a separate “prior” component. In a condensed form, they are preserved in the folder sim_Pi0Histogram/creatingHistogram_fromTTree in the repository specified above.
REANA operates using containers. For the analyses presented here, Docker images created by the PHENIX Collaboration are used. They are hosted in a private registry maintained the BNL SDCC. For the container to work properly, a number of setup step are required, as listed below:
setenv OFFLINE_MAIN /cvmfs/phenix.sdcc.bnl.gov/x8664_sl7/release/release_new/new
setenv ONLINE_MAIN /cvmfs/phenix.sdcc.bnl.gov/x8664_sl7/release/release_new/new
setenv ROOTSYS /cvmfs/phenix.sdcc.bnl.gov/x8664_sl7/opt/phenix/core/root-5.34.36
setenv G4_MAIN /cvmfs/phenix.sdcc.bnl.gov/x8664_sl7/opt/phenix/core/geant4.10.00.p02
source /opt/phenix/core/stow/opt_phenix_scripts/bin/phenix_setup.csh
setenv LD_LIBRARY_PATH .:$LD_LIBRARY_PATH
setenv ODBCINI ${PWD}/afs/rhic.bnl.gov/phenix/etc/odbc.ini
setenv PG_PHENIX_DBNAME Phenix_phnxdbrcf2_C
Since every line in the REANA submission file (formatted in YAML) has its own environment
the setup needs to be performed for every step that needs PHENIX-specific environment
variables. For this reasons the commands are often used with csh
wrapper that
is reponsible for the setup. In the following example, ROOT macros are run from within
the script pi0extraction.csh
.
version: 0.0.1
inputs:
directories:
- ./raw_taxiData
- ./sim_Pi0Histogram
files:
- ./pi0extraction.C
- ./WGRatio.C
- ./generationRM_Pi0.C
- ./VConvolution_Pi0.C
- ./universal.h
- ./setup_env.csh
- ./pi0extraction.csh
workflow:
type: serial
specification:
steps:
- environment: 'registry.sdcc.bnl.gov/sdcc-fabric/rhic_sl7_ext:1.3'
commands:
- mkdir -p output_plots/pdf output_plots/txt output_plots/root
- chmod +x ./pi0extraction.csh
- ls -l > output.txt
- ./pi0extraction.csh >> output.txt
- ls -l >> output.txt
outputs:
files:
- output.txt
REANA implementation of this analysis is based on the workflow illustrated in this partial diagram.
In order for this analysis to run within the REANA framework certain adjustments need to be made, such as
Additional changes include some code cleanup and a minimal amount of renaming of code units, functions and scripts for better readability. The resulting code prepared for REANA is kept in this PHENIX repository on GitHub (Work in Progress): https://github.com/PhenixCollaboration/reana/tree/main/pi0extraction.
The analysis starts with files produced by the Taxi process. For example,
the ROOT macro pioExtraction.cc
takes the Taxi ROOT files as input and generates MB
(min bias)
and ERT
(triggered) data as output. This macro is included in a driver script corrPi0Chain.csh
.
Below is an outline of the analysis sequence with references to “block numbers” in the workflow diagram, along with pointers to relevant REANA components
# Block 1
# condor_Pi0Extraction.cc reformatted and renamed "pi0extraction.C"
root -l -b -q 'pi0extraction.C("MB", "PbSc", 4,5)'
root -l -b -q 'pi0extraction.C("ERT", "PbSc", 4,5)'
root -l -b -q 'WGRatio.C' # Merging MV and ERT spectra of raw pi0 with normalization
# The outputs of this step are placed in the output_plots folder,
# in three subfolders pdf, root, txt
These commands are included in pi0extraction.yaml
.
Currently a folder output_plots
is created with subfolders txt,root,pdf
,
and the folder txt
contains the actual analsys data.
Presented below is the core of Block 2 which includes processing of multiple input files (60 in total):
# Block 2, the original code found in the Condor submission part:
root -l -b <<EOF
.L Pi0EmbedFiles.C
Pi0EmbedFiles t
t.Loop()
EOF
Adaptation of the ROOT macro for REANA, in a separate file pi0run.script:
gSystem->Load("libTHmul.so");
.L Pi0EmbedFiles.C
Pi0EmbedFiles t
t.Loop()
In the REANA script, this is used as follows: cat pi0run.script | root -b
.
Note that a PHENIX-specific ROOT library libTHmul.so
is loaded
in the beginning, as this is necessary for proper operation of the macro.
Please refer to the relevant folder in the PHENIX GitHub repository for access to the actual material.
This is the driver script Pi0EmbedFiles.csh
. Note that symbolic links are created
to feed successive files from a holding folder, to the ROOT macro.
#!/bin/tcsh
source ./setup_env.csh
foreach i (`seq 0 1 $1`)
ln -s gpfs/mnt/gpfs02/phenix/data_preservation/phnxreco/emcal/Pi0/test/simPi0_$i.root pi0_dAuMB.root
echo File: $i
ls -l pi0_dAuMB.root
cat pi0run.script | root -b
mv EmbedPi0dAu.root EmbedPi0dAu_$i.root
rm pi0_dAuMB.root
end
tar -cf embedPi0dAu.tar EmbedPi0dAu_*
Processing of input files takes place sequentially and in this case takes a significant amount of time compared to other steps, i.e. a feew hours.
The results of all emedding runs are bundled together in a tar archive to make downloading easier. Upon
retrieval the data need to be merged using the utility haddPhenix
which is done in Block 3 (see below).
Upon completion of this step the file embedPi0dAu.tar
needs to be downloaded, and put
in the folder from where the next step is launched. An example of the cownload command, assuming the workflow
was named “embed”:
reana-client download -w embed embedPi0dAu.tar
The original macro generationRM_Pi0.cc
was cleaned up (including removal of interactive graphics)
and renamed generationRM_Pi0.C
.
Tar file containing multiple ROOT files (see Block 2 description above) is uploaded as input for this step. Abbreviated contents of driver script look as follows:
#!/bin/tcsh
source ./setup_env.csh
haddPhenix EmbedPi0dAu.root EmbedPi0dAu_*
root -l -b -q 'generationRM_Pi0.C'
The macro generates the file EmbedPi0dAu.root
by merging inputs via haddPhenix
and produces Pion_RM.root
. The complete description is in generationRM_Pi0.yaml
,
which resides with all subsidiary scripts in the folder generationRM
.
The workflow description is as follows:
version: 0.0.1
inputs:
files:
- ./setup_env.csh
- ./generationRM_Pi0.C
- ./generationRM_Pi0.csh
- ./embedPi0dAu.tar
workflow:
type: serial
specification:
steps:
- environment: 'registry.sdcc.bnl.gov/sdcc-fabric/rhic_sl7_ext:1.3'
commands:
- tar xvf embedPi0dAu.tar
- chmod +x ./generationRM_Pi0.csh
- ./generationRM_Pi0.csh > output.txt
- ls -l >> output.txt
outputs:
files:
- output.txt
- Pion_RM.root
The result will need to be downloaded as follows (assuming the worflow was assigned the name “gen” in REANA - can be anything):
reana-client download -w gen Pion_RM.root
This REANA step (final in this analysis)
is accomplished with scripts and macros named VConvolution_Pi0*
. The file Pion_RM.root
serves as input, along with text files residing in output_plots
previosly produced by the macros
pi0extraction.C
and WGRatio.C
:
scaledUEB_rawPi0_ERT_PbSc_0CC88_Chi2_3Sig.txt
scaledUEB_rawPi0_MB_PbSc_0CC88_Chi2_3Sig.txt
scaledUEB_rawPi0_BBCpERT_PbSc_0CC88_Chi2_3Sig.txt
The core of this step looks like this:
# Block 4
root -l -b -q 'VConvolution_Pi0.C'
The REANA submission file has this content:
version: 0.0.1
inputs:
directories:
- ./output_plots
files:
- ./inputTrialFunction_Pi0.txt
- ./setup_env.csh
- ./VConvolution_Pi0.C
- ./VConvolution_Pi0.csh
- ./Pion_RM.root
workflow:
type: serial
specification:
steps:
- environment: 'registry.sdcc.bnl.gov/sdcc-fabric/rhic_sl7_ext:1.3'
commands:
- chmod +x ./VConvolution_Pi0.csh
- ls -l > output.txt
- ./VConvolution_Pi0.csh >> output.txt
- ls -l >> output.txt
outputs:
files:
- output.txt0
Outputs are also written in output_plots/txt
and contain
scaledEB_corrPi0_BBCpERT_PbSc_0CC88_Chi2_3Sig.txt
scaledUEB_corrPi0_BBCpERT_PbSc_0CC88_Chi2_3Sig.txt