Grid

Grid

We can use the Grid for processing large amounts of data. Currently, the appropriate software is only available on polui01.

First, you need to create the proper Grid credentials (certificate).

Then, for each access to the Grid, you need to load these credentials by creating a proxy. Here is the procedure at LLR, on polui01:

# start the proxy management service
 $  /opt/exp_soft/vo.llr.in2p3.fr/tools/proxymanagement --start
# create a local proxy with long life (1000h). this is not associated with a VO role
 $  /opt/exp_soft/vo.llr.in2p3.fr/tools/proxymanagement --proxy --life=1000:00
# create a voms proxy for harpo.cea.fr. lifetime 24h, automatically renewed by the service
 $  $(/opt/exp_soft/vo.llr.in2p3.fr/tools/proxymanagement --voms=harpo.cea.fr)
# create a proxy delegation
 $  /opt/exp_soft/vo.llr.in2p3.fr/tools/proxymanagement --ce=llrcream.in2p3.fr

You can access the ROOT data on the Grid using xrootd. Directly using ROOT:

 $ root root://polgrid4.in2p3.fr//harpo.cea.fr/Reco/2016/01/run1290_reco.root

Using the HARPO framework:

 $ harpoanalysis -a HarpoAnalyseVertex3D -b run1290_hist.root root://polgrid4.in2p3.fr//harpo.cea.fr/Reco/2016/01/run1290_reco.root -n 100

We can submit a full reconstruction job (using processRun.sh). The actual Grid script is runGrid.sh. The procedure is as follows:

# submit job for processing N events of run RUNNO
 $ $HARPO_SCRIPT_DIR/submitRun.sh -r RUNNO -n N
# checking job status
 $ glite-ce-job-status  https://llrcream.in2p3.fr:8443/CREAM977372712
# getting job outputs (logs)
 $ glite-ce-job-output  https://llrcream.in2p3.fr:8443/CREAM977372712
# Accessing a file on the grid
 $ root root://polgrid4.in2p3.fr//harpo.cea.fr/Reco/2016/01/run1421_reco.root

Finally, you can run an analysis on reconstructed data on the Grid using:

 $ $HARPO_SCRIPT_DIR/submitAnaRun.sh -c CONFIGFILE.cfg -j JDLFILE.jdl -O "-a HarpoVertexMatching -n 1000" -r RUNNO -o jobs.jid
 $ glite-ce-job-status jobs.jid
 $ xrdcp root://polgrid4.in2p3.fr//harpo.cea.fr/Reco/2016/01/runRUNNO_hist.root runRUNNO_hist.root

where CONFIGFILE.cfg is the configuration file (default $HARPO_SCRIPT_DIR/HarpoReconstruction.cfg), and JDLFILE.jdl is the job description (default JDLFILE=$HARPO_SCRIPT_DIR/processAnaRun.jdl.in)

You can use a database query to run an analysis on all run with the same properties:

 $ for run in `harpoGetRunList "Ephoton = 72"`; 
   do 
       $HARPO_SCRIPT_DIR/submitAnaRun.sh -O "-a HarpoVertexMatching -n 1000" -r $run -o jobs.jid;
   done
 $ glite-ce-job-status jobs.jid
 $ for run in `harpoGetRunList "Ephoton = 72"`; 
   do
        gridfile=/harpo.cea.fr/Reco/2016/01/run${run}_hist_new.root
        localfile=/data/DATA/data.harpo/2012/Hist/run${run}_hist_new.root
        xrdcp -f -s root://polgrid4.in2p3.fr/$gridfile $localfile; 
   done