|
This page last changed on Apr 14, 2009 by panetta.
How to Fix Halfpipe
Owned by: Philip Hart
Note: Halfpipe processing runs entirely in the pipeline and is subject to the standard problems that occur there.
 |
With the following exceptions, there is little that goes wrong with the pipeline:
- The makeEvt process will, for example, occasionally fail due to network glitches.
- To Fix: Simply roll it back after first checking the pipeline log for errors indicating preliminary action.
- LCI runs which have not been completely received.
- To Fix: If the missing data has been permanently lost, force the launchOnline process to start pipeline online
analysis by touching a file named by the run id in the force directory under:
- /nfs/farm/g/glast/u42/ISOC-flight/Downlinks/force and rolling back the process.
The stuck data will be apparent by looking in .../stage.
- FastCopy Ingest has partially failed due to NFS issues, causing downstream issues
- Symptom: Fastcopy is in INGEST-FAILED. launchL1 has at least one failure. Offline wishes the job to finish, but does not want L1 to launch again.
- Solution: The whole job does not need to be rolled back, only launchL1 (with the actual processing disabled). To disable the processing, touch
- /nfs/farm/g/glast/u42/ISOC-Flight/Downlinks/<id>/haltL1. Then rollback the failed L1 job.
|
For unexpected issues see the code in:
- /afs/slac.stanford.edu/g/glast/isoc/flightOps/volumes/vol5/offline/halfPipe/prod
and installed in:
- /afs/slac/g/glast/isoc/flightOps/offline/halfPipe/
|
|
