Jobsub ID 288557.1@dunegpschedd02.fnal.gov
| Jobsub ID | 288557.1@dunegpschedd02.fnal.gov |
| Workflow ID | 12107 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-01-20 14:48:54 |
| Site | UK_RAL-PPD |
| Entry | CMSHTPC_T2_UK_SGrid_RALPP_hep208 |
| Last heartbeat | 2026-01-20 19:39:27 |
| From worker node | Hostname | heplnc157.pp.rl.ac.uk |
| cpuinfo | AMD EPYC 7763 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2026-01-20 14:50:54 |
| Input files | monte-carlo-012107-000001
|
| Jobscript | Exit code | 0 |
| Real time | 4h (17269s) |
| CPU time | 4h (16681s = 96%) |
| Max RSS bytes | 5217484800 (4975 MiB) |
| Outputting started | 2026-01-20 19:38:44 |
| Output files | |
| Finished | 2026-01-20 19:39:27 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
ning tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 156455 traces, 0 summary
Retagger: tagging trace set: gauss with 121225 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 121225 traces tagged "gauss"
FrameSaver: q=4.28775e+06 n=2169201 tag=gauss
wclsFrameSaver: saving 156455 traces tagged "wiener"
FrameSaver: q=4.50844e+06 n=1919562 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 42642 XUs and 39263 XVs -> 431 XUVs
C:0 T:1 80441 XUs and 81356 XVs -> 7151 XUVs
C:0 T:2 99009 XUs and 114988 XVs -> 3691 XUVs
C:0 T:3 49660 XUs and 51378 XVs -> 344 XUVs
C:0 T:4 11635 XUs and 13272 XVs -> 86 XUVs
C:0 T:5 46913 XUs and 40785 XVs -> 3894 XUVs
C:0 T:6 142789 XUs and 110315 XVs -> 5257 XUVs
C:0 T:7 59476 XUs and 43498 XVs -> 514 XUVs
C:0 T:8 8838 XUs and 10560 XVs -> 158 XUVs
C:0 T:9 59725 XUs and 98220 XVs -> 7437 XUVs
C:0 T:10 82276 XUs and 37019 XVs -> 7839 XUVs
C:0 T:11 344 XUs and 201 XVs -> 27 XUVs
36829 XUVs total
14199 collection wire objects
34552 potential space points
Neighbour search...
3016484 tests to find 1395240 neighbours
Iterating with no regularization...
Begin: 9.93622e+07
0 8.40485e+07
1 8.30538e+07
2 8.29144e+07
3 8.28755e+07
Now with regularization...
Begin: 7.0008e+07
0 6.999e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411853287
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411853287
fLifetime: 30000 [us]
20-Jan-2026 19:38:42 GMT Closed output file "reco_stage1.root"
20-Jan-2026 19:38:42 GMT Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 412.03 555.614 815.621 534.111 119.117 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.0014306 0.00504678 0.00981529 0.00502817 0.00258235 10
reco:ophit:OpHitFinder 0.291678 0.392301 0.495988 0.403454 0.0698065 10
reco:opflash:OpFlashFinder 0.224676 0.283055 0.341887 0.282577 0.044649 10
reco:opslicer:OpSlicer 0.650726 0.97487 1.21151 0.958549 0.164689 10
reco:caldata:DataPrepModule 4.89788 6.46942 8.14894 6.58472 1.07094 10
reco:wclsdatasp:WireCellToolkit 35.3055 41.2106 45.279 42.8483 3.35237 10
reco:gaushit:GausHitFinder 1.32661 1.97334 2.43304 2.04417 0.294408 10
reco:nhitsfilter:NumberOfHitsFilter 0.000513123 0.000815484 0.000967046 0.000835299 0.000118782 10
reco:reco3d:SpacePointSolver 16.0814 29.6306 68.7265 27.4096 14.1208 10
reco:hitpdune:DisambigFromSpacePoints 1.69548 2.88165 4.92703 2.97994 0.922506 10
reco:pandora:StandardPandora 37.4663 69.0109 125.166 56.6988 27.2915 10
reco:pandoraWriter:StandardPandora 0.433043 0.691795 0.943767 0.697531 0.151904 10
reco:pandoraTrack:LArPandoraTrackCreation 4.73922 7.67379 14.3002 6.47143 2.67653 10
reco:pandoraShower:LArPandoraShowerCreation 4.84372 8.19004 15.3753 7.35323 3.01135 10
reco:pandoracalo:Calorimetry 2.86313 4.17728 8.43929 3.60489 1.54704 10
reco:pandoracalonosce:Calorimetry 2.27144 3.95527 7.45095 3.50397 1.38647 10
reco:pandorapid:Chi2ParticleID 0.00555595 0.00912395 0.0143488 0.00912199 0.00281136 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0463564 0.0796479 0.115291 0.0824998 0.0217986 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00552419 0.00865876 0.0132341 0.00816798 0.00236094 10
reco:pandoraShowercalo:ShowerCalorimetry 4.47119 8.32668 16.2491 7.00031 3.17901 10
reco:pandoraShowercalonosce:ShowerCalorimetry 4.11851 8.59612 14.7938 8.48907 2.74927 10
reco:emtrkmichelid:EmTrackMichelId 224.442 278.113 347.17 264.684 39.5929 10
reco:crttag:SingleCRTMatchingProducer 0.0168311 0.0220421 0.0274563 0.021828 0.00294557 10
reco:crtreco:TwoCRTMatchingProducer 0.00987404 0.0139338 0.0172786 0.0142127 0.00248973 10
reco:anodepiercerst0:T0RecoAnodePiercers 0.925757 1.23769 1.54067 1.28546 0.210358 10
reco:pandora2Track:LArPandoraTrackCreation 14.7239 24.7414 42.9074 21.7594 8.2636 10
reco:pandora2calo:Calorimetry 6.53573 11.6869 18.3356 10.269 3.90033 10
reco:pandora2calonosce:Calorimetry 6.17489 12.4854 22.5235 11.358 4.70728 10
reco:pandora2pid:Chi2ParticleID 0.0116314 0.0236188 0.0379291 0.0226789 0.00755861 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0613444 0.106932 0.153963 0.115201 0.0302071 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.0110241 0.0220464 0.0353534 0.0221118 0.00733333 10
reco:pandora2Shower:LArPandoraShowerCreation 4.36683 8.45926 14.6442 8.35374 3.12521 10
reco:pandora2Showercalo:ShowerCalorimetry 6.31938 10.4031 16.8645 10.2117 3.14057 10
reco:pandora2Showercalonosce:ShowerCalorimetry 5.24964 9.42834 14.6027 9.24993 2.8346 10
[art]:TriggerResults:TriggerResultInserter 5.0204e-05 8.64524e-05 0.000200166 8.00305e-05 3.95055e-05 10
end_path:out1:RootOutput 1.092e-05 1.36698e-05 2.1611e-05 1.31145e-05 2.81665e-06 10
end_path:out1:RootOutput(write) 2.89428 4.15568 5.75043 3.98916 0.874963 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5621.99 MB
Peak resident set size usage (VmHWM): 4745.08 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.08 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.29 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.05 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 4.09 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.16 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.05 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 12598760
-rw-r--r-- 1 pltdune04 pltdune 2872800569 Jan 20 18:03 detsim_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2728703202 Jan 20 18:01 detsim_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2635157835 Jan 20 19:38 reco_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2324772868 Jan 20 15:17 g4_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2244526826 Jan 20 14:59 g4_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 94353840 Jan 20 19:31 Pandora_Events.pndr
-rw-r--r-- 1 pltdune04 pltdune 242925 Jan 20 19:38 jobscript.log
-rw-r--r-- 1 pltdune04 pltdune 217088 Jan 20 19:38 mem.db
-rw-r--r-- 1 pltdune04 pltdune 62376 Jan 20 19:38 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune04 pltdune 36864 Jan 20 19:38 time.db
-rw-r--r-- 1 pltdune04 pltdune 519 Jan 20 18:03 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune 519 Jan 20 15:17 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune 519 Jan 20 19:38 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune04 pltdune 52 Jan 20 14:50 all-input-dids.txt
-rw-r--r-- 1 pltdune04 pltdune 7 Jan 20 19:38 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune04 pltdune 0 Jan 20 18:14 ProtoDUNE.log
-rw-r--r-- 1 pltdune04 pltdune 0 Jan 20 18:03 debugprod.log
