Jobsub ID 263383.0@dunegpschedd02.fnal.gov
| Jobsub ID | 263383.0@dunegpschedd02.fnal.gov |
| Workflow ID | 11272 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| HTCondor Group | group_dune |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2025-12-19 07:40:42 |
| Site | IT_CNAF |
| Entry | CMSHTPC_T1_IT_CNAF_condor_ce02 |
| Last heartbeat | 2025-12-19 10:12:29 |
| From worker node | Hostname | wn-s04-05-38-02.cr.cnaf.infn.it |
| cpuinfo | AMD EPYC 7313 16-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2025-12-19 07:42:03 |
| Input files | monte-carlo-011272-000001
|
| Jobscript | Exit code | 0 |
| Real time | 2h (8597s) |
| CPU time | 2h (8296s = 96%) |
| Max RSS bytes | 4355624960 (4153 MiB) |
| Outputting started | 2025-12-19 10:05:21 |
| Output files | |
| Finished | 2025-12-19 10:12:29 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 139240 traces, 0 summary
Retagger: tagging trace set: gauss with 106353 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 106353 traces tagged "gauss"
FrameSaver: q=3.67907e+06 n=1905474 tag=gauss
wclsFrameSaver: saving 139240 traces tagged "wiener"
FrameSaver: q=3.87775e+06 n=1687372 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 34130 XUs and 34088 XVs -> 391 XUVs
C:0 T:1 67397 XUs and 63289 XVs -> 8893 XUVs
C:0 T:2 93580 XUs and 121812 XVs -> 5401 XUVs
C:0 T:3 43454 XUs and 49892 XVs -> 302 XUVs
C:0 T:4 9437 XUs and 7464 XVs -> 31 XUVs
C:0 T:5 27894 XUs and 17398 XVs -> 1687 XUVs
C:0 T:6 90967 XUs and 119273 XVs -> 4717 XUVs
C:0 T:7 36332 XUs and 44958 XVs -> 319 XUVs
C:0 T:8 12315 XUs and 10590 XVs -> 239 XUVs
C:0 T:9 125274 XUs and 113237 XVs -> 17033 XUVs
C:0 T:10 45156 XUs and 37784 XVs -> 5835 XUVs
C:0 T:11 60 XUs and 37 XVs -> 6 XUVs
44854 XUVs total
12404 collection wire objects
41738 potential space points
Neighbour search...
4802124 tests to find 2313418 neighbours
Iterating with no regularization...
Begin: 7.46205e+07
0 6.26476e+07
1 6.18211e+07
2 6.16837e+07
3 6.16425e+07
Now with regularization...
Begin: 5.19074e+07
0 5.18537e+07
1 5.18242e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411209148
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411209148
fLifetime: 30000 [us]
19-Dec-2025 11:05:19 CET Closed output file "reco_stage1.root"
19-Dec-2025 11:05:19 CET Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 164.12 233.743 270.503 240.185 27.4606 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000686989 0.00269359 0.00446342 0.0026963 0.00120898 10
reco:ophit:OpHitFinder 0.198817 0.230421 0.264973 0.223136 0.0192387 10
reco:opflash:OpFlashFinder 0.145106 0.175115 0.216091 0.174554 0.0196656 10
reco:opslicer:OpSlicer 0.378076 0.494117 0.591724 0.489843 0.0641649 10
reco:caldata:DataPrepModule 3.39708 3.50857 3.7008 3.46265 0.103467 10
reco:wclsdatasp:WireCellToolkit 22.4112 23.4206 24.0434 23.5661 0.464634 10
reco:gaushit:GausHitFinder 0.585463 0.797493 0.996185 0.779858 0.115915 10
reco:nhitsfilter:NumberOfHitsFilter 0.000283602 0.00038321 0.000525547 0.00035961 6.41973e-05 10
reco:reco3d:SpacePointSolver 4.89222 13.6298 23.3581 13.8206 4.88563 10
reco:hitpdune:DisambigFromSpacePoints 0.709578 1.2412 1.74522 1.2186 0.258537 10
reco:pandora:StandardPandora 10.998 22.6646 28.1612 22.4933 4.8827 10
reco:pandoraWriter:StandardPandora 0.206279 0.282193 0.325794 0.290496 0.0321304 10
reco:pandoraTrack:LArPandoraTrackCreation 1.68204 2.58051 3.1322 2.63152 0.355008 10
reco:pandoraShower:LArPandoraShowerCreation 1.70319 2.82103 3.77882 2.80657 0.522523 10
reco:pandoracalo:Calorimetry 0.938313 1.52885 1.79664 1.5802 0.217471 10
reco:pandoracalonosce:Calorimetry 0.875152 1.41472 1.67304 1.45055 0.200112 10
reco:pandorapid:Chi2ParticleID 0.00313103 0.00373454 0.00503791 0.00362838 0.00048231 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0258429 0.0350566 0.0415806 0.0357908 0.00445109 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00293246 0.0035019 0.00421602 0.00347243 0.000319558 10
reco:pandoraShowercalo:ShowerCalorimetry 1.66129 3.19033 4.3423 3.26344 0.658308 10
reco:pandoraShowercalonosce:ShowerCalorimetry 1.62222 3.13043 4.2027 3.20863 0.640072 10
reco:emtrkmichelid:EmTrackMichelId 92.995 121.896 141.077 123.366 11.7941 10
reco:crttag:SingleCRTMatchingProducer 0.00637842 0.0108495 0.0159953 0.0101808 0.00329922 10
reco:crtreco:TwoCRTMatchingProducer 0.00393746 0.00556929 0.00657467 0.0055112 0.000712818 10
reco:anodepiercerst0:T0RecoAnodePiercers 0.484667 0.618171 0.754342 0.597892 0.0774181 10
reco:pandora2Track:LArPandoraTrackCreation 4.95214 8.32245 10.7168 8.67324 1.41751 10
reco:pandora2calo:Calorimetry 2.47348 4.50931 5.91893 4.67144 0.838323 10
reco:pandora2calonosce:Calorimetry 2.35414 4.33001 5.62989 4.49823 0.796759 10
reco:pandora2pid:Chi2ParticleID 0.00700165 0.00859444 0.00961614 0.00867409 0.000691124 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0308397 0.0430937 0.0515997 0.0440263 0.00550754 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.00695334 0.00842418 0.00928457 0.00853692 0.000693257 10
reco:pandora2Shower:LArPandoraShowerCreation 1.98369 3.08066 3.895 3.17479 0.474444 10
reco:pandora2Showercalo:ShowerCalorimetry 2.21295 3.98444 5.01097 4.15125 0.701174 10
reco:pandora2Showercalonosce:ShowerCalorimetry 2.05478 3.67362 4.69704 3.86455 0.648178 10
[art]:TriggerResults:TriggerResultInserter 2.5799e-05 4.07907e-05 0.000121118 3.3027e-05 2.69879e-05 10
end_path:out1:RootOutput 5.942e-06 7.6523e-06 1.2112e-05 7.404e-06 1.60305e-06 10
end_path:out1:RootOutput(write) 1.51604 2.0323 2.41116 2.04275 0.227106 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5155.71 MB
Peak resident set size usage (VmHWM): 4355.62 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.05 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.15 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.03 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 2.22 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.08 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.03 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11100324
-rw-r--r-- 1 neutrino002 neutrino 2569281434 Dec 19 10:25 detsim_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 2433401637 Dec 19 10:23 detsim_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 2291004223 Dec 19 11:05 reco_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 2034104171 Dec 19 08:57 g4_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 1955348068 Dec 19 08:48 g4_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 82964000 Dec 19 11:02 Pandora_Events.pndr
-rw-r--r-- 1 neutrino002 neutrino 243202 Dec 19 11:05 jobscript.log
-rw-r--r-- 1 neutrino002 neutrino 217088 Dec 19 11:05 mem.db
-rw-r--r-- 1 neutrino002 neutrino 62376 Dec 19 11:05 Pandora_Geometry.xml
-rw-r--r-- 1 neutrino002 neutrino 36864 Dec 19 11:05 time.db
-rw-r--r-- 1 neutrino002 neutrino 519 Dec 19 10:25 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino 519 Dec 19 08:57 g4_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino 519 Dec 19 11:05 protoDUNE_reco1_hist.root
-rw-r--r-- 1 neutrino002 neutrino 52 Dec 19 08:42 all-input-dids.txt
-rw-r--r-- 1 neutrino002 neutrino 7 Dec 19 11:05 justin-processed-pfns.txt
-rw-r--r-- 1 neutrino002 neutrino 0 Dec 19 10:29 ProtoDUNE.log
-rw-r--r-- 1 neutrino002 neutrino 0 Dec 19 10:25 debugprod.log
