Jobsub ID 257417.0@dunegpschedd01.fnal.gov
| Jobsub ID | 257417.0@dunegpschedd01.fnal.gov |
| Workflow ID | 10684 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| HTCondor Group | group_dune |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2025-11-25 15:08:27 |
| Site | NL_SURFsara |
| Entry | DUNE_SurfSARA_arc03 |
| Last heartbeat | 2025-11-25 19:06:44 |
| From worker node | Hostname | wn-la-19.gina.surf.nl |
| cpuinfo | AMD EPYC 9754 128-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 129600 (36 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2025-11-25 15:09:15 |
| Input files | monte-carlo-010684-000001
|
| Jobscript | Exit code | 0 |
| Real time | 3h (14214s) |
| CPU time | 3h (14021s = 98%) |
| Max RSS bytes | 4208902144 (4013 MiB) |
| Outputting started | 2025-11-25 19:06:10 |
| Output files | |
| Finished | 2025-11-25 19:06:44 |
| List job events (HTCondor job logs unavailable) |
Jobscript log (last 10,000 characters)
tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 131954 traces, 0 summary
Retagger: tagging trace set: gauss with 98288 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 98288 traces tagged "gauss"
FrameSaver: q=3.37616e+06 n=1722038 tag=gauss
wclsFrameSaver: saving 131954 traces tagged "wiener"
FrameSaver: q=3.58167e+06 n=1515396 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 38022 XUs and 34277 XVs -> 547 XUVs
C:0 T:1 59800 XUs and 52533 XVs -> 4562 XUVs
C:0 T:2 119209 XUs and 139686 XVs -> 7234 XUVs
C:0 T:3 54098 XUs and 58226 XVs -> 362 XUVs
C:0 T:4 9062 XUs and 6375 XVs -> 71 XUVs
C:0 T:5 36713 XUs and 19475 XVs -> 5635 XUVs
C:0 T:6 210909 XUs and 149761 XVs -> 28120 XUVs
C:0 T:7 45907 XUs and 37566 XVs -> 785 XUVs
C:0 T:8 5279 XUs and 6930 XVs -> 74 XUVs
C:0 T:9 26816 XUs and 38477 XVs -> 2574 XUVs
C:0 T:10 66694 XUs and 51319 XVs -> 8988 XUVs
C:0 T:11 109 XUs and 93 XVs -> 9 XUVs
58961 XUVs total
12101 collection wire objects
53961 potential space points
Neighbour search...
10613799 tests to find 4396932 neighbours
Iterating with no regularization...
Begin: 6.83542e+07
0 5.86245e+07
1 5.7922e+07
2 5.78129e+07
3 5.77831e+07
Now with regularization...
Begin: 4.77391e+07
0 4.77015e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410732575
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410732575
fLifetime: 30000 [us]
25-Nov-2025 20:06:08 CET Closed output file "reco_stage1.root"
25-Nov-2025 20:06:08 CET Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 263.547 365.845 496.021 325.716 72.9597 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00119187 0.00494215 0.00739319 0.00524507 0.0019998 10
reco:ophit:OpHitFinder 0.322827 0.379746 0.490119 0.377847 0.0463927 10
reco:opflash:OpFlashFinder 0.182167 0.235564 0.344396 0.236599 0.0456058 10
reco:opslicer:OpSlicer 0.613153 0.767424 1.13792 0.733151 0.158908 10
reco:caldata:DataPrepModule 6.11806 6.40367 7.21469 6.34594 0.285067 10
reco:wclsdatasp:WireCellToolkit 43.4604 59.1897 67.9319 59.4237 7.65224 10
reco:gaushit:GausHitFinder 0.841799 1.16981 1.58352 1.14573 0.189015 10
reco:nhitsfilter:NumberOfHitsFilter 0.000359225 0.000589723 0.000998205 0.000576284 0.000184907 10
reco:reco3d:SpacePointSolver 7.4043 16.8165 29.2632 14.5865 7.00903 10
reco:hitpdune:DisambigFromSpacePoints 0.842135 1.32368 2.22843 1.16648 0.400222 10
reco:pandora:StandardPandora 18.0536 35.5912 109.331 22.0698 26.8683 10
reco:pandoraWriter:StandardPandora 0.319743 0.449185 0.637468 0.414506 0.0855665 10
reco:pandoraTrack:LArPandoraTrackCreation 2.58109 4.92517 7.56512 4.49682 1.6438 10
reco:pandoraShower:LArPandoraShowerCreation 2.0394 4.97474 7.63816 4.32491 1.78223 10
reco:pandoracalo:Calorimetry 1.27905 2.52792 4.17435 2.26566 0.894188 10
reco:pandoracalonosce:Calorimetry 1.16006 2.41726 4.09229 2.20223 0.828923 10
reco:pandorapid:Chi2ParticleID 0.00437289 0.00562706 0.00752837 0.00539596 0.000901705 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0380489 0.0476545 0.065991 0.0445461 0.00855994 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.004246 0.00548274 0.00623685 0.0055065 0.000578337 10
reco:pandoraShowercalo:ShowerCalorimetry 1.90119 4.94904 9.66789 3.97341 2.31662 10
reco:pandoraShowercalonosce:ShowerCalorimetry 1.7637 4.60468 8.38437 3.65627 2.08963 10
reco:emtrkmichelid:EmTrackMichelId 128.47 172.924 237.048 164.815 28.6639 10
reco:crttag:SingleCRTMatchingProducer 0.0117723 0.017607 0.0322221 0.0166541 0.00565494 10
reco:crtreco:TwoCRTMatchingProducer 0.00818477 0.0112961 0.0180013 0.0103655 0.00284888 10
reco:anodepiercerst0:T0RecoAnodePiercers 0.718685 0.901901 1.31318 0.872277 0.168582 10
reco:pandora2Track:LArPandoraTrackCreation 6.61291 13.8894 24.0101 11.8441 5.15252 10
reco:pandora2calo:Calorimetry 2.78316 6.6317 11.8821 5.25973 2.81977 10
reco:pandora2calonosce:Calorimetry 2.73181 6.06065 10.62 4.95746 2.44081 10
reco:pandora2pid:Chi2ParticleID 0.00824695 0.0117784 0.0147902 0.0115999 0.00175237 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0457996 0.0596411 0.0800314 0.055357 0.00990026 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.00836273 0.0118313 0.014723 0.0116924 0.00173898 10
reco:pandora2Shower:LArPandoraShowerCreation 2.43558 5.03037 7.90686 4.26565 1.74896 10
reco:pandora2Showercalo:ShowerCalorimetry 2.52591 5.58725 9.04479 4.48076 2.25306 10
reco:pandora2Showercalonosce:ShowerCalorimetry 2.3806 4.77051 8.71135 3.92108 1.87603 10
[art]:TriggerResults:TriggerResultInserter 3.9067e-05 6.28626e-05 0.000157544 5.5458e-05 3.21204e-05 10
end_path:out1:RootOutput 7.44e-06 9.3118e-06 1.4562e-05 8.548e-06 1.9735e-06 10
end_path:out1:RootOutput(write) 2.12269 2.9299 4.16062 2.73577 0.536117 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5008.5 MB
Peak resident set size usage (VmHWM): 4208.9 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.07 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.29 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 3.96 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.17 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9760328
-rw-r--r--. 1 dune009 dune 2306111007 Nov 25 19:03 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2195001523 Nov 25 19:00 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 1950013878 Nov 25 20:06 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1793466541 Nov 25 16:30 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1684635033 Nov 25 16:16 g4_stage1.root
-rw-r--r--. 1 dune009 dune 64727320 Nov 25 20:00 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune 242907 Nov 25 20:06 jobscript.log
-rw-r--r--. 1 dune009 dune 217088 Nov 25 20:06 mem.db
-rw-r--r--. 1 dune009 dune 62376 Nov 25 20:06 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune 36864 Nov 25 20:06 time.db
-rw-r--r--. 1 dune009 dune 519 Nov 25 19:03 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune 519 Nov 25 16:30 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune 519 Nov 25 20:06 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune 52 Nov 25 16:09 all-input-dids.txt
-rw-r--r--. 1 dune009 dune 7 Nov 25 20:06 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune 0 Nov 25 19:10 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune 0 Nov 25 19:03 debugprod.log
