Jobsub ID 251757.1@dunegpschedd02.fnal.gov
| Jobsub ID | 251757.1@dunegpschedd02.fnal.gov |
| Workflow ID | 10687 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| HTCondor Group | group_dune |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2025-11-25 15:08:27 |
| Site | NL_SURFsara |
| Entry | DUNE_SurfSARA_arc02 |
| Last heartbeat | 2025-11-25 19:14:49 |
| From worker node | Hostname | wn-la-12.gina.surf.nl |
| cpuinfo | AMD EPYC 9754 128-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 129600 (36 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2025-11-25 15:09:10 |
| Input files | monte-carlo-010687-000001
|
| Jobscript | Exit code | 0 |
| Real time | 4h (14686s) |
| CPU time | 4h (14485s = 98%) |
| Max RSS bytes | 4137824256 (3946 MiB) |
| Outputting started | 2025-11-25 19:13:57 |
| Output files | |
| Finished | 2025-11-25 19:14:49 |
| List job events (HTCondor job logs unavailable) |
Jobscript log (last 10,000 characters)
nning tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare: Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 95305 traces, 0 summary
Retagger: tagging trace set: gauss with 74117 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 74117 traces tagged "gauss"
FrameSaver: q=2.57319e+06 n=1287206 tag=gauss
wclsFrameSaver: saving 95305 traces tagged "wiener"
FrameSaver: q=2.70971e+06 n=1137792 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 27892 XUs and 32433 XVs -> 548 XUVs
C:0 T:1 51438 XUs and 52345 XVs -> 3803 XUVs
C:0 T:2 41342 XUs and 52132 XVs -> 1626 XUVs
C:0 T:3 24696 XUs and 23900 XVs -> 56 XUVs
C:0 T:4 9362 XUs and 6992 XVs -> 68 XUVs
C:0 T:5 33029 XUs and 21642 XVs -> 3157 XUVs
C:0 T:6 46774 XUs and 44346 XVs -> 1752 XUVs
C:0 T:7 24310 XUs and 20952 XVs -> 292 XUVs
C:0 T:8 5357 XUs and 6060 XVs -> 66 XUVs
C:0 T:9 31095 XUs and 47233 XVs -> 2417 XUVs
C:0 T:10 33709 XUs and 49535 XVs -> 12072 XUVs
C:0 T:11 3411 XUs and 593 XVs -> 307 XUVs
26164 XUVs total
8213 collection wire objects
24687 potential space points
Neighbour search...
3830451 tests to find 1691422 neighbours
Iterating with no regularization...
Begin: 6.27185e+07
0 5.29425e+07
1 5.2252e+07
2 5.21582e+07
3 5.21345e+07
Now with regularization...
Begin: 4.35061e+07
0 4.34724e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410732539
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410732539
fLifetime: 30000 [us]
25-Nov-2025 20:13:55 CET Closed output file "reco_stage1.root"
25-Nov-2025 20:13:55 CET Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 284.199 411.885 668.471 394.408 107.386 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00112529 0.00554529 0.00944846 0.00580525 0.00260943 10
reco:ophit:OpHitFinder 0.343004 0.408884 0.488562 0.404245 0.0485986 10
reco:opflash:OpFlashFinder 0.192547 0.267769 0.371803 0.256075 0.0537232 10
reco:opslicer:OpSlicer 0.563555 0.804641 0.948391 0.843198 0.126171 10
reco:caldata:DataPrepModule 6.18987 6.5259 7.08286 6.44993 0.277921 10
reco:wclsdatasp:WireCellToolkit 42.7552 55.7896 67.011 55.9897 7.36251 10
reco:gaushit:GausHitFinder 0.918059 1.27247 1.79946 1.2284 0.282592 10
reco:nhitsfilter:NumberOfHitsFilter 0.000362173 0.000584245 0.00105417 0.000505611 0.000200056 10
reco:reco3d:SpacePointSolver 8.91695 17.9157 30.7928 15.9638 6.78153 10
reco:hitpdune:DisambigFromSpacePoints 0.883207 1.6555 3.1594 1.55087 0.678162 10
reco:pandora:StandardPandora 17.7345 34.0279 66.3308 32.4063 13.943 10
reco:pandoraWriter:StandardPandora 0.345105 0.480592 0.700774 0.454507 0.104409 10
reco:pandoraTrack:LArPandoraTrackCreation 3.29777 6.48944 14.8786 5.91526 3.09312 10
reco:pandoraShower:LArPandoraShowerCreation 3.40968 7.99721 24.3389 6.85229 5.90208 10
reco:pandoracalo:Calorimetry 1.78678 3.43368 7.50425 3.11984 1.57488 10
reco:pandoracalonosce:Calorimetry 1.62485 3.18034 7.15431 2.92349 1.52635 10
reco:pandorapid:Chi2ParticleID 0.00496924 0.00624885 0.00831244 0.00608306 0.00102037 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.040977 0.0533961 0.0760844 0.0502855 0.0102469 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00479682 0.00593279 0.00796082 0.00592705 0.000835601 10
reco:pandoraShowercalo:ShowerCalorimetry 3.18496 7.62748 20.8643 6.82752 5.0145 10
reco:pandoraShowercalonosce:ShowerCalorimetry 2.96842 7.24558 19.3499 6.40505 4.64841 10
reco:emtrkmichelid:EmTrackMichelId 141.359 190.433 269.254 180.063 35.3382 10
reco:crttag:SingleCRTMatchingProducer 0.00915651 0.0236308 0.0555991 0.020503 0.0128016 10
reco:crtreco:TwoCRTMatchingProducer 0.00790932 0.01348 0.0237084 0.012367 0.00422712 10
reco:anodepiercerst0:T0RecoAnodePiercers 0.802171 1.03174 1.39376 1.03265 0.180636 10
reco:pandora2Track:LArPandoraTrackCreation 10.0984 20.5 44.2523 18.4198 9.76085 10
reco:pandora2calo:Calorimetry 4.86512 9.99341 21.7274 8.73487 5.04273 10
reco:pandora2calonosce:Calorimetry 4.33432 9.39277 19.7689 8.11826 4.70201 10
reco:pandora2pid:Chi2ParticleID 0.0108539 0.0138435 0.019687 0.0137154 0.00246401 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.050623 0.0672619 0.0963817 0.0648604 0.0129014 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.0110583 0.0138775 0.0194213 0.013759 0.00238978 10
reco:pandora2Shower:LArPandoraShowerCreation 3.46752 7.17902 12.5273 6.75769 2.97287 10
reco:pandora2Showercalo:ShowerCalorimetry 3.34963 7.94714 13.9201 6.68168 3.65836 10
reco:pandora2Showercalonosce:ShowerCalorimetry 2.98575 6.58787 10.7638 5.8436 2.65715 10
[art]:TriggerResults:TriggerResultInserter 3.9068e-05 5.4169e-05 0.000127721 4.5768e-05 2.47939e-05 10
end_path:out1:RootOutput 7.741e-06 9.0888e-06 1.5013e-05 8.4225e-06 2.01952e-06 10
end_path:out1:RootOutput(write) 2.40353 3.26482 4.67554 3.04192 0.677287 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4943.12 MB
Peak resident set size usage (VmHWM): 4137.82 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.06 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.29 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 4.06 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.17 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9834624
-rw-r--r--. 1 dune009 dune 2311557979 Nov 25 19:03 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2190692014 Nov 25 19:00 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 1984174940 Nov 25 20:13 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1790238622 Nov 25 16:30 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1721824383 Nov 25 16:16 g4_stage1.root
-rw-r--r--. 1 dune009 dune 71557110 Nov 25 20:10 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune 242612 Nov 25 20:13 jobscript.log
-rw-r--r--. 1 dune009 dune 217088 Nov 25 20:13 mem.db
-rw-r--r--. 1 dune009 dune 62376 Nov 25 20:13 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune 36864 Nov 25 20:13 time.db
-rw-r--r--. 1 dune009 dune 519 Nov 25 19:03 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune 519 Nov 25 16:30 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune 519 Nov 25 20:13 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune 52 Nov 25 16:09 all-input-dids.txt
-rw-r--r--. 1 dune009 dune 7 Nov 25 20:13 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune 0 Nov 25 19:10 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune 0 Nov 25 19:03 debugprod.log
