Jobsub ID 288562.0@dunegpschedd02.fnal.gov
| Jobsub ID | 288562.0@dunegpschedd02.fnal.gov |
| Workflow ID | 12116 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-01-20 14:48:56 |
| Site | UK_Brunel |
| Entry | CMSHTPC_T2_UK_London_Brunel_dc2_26 |
| Last heartbeat | 2026-01-20 17:57:37 |
| From worker node | Hostname | wn-a4-14 |
| cpuinfo | AMD EPYC 7452 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2026-01-20 14:52:28 |
| Input files | monte-carlo-012116-000001
|
| Jobscript | Exit code | 0 |
| Real time | 3h (10995s) |
| CPU time | 2h (10511s = 95%) |
| Max RSS bytes | 4111179776 (3920 MiB) |
| Outputting started | 2026-01-20 17:55:44 |
| Output files | |
| Finished | 2026-01-20 17:57:37 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
awDigitPrepService:prepare: Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare: Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 131618 traces, 0 summary
Retagger: tagging trace set: gauss with 102240 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 102240 traces tagged "gauss"
FrameSaver: q=3.55913e+06 n=1831791 tag=gauss
wclsFrameSaver: saving 131618 traces tagged "wiener"
FrameSaver: q=3.75819e+06 n=1628923 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:1 220421 XUs and 54040 XVs -> 18965 XUVs
C:0 T:2 131351 XUs and 147112 XVs -> 12024 XUVs
C:0 T:3 58844 XUs and 44957 XVs -> 518 XUVs
C:0 T:4 7765 XUs and 8355 XVs -> 71 XUVs
C:0 T:5 47097 XUs and 22539 XVs -> 4878 XUVs
C:0 T:6 114833 XUs and 123061 XVs -> 6145 XUVs
C:0 T:7 43553 XUs and 42284 XVs -> 472 XUVs
C:0 T:8 10483 XUs and 10279 XVs -> 135 XUVs
C:0 T:9 56879 XUs and 70625 XVs -> 5226 XUVs
C:0 T:10 69095 XUs and 70081 XVs -> 8690 XUVs
C:0 T:11 157 XUs and 137 XVs -> 17 XUVs
57141 XUVs total
12090 collection wire objects
52443 potential space points
Neighbour search...
10004837 tests to find 4344008 neighbours
Iterating with no regularization...
Begin: 7.14371e+07
0 6.10055e+07
1 6.03277e+07
2 6.02298e+07
3 6.02e+07
Now with regularization...
Begin: 5.02897e+07
0 5.02555e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411853252
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411853252
fLifetime: 30000 [us]
20-Jan-2026 17:55:42 GMT Closed output file "reco_stage1.root"
20-Jan-2026 17:55:42 GMT Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 239.529 345.907 440.659 359.162 62.5956 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000848242 0.00359668 0.00625105 0.00349329 0.0017224 10
reco:ophit:OpHitFinder 0.241088 0.288056 0.383372 0.280944 0.04029 10
reco:opflash:OpFlashFinder 0.117967 0.168502 0.231052 0.159398 0.0316882 10
reco:opslicer:OpSlicer 0.541737 0.680402 1.00473 0.652988 0.119713 10
reco:caldata:DataPrepModule 4.87084 5.12048 5.29819 5.10738 0.138043 10
reco:wclsdatasp:WireCellToolkit 31.5538 37.4521 41.1263 38.6627 3.37752 10
reco:gaushit:GausHitFinder 0.801459 1.14512 1.90366 1.09818 0.299063 10
reco:nhitsfilter:NumberOfHitsFilter 0.00039673 0.000577038 0.000985555 0.000527614 0.000165498 10
reco:reco3d:SpacePointSolver 6.93268 14.5376 19.6827 15.7747 3.83185 10
reco:hitpdune:DisambigFromSpacePoints 0.867405 1.43248 1.89451 1.49276 0.352344 10
reco:pandora:StandardPandora 14.902 33.5526 55.0992 34.6691 12.7903 10
reco:pandoraWriter:StandardPandora 0.309561 0.428046 0.519467 0.428616 0.0681971 10
reco:pandoraTrack:LArPandoraTrackCreation 1.66743 3.72241 5.62605 4.02684 1.1189 10
reco:pandoraShower:LArPandoraShowerCreation 1.44539 3.95428 6.93995 3.6841 1.65952 10
reco:pandoracalo:Calorimetry 0.934364 1.90021 2.81859 1.88062 0.568154 10
reco:pandoracalonosce:Calorimetry 0.814527 1.71505 2.61444 1.72883 0.539573 10
reco:pandorapid:Chi2ParticleID 0.00383585 0.00544486 0.007166 0.00542878 0.000988853 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0413764 0.0514928 0.0612788 0.0524576 0.00624747 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00379793 0.00517322 0.00614535 0.00542921 0.000757666 10
reco:pandoraShowercalo:ShowerCalorimetry 1.41339 3.83761 6.50903 4.06192 1.52321 10
reco:pandoraShowercalonosce:ShowerCalorimetry 1.33254 3.77577 6.36105 4.10035 1.49912 10
reco:emtrkmichelid:EmTrackMichelId 138.484 192.336 233.843 195.673 28.7549 10
reco:crttag:SingleCRTMatchingProducer 0.00788473 0.0135938 0.02053 0.0130844 0.00413782 10
reco:crtreco:TwoCRTMatchingProducer 0.00551891 0.00904853 0.014397 0.00870284 0.00260399 10
reco:anodepiercerst0:T0RecoAnodePiercers 0.653627 0.843352 1.01702 0.861076 0.142699 10
reco:pandora2Track:LArPandoraTrackCreation 4.71449 12.0823 20.3088 11.4704 4.45746 10
reco:pandora2calo:Calorimetry 2.12266 5.3038 8.28249 5.07332 1.86414 10
reco:pandora2calonosce:Calorimetry 1.9398 4.97754 7.65428 4.79545 1.70139 10
reco:pandora2pid:Chi2ParticleID 0.00839441 0.0133926 0.0172512 0.0137519 0.00255317 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0466266 0.0653766 0.078703 0.0673903 0.0101489 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.00839194 0.0133057 0.0169096 0.0138808 0.0025287 10
reco:pandora2Shower:LArPandoraShowerCreation 1.78526 4.48353 6.47569 4.49073 1.41367 10
reco:pandora2Showercalo:ShowerCalorimetry 1.81313 4.65593 6.7078 4.52571 1.57675 10
reco:pandora2Showercalonosce:ShowerCalorimetry 1.65783 4.18299 6.21473 4.14036 1.42604 10
[art]:TriggerResults:TriggerResultInserter 4.6661e-05 6.60534e-05 0.000152033 5.56915e-05 3.00079e-05 10
end_path:out1:RootOutput 1.145e-05 1.25894e-05 1.7411e-05 1.20405e-05 1.65824e-06 10
end_path:out1:RootOutput(write) 2.20886 3.01773 3.63439 3.17724 0.465756 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4944.81 MB
Peak resident set size usage (VmHWM): 4111.18 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.06 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.21 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 3.17 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.12 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9589572
-rw-r--r--. 1 dune000 dune 2276292993 Jan 20 16:55 detsim_stage2.root
-rw-r--r--. 1 dune000 dune 2163335928 Jan 20 16:53 detsim_stage1.root
-rw-r--r--. 1 dune000 dune 1946872752 Jan 20 17:55 reco_stage1.root
-rw-r--r--. 1 dune000 dune 1714077566 Jan 20 15:10 g4_stage2.root
-rw-r--r--. 1 dune000 dune 1647140905 Jan 20 14:59 g4_stage1.root
-rw-r--r--. 1 dune000 dune 71375610 Jan 20 17:50 Pandora_Events.pndr
-rw-r--r--. 1 dune000 dune 242940 Jan 20 17:55 jobscript.log
-rw-r--r--. 1 dune000 dune 217088 Jan 20 17:55 mem.db
-rw-r--r--. 1 dune000 dune 62376 Jan 20 17:55 Pandora_Geometry.xml
-rw-r--r--. 1 dune000 dune 36864 Jan 20 17:55 time.db
-rw-r--r--. 1 dune000 dune 519 Jan 20 16:55 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune000 dune 519 Jan 20 15:10 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune000 dune 519 Jan 20 17:55 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune000 dune 52 Jan 20 14:52 all-input-dids.txt
-rw-r--r--. 1 dune000 dune 7 Jan 20 17:55 justin-processed-pfns.txt
-rw-r--r--. 1 dune000 dune 0 Jan 20 17:02 ProtoDUNE.log
-rw-r--r--. 1 dune000 dune 0 Jan 20 16:55 debugprod.log
