Jobsub ID 270038.0@dunegpschedd01.fnal.gov
| Jobsub ID | 270038.0@dunegpschedd01.fnal.gov |
| Workflow ID | 11328 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2025-12-24 20:39:26 |
| Site | UK_Bristol |
| Entry | CMSHTPC_T2_UK_SGrid_Bristol_lcgce02 |
| Last heartbeat | 2025-12-25 01:56:51 |
| From worker node | Hostname | hd77.dice.priv |
| cpuinfo | AMD EPYC 7551P 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 259200 (72 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2025-12-24 20:40:17 |
| Input files | monte-carlo-011328-000001
|
| Jobscript | Exit code | 0 |
| Real time | 5h (18971s) |
| CPU time | 5h (18541s = 97%) |
| Max RSS bytes | 4335792128 (4134 MiB) |
| Outputting started | 2025-12-25 01:56:28 |
| Output files | |
| Finished | 2025-12-25 01:56:51 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
oolBasedRawDigitPrepService:prepare: Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 119808 traces, 0 summary
Retagger: tagging trace set: gauss with 93421 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 93421 traces tagged "gauss"
FrameSaver: q=3.32216e+06 n=1704499 tag=gauss
wclsFrameSaver: saving 119808 traces tagged "wiener"
FrameSaver: q=3.50495e+06 n=1520345 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 47002 XUs and 48930 XVs -> 757 XUVs
C:0 T:1 99335 XUs and 118422 XVs -> 8061 XUVs
C:0 T:2 97120 XUs and 131973 XVs -> 4951 XUVs
C:0 T:3 50511 XUs and 61742 XVs -> 469 XUVs
C:0 T:4 10196 XUs and 9997 XVs -> 68 XUVs
C:0 T:5 40634 XUs and 34969 XVs -> 3706 XUVs
C:0 T:6 89342 XUs and 73042 XVs -> 4749 XUVs
C:0 T:7 36397 XUs and 28703 XVs -> 356 XUVs
C:0 T:8 5165 XUs and 4889 XVs -> 53 XUVs
C:0 T:9 39849 XUs and 51799 XVs -> 8118 XUVs
C:0 T:10 73936 XUs and 22891 XVs -> 7953 XUVs
C:0 T:11 335 XUs and 175 XVs -> 63 XUVs
39304 XUVs total
11080 collection wire objects
36040 potential space points
Neighbour search...
4059068 tests to find 1916764 neighbours
Iterating with no regularization...
Begin: 6.76933e+07
0 5.40469e+07
1 5.31761e+07
2 5.30388e+07
3 5.29928e+07
Now with regularization...
Begin: 4.3796e+07
0 4.37507e+07
1 4.3722e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411320478
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411320478
fLifetime: 30000 [us]
25-Dec-2025 01:56:26 UTC Closed output file "reco_stage1.root"
25-Dec-2025 01:56:26 UTC Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 384.791 516.346 588.675 527.904 54.1373 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00116813 0.00396276 0.00612835 0.00400277 0.00165502 10
reco:ophit:OpHitFinder 0.301209 0.364498 0.435471 0.367738 0.0325999 10
reco:opflash:OpFlashFinder 0.182588 0.250729 0.292601 0.252256 0.0287009 10
reco:opslicer:OpSlicer 0.824358 1.06924 1.18933 1.11414 0.123106 10
reco:caldata:DataPrepModule 6.92915 7.22422 7.60133 7.23148 0.166343 10
reco:wclsdatasp:WireCellToolkit 47.1524 51.9419 58.7198 51.3013 3.77241 10
reco:gaushit:GausHitFinder 1.30039 1.71823 1.93612 1.77651 0.189925 10
reco:nhitsfilter:NumberOfHitsFilter 0.000495941 0.000657546 0.000875495 0.000642767 0.000116613 10
reco:reco3d:SpacePointSolver 13.688 20.7432 33.7697 19.6773 6.19985 10
reco:hitpdune:DisambigFromSpacePoints 1.46169 2.17142 2.58634 2.22016 0.288111 10
reco:pandora:StandardPandora 29.9222 50.0463 76.394 50.5451 12.6291 10
reco:pandoraWriter:StandardPandora 0.461797 0.576061 0.643428 0.577852 0.0476487 10
reco:pandoraTrack:LArPandoraTrackCreation 4.4359 7.08131 9.22622 7.21543 1.35319 10
reco:pandoraShower:LArPandoraShowerCreation 3.78193 7.2196 8.33631 7.49392 1.29545 10
reco:pandoracalo:Calorimetry 2.34131 3.80195 4.56495 3.96619 0.668556 10
reco:pandoracalonosce:Calorimetry 2.08395 3.48583 4.29387 3.62322 0.65241 10
reco:pandorapid:Chi2ParticleID 0.00717683 0.00883822 0.0117517 0.00856205 0.00127287 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0705097 0.0831023 0.0977345 0.0830066 0.00723691 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00677765 0.00832168 0.010019 0.00820641 0.000959715 10
reco:pandoraShowercalo:ShowerCalorimetry 3.38559 7.0772 8.65846 7.11714 1.38616 10
reco:pandoraShowercalonosce:ShowerCalorimetry 3.31212 6.89173 8.98843 6.95551 1.44031 10
reco:emtrkmichelid:EmTrackMichelId 215.926 273.533 302.318 277.761 22.4721 10
reco:crttag:SingleCRTMatchingProducer 0.0189255 0.0244911 0.0295416 0.0237628 0.00310585 10
reco:crtreco:TwoCRTMatchingProducer 0.0113996 0.0153421 0.0179217 0.015722 0.00172845 10
reco:anodepiercerst0:T0RecoAnodePiercers 1.07012 1.23561 1.47309 1.20711 0.140864 10
reco:pandora2Track:LArPandoraTrackCreation 11.4852 21.1527 25.3416 21.1039 3.65678 10
reco:pandora2calo:Calorimetry 5.14787 10.4769 12.11 11.3595 2.01432 10
reco:pandora2calonosce:Calorimetry 4.75468 9.9309 11.8629 10.5479 1.94875 10
reco:pandora2pid:Chi2ParticleID 0.0160524 0.0215547 0.0242689 0.0220552 0.00222631 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0806473 0.10431 0.119153 0.107605 0.010206 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.0160461 0.0213118 0.0236902 0.0216806 0.00204889 10
reco:pandora2Shower:LArPandoraShowerCreation 4.58966 7.78676 9.24818 8.00016 1.26468 10
reco:pandora2Showercalo:ShowerCalorimetry 4.84482 8.69992 10.2584 9.15308 1.47395 10
reco:pandora2Showercalonosce:ShowerCalorimetry 3.98158 7.63634 8.9862 7.964 1.33921 10
[art]:TriggerResults:TriggerResultInserter 6.7768e-05 8.344e-05 0.000187984 7.2381e-05 3.49578e-05 10
end_path:out1:RootOutput 1.1141e-05 1.29122e-05 1.8735e-05 1.23885e-05 1.99926e-06 10
end_path:out1:RootOutput(write) 3.07664 3.81062 4.26779 3.85586 0.323352 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5154.74 MB
Peak resident set size usage (VmHWM): 4335.79 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.09 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.26 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.06 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 4.63 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.14 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.06 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10509264
-rw-r--r-- 1 dnpil001 users 2457023754 Dec 25 00:27 detsim_stage2.root
-rw-r--r-- 1 dnpil001 users 2332910281 Dec 25 00:24 detsim_stage1.root
-rw-r--r-- 1 dnpil001 users 2137282905 Dec 25 01:56 reco_stage1.root
-rw-r--r-- 1 dnpil001 users 1914553115 Dec 24 21:06 g4_stage2.root
-rw-r--r-- 1 dnpil001 users 1845759604 Dec 24 20:48 g4_stage1.root
-rw-r--r-- 1 dnpil001 users 73341640 Dec 25 01:49 Pandora_Events.pndr
-rw-r--r-- 1 dnpil001 users 243098 Dec 25 01:56 jobscript.log
-rw-r--r-- 1 dnpil001 users 217088 Dec 25 01:56 mem.db
-rw-r--r-- 1 dnpil001 users 62376 Dec 25 01:56 Pandora_Geometry.xml
-rw-r--r-- 1 dnpil001 users 36864 Dec 25 01:56 time.db
-rw-r--r-- 1 dnpil001 users 519 Dec 25 00:27 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dnpil001 users 519 Dec 24 21:06 g4_protoDUNE_hist.root
-rw-r--r-- 1 dnpil001 users 519 Dec 25 01:56 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dnpil001 users 52 Dec 24 20:40 all-input-dids.txt
-rw-r--r-- 1 dnpil001 users 7 Dec 25 01:56 justin-processed-pfns.txt
-rw-r--r-- 1 dnpil001 users 0 Dec 25 00:36 ProtoDUNE.log
-rw-r--r-- 1 dnpil001 users 0 Dec 25 00:27 debugprod.log
