Jobsub ID 270041.1@dunegpschedd01.fnal.gov
| Jobsub ID | 270041.1@dunegpschedd01.fnal.gov |
| Workflow ID | 11331 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2025-12-24 20:39:27 |
| Site | CERN |
| Entry | CMSHTPC_T2_CH_CERN_ce508 |
| Last heartbeat | 2025-12-25 01:33:29 |
| From worker node | Hostname | b9p04p6805.cern.ch |
| cpuinfo | AMD EPYC 7543 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 343800 (95 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2025-12-24 20:40:47 |
| Input files | monte-carlo-011331-000001
|
| Jobscript | Exit code | 0 |
| Real time | 4h (17537s) |
| CPU time | 4h (16451s = 93%) |
| Max RSS bytes | 4790804480 (4568 MiB) |
| Outputting started | 2025-12-25 01:33:05 |
| Output files | |
| Finished | 2025-12-25 01:33:29 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
ol adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 144642 traces, 0 summary
Retagger: tagging trace set: gauss with 110133 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 110133 traces tagged "gauss"
FrameSaver: q=3.79492e+06 n=1937551 tag=gauss
wclsFrameSaver: saving 144642 traces tagged "wiener"
FrameSaver: q=4.01399e+06 n=1712559 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 33288 XUs and 42532 XVs -> 443 XUVs
C:0 T:1 70716 XUs and 89226 XVs -> 7276 XUVs
C:0 T:2 138610 XUs and 140407 XVs -> 9963 XUVs
C:0 T:3 67730 XUs and 50442 XVs -> 483 XUVs
C:0 T:4 7685 XUs and 11332 XVs -> 400 XUVs
C:0 T:5 25129 XUs and 24486 XVs -> 2336 XUVs
C:0 T:6 113415 XUs and 118630 XVs -> 7182 XUVs
C:0 T:7 50059 XUs and 49019 XVs -> 461 XUVs
C:0 T:8 8953 XUs and 10616 XVs -> 114 XUVs
C:0 T:9 56800 XUs and 79677 XVs -> 3645 XUVs
C:0 T:10 36883 XUs and 31577 XVs -> 3951 XUVs
C:0 T:11 18 XUs and 13 XVs -> 3 XUVs
36257 XUVs total
12312 collection wire objects
33249 potential space points
Neighbour search...
2560767 tests to find 1308854 neighbours
Iterating with no regularization...
Begin: 6.89588e+07
0 5.7986e+07
1 5.72161e+07
2 5.71116e+07
3 5.70827e+07
Now with regularization...
Begin: 4.68884e+07
0 4.6892e+07
Warning: metric increased
run: 1 ; subrun: 0 ; event: 10
evttime: 411320492
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411320492
fLifetime: 30000 [us]
25-Dec-2025 02:33:02 CET Closed output file "reco_stage1.root"
25-Dec-2025 02:33:02 CET Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 375.989 470.191 744.352 445.368 97.3833 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00115582 0.00412276 0.00718858 0.00407299 0.00185649 10
reco:ophit:OpHitFinder 0.262991 0.3706 0.440008 0.377891 0.0477027 10
reco:opflash:OpFlashFinder 0.173433 0.258343 0.333022 0.272469 0.0502307 10
reco:opslicer:OpSlicer 0.604257 0.877873 1.17506 0.872843 0.16157 10
reco:caldata:DataPrepModule 4.75819 6.01439 7.26407 5.93345 0.856315 10
reco:wclsdatasp:WireCellToolkit 34.6599 37.2095 39.6699 37.3136 1.63081 10
reco:gaushit:GausHitFinder 1.04469 1.59725 2.03261 1.67107 0.337417 10
reco:nhitsfilter:NumberOfHitsFilter 0.000461008 0.000691554 0.000950665 0.000688267 0.000155292 10
reco:reco3d:SpacePointSolver 15.0825 26.1632 60.7493 24.7245 12.4361 10
reco:hitpdune:DisambigFromSpacePoints 2.00138 2.78422 4.4806 2.58422 0.681024 10
reco:pandora:StandardPandora 32.6532 56.9617 187.569 42.9256 43.9429 10
reco:pandoraWriter:StandardPandora 0.335164 0.566228 1.03966 0.591351 0.198789 10
reco:pandoraTrack:LArPandoraTrackCreation 3.12209 5.11666 9.83179 4.69418 1.83724 10
reco:pandoraShower:LArPandoraShowerCreation 3.36194 5.53231 9.37465 5.10598 1.57105 10
reco:pandoracalo:Calorimetry 2.12738 2.93672 4.80266 2.54817 0.849106 10
reco:pandoracalonosce:Calorimetry 1.74605 2.45184 3.32675 2.2989 0.497508 10
reco:pandorapid:Chi2ParticleID 0.0043548 0.00696628 0.0139663 0.00583105 0.00278032 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.043034 0.0638674 0.110673 0.0502108 0.0225135 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00431489 0.00631521 0.00916531 0.0061441 0.00152391 10
reco:pandoraShowercalo:ShowerCalorimetry 3.81693 5.61778 7.94561 5.43439 1.07752 10
reco:pandoraShowercalonosce:ShowerCalorimetry 4.38194 6.009 9.69226 5.44764 1.6317 10
reco:emtrkmichelid:EmTrackMichelId 214.471 251.302 323.013 241.789 28.4539 10
reco:crttag:SingleCRTMatchingProducer 0.0129498 0.0198003 0.0329273 0.0192257 0.00539351 10
reco:crtreco:TwoCRTMatchingProducer 0.00701535 0.011378 0.017341 0.0116173 0.00283307 10
reco:anodepiercerst0:T0RecoAnodePiercers 0.771602 1.03715 1.22295 1.04464 0.137309 10
reco:pandora2Track:LArPandoraTrackCreation 12.6054 16.484 19.6642 16.6935 2.33745 10
reco:pandora2calo:Calorimetry 6.13661 8.31247 10.6133 8.16334 1.41183 10
reco:pandora2calonosce:Calorimetry 6.20168 8.04469 11.9447 7.55814 1.81439 10
reco:pandora2pid:Chi2ParticleID 0.0100635 0.017362 0.0250795 0.0174112 0.00555816 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0563248 0.0857011 0.111626 0.089047 0.0221855 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.0105076 0.0164587 0.0237592 0.0157277 0.0047742 10
reco:pandora2Shower:LArPandoraShowerCreation 3.66407 6.16153 10.0149 6.14184 1.72629 10
reco:pandora2Showercalo:ShowerCalorimetry 5.2367 7.51456 11.8042 6.77569 1.75143 10
reco:pandora2Showercalonosce:ShowerCalorimetry 4.76527 6.636 8.37767 6.67171 1.40557 10
[art]:TriggerResults:TriggerResultInserter 5.022e-05 7.95333e-05 0.000179513 6.9291e-05 3.4999e-05 10
end_path:out1:RootOutput 1.141e-05 1.47894e-05 2.3321e-05 1.37855e-05 3.59604e-06 10
end_path:out1:RootOutput(write) 2.59412 3.88111 5.00194 3.92247 0.638093 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5616.53 MB
Peak resident set size usage (VmHWM): 4790.8 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.07 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.26 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 3.83 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.15 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.05 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 12178780
-rw-r--r--. 1 duneprd np-comp 2791443580 Dec 25 01:11 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2646050665 Dec 25 01:08 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2548694611 Dec 25 02:33 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 2235605425 Dec 24 22:13 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 2156097377 Dec 24 21:51 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp 92565570 Dec 25 02:27 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp 243438 Dec 25 02:33 jobscript.log
-rw-r--r--. 1 duneprd np-comp 217088 Dec 25 02:33 mem.db
-rw-r--r--. 1 duneprd np-comp 62376 Dec 25 02:33 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp 36864 Dec 25 02:33 time.db
-rw-r--r--. 1 duneprd np-comp 519 Dec 25 01:11 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp 519 Dec 24 22:13 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp 519 Dec 25 02:33 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp 52 Dec 24 21:40 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp 7 Dec 25 02:33 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp 0 Dec 25 01:18 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp 0 Dec 25 01:11 debugprod.log
