Jobsub ID 246357.0@dunegpschedd02.fnal.gov
| Jobsub ID | 246357.0@dunegpschedd02.fnal.gov |
| Workflow ID | 10220 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2025-11-15 16:43:52 |
| Site | US_FNAL-FermiGrid |
| Entry | FNAL_GPGrid_ce03_mcore_op_duneonly |
| Last heartbeat | 2025-11-15 18:35:57 |
| From worker node | Hostname | dunegli-7611256-0-fnpc23022.fnal.gov |
| cpuinfo | AMD EPYC 7543 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 172800 (48 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | stalled |
| Started | 2025-11-15 16:45:56 |
| Input files | monte-carlo-010220-000001
|
| Outputting started | 2025-11-15 18:33:45 |
| Output files | |
| Finished | 2025-11-15 19:23:45 |
| List job events (HTCondor job logs unavailable) |
Jobscript log (last 10,000 characters)
Service:prepare: Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 122170 traces, 0 summary
Retagger: tagging trace set: gauss with 95980 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 95980 traces tagged "gauss"
FrameSaver: q=3.45776e+06 n=1768936 tag=gauss
wclsFrameSaver: saving 122170 traces tagged "wiener"
FrameSaver: q=3.6228e+06 n=1575442 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 34015 XUs and 24547 XVs -> 514 XUVs
C:0 T:1 111255 XUs and 57752 XVs -> 15724 XUVs
C:0 T:2 109451 XUs and 129638 XVs -> 5664 XUVs
C:0 T:3 54181 XUs and 48395 XVs -> 402 XUVs
C:0 T:4 8413 XUs and 5841 XVs -> 42 XUVs
C:0 T:5 24916 XUs and 14537 XVs -> 2424 XUVs
C:0 T:6 157074 XUs and 153111 XVs -> 8048 XUVs
C:0 T:7 60335 XUs and 51078 XVs -> 647 XUVs
C:0 T:8 8748 XUs and 8463 XVs -> 80 XUVs
C:0 T:9 59945 XUs and 81916 XVs -> 5029 XUVs
C:0 T:10 47745 XUs and 78469 XVs -> 14261 XUVs
C:0 T:11 52 XUs and 18 XVs -> 3 XUVs
52838 XUVs total
11467 collection wire objects
49244 potential space points
Neighbour search...
9819414 tests to find 4182496 neighbours
Iterating with no regularization...
Begin: 8.49257e+07
0 7.34235e+07
1 7.26308e+07
2 7.25122e+07
3 7.2479e+07
Now with regularization...
Begin: 6.25081e+07
0 6.24815e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410532741
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410532741
fLifetime: 30000 [us]
15-Nov-2025 18:33:44 UTC Closed output file "reco_stage1.root"
15-Nov-2025 18:33:44 UTC Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 115.144 186.941 233.315 204.208 42.262 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00061342 0.00253851 0.00447123 0.00278225 0.00116615 10
reco:ophit:OpHitFinder 0.151845 0.181485 0.207833 0.187167 0.0192143 10
reco:opflash:OpFlashFinder 0.0714433 0.105598 0.13032 0.114019 0.0211848 10
reco:opslicer:OpSlicer 0.274555 0.389074 0.476758 0.413195 0.0689673 10
reco:caldata:DataPrepModule 3.35893 3.48989 3.59136 3.51167 0.0691986 10
reco:wclsdatasp:WireCellToolkit 21.5939 22.8324 24.0634 22.8995 0.753783 10
reco:gaushit:GausHitFinder 0.418895 0.683994 0.862031 0.735785 0.140309 10
reco:nhitsfilter:NumberOfHitsFilter 0.000233215 0.000289908 0.000336742 0.000297458 3.74248e-05 10
reco:reco3d:SpacePointSolver 3.22795 8.07119 13.137 8.81258 3.20353 10
reco:hitpdune:DisambigFromSpacePoints 0.383279 0.851376 1.16153 0.923638 0.279723 10
reco:pandora:StandardPandora 7.1332 17.4517 30.0044 17.9568 7.08857 10
reco:pandoraWriter:StandardPandora 0.129646 0.225425 0.281463 0.245025 0.0519301 10
reco:pandoraTrack:LArPandoraTrackCreation 1.00512 1.80487 2.60528 2.0117 0.576263 10
reco:pandoraShower:LArPandoraShowerCreation 0.622089 1.94993 2.7983 2.20131 0.772789 10
reco:pandoracalo:Calorimetry 0.615829 1.09026 1.51346 1.23583 0.351959 10
reco:pandoracalonosce:Calorimetry 0.560891 1.00316 1.41735 1.14101 0.334241 10
reco:pandorapid:Chi2ParticleID 0.00261191 0.00331655 0.00389529 0.00333109 0.000376914 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0178556 0.0287738 0.0377112 0.0305355 0.00611887 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00217165 0.0029357 0.00360857 0.00303484 0.000466958 10
reco:pandoraShowercalo:ShowerCalorimetry 0.627371 2.20559 3.29816 2.546 0.941647 10
reco:pandoraShowercalonosce:ShowerCalorimetry 0.610556 2.14851 3.23797 2.47919 0.924496 10
reco:emtrkmichelid:EmTrackMichelId 64.2814 100.835 123.435 109.288 20.4133 10
reco:crttag:SingleCRTMatchingProducer 0.00470234 0.00768143 0.011472 0.00792566 0.00225434 10
reco:crtreco:TwoCRTMatchingProducer 0.00284716 0.00438188 0.00539518 0.00445474 0.000860155 10
reco:anodepiercerst0:T0RecoAnodePiercers 0.334902 0.562093 0.82327 0.5497 0.158099 10
reco:pandora2Track:LArPandoraTrackCreation 2.22733 5.69785 8.30552 6.39435 2.18813 10
reco:pandora2calo:Calorimetry 1.17609 3.12487 4.634 3.56397 1.25181 10
reco:pandora2calonosce:Calorimetry 1.1045 2.97403 4.41997 3.39891 1.20603 10
reco:pandora2pid:Chi2ParticleID 0.00470389 0.00755564 0.00958825 0.00835462 0.00166246 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0222981 0.0369183 0.0475341 0.0399509 0.00803307 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.00453339 0.00725367 0.00930009 0.00795029 0.00158469 10
reco:pandora2Shower:LArPandoraShowerCreation 0.889365 2.11669 3.0346 2.35169 0.769485 10
reco:pandora2Showercalo:ShowerCalorimetry 1.03903 2.75241 4.13689 3.12161 1.10619 10
reco:pandora2Showercalonosce:ShowerCalorimetry 0.974232 2.53184 3.81954 2.84209 1.00612 10
[art]:TriggerResults:TriggerResultInserter 2.9105e-05 4.17226e-05 7.9963e-05 3.53175e-05 1.44047e-05 10
end_path:out1:RootOutput 8.366e-06 1.14169e-05 1.3756e-05 1.1862e-05 1.68194e-06 10
end_path:out1:RootOutput(write) 1.25021 1.70836 2.1276 1.77882 0.323244 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4815.34 MB
Peak resident set size usage (VmHWM): 4006.47 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.15 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.03 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 2.19 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.08 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.03 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9080812
-rw-r--r-- 1 dunegli fnalgrid 2180211475 Nov 15 18:01 detsim_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2077093800 Nov 15 18:00 detsim_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1832765212 Nov 15 18:33 reco_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1601805973 Nov 15 16:55 g4_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 1538998681 Nov 15 16:48 g4_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 67257430 Nov 15 18:31 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid 242764 Nov 15 18:33 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid 217088 Nov 15 18:33 mem.db
-rw-r--r-- 1 dunegli fnalgrid 62376 Nov 15 18:33 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid 36864 Nov 15 18:33 time.db
-rw-r--r-- 1 dunegli fnalgrid 519 Nov 15 18:01 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid 519 Nov 15 16:55 g4_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid 519 Nov 15 18:33 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dunegli fnalgrid 52 Nov 15 16:45 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid 7 Nov 15 18:33 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid 0 Nov 15 18:03 ProtoDUNE.log
-rw-r--r-- 1 dunegli fnalgrid 0 Nov 15 18:01 debugprod.log
