Jobsub ID 238779.1@dunegpschedd01.fnal.gov
| Jobsub ID | 238779.1@dunegpschedd01.fnal.gov | 
| Workflow ID | 9367 | 
| Stage ID | 1 | 
| User name | pmelas@fnal.gov | 
| HTCondor Group | group_dune | 
| Requested | Processors | 1 | 
| GPU | No | 
| RSS bytes | 8388608000 (8000 MiB) | 
| Wall seconds limit | 86400 (24 hours) | 
| Submitted time | 2025-10-27 06:10:32 | 
| Site | US_FNAL-FermiGrid | 
| Entry | FNAL_GPGrid_ce03_mcore_op_duneonly | 
| Last heartbeat | 2025-10-27 08:13:20 | 
| From worker node | Hostname | dunegli-7381652-0-fnpc23042.fnal.gov | 
| cpuinfo | AMD EPYC 7543 32-Core Processor | 
| OS release | Scientific Linux release 7.9 (Nitrogen) | 
| Processors | 1 | 
| RSS bytes | 10485760000 (10000 MiB) | 
| Wall seconds limit | 172800 (48 hours) | 
| GPU |  | 
| Inner Apptainer? | True | 
| Job state | finished | 
| Started | 2025-10-27 06:11:56 | 
| Input files | monte-carlo-009367-000001
  | 
| Jobscript | Exit code | 0 | 
| Real time | 2h (7270s) | 
| CPU time | 2h (7206s = 99%) | 
| Max RSS bytes | 4244176896 (4047 MiB) | 
| Outputting started | 2025-10-27 08:13:06 | 
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09367/1/001/reco_stage1.root
  | 
| Finished | 2025-10-27 08:13:20 | 
| Saved logs | justin-logs:238779.1-dunegpschedd01.fnal.gov.logs.tgz | 
| List job events     Cached HTCondor job logs | 
Jobscript log (last 10,000 characters)
g tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 113850 traces, 0 summary
Retagger: tagging trace set: gauss with 83837 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 83837 traces tagged "gauss"
FrameSaver: q=2.82069e+06 n=1437528 tag=gauss
wclsFrameSaver: saving 113850 traces tagged "wiener"
FrameSaver: q=3.00463e+06 n=1247497 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 18139 XUs and 17073 XVs -> 83 XUVs
C:0 T:1 34023 XUs and 33171 XVs -> 2255 XUVs
C:0 T:2 141501 XUs and 141099 XVs -> 7455 XUVs
C:0 T:3 58363 XUs and 48827 XVs -> 379 XUVs
C:0 T:4 3777 XUs and 3910 XVs -> 5 XUVs
C:0 T:5 12400 XUs and 7975 XVs -> 1283 XUVs
C:0 T:6 77571 XUs and 138840 XVs -> 4572 XUVs
C:0 T:7 30153 XUs and 41571 XVs -> 311 XUVs
C:0 T:8 7130 XUs and 5326 XVs -> 175 XUVs
C:0 T:9 60246 XUs and 61994 XVs -> 8509 XUVs
C:0 T:10 35801 XUs and 33921 XVs -> 4240 XUVs
C:0 T:11 59 XUs and 50 XVs -> 7 XUVs
29274 XUVs total
10595 collection wire objects
26823 potential space points
Neighbour search...
2072473 tests to find 1098114 neighbours
Iterating with no regularization...
Begin: 5.31573e+07
0 3.93109e+07
1 3.85019e+07
2 3.83738e+07
3 3.83321e+07
4 3.83121e+07
Now with regularization...
Begin: 3.11153e+07
0 3.10724e+07
1 3.10517e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410141669
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410141669
fLifetime: 30000 [us]
27-Oct-2025 08:13:05 UTC  Closed output file "reco_stage1.root"
27-Oct-2025 08:13:05 UTC  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          164.662       211.491       269.206       207.193       30.8682        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000687689   0.00289314    0.00466167     0.0030279     0.0012756       10     
reco:ophit:OpHitFinder                             0.169264      0.204337      0.232845      0.207973      0.0207445       10     
reco:opflash:OpFlashFinder                         0.112422      0.141002      0.168635      0.145483      0.0185763       10     
reco:opslicer:OpSlicer                             0.295626      0.422744      0.541183      0.436682      0.0759093       10     
reco:caldata:DataPrepModule                         3.34464       3.44605       3.65135       3.43976      0.080032        10     
reco:wclsdatasp:WireCellToolkit                     22.1085       23.6869       26.1465       23.5662       1.25092        10     
reco:gaushit:GausHitFinder                         0.564481      0.768914      0.990075      0.753493      0.123759        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000273516   0.000343665   0.000462563   0.000324873   5.52119e-05      10     
reco:reco3d:SpacePointSolver                        6.5581        9.85448       22.0092       8.55297       4.30354        10     
reco:hitpdune:DisambigFromSpacePoints              0.715447       1.04666       1.58546       1.00859      0.252454        10     
reco:pandora:StandardPandora                        11.5815       20.2929       33.137        17.6829       6.48989        10     
reco:pandoraWriter:StandardPandora                  0.20213      0.253259      0.325036      0.249567      0.0361057       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.62368       2.14473       2.77339       2.17797      0.307682        10     
reco:pandoraShower:LArPandoraShowerCreation         1.48036       2.35704       4.17171       2.20357      0.707849        10     
reco:pandoracalo:Calorimetry                       0.881416       1.25934       1.64498       1.28384      0.198797        10     
reco:pandoracalonosce:Calorimetry                  0.805913       1.16179       1.54657       1.1839       0.190069        10     
reco:pandorapid:Chi2ParticleID                     0.0028478    0.00334167    0.00429369    0.00331556    0.000438976      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0277725     0.0325284     0.0410978     0.0318673    0.00409942       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00275932    0.00309174    0.00354802    0.00314503    0.000221841      10     
reco:pandoraShowercalo:ShowerCalorimetry            1.41784       2.68788       4.7987        2.5852       0.857675        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.37928       2.62569       4.70944       2.49304      0.846324        10     
reco:emtrkmichelid:EmTrackMichelId                  91.9485       112.889       139.702       113.808       13.8991        10     
reco:crttag:SingleCRTMatchingProducer             0.00549612    0.00790514     0.0120491    0.00778773    0.00186918       10     
reco:crtreco:TwoCRTMatchingProducer               0.00391404     0.0049247    0.00601023    0.00486006    0.000644233      10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.522064      0.626857      0.811791      0.598406      0.0894205       10     
reco:pandora2Track:LArPandoraTrackCreation          4.72101       7.24115       11.6996       6.71745       1.84236        10     
reco:pandora2calo:Calorimetry                       2.23055       3.75756       6.06712       3.59077      0.973289        10     
reco:pandora2calonosce:Calorimetry                  2.12574       3.60991       5.75757       3.42697      0.931898        10     
reco:pandora2pid:Chi2ParticleID                   0.00643339    0.00800406     0.0108441    0.00788771    0.00119383       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0336569     0.0400251     0.0532417     0.0387813    0.00584343       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00632523     0.0078178     0.0104829    0.00777798    0.00113841       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.8353        2.5598        3.75812       2.37214      0.525611        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.97372       3.35606       5.05633       3.16972      0.812836        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.81086        3.066        4.51119       2.87039      0.704194        10     
[art]:TriggerResults:TriggerResultInserter        2.9585e-05    4.1497e-05    9.3336e-05    3.66195e-05   1.81405e-05      10     
end_path:out1:RootOutput                           6.392e-06    8.2876e-06    1.2013e-05     8.386e-06    1.53738e-06      10     
end_path:out1:RootOutput(write)                     1.52921       1.86291       2.36393       1.79929      0.275055        10     
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
  Peak virtual memory usage (VmPeak)  : 5057.98 MB
  Peak resident set size usage (VmHWM): 4244.18 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.16 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10283188
-rw-r--r-- 1 dunegli fnalgrid 2423743566 Oct 27 07:36 detsim_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2302312010 Oct 27 07:35 detsim_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 2118610071 Oct 27 08:13 reco_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1844229074 Oct 27 06:22 g4_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 1763363278 Oct 27 06:15 g4_stage1.root
-rw-r--r-- 1 dunegli fnalgrid   77111120 Oct 27 08:11 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid     242787 Oct 27 08:13 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid     217088 Oct 27 08:13 mem.db
-rw-r--r-- 1 dunegli fnalgrid      62376 Oct 27 08:13 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid      36864 Oct 27 08:13 time.db
-rw-r--r-- 1 dunegli fnalgrid        519 Oct 27 07:36 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Oct 27 06:22 g4_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Oct 27 08:13 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dunegli fnalgrid         52 Oct 27 06:11 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid          7 Oct 27 08:13 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid          0 Oct 27 07:40 ProtoDUNE.log
-rw-r--r-- 1 dunegli fnalgrid          0 Oct 27 07:36 debugprod.log