justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 238861.0@dunegpschedd01.fnal.gov

Jobsub ID238861.0@dunegpschedd01.fnal.gov
Workflow ID9379
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-27 14:33:04
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-10-27 17:02:08
From worker nodeHostnamedunegli-7542233-0-fnpc22002.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-27 14:39:21
Input filesmonte-carlo-009379-000001
JobscriptExit code0
Real time2h (8543s)
CPU time2h (8455s = 98%)
Max RSS bytes4609146880 (4395 MiB)
Outputting started2025-10-27 17:01:45
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09379/1/001/reco_stage1.root
Finished2025-10-27 17:02:08
Saved logsjustin-logs:238861.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ing as is
Retagger: tagging trace set: wiener with 158927 traces, 0 summary
Retagger: tagging trace set: gauss with 122206 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 122206 traces tagged "gauss"
FrameSaver: q=4.6607e+06 n=2194398 tag=gauss
wclsFrameSaver: saving 158927 traces tagged "wiener"
FrameSaver: q=4.97736e+06 n=1953572 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 33575 XUs and 39747 XVs -> 326 XUVs
C:0 T:1 62104 XUs and 65666 XVs -> 3559 XUVs
C:0 T:2 174191 XUs and 175688 XVs -> 10897 XUVs
C:0 T:3 70883 XUs and 60941 XVs -> 679 XUVs
C:0 T:4 14924 XUs and 12273 XVs -> 605 XUVs
C:0 T:5 63603 XUs and 70182 XVs -> 7562 XUVs
C:0 T:6 167010 XUs and 136985 XVs -> 8635 XUVs
C:0 T:7 61640 XUs and 47057 XVs -> 614 XUVs
C:0 T:8 12911 XUs and 13191 XVs -> 187 XUVs
C:0 T:9 128924 XUs and 145782 XVs -> 11415 XUVs
C:0 T:10 51567 XUs and 34934 XVs -> 5296 XUVs
C:0 T:11 177 XUs and 111 XVs -> 16 XUVs
49791 XUVs total
15623 collection wire objects
45254 potential space points
Neighbour search...
4975670 tests to find 1919134 neighbours
Iterating with no regularization...
Begin: 2.20317e+08
0 1.39313e+08
1 1.31981e+08
2 1.30863e+08
3 1.30571e+08
4 1.30461e+08
Now with regularization...
Begin: 1.05234e+08
0 1.04911e+08
1 1.04812e+08
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410148684
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410148684
fLifetime: 30000 [us]
27-Oct-2025 17:01:44 UTC  Closed output file "reco_stage1.root"
27-Oct-2025 17:01:44 UTC  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          202.112       268.368       321.57        270.426       34.837         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000571646   0.00292535    0.00462798    0.00315915    0.00124055       10     
reco:ophit:OpHitFinder                              0.20226      0.232311      0.264449      0.224448      0.0191787       10     
reco:opflash:OpFlashFinder                         0.134943      0.173276      0.214732      0.166249      0.0240927       10     
reco:opslicer:OpSlicer                             0.470122      0.553417      0.671876      0.538091      0.055912        10     
reco:caldata:DataPrepModule                         3.40864       3.50806       3.7334        3.49488      0.0864679       10     
reco:wclsdatasp:WireCellToolkit                     22.8938       23.7183       24.8899       23.5506      0.537506        10     
reco:gaushit:GausHitFinder                         0.765702      0.935679       1.05188       0.94171      0.0842203       10     
reco:nhitsfilter:NumberOfHitsFilter               0.000311442   0.000412564   0.000756146   0.000386349   0.000118845      10     
reco:reco3d:SpacePointSolver                        7.43342       14.7069       21.5362       14.956        3.92708        10     
reco:hitpdune:DisambigFromSpacePoints              0.895861       1.42541       1.85606       1.43678      0.248061        10     
reco:pandora:StandardPandora                        21.1699       35.1913       59.7418       32.469        10.9237        10     
reco:pandoraWriter:StandardPandora                 0.253214      0.334058      0.396073      0.329308      0.041956        10     
reco:pandoraTrack:LArPandoraTrackCreation           2.15417       3.08977       4.07468       3.07106      0.576502        10     
reco:pandoraShower:LArPandoraShowerCreation         1.97456       3.37043       4.71917       3.3025       0.807605        10     
reco:pandoracalo:Calorimetry                         1.407        1.80411       2.30205       1.88556       0.26316        10     
reco:pandoracalonosce:Calorimetry                   1.27575       1.70313       2.16298       1.73964      0.288299        10     
reco:pandorapid:Chi2ParticleID                    0.00336276    0.00413341    0.00507387    0.00402655    0.000498511      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0318111     0.0378913     0.0412603     0.038297     0.00270566       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00319058    0.00390109    0.00457614    0.00392389    0.000382367      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.30075       3.79577       5.4725        3.76309      0.896805        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.23408       3.83837       6.37733       3.70066       1.07906        10     
reco:emtrkmichelid:EmTrackMichelId                  106.517       133.578       157.132       134.769       12.8636        10     
reco:crttag:SingleCRTMatchingProducer             0.00863589     0.0106437     0.0148404     0.0103332    0.00190665       10     
reco:crtreco:TwoCRTMatchingProducer               0.00554667    0.00628509    0.00747721    0.00620202    0.000637079      10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.621199      0.687263      0.791856      0.660756      0.0559467       10     
reco:pandora2Track:LArPandoraTrackCreation           7.59         10.3671       13.5061       9.60898       1.9379         10     
reco:pandora2calo:Calorimetry                       3.9839        5.34717       6.96897       5.10464      0.963581        10     
reco:pandora2calonosce:Calorimetry                  3.56613       5.14936       6.94328       4.81414       1.02364        10     
reco:pandora2pid:Chi2ParticleID                   0.00792027    0.00975081     0.012238     0.00959452     0.0013534       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0368927     0.0484579     0.0563257     0.0482252    0.00493159       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00812065    0.00966799     0.0124099    0.00952668    0.00134785       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.30254       3.65337       5.18418       3.48577       0.82665        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.00991       4.58825       6.18277       4.3966       0.914501        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.68291       4.14232       5.57162       4.01904      0.791787        10     
[art]:TriggerResults:TriggerResultInserter        3.9706e-05    4.89512e-05   9.9319e-05    4.34925e-05   1.69437e-05      10     
end_path:out1:RootOutput                           4.198e-06    8.7957e-06    1.3255e-05    8.5465e-06    2.14965e-06      10     
end_path:out1:RootOutput(write)                     1.77558       2.25072       2.73272       2.25705      0.232344        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5417.47 MB
  Peak resident set size usage (VmHWM): 4609.15 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.21 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11430532
-rw-r--r-- 1 dunegli fnalgrid 2651310018 Oct 27 16:15 detsim_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2512923048 Oct 27 16:14 detsim_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 2403294441 Oct 27 17:01 reco_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 2064705851 Oct 27 14:51 g4_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 1980526292 Oct 27 14:42 g4_stage1.root
-rw-r--r-- 1 dunegli fnalgrid   91480970 Oct 27 16:58 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid     243609 Oct 27 17:01 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid     217088 Oct 27 17:01 mem.db
-rw-r--r-- 1 dunegli fnalgrid      62376 Oct 27 17:01 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid      36864 Oct 27 17:01 time.db
-rw-r--r-- 1 dunegli fnalgrid        519 Oct 27 16:15 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Oct 27 14:51 g4_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Oct 27 17:01 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dunegli fnalgrid         52 Oct 27 14:39 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid          7 Oct 27 17:01 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid          0 Oct 27 16:20 ProtoDUNE.log
-rw-r--r-- 1 dunegli fnalgrid          0 Oct 27 16:15 debugprod.log
justIN time: 2025-11-04 01:29:11 UTC       justIN version: 01.05.01