justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 36629.0@dunegpschedd02.fnal.gov

Jobsub ID36629.0@dunegpschedd02.fnal.gov
Workflow ID2355
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-09-08 20:39:15
SiteUS_FNAL-T1
EntryCMSHTPC_T1_US_FNAL_condce_opp1_whole
Last heartbeat2025-09-08 22:45:07
From worker nodeHostnamedunegli-45666-0-cmswn4014.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-08 20:41:25
Input filesmonte-carlo-002355-000001
JobscriptExit code0
Real time2h (7408s)
CPU time2h (7354s = 99%)
Max RSS bytes4124778496 (3933 MiB)
Outputting started2025-09-08 22:44:53
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/02355/1/001/reco_stage1.root
Finished2025-09-08 22:45:07
Saved logsjustin-logs:36629.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 167334 traces, 0 summary
Retagger: tagging trace set: gauss with 120036 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 120036 traces tagged "gauss"
FrameSaver: q=4.22191e+06 n=2076467 tag=gauss
wclsFrameSaver: saving 167334 traces tagged "wiener"
FrameSaver: q=4.52669e+06 n=1813734 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 34849 XUs and 52148 XVs -> 823 XUVs
C:0 T:1 80933 XUs and 105146 XVs -> 8116 XUVs
C:0 T:2 128862 XUs and 123900 XVs -> 7990 XUVs
C:0 T:3 53358 XUs and 43826 XVs -> 372 XUVs
C:0 T:4 10594 XUs and 13623 XVs -> 207 XUVs
C:0 T:5 46587 XUs and 59112 XVs -> 6548 XUVs
C:0 T:6 247685 XUs and 189720 XVs -> 23373 XUVs
C:0 T:7 60983 XUs and 60721 XVs -> 904 XUVs
C:0 T:8 7739 XUs and 8626 XVs -> 279 XUVs
C:0 T:9 57267 XUs and 94316 XVs -> 9086 XUVs
C:0 T:10 112914 XUs and 84368 XVs -> 21147 XUVs
C:0 T:11 771 XUs and 672 XVs -> 101 XUVs
78946 XUVs total
15342 collection wire objects
72879 potential space points
Neighbour search...
11961867 tests to find 5342498 neighbours
Iterating with no regularization...
Begin: 6.36601e+07
0 4.58706e+07
1 4.45638e+07
2 4.43247e+07
3 4.42458e+07
4 4.42093e+07
Now with regularization...
Begin: 3.2188e+07
0 3.20959e+07
1 3.20413e+07
2 3.20007e+07
3 3.19683e+07
4 3.19415e+07
TestBeamParticleCreationAlgorithm::SetupTestBeamVertex - Test beam particle has no initial vertex
run: 1 ; subrun: 0 ; event: 10
evttime: 409168130
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 409168130
fLifetime: 30000 [us]
08-Sep-2025 22:44:52 UTC  Closed output file "reco_stage1.root"
08-Sep-2025 22:44:52 UTC  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          139.482       212.409       257.605       218.736       35.2387        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000707542    0.0029781     0.0048669    0.00298262    0.00137079       10     
reco:ophit:OpHitFinder                             0.173222      0.204484       0.2314       0.204497      0.0191126       10     
reco:opflash:OpFlashFinder                         0.0892228     0.126566      0.165134      0.119137      0.0256918       10     
reco:opslicer:OpSlicer                             0.366602      0.459888      0.571511      0.439053      0.0665896       10     
reco:caldata:DataPrepModule                         3.33656       3.46349       3.63471       3.45628      0.0708399       10     
reco:wclsdatasp:WireCellToolkit                     22.3347       23.0518       24.423        22.824       0.664409        10     
reco:gaushit:GausHitFinder                         0.512862       0.73383       0.89243       0.74802       0.11608        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000229515   0.000337882   0.000465834   0.000331829   6.47688e-05      10     
reco:reco3d:SpacePointSolver                        4.86468       11.2629       16.575        12.5878       4.03061        10     
reco:hitpdune:DisambigFromSpacePoints              0.504544       1.00879       1.35184      0.985524      0.264564        10     
reco:pandora:StandardPandora                        8.79459       20.6737       28.9105       21.5412       5.96317        10     
reco:pandoraWriter:StandardPandora                 0.179008      0.263177      0.305893      0.268068      0.0362778       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.24713       2.13938       2.82331       2.14986      0.459502        10     
reco:pandoraShower:LArPandoraShowerCreation         1.09878       2.46109       3.40972       2.68141       0.68258        10     
reco:pandoracalo:Calorimetry                        0.73293       1.29917       1.65279       1.34931      0.271723        10     
reco:pandoracalonosce:Calorimetry                  0.663193       1.20501       1.53849       1.25392      0.259771        10     
reco:pandorapid:Chi2ParticleID                    0.00266241    0.00343237    0.00487488    0.00325848    0.000610443      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0250745     0.0324909     0.0367072     0.0336346    0.00349836       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00248534     0.0030732    0.00371486    0.00310165    0.000343845      10     
reco:pandoraShowercalo:ShowerCalorimetry             1.134        2.7985        3.79495       3.10494      0.799881        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.10455       2.72395       3.70743       3.01607      0.792244        10     
reco:emtrkmichelid:EmTrackMichelId                  78.8857       111.804       134.071       113.78        15.9151        10     
reco:crttag:SingleCRTMatchingProducer             0.00678196    0.00998019     0.0135532    0.00909587    0.00228089       10     
reco:crtreco:TwoCRTMatchingProducer               0.00342489    0.00508333    0.00606229    0.00517527    0.000731352      10     
reco:anodepiercerst0:T0RecoAnodePiercers            0.41608      0.594902      0.803972      0.565997      0.107372        10     
reco:pandora2Track:LArPandoraTrackCreation          3.42029       7.0753        10.3454       7.62241       1.92201        10     
reco:pandora2calo:Calorimetry                       1.78877       3.88281       5.26074       4.26857       1.01659        10     
reco:pandora2calonosce:Calorimetry                  1.68574       3.72229       5.04156       4.09178      0.984291        10     
reco:pandora2pid:Chi2ParticleID                   0.00580178    0.00811089     0.0106397    0.00827663    0.00127447       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0304453     0.0406374     0.0480304     0.0421006    0.00525685       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00552487    0.00771346    0.00936688    0.00790641     0.0010779       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.3472        2.66894       3.58065       2.9221       0.656662        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.58635       3.46609       4.6016        3.88403       0.9046         10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.49018       3.23455       4.25975       3.65801      0.839509        10     
[art]:TriggerResults:TriggerResultInserter        3.0308e-05    4.68968e-05   0.00013723    3.8363e-05    3.02889e-05      10     
end_path:out1:RootOutput                           6.222e-06    8.6062e-06    1.7333e-05    7.6545e-06    3.06608e-06      10     
end_path:out1:RootOutput(write)                     1.34059       1.90702       2.29841       1.95023      0.295297        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4949.33 MB
  Peak resident set size usage (VmHWM): 4124.78 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.18 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10301688
-rw-r--r-- 1 dunegli fnalgrid 2403669572 Sep  8 22:08 detsim_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2282936211 Sep  8 22:07 detsim_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 2101331639 Sep  8 22:44 reco_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1875958954 Sep  8 20:52 g4_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 1806873185 Sep  8 20:44 g4_stage1.root
-rw-r--r-- 1 dunegli fnalgrid   77544850 Sep  8 22:41 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid     243123 Sep  8 22:44 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid     217088 Sep  8 22:44 mem.db
-rw-r--r-- 1 dunegli fnalgrid      62376 Sep  8 22:44 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid      36864 Sep  8 22:44 time.db
-rw-r--r-- 1 dunegli fnalgrid        519 Sep  8 22:08 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Sep  8 20:52 g4_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Sep  8 22:44 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dunegli fnalgrid         52 Sep  8 20:41 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid          7 Sep  8 22:44 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid          0 Sep  8 22:11 ProtoDUNE.log
-rw-r--r-- 1 dunegli fnalgrid          0 Sep  8 22:08 debugprod.log
justIN time: 2025-09-18 22:20:58 UTC       justIN version: 01.05.00