justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 231482.1@dunegpschedd02.fnal.gov

Jobsub ID231482.1@dunegpschedd02.fnal.gov
Workflow ID9226
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-20 23:10:23
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce04
Last heartbeat2025-10-21 02:51:58
From worker nodeHostnameccwcondor0046
cpuinfoAMD EPYC 9334 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-20 23:10:54
Input filesmonte-carlo-009226-000001
JobscriptExit code0
Real time3h (13177s)
CPU time3h (12771s = 96%)
Max RSS bytes4290945024 (4092 MiB)
Outputting started2025-10-21 02:50:32
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09226/1/001/reco_stage1.root
Finished2025-10-21 02:51:58
Saved logsjustin-logs:231482.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 141979 traces, 0 summary
Retagger: tagging trace set: gauss with 108834 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 108834 traces tagged "gauss"
FrameSaver: q=3.80652e+06 n=1894807 tag=gauss
wclsFrameSaver: saving 141979 traces tagged "wiener"
FrameSaver: q=4.04108e+06 n=1683215 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 23253 XUs and 25212 XVs -> 256 XUVs
C:0 T:1 37835 XUs and 37601 XVs -> 2645 XUVs
C:0 T:2 115628 XUs and 114096 XVs -> 4643 XUVs
C:0 T:3 57135 XUs and 46882 XVs -> 358 XUVs
C:0 T:4 9746 XUs and 11598 XVs -> 143 XUVs
C:0 T:5 42442 XUs and 42138 XVs -> 11067 XUVs
C:0 T:6 173102 XUs and 166561 XVs -> 6485 XUVs
C:0 T:7 65049 XUs and 61429 XVs -> 752 XUVs
C:0 T:8 6497 XUs and 5473 XVs -> 63 XUVs
C:0 T:9 42013 XUs and 46525 XVs -> 2929 XUVs
C:0 T:10 65565 XUs and 43390 XVs -> 5554 XUVs
C:0 T:11 386 XUs and 237 XVs -> 44 XUVs
34939 XUVs total
11701 collection wire objects
31885 potential space points
Neighbour search...
3096715 tests to find 1414652 neighbours
Iterating with no regularization...
Begin: 7.08385e+07
0 5.92594e+07
1 5.8392e+07
2 5.82745e+07
3 5.82461e+07
Now with regularization...
Begin: 4.71371e+07
0 4.71054e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410015086
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410015086
fLifetime: 30000 [us]
21-Oct-2025 04:50:29 CEST  Closed output file "reco_stage1.root"
21-Oct-2025 04:50:29 CEST  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          345.493       442.177       543.313       437.243        55.75         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00152986    0.00458413     0.0068824    0.00460594    0.00153464       10     
reco:ophit:OpHitFinder                             0.253563      0.315542      0.373497      0.327081      0.0367124       10     
reco:opflash:OpFlashFinder                         0.184577      0.219649      0.284514      0.214724      0.0273149       10     
reco:opslicer:OpSlicer                             0.534091      0.726068      0.913113      0.747128      0.112317        10     
reco:caldata:DataPrepModule                         4.34171       5.64518       6.46629       5.73018       0.60132        10     
reco:wclsdatasp:WireCellToolkit                     31.6814       37.1527       44.9727       36.5189       3.54985        10     
reco:gaushit:GausHitFinder                          1.16182       1.39206       1.85749       1.33659      0.220406        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000545406   0.00119401    0.00312711    0.000731843   0.000867341      10     
reco:reco3d:SpacePointSolver                        12.6569       20.2582       30.3005       19.6003       4.56093        10     
reco:hitpdune:DisambigFromSpacePoints               1.43381       1.92628       2.34075       1.92721      0.324356        10     
reco:pandora:StandardPandora                        27.6388       38.0159       58.0506       37.1653       7.8345         10     
reco:pandoraWriter:StandardPandora                 0.461301      0.603066      0.727263      0.604821      0.0900374       10     
reco:pandoraTrack:LArPandoraTrackCreation           5.18686       7.98201       10.8107       7.0622        2.04359        10     
reco:pandoraShower:LArPandoraShowerCreation         4.44298       9.48593       14.3358       9.00206       2.9642         10     
reco:pandoracalo:Calorimetry                        3.06183       4.44816       5.81799       3.93122      0.989583        10     
reco:pandoracalonosce:Calorimetry                   2.38499       4.22603       5.96978       4.02078       1.11484        10     
reco:pandorapid:Chi2ParticleID                    0.00560696    0.00730355     0.0101204    0.00713799    0.00151767       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0326978     0.0708323     0.0968172     0.0720749     0.0177081       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0042399    0.00664634     0.0085154     0.006874     0.00133607       10     
reco:pandoraShowercalo:ShowerCalorimetry            4.82141       9.34123       13.3881       8.92573       2.15558        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       5.23033       9.46284       13.0894       9.49447       2.13981        10     
reco:emtrkmichelid:EmTrackMichelId                  174.417       211.437       241.559       213.807       20.4366        10     
reco:crttag:SingleCRTMatchingProducer              0.0116043     0.0220506     0.0333137     0.0227361    0.00660964       10     
reco:crtreco:TwoCRTMatchingProducer               0.00830893     0.0156343     0.0222957     0.0163417    0.00432848       10     
reco:anodepiercerst0:T0RecoAnodePiercers            0.7629        1.01625       1.25485       1.0227       0.167273        10     
reco:pandora2Track:LArPandoraTrackCreation          13.648        24.8488       32.9972       24.8443       5.55511        10     
reco:pandora2calo:Calorimetry                       5.79522       12.5741       19.0781       11.8872       3.81722        10     
reco:pandora2calonosce:Calorimetry                  6.22361       11.4582       16.9129       11.237        3.07933        10     
reco:pandora2pid:Chi2ParticleID                   0.00722807      0.01661      0.0219882     0.0172934    0.00362561       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0434601     0.0856265     0.110007      0.0876345     0.0181579       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00725492     0.0152648     0.018812      0.0158931    0.00326663       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.86111       8.52953       11.6628       8.36262        1.74          10     
reco:pandora2Showercalo:ShowerCalorimetry           6.30241       9.33987       11.3718       9.88342       1.5888         10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.48433       8.22901       12.2181       7.90368       2.09542        10     
[art]:TriggerResults:TriggerResultInserter        6.5489e-05    0.000100339   0.000215166   9.10775e-05   4.16476e-05      10     
end_path:out1:RootOutput                          1.1527e-05    1.5765e-05    2.1913e-05    1.49125e-05   2.93223e-06      10     
end_path:out1:RootOutput(write)                     2.33916       3.05261       3.6167        3.17663      0.380154        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5140.9 MB
  Peak resident set size usage (VmHWM): 4290.95 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.29 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.47 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10337708
-rw-r--r-- 1 dune001 lbno 2428584041 Oct 21 03:34 detsim_stage2.root
-rw-r--r-- 1 dune001 lbno 2300252627 Oct 21 03:32 detsim_stage1.root
-rw-r--r-- 1 dune001 lbno 2137880864 Oct 21 04:50 reco_stage1.root
-rw-r--r-- 1 dune001 lbno 1855352790 Oct 21 01:31 g4_stage2.root
-rw-r--r-- 1 dune001 lbno 1782653784 Oct 21 01:17 g4_stage1.root
-rw-r--r-- 1 dune001 lbno   80472720 Oct 21 04:45 Pandora_Events.pndr
-rw-r--r-- 1 dune001 lbno     243188 Oct 21 04:50 jobscript.log
-rw-r--r-- 1 dune001 lbno     217088 Oct 21 04:50 mem.db
-rw-r--r-- 1 dune001 lbno      62376 Oct 21 04:50 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 lbno      36864 Oct 21 04:50 time.db
-rw-r--r-- 1 dune001 lbno        519 Oct 21 03:34 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Oct 21 01:31 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Oct 21 04:50 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 lbno         52 Oct 21 01:10 all-input-dids.txt
-rw-r--r-- 1 dune001 lbno          7 Oct 21 04:50 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 lbno          0 Oct 21 03:42 ProtoDUNE.log
-rw-r--r-- 1 dune001 lbno          0 Oct 21 03:34 debugprod.log
justIN time: 2025-11-03 18:30:07 UTC       justIN version: 01.05.01