justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 273481.1@dunegpschedd01.fnal.gov

Jobsub ID273481.1@dunegpschedd01.fnal.gov
Workflow ID11676
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-10 10:41:26
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce02
Last heartbeat2026-01-10 15:37:41
From worker nodeHostnameccwcondor0750
cpuinfoAMD EPYC 7302 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-10 10:42:53
Input filesmonte-carlo-011676-000001
JobscriptExit code0
Real time4h (17585s)
CPU time4h (17010s = 96%)
Max RSS bytes4470861824 (4263 MiB)
Outputting started2026-01-10 15:36:00
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11676/1/001/reco_stage1.root
Finished2026-01-10 15:37:41
Saved logsjustin-logs:273481.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

caleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 166763 traces, 0 summary
Retagger: tagging trace set: gauss with 127559 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 127559 traces tagged "gauss"
FrameSaver: q=4.40188e+06 n=2313563 tag=gauss
wclsFrameSaver: saving 166763 traces tagged "wiener"
FrameSaver: q=4.65701e+06 n=2057366 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 30532 XUs and 30347 XVs -> 283 XUVs
C:0 T:1 69610 XUs and 52034 XVs -> 5756 XUVs
C:0 T:2 209364 XUs and 200265 XVs -> 17377 XUVs
C:0 T:3 88080 XUs and 64662 XVs -> 1005 XUVs
C:0 T:4 24074 XUs and 20255 XVs -> 249 XUVs
C:0 T:5 124657 XUs and 91508 XVs -> 8111 XUVs
C:0 T:6 189346 XUs and 203872 XVs -> 11053 XUVs
C:0 T:7 67672 XUs and 71936 XVs -> 745 XUVs
C:0 T:8 9044 XUs and 6274 XVs -> 144 XUVs
C:0 T:9 60568 XUs and 58551 XVs -> 6619 XUVs
C:0 T:10 35623 XUs and 16591 XVs -> 3580 XUVs
C:0 T:11 245 XUs and 153 XVs -> 23 XUVs
54945 XUVs total
15785 collection wire objects
50248 potential space points
Neighbour search...
5696580 tests to find 2730570 neighbours
Iterating with no regularization...
Begin: 7.16887e+07
0 5.80905e+07
1 5.70636e+07
2 5.68881e+07
3 5.6837e+07
Now with regularization...
Begin: 4.4779e+07
0 4.47315e+07
1 4.46958e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411653915
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411653915
fLifetime: 30000 [us]
10-Jan-2026 16:35:57 CET  Closed output file "reco_stage1.root"
10-Jan-2026 16:35:57 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          484.372       604.991       787.954       575.598       98.9151        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                             0.0013403    0.00892132     0.0392355    0.00612426     0.0103371       10     
reco:ophit:OpHitFinder                             0.329003      0.394112      0.431287      0.411974      0.0379886       10     
reco:opflash:OpFlashFinder                         0.202475      0.251149      0.276067      0.255412      0.0212295       10     
reco:opslicer:OpSlicer                             0.868673       1.07608       1.25652       1.09953      0.132375        10     
reco:caldata:DataPrepModule                         5.53818       6.47633       6.98212       6.65247       0.41613        10     
reco:wclsdatasp:WireCellToolkit                     41.9696       46.0703       52.0324       45.4598       3.36047        10     
reco:gaushit:GausHitFinder                          1.40538       1.77344       2.23311       1.77908      0.278625        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000767444   0.000890172   0.00102179    0.000894208   8.50539e-05      10     
reco:reco3d:SpacePointSolver                        18.362        25.8711       36.8344       26.2413       6.17957        10     
reco:hitpdune:DisambigFromSpacePoints               1.98899       2.73476       4.00334       2.35899      0.695836        10     
reco:pandora:StandardPandora                        38.6704       67.2077       110.607       66.5123       21.5075        10     
reco:pandoraWriter:StandardPandora                 0.552204      0.673189       0.77636      0.680454      0.0676579       10     
reco:pandoraTrack:LArPandoraTrackCreation           5.00476       7.34211       11.4922       6.88204       1.8632         10     
reco:pandoraShower:LArPandoraShowerCreation         5.44677       9.02138       15.0642       8.32574       3.05656        10     
reco:pandoracalo:Calorimetry                        2.61778       3.70882       4.96379       3.67889      0.698505        10     
reco:pandoracalonosce:Calorimetry                   1.95612       3.30578       4.69924       3.22685      0.827626        10     
reco:pandorapid:Chi2ParticleID                    0.00683955    0.00854256    0.00969328    0.00845566    0.000913621      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0567837     0.087258      0.111156      0.0870873     0.0139061       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00475719    0.00790649    0.00908782    0.00818224    0.00117122       10     
reco:pandoraShowercalo:ShowerCalorimetry            5.46012       8.22588       12.7592       7.19113       2.50861        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.89316       8.36216       12.8629       7.68598       2.42444        10     
reco:emtrkmichelid:EmTrackMichelId                  274.879       332.857       408.311       326.463       43.6039        10     
reco:crttag:SingleCRTMatchingProducer              0.0140442     0.0192622     0.0271554     0.0180829    0.00427528       10     
reco:crtreco:TwoCRTMatchingProducer               0.00864517     0.0130486     0.0197833     0.0127192    0.00333923       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.852371       1.21464       1.58376       1.28595       0.21479        10     
reco:pandora2Track:LArPandoraTrackCreation          14.5815       24.488        35.214        21.4198       7.03246        10     
reco:pandora2calo:Calorimetry                       8.29535       11.5255       17.9467       9.6983        3.18487        10     
reco:pandora2calonosce:Calorimetry                  7.78311       10.9396       15.7411       9.49246       2.88823        10     
reco:pandora2pid:Chi2ParticleID                    0.0133906     0.0225456     0.0285238     0.0221815    0.00471841       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0617899     0.111086      0.140357      0.110709      0.0213613       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0131688     0.0213652     0.0271627     0.0217581    0.00406652       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.93337       8.68051       12.168        8.24593       2.16517        10     
reco:pandora2Showercalo:ShowerCalorimetry           6.95146       9.49048       14.2544       8.73026       2.3671         10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.80645       8.36117       12.3943       7.70008       2.01069        10     
[art]:TriggerResults:TriggerResultInserter        5.1898e-05    7.28041e-05   0.000147738   6.5388e-05    2.55533e-05      10     
end_path:out1:RootOutput                           9.077e-06    1.19486e-05   1.6632e-05    1.19075e-05   1.8313e-06       10     
end_path:out1:RootOutput(write)                     3.82892       4.50394       5.6903        4.31406      0.563875        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5285.79 MB
  Peak resident set size usage (VmHWM): 4470.86 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.27 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.07 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11704592
-rw-r--r-- 1 dune001 lbno 2684944486 Jan 10 14:52 detsim_stage2.root
-rw-r--r-- 1 dune001 lbno 2539849623 Jan 10 14:49 detsim_stage1.root
-rw-r--r-- 1 dune001 lbno 2443135257 Jan 10 16:35 reco_stage1.root
-rw-r--r-- 1 dune001 lbno 2160156271 Jan 10 12:13 g4_stage2.root
-rw-r--r-- 1 dune001 lbno 2064920604 Jan 10 11:52 g4_stage1.root
-rw-r--r-- 1 dune001 lbno   91869270 Jan 10 16:27 Pandora_Events.pndr
-rw-r--r-- 1 dune001 lbno     243415 Jan 10 16:35 jobscript.log
-rw-r--r-- 1 dune001 lbno     217088 Jan 10 16:35 mem.db
-rw-r--r-- 1 dune001 lbno      62376 Jan 10 16:35 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 lbno      36864 Jan 10 16:35 time.db
-rw-r--r-- 1 dune001 lbno        519 Jan 10 14:52 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan 10 12:13 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan 10 16:35 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 lbno         52 Jan 10 11:42 all-input-dids.txt
-rw-r--r-- 1 dune001 lbno          7 Jan 10 16:35 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 lbno          0 Jan 10 15:00 ProtoDUNE.log
-rw-r--r-- 1 dune001 lbno          0 Jan 10 14:52 debugprod.log
justIN time: 2026-02-06 13:00:50 UTC       justIN version: 01.06.00