justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 294674.0@dunegpschedd02.fnal.gov

Jobsub ID294674.0@dunegpschedd02.fnal.gov
Workflow ID12382
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-24 21:42:03
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce503
Last heartbeat2026-01-25 00:19:14
From worker nodeHostnameb9p34p1036.cern.ch
cpuinfoAMD EPYC 9655 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-24 21:42:41
Input filesmonte-carlo-012382-000001
JobscriptExit code0
Real time2h (9310s)
CPU time2h (8776s = 94%)
Max RSS bytes4552331264 (4341 MiB)
Outputting started2026-01-25 00:17:52
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12382/1/001/reco_stage1.root
Finished2026-01-25 00:19:14
Saved logsjustin-logs:294674.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

eSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 159388 traces, 0 summary
Retagger: tagging trace set: gauss with 121751 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 121751 traces tagged "gauss"
FrameSaver: q=4.38493e+06 n=2186633 tag=gauss
wclsFrameSaver: saving 159388 traces tagged "wiener"
FrameSaver: q=4.6419e+06 n=1945741 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 43963 XUs and 43630 XVs -> 1325 XUVs
C:0 T:1 111052 XUs and 117366 XVs -> 22413 XUVs
C:0 T:2 89171 XUs and 104197 XVs -> 4488 XUVs
C:0 T:3 44471 XUs and 44894 XVs -> 508 XUVs
C:0 T:4 13681 XUs and 10676 XVs -> 97 XUVs
C:0 T:5 44210 XUs and 26750 XVs -> 3016 XUVs
C:0 T:6 231727 XUs and 182855 XVs -> 20944 XUVs
C:0 T:7 53989 XUs and 70544 XVs -> 813 XUVs
C:0 T:8 14259 XUs and 16670 XVs -> 261 XUVs
C:0 T:9 101802 XUs and 127856 XVs -> 6769 XUVs
C:0 T:10 62985 XUs and 26034 XVs -> 4918 XUVs
C:0 T:11 181 XUs and 89 XVs -> 14 XUVs
65566 XUVs total
13605 collection wire objects
59078 potential space points
Neighbour search...
11869864 tests to find 4955252 neighbours
Iterating with no regularization...
Begin: 8.76421e+07
0 7.47022e+07
1 7.38159e+07
2 7.37057e+07
3 7.36745e+07
Now with regularization...
Begin: 6.09726e+07
0 6.09154e+07
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411945081
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411945081
fLifetime: 30000 [us]
25-Jan-2026 01:17:50 CET  Closed output file "reco_stage1.root"
25-Jan-2026 01:17:50 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          194.994       303.344       444.965       294.637       66.1132        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00152252    0.00384223    0.00663918    0.00380452     0.0015435       10     
reco:ophit:OpHitFinder                             0.229368      0.257043      0.336159      0.244755      0.0312142       10     
reco:opflash:OpFlashFinder                         0.120468      0.142775      0.209871      0.136369      0.0248265       10     
reco:opslicer:OpSlicer                             0.481721      0.601305      0.845535      0.584847      0.0938393       10     
reco:caldata:DataPrepModule                         3.34658       4.06509       4.73087       4.06973      0.338287        10     
reco:wclsdatasp:WireCellToolkit                     23.2277       25.1753       28.3739       25.1847       1.36864        10     
reco:gaushit:GausHitFinder                         0.752736       1.06847       1.44801       1.07596      0.214124        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000257746   0.000447072   0.000807772   0.000400986   0.000163467      10     
reco:reco3d:SpacePointSolver                        7.91393       17.1733       23.8561       18.2217       5.12674        10     
reco:hitpdune:DisambigFromSpacePoints               1.07835       1.69553       2.52554       1.57335      0.488317        10     
reco:pandora:StandardPandora                        16.8076       37.2911       77.1221       33.8109       16.9221        10     
reco:pandoraWriter:StandardPandora                 0.389854      0.500663      0.680029      0.491554      0.0929026       10     
reco:pandoraTrack:LArPandoraTrackCreation           2.78656       4.52138       8.25975       4.29398       1.61257        10     
reco:pandoraShower:LArPandoraShowerCreation         2.20791       4.80059       8.82405       4.46033       1.93123        10     
reco:pandoracalo:Calorimetry                        1.44148       2.2508        3.45395       2.24089      0.583978        10     
reco:pandoracalonosce:Calorimetry                   1.23466       2.09758       3.40267       2.00722      0.585117        10     
reco:pandorapid:Chi2ParticleID                     0.0040433    0.00500938    0.00575649    0.00509057    0.000510286      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0392089     0.054874      0.0692059     0.0565567     0.0109666       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0039485     0.0049394    0.00697888    0.00479137    0.000820375      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.00498       4.82018       7.84361       4.76235       1.62666        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.86496       4.85245       8.01207       4.76772       1.5362         10     
reco:emtrkmichelid:EmTrackMichelId                  105.062        144.1        182.341       141.899       22.2568        10     
reco:crttag:SingleCRTMatchingProducer              0.0118426     0.0178368     0.0285402     0.0163725    0.00508247       10     
reco:crtreco:TwoCRTMatchingProducer               0.00495395    0.00938513     0.0138603    0.00933573    0.00223911       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.413025      0.613844      0.870823      0.579262       0.14085        10     
reco:pandora2Track:LArPandoraTrackCreation          7.49177       15.8714       33.0768       13.6121       7.51258        10     
reco:pandora2calo:Calorimetry                       2.73263       6.7152        11.5342       6.55343       2.50113        10     
reco:pandora2calonosce:Calorimetry                  2.59396       6.25521       10.4937       6.21012       2.24999        10     
reco:pandora2pid:Chi2ParticleID                   0.00819868     0.011827      0.0155109     0.0119875    0.00227314       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0417449     0.0637052     0.0901157     0.063372      0.0145064       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00738647     0.0115574     0.0139934     0.0121927    0.00226363       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.27993       4.89168       9.33894       4.66334       1.90766        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.38003       5.59499       9.97992       5.4823        1.88778        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.23583       4.96213       8.76662       4.91307       1.65643        10     
[art]:TriggerResults:TriggerResultInserter        3.8017e-05    6.54371e-05   0.000136635   5.4702e-05    2.83392e-05      10     
end_path:out1:RootOutput                           9.805e-06    1.11318e-05   1.6105e-05    1.08165e-05   1.73926e-06      10     
end_path:out1:RootOutput(write)                     1.92833       2.68565       3.53387       2.73673      0.441979        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5374.1 MB
  Peak resident set size usage (VmHWM): 4552.33 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.18 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.55 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11494580
-rw-r--r--. 1 duneprd np-comp 2646919739 Jan 25 00:25 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2514085286 Jan 25 00:23 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2388548121 Jan 25 01:17 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 2112437306 Jan 24 23:05 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 2019670036 Jan 24 22:48 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   88171180 Jan 25 01:14 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     243275 Jan 25 01:17 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan 25 01:17 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan 25 01:17 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan 25 01:17 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan 25 00:25 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 24 23:05 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 25 01:17 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan 24 22:42 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan 25 01:17 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan 25 00:30 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan 25 00:25 debugprod.log
justIN time: 2026-02-04 22:17:52 UTC       justIN version: 01.06.00