justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 282177.0@dunegpschedd01.fnal.gov

Jobsub ID282177.0@dunegpschedd01.fnal.gov
Workflow ID12102
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-19 14:19:12
SiteIT_CNAF
EntryCMSHTPC_T1_IT_CNAF_condor_ce02
Last heartbeat2026-01-19 17:14:27
From worker nodeHostnamewn-s04-05-09-01.cr.cnaf.infn.it
cpuinfoAMD EPYC 7313 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-19 14:20:10
Input filesmonte-carlo-012102-000001
JobscriptExit code0
Real time2h (10135s)
CPU time2h (9706s = 95%)
Max RSS bytes4409958400 (4205 MiB)
Outputting started2026-01-19 17:09:06
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12102/1/001/reco_stage1.root
Finished2026-01-19 17:14:27
Saved logsjustin-logs:282177.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 183253 traces, 0 summary
Retagger: tagging trace set: gauss with 135143 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 135143 traces tagged "gauss"
FrameSaver: q=4.54282e+06 n=2357128 tag=gauss
wclsFrameSaver: saving 183253 traces tagged "wiener"
FrameSaver: q=4.84778e+06 n=2051638 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 60078 XUs and 61651 XVs -> 1138 XUVs
C:0 T:1 148205 XUs and 161966 XVs -> 14306 XUVs
C:0 T:2 153492 XUs and 189342 XVs -> 12723 XUVs
C:0 T:3 62290 XUs and 66230 XVs -> 656 XUVs
C:0 T:4 16107 XUs and 18776 XVs -> 277 XUVs
C:0 T:5 109100 XUs and 196012 XVs -> 28042 XUVs
C:0 T:6 195425 XUs and 179390 XVs -> 23822 XUVs
C:0 T:7 56625 XUs and 53747 XVs -> 1086 XUVs
C:0 T:8 8247 XUs and 9162 XVs -> 143 XUVs
C:0 T:9 107987 XUs and 234315 XVs -> 29466 XUVs
C:0 T:10 65922 XUs and 37666 XVs -> 12175 XUVs
C:0 T:11 153 XUs and 69 XVs -> 15 XUVs
123849 XUVs total
19688 collection wire objects
114019 potential space points
Neighbour search...
33088885 tests to find 13280328 neighbours
Iterating with no regularization...
Begin: 8.40515e+07
0 6.60696e+07
1 6.48221e+07
2 6.46452e+07
3 6.45922e+07
Now with regularization...
Begin: 5.22511e+07
0 5.21334e+07
1 5.20339e+07
2 5.19545e+07
3 5.18884e+07
4 5.18308e+07
5 5.17823e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411838298
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411838298
fLifetime: 30000 [us]
19-Jan-2026 18:09:04 CET  Closed output file "reco_stage1.root"
19-Jan-2026 18:09:04 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          140.655       303.137       410.66        332.114       100.692        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000766508    0.0031147    0.00639132    0.00328621    0.00154969       10     
reco:ophit:OpHitFinder                             0.155159      0.247311      0.373168      0.227959      0.0614685       10     
reco:opflash:OpFlashFinder                         0.0778653     0.158046      0.214959      0.159808      0.0358034       10     
reco:opslicer:OpSlicer                             0.277698      0.531367      0.779682      0.512961      0.143858        10     
reco:caldata:DataPrepModule                         3.30574       4.10524       6.16427       3.60545      0.945936        10     
reco:wclsdatasp:WireCellToolkit                     22.5705       26.283        31.8321       24.577        3.39458        10     
reco:gaushit:GausHitFinder                         0.566217       1.02481       1.71809      0.970006      0.371009        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000196309   0.000430694   0.000678905   0.000393338   0.000148435      10     
reco:reco3d:SpacePointSolver                        5.05913       15.5109       45.3906       11.6492       11.1364        10     
reco:hitpdune:DisambigFromSpacePoints              0.447423       1.53349       2.73819       1.48937      0.737543        10     
reco:pandora:StandardPandora                        6.66708       41.0514       85.1419       32.6955       28.631         10     
reco:pandoraWriter:StandardPandora                 0.156853      0.370644      0.636417       0.33646      0.163415        10     
reco:pandoraTrack:LArPandoraTrackCreation          0.995541       3.44331       6.22822       3.04035       1.84079        10     
reco:pandoraShower:LArPandoraShowerCreation        0.941721       3.84757       8.10208       3.61266       2.43565        10     
reco:pandoracalo:Calorimetry                       0.615378       1.93595       3.3284        1.62233       1.02095        10     
reco:pandoracalonosce:Calorimetry                  0.554728       1.79754       2.96383       1.51061      0.952615        10     
reco:pandorapid:Chi2ParticleID                    0.00235881    0.00520921    0.00802683    0.00488824    0.00205098       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0217206     0.0468328     0.074606      0.0402552     0.0197175       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0021751    0.00477224    0.00817449    0.00399994    0.00207849       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.05528       3.82687       7.01466       4.13539       2.1322         10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.17244       3.75953       8.44302       3.9179        2.22876        10     
reco:emtrkmichelid:EmTrackMichelId                  75.574        158.471       207.672       184.021       44.1909        10     
reco:crttag:SingleCRTMatchingProducer             0.00702446     0.0100364     0.0144297    0.00951329    0.00228467       10     
reco:crtreco:TwoCRTMatchingProducer               0.00402868    0.00564693    0.00920005    0.00532201    0.00131301       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.530166      0.701207      0.856069      0.702536      0.110789        10     
reco:pandora2Track:LArPandoraTrackCreation          4.15073       10.1964       15.824        10.823        4.58061        10     
reco:pandora2calo:Calorimetry                       2.06548       5.18441       8.49436       5.14652       2.45069        10     
reco:pandora2calonosce:Calorimetry                  1.92889       4.93128       8.20154       4.89875       2.36004        10     
reco:pandora2pid:Chi2ParticleID                   0.00679542     0.0123065     0.0195578     0.0116465    0.00391683       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0301845     0.0575065     0.0916557     0.0531523     0.0182115       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00662339     0.0132893     0.0220967     0.0113068     0.0053077       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.45464       3.43805       5.98548       3.16665       1.47439        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.64897       4.12751       6.92193       4.37021       1.91139        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.51253       4.06284       6.6056        4.45133       1.93075        10     
[art]:TriggerResults:TriggerResultInserter        2.2522e-05    8.22013e-05    0.0004578    3.88225e-05   0.000126345      10     
end_path:out1:RootOutput                           5.76e-06     9.2083e-06    1.7443e-05    8.1205e-06    3.27825e-06      10     
end_path:out1:RootOutput(write)                     1.41498       2.37194       3.3645        2.32041      0.652443        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5211.37 MB
  Peak resident set size usage (VmHWM): 4409.96 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.18 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.55 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11059876
-rw-r--r-- 1 neutrino002 neutrino 2569705264 Jan 19 17:16 detsim_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 2444432462 Jan 19 17:15 detsim_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 2277000501 Jan 19 18:09 reco_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 2024780137 Jan 19 15:41 g4_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 1927717658 Jan 19 15:29 g4_stage1.root
-rw-r--r-- 1 neutrino002 neutrino   81040540 Jan 19 18:04 Pandora_Events.pndr
-rw-r--r-- 1 neutrino002 neutrino     243203 Jan 19 18:09 jobscript.log
-rw-r--r-- 1 neutrino002 neutrino     217088 Jan 19 18:09 mem.db
-rw-r--r-- 1 neutrino002 neutrino      62376 Jan 19 18:09 Pandora_Geometry.xml
-rw-r--r-- 1 neutrino002 neutrino      36864 Jan 19 18:09 time.db
-rw-r--r-- 1 neutrino002 neutrino        519 Jan 19 17:16 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Jan 19 15:41 g4_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Jan 19 18:09 protoDUNE_reco1_hist.root
-rw-r--r-- 1 neutrino002 neutrino         52 Jan 19 15:20 all-input-dids.txt
-rw-r--r-- 1 neutrino002 neutrino          7 Jan 19 18:09 justin-processed-pfns.txt
-rw-r--r-- 1 neutrino002 neutrino          0 Jan 19 17:20 ProtoDUNE.log
-rw-r--r-- 1 neutrino002 neutrino          0 Jan 19 17:16 debugprod.log
justIN time: 2026-02-05 17:08:42 UTC       justIN version: 01.06.00