justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 282176.1@dunegpschedd01.fnal.gov

Jobsub ID282176.1@dunegpschedd01.fnal.gov
Workflow ID12101
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-19 14:19:12
SiteIT_CNAF
EntryCMSHTPC_T1_IT_CNAF_condor_ce02
Last heartbeat2026-01-19 17:17:22
From worker nodeHostnamewn-s04-05-09-01.cr.cnaf.infn.it
cpuinfoAMD EPYC 7313 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-19 14:20:02
Input filesmonte-carlo-012101-000001
JobscriptExit code0
Real time2h (10317s)
CPU time2h (9909s = 96%)
Max RSS bytes4426420224 (4221 MiB)
Outputting started2026-01-19 17:12:00
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12101/1/001/reco_stage1.root
Finished2026-01-19 17:17:22
Saved logsjustin-logs:282176.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 147489 traces, 0 summary
Retagger: tagging trace set: gauss with 110461 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 110461 traces tagged "gauss"
FrameSaver: q=3.89092e+06 n=1926086 tag=gauss
wclsFrameSaver: saving 147489 traces tagged "wiener"
FrameSaver: q=4.13935e+06 n=1699089 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 23596 XUs and 27622 XVs -> 534 XUVs
C:0 T:1 42739 XUs and 45331 XVs -> 6486 XUVs
C:0 T:2 121642 XUs and 171742 XVs -> 10882 XUVs
C:0 T:3 45548 XUs and 61174 XVs -> 544 XUVs
C:0 T:4 14080 XUs and 14115 XVs -> 244 XUVs
C:0 T:5 58615 XUs and 49358 XVs -> 5399 XUVs
C:0 T:6 118137 XUs and 161438 XVs -> 14985 XUVs
C:0 T:7 38867 XUs and 52790 XVs -> 1334 XUVs
C:0 T:8 11005 XUs and 8331 XVs -> 122 XUVs
C:0 T:9 63673 XUs and 63617 XVs -> 4112 XUVs
C:0 T:10 34804 XUs and 26586 XVs -> 3829 XUVs
C:0 T:11 231 XUs and 196 XVs -> 19 XUVs
48490 XUVs total
12370 collection wire objects
41849 potential space points
Neighbour search...
4841193 tests to find 2315014 neighbours
Iterating with no regularization...
Begin: 6.77044e+07
0 5.46494e+07
1 5.35185e+07
2 5.33316e+07
3 5.32797e+07
Now with regularization...
Begin: 4.25517e+07
0 4.24878e+07
1 4.24512e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411838286
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411838286
fLifetime: 30000 [us]
19-Jan-2026 18:11:58 CET  Closed output file "reco_stage1.root"
19-Jan-2026 18:11:58 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          224.026       277.05        349.466       256.335       45.2938        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000943932   0.00321274    0.00474405    0.00357027     0.0012995       10     
reco:ophit:OpHitFinder                             0.204989      0.247407      0.357536       0.2381       0.0395028       10     
reco:opflash:OpFlashFinder                         0.119562      0.168069      0.196741      0.170391      0.023203        10     
reco:opslicer:OpSlicer                             0.411033      0.538251       0.79571      0.509407      0.117133        10     
reco:caldata:DataPrepModule                         3.48613       4.24142       5.94282       3.81056      0.866691        10     
reco:wclsdatasp:WireCellToolkit                     23.4627       26.4221       31.8675       24.9556       2.94093        10     
reco:gaushit:GausHitFinder                         0.693993       0.9869        1.38023      0.877356      0.260683        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000342974   0.000474334   0.000752012   0.000438223   0.000130785      10     
reco:reco3d:SpacePointSolver                        7.76186       16.6888       40.2288       13.2056       9.13241        10     
reco:hitpdune:DisambigFromSpacePoints              0.963427       1.52207       2.28299       1.55464      0.404712        10     
reco:pandora:StandardPandora                        17.372        27.7688       44.4195       25.1373       8.96406        10     
reco:pandoraWriter:StandardPandora                  0.27956      0.356249       0.49796      0.349195      0.0604231       10     
reco:pandoraTrack:LArPandoraTrackCreation           2.27681       2.82361       4.18921       2.55245      0.575691        10     
reco:pandoraShower:LArPandoraShowerCreation         2.38968       3.17451       4.46962       3.13894      0.619007        10     
reco:pandoracalo:Calorimetry                        1.40301       1.66349       2.50269       1.52878      0.335774        10     
reco:pandoracalonosce:Calorimetry                   1.27949       1.52246       2.0598        1.46093      0.239049        10     
reco:pandorapid:Chi2ParticleID                    0.00341815    0.00399039    0.00541963    0.00364387    0.000725638      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0331602     0.0400689     0.0624518     0.0354421    0.00880653       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00331912    0.00367613    0.00520559    0.00348332    0.000533904      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.77238       3.52496       4.45937       3.41578      0.618642        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.69786       3.67411       5.57639       3.26353       1.01235        10     
reco:emtrkmichelid:EmTrackMichelId                  118.903       146.746       193.967       136.555        24.88         10     
reco:crttag:SingleCRTMatchingProducer             0.00645808     0.0109195     0.016179      0.0103794    0.00317687       10     
reco:crtreco:TwoCRTMatchingProducer               0.00498983    0.00623226    0.00758132    0.00606927    0.00098819       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.596152      0.718902       0.9031       0.674214      0.113292        10     
reco:pandora2Track:LArPandoraTrackCreation          6.91285       9.66543       13.0527       9.68852       2.04603        10     
reco:pandora2calo:Calorimetry                       3.76372       5.18516       6.64675       5.31396      0.923027        10     
reco:pandora2calonosce:Calorimetry                  3.59212       4.90497       6.23482       4.96543      0.837126        10     
reco:pandora2pid:Chi2ParticleID                   0.00730466    0.00954486     0.0157991    0.00892625     0.0023235       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0389031     0.0495818     0.0814535     0.046843      0.0118453       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00707403    0.00944231     0.0165071    0.00873747    0.00252054       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.57222       3.40047       4.7104        3.1469       0.726622        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.51892       4.35838       5.73471       4.09504      0.793294        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.2082        4.07547       5.10307       3.9455       0.674234        10     
[art]:TriggerResults:TriggerResultInserter        3.2792e-05    6.62221e-05   0.000212549   5.12655e-05   5.04531e-05      10     
end_path:out1:RootOutput                           8.485e-06    1.31335e-05   2.5598e-05    1.1341e-05    5.29706e-06      10     
end_path:out1:RootOutput(write)                     1.88096       2.45293       3.02979       2.5291       0.360112        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5230.99 MB
  Peak resident set size usage (VmHWM): 4426.42 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.20 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.66 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11162824
-rw-r--r-- 1 neutrino002 neutrino 2584723942 Jan 19 17:24 detsim_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 2449396912 Jan 19 17:22 detsim_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 2316952128 Jan 19 18:11 reco_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 2042719235 Jan 19 15:42 g4_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 1950972213 Jan 19 15:29 g4_stage1.root
-rw-r--r-- 1 neutrino002 neutrino   85334610 Jan 19 18:08 Pandora_Events.pndr
-rw-r--r-- 1 neutrino002 neutrino     243806 Jan 19 18:11 jobscript.log
-rw-r--r-- 1 neutrino002 neutrino     217088 Jan 19 18:11 mem.db
-rw-r--r-- 1 neutrino002 neutrino      62376 Jan 19 18:11 Pandora_Geometry.xml
-rw-r--r-- 1 neutrino002 neutrino      36864 Jan 19 18:11 time.db
-rw-r--r-- 1 neutrino002 neutrino        519 Jan 19 17:24 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Jan 19 15:42 g4_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Jan 19 18:11 protoDUNE_reco1_hist.root
-rw-r--r-- 1 neutrino002 neutrino         52 Jan 19 15:20 all-input-dids.txt
-rw-r--r-- 1 neutrino002 neutrino          7 Jan 19 18:11 justin-processed-pfns.txt
-rw-r--r-- 1 neutrino002 neutrino          0 Jan 19 17:29 ProtoDUNE.log
-rw-r--r-- 1 neutrino002 neutrino          0 Jan 19 17:24 debugprod.log
justIN time: 2026-02-05 17:08:09 UTC       justIN version: 01.06.00