justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 296253.0@dunegpschedd02.fnal.gov

Jobsub ID296253.0@dunegpschedd02.fnal.gov
Workflow ID12464
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-28 08:06:25
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce512
Last heartbeat2026-01-28 11:17:27
From worker nodeHostnameb9p02p8722.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-28 08:07:54
Input filesmonte-carlo-012464-000001
JobscriptExit code0
Real time3h (11295s)
CPU time2h (10693s = 94%)
Max RSS bytes4047990784 (3860 MiB)
Outputting started2026-01-28 11:16:10
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12464/1/001/reco_stage1.root
Finished2026-01-28 11:17:27
Saved logsjustin-logs:296253.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

tPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 121185 traces, 0 summary
Retagger: tagging trace set: gauss with 95055 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 95055 traces tagged "gauss"
FrameSaver: q=3.19157e+06 n=1689565 tag=gauss
wclsFrameSaver: saving 121185 traces tagged "wiener"
FrameSaver: q=3.35359e+06 n=1492862 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 36449 XUs and 38765 XVs -> 350 XUVs
C:0 T:1 63264 XUs and 61938 XVs -> 3579 XUVs
C:0 T:2 145800 XUs and 121384 XVs -> 5612 XUVs
C:0 T:3 67426 XUs and 44340 XVs -> 520 XUVs
C:0 T:4 9609 XUs and 7292 XVs -> 72 XUVs
C:0 T:5 23606 XUs and 17410 XVs -> 1564 XUVs
C:0 T:6 83644 XUs and 74193 XVs -> 3215 XUVs
C:0 T:7 34255 XUs and 29650 XVs -> 185 XUVs
C:0 T:8 7472 XUs and 5593 XVs -> 67 XUVs
C:0 T:9 56936 XUs and 53619 XVs -> 4358 XUVs
C:0 T:10 39887 XUs and 15551 XVs -> 3660 XUVs
C:0 T:11 69 XUs and 36 XVs -> 11 XUVs
23193 XUVs total
11287 collection wire objects
21135 potential space points
Neighbour search...
661313 tests to find 363980 neighbours
Iterating with no regularization...
Begin: 6.33427e+07
0 5.36654e+07
1 5.30991e+07
2 5.30225e+07
3 5.30008e+07
Now with regularization...
Begin: 4.3918e+07
0 4.39097e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 412014098
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 412014098
fLifetime: 30000 [us]
28-Jan-2026 12:16:08 CET  Closed output file "reco_stage1.root"
28-Jan-2026 12:16:08 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          206.977       284.054       345.189       293.018       53.529         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00109808    0.00344126    0.00629602    0.00343941    0.00163149       10     
reco:ophit:OpHitFinder                             0.197782       0.26129      0.342129      0.250778      0.0463351       10     
reco:opflash:OpFlashFinder                         0.0997639      0.14508      0.183782      0.148798      0.0266351       10     
reco:opslicer:OpSlicer                             0.382029      0.560419      0.721881      0.567157      0.104264        10     
reco:caldata:DataPrepModule                         4.04816       4.85836       5.60532       4.8644       0.522383        10     
reco:wclsdatasp:WireCellToolkit                     26.8988       30.4766       33.1091       30.0966       1.84115        10     
reco:gaushit:GausHitFinder                         0.648212       1.03124       1.26227      0.986624      0.177361        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000296303   0.000438514   0.000578357   0.000429459   7.5323e-05       10     
reco:reco3d:SpacePointSolver                        6.91338       12.5204       19.9265       13.2819       3.93489        10     
reco:hitpdune:DisambigFromSpacePoints              0.661944       1.36365       1.95596       1.39304      0.469893        10     
reco:pandora:StandardPandora                        10.5372       25.6567       39.7315       27.5252       8.22972        10     
reco:pandoraWriter:StandardPandora                 0.193041      0.344755      0.551863      0.357232      0.093611        10     
reco:pandoraTrack:LArPandoraTrackCreation           1.34889       2.75078       3.92818       3.03794       0.85215        10     
reco:pandoraShower:LArPandoraShowerCreation         1.22237       3.02247       4.44721       3.09603       1.09272        10     
reco:pandoracalo:Calorimetry                       0.735494       1.63414       2.70615       1.74174      0.628623        10     
reco:pandoracalonosce:Calorimetry                  0.753986       1.55757       2.74845       1.55188      0.625233        10     
reco:pandorapid:Chi2ParticleID                    0.00269668     0.0047253    0.00842174    0.00430695    0.00161883       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0220126     0.0436295     0.0755619     0.038618      0.0168714       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00284693    0.00437524    0.00782624    0.00393852    0.00148837       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.26628       3.55531       6.45851        3.607        1.53845        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.32079       3.45465       4.74951       3.89795       1.28702        10     
reco:emtrkmichelid:EmTrackMichelId                  118.229       156.267         185          162.1        25.9466        10     
reco:crttag:SingleCRTMatchingProducer             0.00932894     0.0126414     0.0163943     0.0126424    0.00275726       10     
reco:crtreco:TwoCRTMatchingProducer               0.00555697    0.00763308     0.0110815    0.00706754     0.0018961       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.553352      0.855131       1.14277      0.851034      0.197889        10     
reco:pandora2Track:LArPandoraTrackCreation          4.51221       9.44406       13.9829       9.53813       3.24467        10     
reco:pandora2calo:Calorimetry                       2.62645       5.08523       7.42633       5.34302       1.67455        10     
reco:pandora2calonosce:Calorimetry                  2.36529       4.75707       7.12943       4.69282       1.74511        10     
reco:pandora2pid:Chi2ParticleID                   0.00648811     0.0122835     0.0228371     0.0103041    0.00489303       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0370593     0.0605257     0.103743      0.0516084     0.0217393       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00685934     0.0118971     0.0202759     0.0102911    0.00448548       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.52469       3.31252       4.84474       3.47644       1.20745        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.92027       4.42543       6.55981       4.88011       1.57038        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.80416       4.08087       5.88525       4.34568       1.41481        10     
[art]:TriggerResults:TriggerResultInserter        4.5881e-05    7.01106e-05   0.000168271   6.14905e-05   3.34785e-05      10     
end_path:out1:RootOutput                            9.6e-06     1.2195e-05     1.91e-05     1.1415e-05    2.5454e-06       10     
end_path:out1:RootOutput(write)                     1.59084       2.38081       3.32121       2.31378      0.564326        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4908.47 MB
  Peak resident set size usage (VmHWM): 4047.99 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.21 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.07 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9582988
-rw-r--r--. 1 duneprd np-comp 2276582666 Jan 28 11:26 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2165048960 Jan 28 11:24 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 1948751940 Jan 28 12:16 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1707742343 Jan 28 09:29 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1642670855 Jan 28 09:17 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   71567890 Jan 28 12:12 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     242655 Jan 28 12:16 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan 28 12:16 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan 28 12:16 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan 28 12:16 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan 28 11:26 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 28 09:29 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 28 12:16 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan 28 09:08 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan 28 12:16 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan 28 11:32 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan 28 11:26 debugprod.log
justIN time: 2026-02-04 18:09:27 UTC       justIN version: 01.06.00