justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 277581.0@dunegpschedd01.fnal.gov

Jobsub ID277581.0@dunegpschedd01.fnal.gov
Workflow ID11882
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-14 06:32:15
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce15-multicore
Last heartbeat2026-01-14 11:51:52
From worker nodeHostnamehnode62.pic.es
cpuinfoAMD EPYC 7402P 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-14 06:33:10
Input filesmonte-carlo-011882-000001
JobscriptExit code0
Real time5h (19052s)
CPU time3h (12822s = 67%)
Max RSS bytes4428181504 (4223 MiB)
Outputting started2026-01-14 11:50:43
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11882/1/001/reco_stage1.root
Finished2026-01-14 11:51:52
Saved logsjustin-logs:277581.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

are:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 148680 traces, 0 summary
Retagger: tagging trace set: gauss with 115292 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 115292 traces tagged "gauss"
FrameSaver: q=4.11913e+06 n=2080841 tag=gauss
wclsFrameSaver: saving 148680 traces tagged "wiener"
FrameSaver: q=4.34681e+06 n=1852631 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 36311 XUs and 37447 XVs -> 448 XUVs
C:0 T:1 80411 XUs and 75734 XVs -> 6591 XUVs
C:0 T:2 85126 XUs and 109831 XVs -> 3783 XUVs
C:0 T:3 42408 XUs and 51300 XVs -> 319 XUVs
C:0 T:4 18627 XUs and 17863 XVs -> 352 XUVs
C:0 T:5 95852 XUs and 92001 XVs -> 10315 XUVs
C:0 T:6 181858 XUs and 193386 XVs -> 11848 XUVs
C:0 T:7 64074 XUs and 65537 XVs -> 682 XUVs
C:0 T:8 9079 XUs and 7769 XVs -> 97 XUVs
C:0 T:9 61498 XUs and 70837 XVs -> 4868 XUVs
C:0 T:10 16653 XUs and 16975 XVs -> 2850 XUVs
C:0 T:11 8 XUs and 4 XVs -> 0 XUVs
42153 XUVs total
14052 collection wire objects
38385 potential space points
Neighbour search...
3392587 tests to find 1697204 neighbours
Iterating with no regularization...
Begin: 7.91014e+07
0 6.66593e+07
1 6.57121e+07
2 6.55749e+07
3 6.55365e+07
Now with regularization...
Begin: 5.32554e+07
0 5.32154e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411731020
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411731020
fLifetime: 30000 [us]
14-Jan-2026 12:50:42 CET  Closed output file "reco_stage1.root"
14-Jan-2026 12:50:42 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          308.918       545.165        953.3        451.215       216.677        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                             0.0011391    0.00329107    0.00463291     0.0035756    0.00102439       10     
reco:ophit:OpHitFinder                             0.246439      0.281627      0.344796      0.272763      0.027798        10     
reco:opflash:OpFlashFinder                         0.148067      0.204271       0.29165      0.206151      0.0395948       10     
reco:opslicer:OpSlicer                              0.55977      0.696712      0.925195      0.694082      0.0939028       10     
reco:caldata:DataPrepModule                         4.26914       4.4819        4.75795       4.49121      0.155768        10     
reco:wclsdatasp:WireCellToolkit                     28.3793       33.6237       40.724        33.6487       4.07406        10     
reco:gaushit:GausHitFinder                         0.961266       1.2821        1.70992       1.21756      0.234128        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000514808   0.00084777    0.00161547    0.000769243   0.000313363      10     
reco:reco3d:SpacePointSolver                        7.8307        16.7236       37.4208       14.9803       7.84972        10     
reco:hitpdune:DisambigFromSpacePoints               1.17368       1.75135       2.6413        1.73268      0.463372        10     
reco:pandora:StandardPandora                        20.3667       45.4829       107.775       32.9895       27.6167        10     
reco:pandoraWriter:StandardPandora                 0.369518      0.496461       0.90165      0.430886      0.157197        10     
reco:pandoraTrack:LArPandoraTrackCreation           3.28268       6.20712       10.5606       4.45714       3.03855        10     
reco:pandoraShower:LArPandoraShowerCreation         2.94627       7.80558       15.9635       4.97864       5.19585        10     
reco:pandoracalo:Calorimetry                        1.64834       3.53612       6.89194       2.13872       2.11746        10     
reco:pandoracalonosce:Calorimetry                   1.49826       2.88119       5.7743        2.03985       1.47686        10     
reco:pandorapid:Chi2ParticleID                    0.00507432    0.00609711    0.00978931    0.00568505    0.00135997       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP          0.04409      0.0531806     0.0773442     0.0488357    0.00992067       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00478287    0.00585863    0.00770351    0.00549505    0.00102398       10     
reco:pandoraShowercalo:ShowerCalorimetry            3.01571       8.46238       20.6285       5.41756       5.90763        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.94137       8.84839       16.0557       7.55304       5.43047        10     
reco:emtrkmichelid:EmTrackMichelId                  183.816       322.339       645.434       246.573       160.622        10     
reco:crttag:SingleCRTMatchingProducer             0.00851253      0.02172      0.0330523     0.0224315    0.00767176       10     
reco:crtreco:TwoCRTMatchingProducer               0.00686664     0.0155336     0.0237486     0.0163072    0.00497888       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.707801      0.919003       1.2511       0.834129      0.170244        10     
reco:pandora2Track:LArPandoraTrackCreation          10.6186       33.8253       150.012       20.0891       39.4181        10     
reco:pandora2calo:Calorimetry                       4.74366       9.77452       19.6426       9.09258       3.90103        10     
reco:pandora2calonosce:Calorimetry                  4.50729       8.94001       12.3848       8.31673       2.59795        10     
reco:pandora2pid:Chi2ParticleID                    0.0117118     0.0156465     0.0204592     0.0158827    0.00252578       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0545872     0.071645      0.103224      0.0688695     0.0145989       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0115468     0.015231      0.0180127     0.0160315    0.00209019       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.11473       7.43983       12.3797       6.98972       2.28296        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.24674       8.36428       14.8085       8.24615       3.0023         10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.95303       7.32034       12.5685       7.04566       2.49126        10     
[art]:TriggerResults:TriggerResultInserter        2.3311e-05    3.52263e-05   9.3414e-05    2.7481e-05    2.00198e-05      10     
end_path:out1:RootOutput                           4.91e-06     5.9492e-06     9.221e-06     5.595e-06    1.17225e-06      10     
end_path:out1:RootOutput(write)                     2.53208       3.10337       3.82318       3.0837       0.437615        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5239.21 MB
  Peak resident set size usage (VmHWM): 4428.18 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.19 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.81 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11380876
-rw-r--r-- 1 dune001 dune 2627406365 Jan 14 11:17 detsim_stage2.root
-rw-r--r-- 1 dune001 dune 2491694104 Jan 14 11:13 detsim_stage1.root
-rw-r--r-- 1 dune001 dune 2368027640 Jan 14 12:50 reco_stage1.root
-rw-r--r-- 1 dune001 dune 2075747814 Jan 14 07:54 g4_stage2.root
-rw-r--r-- 1 dune001 dune 2002211215 Jan 14 07:39 g4_stage1.root
-rw-r--r-- 1 dune001 dune   88299110 Jan 14 12:38 Pandora_Events.pndr
-rw-r--r-- 1 dune001 dune     243199 Jan 14 12:50 jobscript.log
-rw-r--r-- 1 dune001 dune     217088 Jan 14 12:50 mem.db
-rw-r--r-- 1 dune001 dune      62376 Jan 14 12:50 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 dune      36864 Jan 14 12:50 time.db
-rw-r--r-- 1 dune001 dune        519 Jan 14 11:17 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Jan 14 07:54 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Jan 14 12:50 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 dune         52 Jan 14 07:33 all-input-dids.txt
-rw-r--r-- 1 dune001 dune          7 Jan 14 12:50 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune          0 Jan 14 11:33 ProtoDUNE.log
-rw-r--r-- 1 dune001 dune          0 Jan 14 11:17 debugprod.log
justIN time: 2026-02-06 03:24:59 UTC       justIN version: 01.06.00