justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 270839.0@dunegpschedd01.fnal.gov

Jobsub ID270839.0@dunegpschedd01.fnal.gov
Workflow ID11404
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-29 21:07:45
SiteIT_CNAF
EntryCMSHTPC_T1_IT_CNAF_condor_ce02
Last heartbeat2025-12-30 00:05:14
From worker nodeHostnamewn-s04-05-07-02.cr.cnaf.infn.it
cpuinfoAMD EPYC 7313 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-29 21:08:35
Input filesmonte-carlo-011404-000001
JobscriptExit code0
Real time2h (10278s)
CPU time2h (9886s = 96%)
Max RSS bytes4274462720 (4076 MiB)
Outputting started2025-12-29 23:59:54
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11404/1/001/reco_stage1.root
Finished2025-12-30 00:05:14
Saved logsjustin-logs:270839.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 146953 traces, 0 summary
Retagger: tagging trace set: gauss with 111176 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 111176 traces tagged "gauss"
FrameSaver: q=3.91147e+06 n=1951941 tag=gauss
wclsFrameSaver: saving 146953 traces tagged "wiener"
FrameSaver: q=4.15056e+06 n=1717006 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 57837 XUs and 50290 XVs -> 1108 XUVs
C:0 T:1 153777 XUs and 129308 XVs -> 12010 XUVs
C:0 T:2 86254 XUs and 131509 XVs -> 6519 XUVs
C:0 T:3 42612 XUs and 51801 XVs -> 371 XUVs
C:0 T:4 15793 XUs and 13515 XVs -> 131 XUVs
C:0 T:5 61214 XUs and 48850 XVs -> 4912 XUVs
C:0 T:6 86149 XUs and 92329 XVs -> 6763 XUVs
C:0 T:7 34879 XUs and 37007 XVs -> 751 XUVs
C:0 T:8 5923 XUs and 9315 XVs -> 165 XUVs
C:0 T:9 42171 XUs and 76011 XVs -> 7073 XUVs
C:0 T:10 27750 XUs and 20082 XVs -> 3429 XUVs
C:0 T:11 63 XUs and 30 XVs -> 7 XUVs
43239 XUVs total
13502 collection wire objects
37876 potential space points
Neighbour search...
2767558 tests to find 1488684 neighbours
Iterating with no regularization...
Begin: 6.23364e+07
0 5.02662e+07
1 4.9372e+07
2 4.92455e+07
3 4.92088e+07
Now with regularization...
Begin: 3.78964e+07
0 3.78573e+07
1 3.7833e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411421574
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411421574
fLifetime: 30000 [us]
30-Dec-2025 00:59:52 CET  Closed output file "reco_stage1.root"
30-Dec-2025 00:59:52 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          264.971       328.589       391.155       331.484       42.9403        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00122992    0.00370382    0.00675372    0.00424212    0.00162227       10     
reco:ophit:OpHitFinder                             0.250656      0.305181      0.348778      0.308688      0.0235189       10     
reco:opflash:OpFlashFinder                         0.135353      0.169346      0.203025       0.16679      0.0195197       10     
reco:opslicer:OpSlicer                             0.417148      0.691708      0.874327      0.703402      0.132383        10     
reco:caldata:DataPrepModule                         4.14212       5.46052       6.24348       5.70082      0.640641        10     
reco:wclsdatasp:WireCellToolkit                     26.7292       32.522        35.461        34.1041       3.06659        10     
reco:gaushit:GausHitFinder                         0.855121       1.28032       1.52907       1.30336      0.181458        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000389121   0.000596938   0.00109559    0.000552938   0.000189752      10     
reco:reco3d:SpacePointSolver                        11.3052       16.6971       22.6714       16.6049       3.90269        10     
reco:hitpdune:DisambigFromSpacePoints               1.21918       1.75277       2.16689       1.73289      0.341449        10     
reco:pandora:StandardPandora                        16.5485       31.4598       44.8872       33.6885       8.08423        10     
reco:pandoraWriter:StandardPandora                 0.271243      0.461053      0.561549      0.498467      0.0841629       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.73362       3.46159       5.19026       3.69216      0.883706        10     
reco:pandoraShower:LArPandoraShowerCreation         2.32742       3.7485        4.74757       3.82774      0.755586        10     
reco:pandoracalo:Calorimetry                        1.19323       2.07877       3.0102        2.19712      0.474089        10     
reco:pandoracalonosce:Calorimetry                  0.984388       1.84918       2.91016       1.84198      0.476792        10     
reco:pandorapid:Chi2ParticleID                    0.00306085    0.00596734    0.00765299    0.00648381    0.00146109       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0321943     0.0594604     0.0740067     0.0642179     0.0133514       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00333596    0.00565113    0.00749725    0.00592927     0.0012811       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.60957       4.01236       4.95721       4.02422      0.788727        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.55613       3.91121       4.81967       3.94887      0.798149        10     
reco:emtrkmichelid:EmTrackMichelId                  135.328       177.918       203.787       186.287       21.9591        10     
reco:crttag:SingleCRTMatchingProducer             0.00955425     0.0153604     0.0187711     0.0161369     0.0026261       10     
reco:crtreco:TwoCRTMatchingProducer               0.00721994    0.00846262     0.0106132    0.00837075    0.000955731      10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.730273      0.876694       1.1276        0.86119      0.110403        10     
reco:pandora2Track:LArPandoraTrackCreation          8.30085       11.4317       14.5968       11.4335       2.11727        10     
reco:pandora2calo:Calorimetry                       4.61171       5.94664       7.57745       6.17978       1.01993        10     
reco:pandora2calonosce:Calorimetry                  4.1275        5.59186       7.33613       5.5563        1.04361        10     
reco:pandora2pid:Chi2ParticleID                   0.00747609     0.0142126     0.0196547     0.014967     0.00360878       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.041908      0.0732166     0.0907306     0.0786934     0.0154264       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00887229     0.0142318     0.0184909     0.0141269    0.00251971       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.59285       3.98635       5.23729       4.11334      0.821167        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.67045       5.06283       6.27585       5.35236      0.934502        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.50558       4.81513       5.82932       5.00024      0.765354        10     
[art]:TriggerResults:TriggerResultInserter        4.5776e-05    8.82769e-05   0.000190618   7.4385e-05     4.081e-05       10     
end_path:out1:RootOutput                           9.648e-06    1.37758e-05   2.5879e-05    1.20975e-05   4.86446e-06      10     
end_path:out1:RootOutput(write)                     2.11557       2.79291       3.32846       2.85341       0.38593        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5083.71 MB
  Peak resident set size usage (VmHWM): 4274.46 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.25 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.29 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10565636
-rw-r--r-- 1 neutrino002 neutrino 2468287419 Dec 30 00:03 detsim_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 2340197979 Dec 30 00:01 detsim_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 2179128911 Dec 30 00:59 reco_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 1917536454 Dec 29 22:25 g4_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 1832899525 Dec 29 22:16 g4_stage1.root
-rw-r--r-- 1 neutrino002 neutrino   80531240 Dec 30 00:56 Pandora_Events.pndr
-rw-r--r-- 1 neutrino002 neutrino     242956 Dec 30 00:59 jobscript.log
-rw-r--r-- 1 neutrino002 neutrino     217088 Dec 30 00:59 mem.db
-rw-r--r-- 1 neutrino002 neutrino      62376 Dec 30 00:59 Pandora_Geometry.xml
-rw-r--r-- 1 neutrino002 neutrino      36864 Dec 30 00:59 time.db
-rw-r--r-- 1 neutrino002 neutrino        519 Dec 30 00:03 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Dec 29 22:25 g4_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Dec 30 00:59 protoDUNE_reco1_hist.root
-rw-r--r-- 1 neutrino002 neutrino         52 Dec 29 22:08 all-input-dids.txt
-rw-r--r-- 1 neutrino002 neutrino          7 Dec 30 00:59 justin-processed-pfns.txt
-rw-r--r-- 1 neutrino002 neutrino          0 Dec 30 00:08 ProtoDUNE.log
-rw-r--r-- 1 neutrino002 neutrino          0 Dec 30 00:03 debugprod.log
justIN time: 2026-02-07 02:18:57 UTC       justIN version: 01.06.00