justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 232178.0@dunegpschedd02.fnal.gov

Jobsub ID232178.0@dunegpschedd02.fnal.gov
Workflow ID9272
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-22 21:28:55
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce16-multicore
Last heartbeat2025-10-23 02:00:34
From worker nodeHostnametds500.pic.es
cpuinfoAMD EPYC 7513 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-22 21:30:04
Input filesmonte-carlo-009272-000001
JobscriptExit code0
Real time4h (16181s)
CPU time4h (15353s = 94%)
Max RSS bytes4402343936 (4198 MiB)
Outputting started2025-10-23 01:59:46
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09272/1/001/reco_stage1.root
Finished2025-10-23 02:00:34
Saved logsjustin-logs:232178.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

e:prepare:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 95822 traces, 0 summary
Retagger: tagging trace set: gauss with 78723 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 78723 traces tagged "gauss"
FrameSaver: q=2.57045e+06 n=1507024 tag=gauss
wclsFrameSaver: saving 95822 traces tagged "wiener"
FrameSaver: q=2.68843e+06 n=1375343 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 18077 XUs and 13134 XVs -> 220 XUVs
C:0 T:1 23597 XUs and 21054 XVs -> 1804 XUVs
C:0 T:2 79276 XUs and 111908 XVs -> 3510 XUVs
C:0 T:3 41171 XUs and 49019 XVs -> 364 XUVs
C:0 T:4 10780 XUs and 7954 XVs -> 57 XUVs
C:0 T:5 29544 XUs and 16254 XVs -> 2123 XUVs
C:0 T:6 134454 XUs and 174403 XVs -> 8986 XUVs
C:0 T:7 50605 XUs and 59458 XVs -> 723 XUVs
C:0 T:8 8314 XUs and 6668 XVs -> 69 XUVs
C:0 T:9 45645 XUs and 59426 XVs -> 3929 XUVs
C:0 T:10 22603 XUs and 10088 XVs -> 2293 XUVs
C:0 T:11 16 XUs and 10 XVs -> 1 XUVs
24079 XUVs total
8796 collection wire objects
21586 potential space points
Neighbour search...
1421548 tests to find 730170 neighbours
Iterating with no regularization...
Begin: 4.38469e+07
0 3.77145e+07
1 3.72975e+07
2 3.72267e+07
3 3.72048e+07
Now with regularization...
Begin: 3.13801e+07
0 3.13691e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410053904
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410053904
fLifetime: 30000 [us]
23-Oct-2025 03:59:44 CEST  Closed output file "reco_stage1.root"
23-Oct-2025 03:59:44 CEST  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          349.559       544.051       782.36        575.207       117.957        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00135892    0.00448509    0.00725988     0.0040027    0.00211125       10     
reco:ophit:OpHitFinder                             0.283474      0.428197      0.519702       0.43331      0.0621503       10     
reco:opflash:OpFlashFinder                         0.200343      0.264013      0.373143      0.249591      0.0495404       10     
reco:opslicer:OpSlicer                             0.635309       1.00365       1.33381       1.0196       0.187457        10     
reco:caldata:DataPrepModule                         5.72519       7.66168       8.26088       7.73472      0.707477        10     
reco:wclsdatasp:WireCellToolkit                     34.1031       41.5108       45.646        42.8733       3.74064        10     
reco:gaushit:GausHitFinder                         0.934629       1.92891       2.62438       2.17633      0.548621        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000371818   0.000763549   0.00104332    0.000805333   0.000222003      10     
reco:reco3d:SpacePointSolver                        14.606        29.2357       46.3322       31.2401       8.92283        10     
reco:hitpdune:DisambigFromSpacePoints               1.41698       2.97856       5.12325       3.03304       1.06877        10     
reco:pandora:StandardPandora                         26.33        63.1289       118.732       60.8584       25.6184        10     
reco:pandoraWriter:StandardPandora                 0.503199      0.752969       0.95936      0.785978      0.132091        10     
reco:pandoraTrack:LArPandoraTrackCreation           3.63494       7.73971       13.4884       8.14321       2.79883        10     
reco:pandoraShower:LArPandoraShowerCreation         3.18648       8.13031       16.6492       7.97816       3.44337        10     
reco:pandoracalo:Calorimetry                        1.77028       3.68507       5.81248       3.95434       1.21658        10     
reco:pandoracalonosce:Calorimetry                   1.96677       3.54902       5.13082       4.00562       1.03541        10     
reco:pandorapid:Chi2ParticleID                    0.00484226    0.00860787     0.0127379     0.0079277    0.00214156       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0531662     0.0882296     0.111502      0.0892507     0.0216219       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00515464    0.00820401     0.0102395    0.00832563    0.00152134       10     
reco:pandoraShowercalo:ShowerCalorimetry            3.59738       9.02608       17.1986       8.41501       3.53597        10     
reco:pandoraShowercalonosce:ShowerCalorimetry        3.285        7.91721       14.4052       7.48053       2.99526        10     
reco:emtrkmichelid:EmTrackMichelId                  198.932       273.544       357.651       282.409       44.1731        10     
reco:crttag:SingleCRTMatchingProducer              0.0137051     0.0227697     0.0297107     0.0259842    0.00562513       10     
reco:crtreco:TwoCRTMatchingProducer               0.00840287     0.0153282     0.0199896     0.0164665    0.00421022       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.668753       1.21058       1.77343       1.23164      0.302821        10     
reco:pandora2Track:LArPandoraTrackCreation          10.8632       23.2685       45.0447       22.4398       9.21574        10     
reco:pandora2calo:Calorimetry                       5.04465       12.7087        24.41        11.8452       4.9871         10     
reco:pandora2calonosce:Calorimetry                  4.81119       11.9255       20.043        11.1555       4.51931        10     
reco:pandora2pid:Chi2ParticleID                    0.012639      0.0212927     0.0367105     0.0211091    0.00662034       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0740105     0.113385      0.170406      0.113728      0.0261709       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0108763     0.0207337     0.0344114     0.0212228     0.0064155       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.60788       8.58879       13.7233       8.58933       2.64479        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.85967       10.3523       16.4653       10.1195       3.50295        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.83445       8.83682       14.0071       8.71696       2.35679        10     
[art]:TriggerResults:TriggerResultInserter        6.7206e-05    9.40104e-05   0.000159289   7.64135e-05   3.19182e-05      10     
end_path:out1:RootOutput                          1.1211e-05    1.3316e-05    1.8475e-05    1.2198e-05    2.33971e-06      10     
end_path:out1:RootOutput(write)                     2.65481       4.18779       6.15683       4.05502       0.91063        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5247.55 MB
  Peak resident set size usage (VmHWM): 4402.34 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.35 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.96 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.21 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11147216
-rw-r--r-- 1 dune001 dune 2582791409 Oct 23 02:26 detsim_stage2.root
-rw-r--r-- 1 dune001 dune 2447702965 Oct 23 02:24 detsim_stage1.root
-rw-r--r-- 1 dune001 dune 2330159480 Oct 23 03:59 reco_stage1.root
-rw-r--r-- 1 dune001 dune 2018338488 Oct 22 23:56 g4_stage2.root
-rw-r--r-- 1 dune001 dune 1945397483 Oct 22 23:40 g4_stage1.root
-rw-r--r-- 1 dune001 dune   89730540 Oct 23 03:55 Pandora_Events.pndr
-rw-r--r-- 1 dune001 dune     244297 Oct 23 03:59 jobscript.log
-rw-r--r-- 1 dune001 dune     217088 Oct 23 03:59 mem.db
-rw-r--r-- 1 dune001 dune      62376 Oct 23 03:59 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 dune      36864 Oct 23 03:59 time.db
-rw-r--r-- 1 dune001 dune        519 Oct 23 02:26 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Oct 22 23:56 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Oct 23 03:59 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 dune         52 Oct 22 23:30 all-input-dids.txt
-rw-r--r-- 1 dune001 dune          7 Oct 23 03:59 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune          0 Oct 23 02:35 ProtoDUNE.log
-rw-r--r-- 1 dune001 dune          0 Oct 23 02:26 debugprod.log
justIN time: 2025-11-05 04:54:36 UTC       justIN version: 01.05.01