justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 254429.0@dunegpschedd01.fnal.gov

Jobsub ID254429.0@dunegpschedd01.fnal.gov
Workflow ID10393
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-20 10:00:58
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2025-11-20 12:55:52
From worker nodeHostnamecn050.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-20 10:01:50
Input filesmonte-carlo-010393-000001
JobscriptExit code0
Real time2h (10249s)
CPU time2h (10087s = 98%)
Max RSS bytes4060807168 (3872 MiB)
Outputting started2025-11-20 12:52:40
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10393/1/001/reco_stage1.root
Finished2025-11-20 12:55:52
Saved logsjustin-logs:254429.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

e:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 104728 traces, 0 summary
Retagger: tagging trace set: gauss with 78623 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 78623 traces tagged "gauss"
FrameSaver: q=2.7571e+06 n=1363312 tag=gauss
wclsFrameSaver: saving 104728 traces tagged "wiener"
FrameSaver: q=2.91363e+06 n=1201721 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 19759 XUs and 24255 XVs -> 225 XUVs
C:0 T:1 33758 XUs and 38017 XVs -> 3085 XUVs
C:0 T:2 70590 XUs and 91936 XVs -> 7127 XUVs
C:0 T:3 29410 XUs and 33261 XVs -> 202 XUVs
C:0 T:4 8563 XUs and 7955 XVs -> 139 XUVs
C:0 T:5 23084 XUs and 15311 XVs -> 2062 XUVs
C:0 T:6 114303 XUs and 125767 XVs -> 6796 XUVs
C:0 T:7 42763 XUs and 44840 XVs -> 577 XUVs
C:0 T:8 6313 XUs and 5986 XVs -> 91 XUVs
C:0 T:9 27821 XUs and 33757 XVs -> 1934 XUVs
C:0 T:10 21688 XUs and 20135 XVs -> 2638 XUVs
C:0 T:11 184 XUs and 111 XVs -> 17 XUVs
24893 XUVs total
8474 collection wire objects
22527 potential space points
Neighbour search...
1789717 tests to find 906340 neighbours
Iterating with no regularization...
Begin: 4.52447e+07
0 3.8732e+07
1 3.82175e+07
2 3.81383e+07
3 3.81148e+07
Now with regularization...
Begin: 3.01631e+07
0 3.01462e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410627720
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410627720
fLifetime: 30000 [us]
20-Nov-2025 12:52:38 GMT  Closed output file "reco_stage1.root"
20-Nov-2025 12:52:38 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          298.332       436.49        567.558       439.303       71.1835        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00101458    0.00361022    0.00639186    0.00372991    0.00153667       10     
reco:ophit:OpHitFinder                             0.218021      0.247913      0.279983      0.240333      0.0216742       10     
reco:opflash:OpFlashFinder                         0.147273      0.182781      0.228292       0.17386      0.0275738       10     
reco:opslicer:OpSlicer                              0.47584      0.582321      0.675809      0.591525      0.0620715       10     
reco:caldata:DataPrepModule                         5.35895       5.48728       5.65519       5.48229      0.0847752       10     
reco:wclsdatasp:WireCellToolkit                     45.9861       48.2271       51.0241       48.1843       1.38137        10     
reco:gaushit:GausHitFinder                         0.936454       1.36184       1.83625       1.37708      0.233456        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000722536   0.000887492   0.00109499    0.000879049   0.000111608      10     
reco:reco3d:SpacePointSolver                        8.50921       12.6289       17.5403       12.5662       2.92677        10     
reco:hitpdune:DisambigFromSpacePoints               1.16912       1.88544       2.79867       1.81441      0.439166        10     
reco:pandora:StandardPandora                        22.7686       37.8865       70.9916       36.5991       13.4872        10     
reco:pandoraWriter:StandardPandora                  0.38268      0.483312      0.598492      0.464477       0.06214        10     
reco:pandoraTrack:LArPandoraTrackCreation           5.93319       10.6565       14.4221       10.9312       2.0915         10     
reco:pandoraShower:LArPandoraShowerCreation         5.14477       12.7821       18.8842       13.205        3.64887        10     
reco:pandoracalo:Calorimetry                        2.34763       4.75457       6.78639       4.78964       1.07924        10     
reco:pandoracalonosce:Calorimetry                   2.38755       4.60062       6.3782        4.6373       0.995279        10     
reco:pandorapid:Chi2ParticleID                    0.00443677    0.00572486    0.00687799    0.00569881    0.00069292       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.050116      0.0606397     0.0710617     0.0600646     0.0061945       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00435892    0.00546133    0.00676324    0.00542694    0.000577088      10     
reco:pandoraShowercalo:ShowerCalorimetry            3.83377       11.3933       16.6357       11.9606       3.37396        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.84178       10.8107       15.7107       11.3057       3.31021        10     
reco:emtrkmichelid:EmTrackMichelId                  138.532       173.973       217.109       168.74        22.265         10     
reco:crttag:SingleCRTMatchingProducer              0.0239964     0.0301638     0.0417885     0.0289424    0.00540276       10     
reco:crtreco:TwoCRTMatchingProducer                0.0171087     0.0245226     0.0390616     0.0229645     0.0058732       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.996764       1.24147       1.55156       1.20813      0.159717        10     
reco:pandora2Track:LArPandoraTrackCreation          17.0688       34.8909       45.3618       36.5023       8.10808        10     
reco:pandora2calo:Calorimetry                       6.61413       14.2901       18.6069       15.2817       3.32862        10     
reco:pandora2calonosce:Calorimetry                  6.63559       13.7478       18.8392       14.1584       3.30546        10     
reco:pandora2pid:Chi2ParticleID                   0.00918041     0.0130174     0.0156769     0.0131377    0.00182468       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.056994      0.0763944     0.0949854     0.0762794     0.0108166       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00924118     0.0128913     0.0154599     0.0131206    0.00165406       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.90408       10.046         12.31        10.6254       2.19545        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.77729       11.5902       14.8806       11.7584       2.7583         10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.38488       9.83135       13.1891       9.99093       2.14471        10     
[art]:TriggerResults:TriggerResultInserter        3.9666e-05    5.73855e-05   0.000118912   5.1304e-05    2.12411e-05      10     
end_path:out1:RootOutput                           7.24e-06     9.2417e-06    1.4875e-05    8.6905e-06    1.99675e-06      10     
end_path:out1:RootOutput(write)                     1.88487       2.49347       3.17434       2.4106       0.337248        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4866.78 MB
  Peak resident set size usage (VmHWM): 4060.81 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.21 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.37 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.19 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9568628
-rw-r--r-- 1 pildune32 pildune 2277430027 Nov 20 11:38 detsim_stage2.root
-rw-r--r-- 1 pildune32 pildune 2161684517 Nov 20 11:36 detsim_stage1.root
-rw-r--r-- 1 pildune32 pildune 1953025674 Nov 20 12:52 reco_stage1.root
-rw-r--r-- 1 pildune32 pildune 1712381984 Nov 20 10:19 g4_stage2.root
-rw-r--r-- 1 pildune32 pildune 1620460528 Nov 20 10:07 g4_stage1.root
-rw-r--r-- 1 pildune32 pildune   72673060 Nov 20 12:49 Pandora_Events.pndr
-rw-r--r-- 1 pildune32 pildune     242951 Nov 20 12:52 jobscript.log
-rw-r--r-- 1 pildune32 pildune     217088 Nov 20 12:52 mem.db
-rw-r--r-- 1 pildune32 pildune      62376 Nov 20 12:52 Pandora_Geometry.xml
-rw-r--r-- 1 pildune32 pildune      36864 Nov 20 12:52 time.db
-rw-r--r-- 1 pildune32 pildune        519 Nov 20 11:38 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Nov 20 10:19 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Nov 20 12:52 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune32 pildune         52 Nov 20 10:01 all-input-dids.txt
-rw-r--r-- 1 pildune32 pildune          7 Nov 20 12:52 justin-processed-pfns.txt
-rw-r--r-- 1 pildune32 pildune          0 Nov 20 11:43 ProtoDUNE.log
-rw-r--r-- 1 pildune32 pildune          0 Nov 20 11:38 debugprod.log
justIN time: 2026-02-09 03:29:06 UTC       justIN version: 01.06.00