justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 273627.1@dunegpschedd01.fnal.gov

Jobsub ID273627.1@dunegpschedd01.fnal.gov
Workflow ID11691
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-10 17:45:48
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2026-01-10 21:52:43
From worker nodeHostnamecn514.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-10 17:46:22
Input filesmonte-carlo-011691-000001
JobscriptExit code0
Real time4h (14530s)
CPU time3h (14328s = 98%)
Max RSS bytes4328407040 (4127 MiB)
Outputting started2026-01-10 21:48:33
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11691/1/001/reco_stage1.root
Finished2026-01-10 21:52:43
Saved logsjustin-logs:273627.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 113806 traces, 0 summary
Retagger: tagging trace set: gauss with 87949 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 87949 traces tagged "gauss"
FrameSaver: q=3.11305e+06 n=1510387 tag=gauss
wclsFrameSaver: saving 113806 traces tagged "wiener"
FrameSaver: q=3.2705e+06 n=1329077 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 20630 XUs and 16775 XVs -> 193 XUVs
C:0 T:1 28991 XUs and 24918 XVs -> 1451 XUVs
C:0 T:2 76489 XUs and 86936 XVs -> 5382 XUVs
C:0 T:3 35047 XUs and 35393 XVs -> 352 XUVs
C:0 T:4 14768 XUs and 12287 XVs -> 106 XUVs
C:0 T:5 47864 XUs and 34969 XVs -> 3580 XUVs
C:0 T:6 69168 XUs and 52149 XVs -> 2190 XUVs
C:0 T:7 32363 XUs and 22220 XVs -> 226 XUVs
C:0 T:8 9074 XUs and 10009 XVs -> 137 XUVs
C:0 T:9 54632 XUs and 74373 XVs -> 4657 XUVs
C:0 T:10 18790 XUs and 11612 XVs -> 2429 XUVs
C:0 T:11 206 XUs and 143 XVs -> 26 XUVs
20729 XUVs total
9266 collection wire objects
19296 potential space points
Neighbour search...
789996 tests to find 419856 neighbours
Iterating with no regularization...
Begin: 6.6529e+07
0 5.83268e+07
1 5.77367e+07
2 5.76605e+07
3 5.76398e+07
Now with regularization...
Begin: 4.81716e+07
0 4.81533e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411660112
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411660112
fLifetime: 30000 [us]
10-Jan-2026 21:48:29 GMT  Closed output file "reco_stage1.root"
10-Jan-2026 21:48:29 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          498.533       659.684       972.262       625.137       143.339        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00145064    0.00460479    0.00745621    0.00419179    0.00214385       10     
reco:ophit:OpHitFinder                             0.425747      0.564769      0.673469      0.594177      0.0894032       10     
reco:opflash:OpFlashFinder                         0.263233      0.355728      0.410879      0.374249      0.0493163       10     
reco:opslicer:OpSlicer                             0.860795       1.17801       1.45776       1.17847      0.181916        10     
reco:caldata:DataPrepModule                         7.02571       9.97418       11.5921       11.0204       1.76538        10     
reco:wclsdatasp:WireCellToolkit                     53.7924        67.35        82.8259       66.173        8.19681        10     
reco:gaushit:GausHitFinder                          1.51668       2.08368       2.77812       2.10229      0.391239        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000678738   0.000886395   0.00113306    0.000827458   0.000165972      10     
reco:reco3d:SpacePointSolver                        15.4262       21.9177       29.9626       21.4356       5.17971        10     
reco:hitpdune:DisambigFromSpacePoints               1.8502        3.10094       4.20648       3.12584      0.780699        10     
reco:pandora:StandardPandora                        26.5278       55.0436       86.5909       50.0851       17.8183        10     
reco:pandoraWriter:StandardPandora                 0.462465      0.625427      0.884395      0.589154      0.129709        10     
reco:pandoraTrack:LArPandoraTrackCreation           6.8434        11.7979       20.0764       10.7053       3.87578        10     
reco:pandoraShower:LArPandoraShowerCreation         6.49252       11.9051       18.5346       11.8273       4.00435        10     
reco:pandoracalo:Calorimetry                        2.74071       5.45014       9.60896       4.94518       1.88116        10     
reco:pandoracalonosce:Calorimetry                   2.61977       4.93445       8.15537        4.485        1.63746        10     
reco:pandorapid:Chi2ParticleID                    0.00605311     0.0089341     0.0126185    0.00875791    0.00230091       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0574096     0.0842666     0.109719      0.0847533     0.0160919       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00565268    0.00864412     0.0125644    0.00805107    0.00224254       10     
reco:pandoraShowercalo:ShowerCalorimetry            5.18874       10.8066       18.1648       10.909        4.0051         10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.70822       10.3161       17.8175       9.77569       3.87901        10     
reco:emtrkmichelid:EmTrackMichelId                  213.439       314.52        487.01        308.451       76.914         10     
reco:crttag:SingleCRTMatchingProducer              0.0205812     0.0388577     0.0577853     0.0383551     0.0109233       10     
reco:crtreco:TwoCRTMatchingProducer                0.0170033     0.0266857     0.0364601     0.0273658    0.00512639       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.43565       1.95004       2.58149       1.94883       0.40298        10     
reco:pandora2Track:LArPandoraTrackCreation          28.6523       40.9502       69.7126       36.4758       13.1234        10     
reco:pandora2calo:Calorimetry                       12.2385       18.4518       30.934        17.571        5.93153        10     
reco:pandora2calonosce:Calorimetry                  10.6415       18.026        28.7411       16.8706       5.3869         10     
reco:pandora2pid:Chi2ParticleID                    0.017734      0.0271813     0.035858      0.0262081    0.00567319       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0953935      0.13627      0.185867      0.128017      0.0254526       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0182947     0.0274479     0.0365398     0.0268518     0.0057931       10     
reco:pandora2Shower:LArPandoraShowerCreation        8.89037       13.9776       19.818        12.348        3.7921         10     
reco:pandora2Showercalo:ShowerCalorimetry           8.89821       15.483        25.0311       14.3688       5.18815        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      7.86212       14.0262       24.686        11.9768       5.24264        10     
[art]:TriggerResults:TriggerResultInserter         5.513e-05    8.12836e-05   0.000133959   7.69685e-05   2.11113e-05      10     
end_path:out1:RootOutput                           6.912e-06    1.04125e-05   1.4042e-05    1.08645e-05   2.20607e-06      10     
end_path:out1:RootOutput(write)                     2.96833       4.37385       6.87622       4.23373       1.06186        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5139.48 MB
  Peak resident set size usage (VmHWM): 4328.41 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.37 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   6.03 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.34 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10417096
-rw-r--r-- 1 pildune32 pildune 2445594664 Jan 10 19:55 detsim_stage2.root
-rw-r--r-- 1 pildune32 pildune 2318467434 Jan 10 19:52 detsim_stage1.root
-rw-r--r-- 1 pildune32 pildune 2124888820 Jan 10 21:48 reco_stage1.root
-rw-r--r-- 1 pildune32 pildune 1890372751 Jan 10 18:10 g4_stage2.root
-rw-r--r-- 1 pildune32 pildune 1813742470 Jan 10 17:53 g4_stage1.root
-rw-r--r-- 1 pildune32 pildune   73419520 Jan 10 21:41 Pandora_Events.pndr
-rw-r--r-- 1 pildune32 pildune     242851 Jan 10 21:48 jobscript.log
-rw-r--r-- 1 pildune32 pildune     217088 Jan 10 21:48 mem.db
-rw-r--r-- 1 pildune32 pildune      62376 Jan 10 21:48 Pandora_Geometry.xml
-rw-r--r-- 1 pildune32 pildune      36864 Jan 10 21:48 time.db
-rw-r--r-- 1 pildune32 pildune        519 Jan 10 19:55 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Jan 10 18:10 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Jan 10 21:48 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune32 pildune         52 Jan 10 17:46 all-input-dids.txt
-rw-r--r-- 1 pildune32 pildune          7 Jan 10 21:48 justin-processed-pfns.txt
-rw-r--r-- 1 pildune32 pildune          0 Jan 10 20:04 ProtoDUNE.log
-rw-r--r-- 1 pildune32 pildune          0 Jan 10 19:55 debugprod.log
justIN time: 2026-02-06 11:40:52 UTC       justIN version: 01.06.00