justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 271462.0@dunegpschedd01.fnal.gov

Jobsub ID271462.0@dunegpschedd01.fnal.gov
Workflow ID11496
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-03 20:22:11
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2026-01-03 23:10:27
From worker nodeHostnamecn001.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-03 20:24:03
Input filesmonte-carlo-011496-000001
JobscriptExit code0
Real time2h (9781s)
CPU time2h (9625s = 98%)
Max RSS bytes3985641472 (3801 MiB)
Outputting started2026-01-03 23:07:06
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11496/1/001/reco_stage1.root
Finished2026-01-03 23:10:27
Saved logsjustin-logs:271462.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 116864 traces, 0 summary
Retagger: tagging trace set: gauss with 89667 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 89667 traces tagged "gauss"
FrameSaver: q=3.0786e+06 n=1623540 tag=gauss
wclsFrameSaver: saving 116864 traces tagged "wiener"
FrameSaver: q=3.25505e+06 n=1442970 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 26695 XUs and 27130 XVs -> 213 XUVs
C:0 T:1 43266 XUs and 41760 XVs -> 2506 XUVs
C:0 T:2 64825 XUs and 115867 XVs -> 7626 XUVs
C:0 T:3 28069 XUs and 40683 XVs -> 350 XUVs
C:0 T:4 9408 XUs and 6619 XVs -> 80 XUVs
C:0 T:5 48308 XUs and 19025 XVs -> 5956 XUVs
C:0 T:6 156662 XUs and 135157 XVs -> 10445 XUVs
C:0 T:7 56419 XUs and 52200 XVs -> 962 XUVs
C:0 T:8 11626 XUs and 6757 XVs -> 108 XUVs
C:0 T:9 78863 XUs and 64505 XVs -> 5385 XUVs
C:0 T:10 56180 XUs and 31731 XVs -> 5461 XUVs
C:0 T:11 85 XUs and 62 XVs -> 16 XUVs
39108 XUVs total
10987 collection wire objects
34744 potential space points
Neighbour search...
3607262 tests to find 1802524 neighbours
Iterating with no regularization...
Begin: 6.52869e+07
0 5.28764e+07
1 5.21221e+07
2 5.20179e+07
3 5.1984e+07
Now with regularization...
Begin: 4.40488e+07
0 4.39797e+07
1 4.39516e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411520552
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411520552
fLifetime: 30000 [us]
03-Jan-2026 23:07:03 GMT  Closed output file "reco_stage1.root"
03-Jan-2026 23:07:03 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          275.334       404.037       508.767       420.288       64.2847        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00115647    0.00317258    0.00462126    0.00337541    0.00102718       10     
reco:ophit:OpHitFinder                             0.208885      0.247571      0.294472      0.247305      0.0244523       10     
reco:opflash:OpFlashFinder                         0.130591      0.185138      0.244987      0.176532      0.0370539       10     
reco:opslicer:OpSlicer                             0.468166      0.554107      0.684512      0.544916      0.0625081       10     
reco:caldata:DataPrepModule                         5.03812       5.28158       5.42181       5.28217      0.0989068       10     
reco:wclsdatasp:WireCellToolkit                     44.311        47.5807       49.1088       48.226        1.60749        10     
reco:gaushit:GausHitFinder                          1.03189       1.28696       1.53136       1.30248      0.140179        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000569546   0.00077913    0.000905568   0.000798836   8.93401e-05      10     
reco:reco3d:SpacePointSolver                        8.8505        12.0564       16.931        11.5745       2.24595        10     
reco:hitpdune:DisambigFromSpacePoints               1.17211       1.79339       2.29666       1.83279      0.329715        10     
reco:pandora:StandardPandora                        20.9548       31.817        42.4099       30.0889       6.79806        10     
reco:pandoraWriter:StandardPandora                 0.341688      0.453361      0.525571       0.45877      0.0552659       10     
reco:pandoraTrack:LArPandoraTrackCreation           5.86872       9.40559       11.7775       9.83592       1.89285        10     
reco:pandoraShower:LArPandoraShowerCreation         5.19981       11.5781       18.4654       12.6292       3.53632        10     
reco:pandoracalo:Calorimetry                        2.40373       4.21225       5.5705        4.36149       0.94864        10     
reco:pandoracalonosce:Calorimetry                   2.14692       3.91696       5.20945       4.00894      0.902867        10     
reco:pandorapid:Chi2ParticleID                    0.00434659    0.00522369    0.00586271    0.00514893    0.000436199      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0397351     0.0556057     0.0639197     0.0565423    0.00669226       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00428808     0.0050325    0.00585825    0.00503399    0.000401889      10     
reco:pandoraShowercalo:ShowerCalorimetry            3.79776       10.2057       16.4212       10.7393       3.44311        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.56482       9.85163       15.6594       10.4063       3.2558         10     
reco:emtrkmichelid:EmTrackMichelId                  121.64        162.034       186.237       165.08        18.949         10     
reco:crttag:SingleCRTMatchingProducer              0.0183582     0.0267504     0.0383448     0.0261858    0.00585357       10     
reco:crtreco:TwoCRTMatchingProducer                0.0154349     0.0235016     0.0345069     0.0229203    0.00613249       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.864402       1.13726       1.73714       1.04628      0.244787        10     
reco:pandora2Track:LArPandoraTrackCreation          14.9787       31.6676       47.2179       34.0022       8.66161        10     
reco:pandora2calo:Calorimetry                       5.7093        13.3813       19.1793       14.2919       3.78606        10     
reco:pandora2calonosce:Calorimetry                  5.88131       12.8761       20.9576       13.3748       3.97864        10     
reco:pandora2pid:Chi2ParticleID                   0.00855242     0.012225      0.0151609     0.012517     0.00172055       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0477902     0.0707384     0.084389      0.0722948    0.00979287       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00884183     0.0121166     0.0150571     0.0122975    0.00161224       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.59908       9.69692       15.4834       9.48901       2.76352        10     
reco:pandora2Showercalo:ShowerCalorimetry           5.31582       10.995        16.8518       11.2093       3.2618         10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.49707       9.11566       14.2535       9.00825       2.85058        10     
[art]:TriggerResults:TriggerResultInserter        4.5009e-05    5.53096e-05   0.000107736   5.02775e-05   1.77828e-05      10     
end_path:out1:RootOutput                           5.52e-06     8.3298e-06    1.7145e-05     7.639e-06    3.01771e-06      10     
end_path:out1:RootOutput(write)                     1.76768       2.3445        2.82202       2.32151      0.287299        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4797.08 MB
  Peak resident set size usage (VmHWM): 3985.64 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.20 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.24 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.18 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9414500
-rw-r--r-- 1 pildune32 pildune 2239962851 Jan  3 21:58 detsim_stage2.root
-rw-r--r-- 1 pildune32 pildune 2124801988 Jan  3 21:56 detsim_stage1.root
-rw-r--r-- 1 pildune32 pildune 1907566522 Jan  3 23:07 reco_stage1.root
-rw-r--r-- 1 pildune32 pildune 1684070698 Jan  3 20:40 g4_stage2.root
-rw-r--r-- 1 pildune32 pildune 1612763903 Jan  3 20:29 g4_stage1.root
-rw-r--r-- 1 pildune32 pildune   70665560 Jan  3 23:01 Pandora_Events.pndr
-rw-r--r-- 1 pildune32 pildune     242846 Jan  3 23:07 jobscript.log
-rw-r--r-- 1 pildune32 pildune     217088 Jan  3 23:07 mem.db
-rw-r--r-- 1 pildune32 pildune      62376 Jan  3 23:07 Pandora_Geometry.xml
-rw-r--r-- 1 pildune32 pildune      36864 Jan  3 23:07 time.db
-rw-r--r-- 1 pildune32 pildune        519 Jan  3 21:58 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Jan  3 20:40 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Jan  3 23:07 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune32 pildune         52 Jan  3 20:24 all-input-dids.txt
-rw-r--r-- 1 pildune32 pildune          7 Jan  3 23:07 justin-processed-pfns.txt
-rw-r--r-- 1 pildune32 pildune          0 Jan  3 22:03 ProtoDUNE.log
-rw-r--r-- 1 pildune32 pildune          0 Jan  3 21:58 debugprod.log
justIN time: 2026-02-06 21:02:48 UTC       justIN version: 01.06.00