justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 271135.0@dunegpschedd01.fnal.gov

Jobsub ID271135.0@dunegpschedd01.fnal.gov
Workflow ID11446
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-31 23:20:28
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2026-01-01 05:04:17
From worker nodeHostnamecn519.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-31 23:21:35
Input filesmonte-carlo-011446-000001
JobscriptExit code0
Real time5h (20352s)
CPU time5h (20094s = 98%)
Max RSS bytes4186755072 (3992 MiB)
Outputting started2026-01-01 05:00:48
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11446/1/001/reco_stage1.root
Finished2026-01-01 05:04:17
Saved logsjustin-logs:271135.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

KeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 141132 traces, 0 summary
Retagger: tagging trace set: gauss with 106275 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 106275 traces tagged "gauss"
FrameSaver: q=3.61068e+06 n=1850130 tag=gauss
wclsFrameSaver: saving 141132 traces tagged "wiener"
FrameSaver: q=3.82559e+06 n=1622320 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 36965 XUs and 35152 XVs -> 548 XUVs
C:0 T:1 69682 XUs and 57792 XVs -> 6479 XUVs
C:0 T:2 140380 XUs and 153311 XVs -> 6826 XUVs
C:0 T:3 64907 XUs and 58197 XVs -> 771 XUVs
C:0 T:4 11044 XUs and 12760 XVs -> 142 XUVs
C:0 T:5 63273 XUs and 82852 XVs -> 16570 XUVs
C:0 T:6 57679 XUs and 72803 XVs -> 2516 XUVs
C:0 T:7 23034 XUs and 29329 XVs -> 162 XUVs
C:0 T:8 6226 XUs and 9305 XVs -> 181 XUVs
C:0 T:9 71236 XUs and 141683 XVs -> 20653 XUVs
C:0 T:10 37295 XUs and 32476 XVs -> 3540 XUVs
C:0 T:11 221 XUs and 178 XVs -> 26 XUVs
58414 XUVs total
12676 collection wire objects
54172 potential space points
Neighbour search...
12959868 tests to find 5120024 neighbours
Iterating with no regularization...
Begin: 8.30888e+07
0 6.74599e+07
1 6.65657e+07
2 6.64305e+07
3 6.63929e+07
Now with regularization...
Begin: 5.59145e+07
0 5.5858e+07
1 5.5825e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411463686
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411463686
fLifetime: 30000 [us]
01-Jan-2026 05:00:45 GMT  Closed output file "reco_stage1.root"
01-Jan-2026 05:00:45 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          754.528       858.962       987.614       829.269       73.8919        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00168511    0.00450826    0.00707225    0.00484449     0.0014801       10     
reco:ophit:OpHitFinder                             0.442543      0.514259      0.602661      0.515506      0.041984        10     
reco:opflash:OpFlashFinder                          0.26291      0.327022      0.368594      0.330827      0.0284866       10     
reco:opslicer:OpSlicer                             0.970372       1.25556       1.45655       1.2881       0.134158        10     
reco:caldata:DataPrepModule                         10.0824       10.8049       12.0103       10.5583      0.608199        10     
reco:wclsdatasp:WireCellToolkit                     71.699        80.1972       88.1007       79.8947       5.24651        10     
reco:gaushit:GausHitFinder                          2.33445       2.81377        3.33         2.79075      0.297398        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000775108   0.00108855    0.00149367    0.00102718    0.000214124      10     
reco:reco3d:SpacePointSolver                        22.7633       29.871        37.219        30.6046       4.48308        10     
reco:hitpdune:DisambigFromSpacePoints               3.75028       4.76387       6.79909       4.65515      0.830157        10     
reco:pandora:StandardPandora                        57.0694       70.6093       89.4753       66.5118       10.1254        10     
reco:pandoraWriter:StandardPandora                 0.842268      0.926118       1.01899      0.913108      0.057713        10     
reco:pandoraTrack:LArPandoraTrackCreation           13.0901       15.6084       20.2499       14.5362       2.25179        10     
reco:pandoraShower:LArPandoraShowerCreation         12.1803       16.583        23.624        16.3381       2.90529        10     
reco:pandoracalo:Calorimetry                        6.41132        7.338        9.03408       6.91122      0.930617        10     
reco:pandoracalonosce:Calorimetry                   5.98603       6.96327       8.42475       6.58921      0.860361        10     
reco:pandorapid:Chi2ParticleID                     0.0109815     0.0125686     0.0144331     0.012218     0.00105491       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.101919      0.118288      0.129675      0.118413     0.00780345       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0108857     0.0121849     0.0139228     0.0118578    0.00100771       10     
reco:pandoraShowercalo:ShowerCalorimetry            12.7169       17.0892       23.1964       16.9878       2.68179        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       12.5584       16.6725       22.6311       16.415        2.58609        10     
reco:emtrkmichelid:EmTrackMichelId                  365.889       411.507       465.701       400.32        36.3551        10     
reco:crttag:SingleCRTMatchingProducer              0.0344838     0.0434585     0.0536388     0.0434138     0.0050474       10     
reco:crtreco:TwoCRTMatchingProducer                0.0279695     0.0360989     0.0431337     0.0352383    0.00431874       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.5166        2.21372       2.88495       2.17564      0.429755        10     
reco:pandora2Track:LArPandoraTrackCreation          42.156        52.9024       69.3875       52.1605       8.02536        10     
reco:pandora2calo:Calorimetry                       18.9641       24.3239       32.4024       23.0748       3.94641        10     
reco:pandora2calonosce:Calorimetry                  18.0099       23.2373       31.4531       21.9617       3.95282        10     
reco:pandora2pid:Chi2ParticleID                    0.0265693     0.0318653     0.0386698     0.0309473     0.0036186       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.141802      0.155549      0.180824      0.154021      0.0111064       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0246568     0.0304409     0.0362068     0.0301847    0.00300553       10     
reco:pandora2Shower:LArPandoraShowerCreation        13.5731       16.9037        22.08        16.3587       2.50978        10     
reco:pandora2Showercalo:ShowerCalorimetry           16.5426       20.8544       25.9765       20.2489       2.95504        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      14.4227       18.7203       22.1366       18.5411       2.59233        10     
[art]:TriggerResults:TriggerResultInserter        8.9385e-05    0.000127842   0.000269622   0.000110667   5.10916e-05      10     
end_path:out1:RootOutput                           9.73e-06     1.36784e-05   2.3792e-05    1.18525e-05   4.00594e-06      10     
end_path:out1:RootOutput(write)                     4.42299       5.30223       6.08398       5.47391      0.589891        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4998.09 MB
  Peak resident set size usage (VmHWM): 4186.76 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.41 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   6.84 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.37 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10141216
-rw-r--r-- 1 pildune32 pildune 2381022069 Jan  1 02:34 detsim_stage2.root
-rw-r--r-- 1 pildune32 pildune 2258760193 Jan  1 02:30 detsim_stage1.root
-rw-r--r-- 1 pildune32 pildune 2072808439 Jan  1 05:00 reco_stage1.root
-rw-r--r-- 1 pildune32 pildune 1836712065 Dec 31 23:57 g4_stage2.root
-rw-r--r-- 1 pildune32 pildune 1758732005 Dec 31 23:32 g4_stage1.root
-rw-r--r-- 1 pildune32 pildune   75953370 Jan  1 04:48 Pandora_Events.pndr
-rw-r--r-- 1 pildune32 pildune     242918 Jan  1 05:00 jobscript.log
-rw-r--r-- 1 pildune32 pildune     217088 Jan  1 05:00 mem.db
-rw-r--r-- 1 pildune32 pildune      62376 Jan  1 05:00 Pandora_Geometry.xml
-rw-r--r-- 1 pildune32 pildune      36864 Jan  1 05:00 time.db
-rw-r--r-- 1 pildune32 pildune        519 Jan  1 02:34 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Dec 31 23:57 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Jan  1 05:00 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune32 pildune         52 Dec 31 23:21 all-input-dids.txt
-rw-r--r-- 1 pildune32 pildune          7 Jan  1 05:00 justin-processed-pfns.txt
-rw-r--r-- 1 pildune32 pildune          0 Jan  1 02:46 ProtoDUNE.log
-rw-r--r-- 1 pildune32 pildune          0 Jan  1 02:34 debugprod.log
justIN time: 2026-02-06 23:33:07 UTC       justIN version: 01.06.00