justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 280541.0@dunegpschedd01.fnal.gov

Jobsub ID280541.0@dunegpschedd01.fnal.gov
Workflow ID12042
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-18 06:05:29
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce02
Last heartbeat2026-01-18 10:24:09
From worker nodeHostnamecn104.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-18 06:06:26
Input filesmonte-carlo-012042-000001
JobscriptExit code0
Real time4h (15249s)
CPU time4h (15046s = 98%)
Max RSS bytes4483215360 (4275 MiB)
Outputting started2026-01-18 10:20:36
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12042/1/001/reco_stage1.root
Finished2026-01-18 10:24:09
Saved logsjustin-logs:280541.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

repare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 145563 traces, 0 summary
Retagger: tagging trace set: gauss with 108426 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 108426 traces tagged "gauss"
FrameSaver: q=3.97848e+06 n=1892426 tag=gauss
wclsFrameSaver: saving 145563 traces tagged "wiener"
FrameSaver: q=4.23644e+06 n=1671014 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 36954 XUs and 34210 XVs -> 733 XUVs
C:0 T:1 73723 XUs and 64892 XVs -> 12959 XUVs
C:0 T:2 107552 XUs and 109607 XVs -> 15366 XUVs
C:0 T:3 39620 XUs and 38311 XVs -> 1362 XUVs
C:0 T:4 13309 XUs and 11891 XVs -> 135 XUVs
C:0 T:5 46521 XUs and 35866 XVs -> 4308 XUVs
C:0 T:6 135063 XUs and 132795 XVs -> 15482 XUVs
C:0 T:7 37491 XUs and 40238 XVs -> 1071 XUVs
C:0 T:8 15203 XUs and 14822 XVs -> 405 XUVs
C:0 T:9 153683 XUs and 182052 XVs -> 20401 XUVs
C:0 T:10 41773 XUs and 31717 XVs -> 9629 XUVs
C:0 T:11 211 XUs and 134 XVs -> 39 XUVs
81890 XUVs total
12629 collection wire objects
73082 potential space points
Neighbour search...
12613050 tests to find 5850656 neighbours
Iterating with no regularization...
Begin: 7.97051e+07
0 6.55238e+07
1 6.44889e+07
2 6.43135e+07
3 6.42576e+07
Now with regularization...
Begin: 5.21509e+07
0 5.2056e+07
1 5.19878e+07
2 5.19345e+07
3 5.18904e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411811301
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411811301
fLifetime: 30000 [us]
18-Jan-2026 10:20:33 GMT  Closed output file "reco_stage1.root"
18-Jan-2026 10:20:33 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          430.149       626.851       881.654       598.443       136.947        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00149053    0.00479367     0.0121885    0.00444213    0.00277315       10     
reco:ophit:OpHitFinder                             0.234253      0.332406       0.47073      0.301147      0.0797897       10     
reco:opflash:OpFlashFinder                         0.190883      0.270487      0.355775      0.261286      0.0519102       10     
reco:opslicer:OpSlicer                             0.510996      0.855115       1.39172      0.722998      0.312807        10     
reco:caldata:DataPrepModule                         5.12053       6.41618       9.33504       5.6594        1.52524        10     
reco:wclsdatasp:WireCellToolkit                     45.513        51.8936       72.5377       49.1933       7.43029        10     
reco:gaushit:GausHitFinder                          1.03345       1.63236       2.21214       1.58573       0.36966        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000704747   0.000930129   0.00114518    0.00092914    0.000147801      10     
reco:reco3d:SpacePointSolver                        9.44773       20.4549       33.1827       18.3171       7.00989        10     
reco:hitpdune:DisambigFromSpacePoints               1.58754       2.71971       4.14808       2.63122      0.777521        10     
reco:pandora:StandardPandora                        22.241        58.5098       124.698       47.0591       28.819         10     
reco:pandoraWriter:StandardPandora                  0.39918      0.556647      0.681197      0.553921      0.0881239       10     
reco:pandoraTrack:LArPandoraTrackCreation           11.2274       14.2398       19.6926       13.8631       2.84346        10     
reco:pandoraShower:LArPandoraShowerCreation         9.32921       18.4879       27.4288       17.7524       5.14902        10     
reco:pandoracalo:Calorimetry                        4.74063       6.49631       8.91767       6.39017       1.34422        10     
reco:pandoracalonosce:Calorimetry                   4.57975       6.27474       8.95592       6.07927       1.38286        10     
reco:pandorapid:Chi2ParticleID                    0.00576547    0.00669986    0.00782812    0.00646304    0.000784827      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0553426     0.071097      0.0884046     0.0724602    0.00932104       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00548712    0.00628704    0.00736235    0.00629882    0.000631767      10     
reco:pandoraShowercalo:ShowerCalorimetry            8.22586       17.2959       26.0108       16.6996       5.01722        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       7.8671        16.7363       24.8397       16.2657       4.92521        10     
reco:emtrkmichelid:EmTrackMichelId                  163.384       245.146       363.186       232.767       56.8568        10     
reco:crttag:SingleCRTMatchingProducer              0.0260203     0.0360656     0.0489385     0.0342297    0.00679932       10     
reco:crtreco:TwoCRTMatchingProducer                0.024025      0.0321371     0.039732      0.0307457    0.00474026       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.18699       1.54945       2.19243       1.52041       0.29511        10     
reco:pandora2Track:LArPandoraTrackCreation          32.308        53.5814       71.0041       50.7487       13.2528        10     
reco:pandora2calo:Calorimetry                       14.0068       23.5965        30.69        22.3772       5.86612        10     
reco:pandora2calonosce:Calorimetry                  13.2603       22.2563       29.5746       21.1097       5.51543        10     
reco:pandora2pid:Chi2ParticleID                    0.0119676     0.0181774     0.0256791     0.0167228     0.0043089       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0695924     0.0998053     0.139367      0.0944484     0.0228298       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0119916     0.0176843     0.0266838     0.016236     0.00433238       10     
reco:pandora2Shower:LArPandoraShowerCreation        10.7911       16.5593       26.2195       15.3365       4.71696        10     
reco:pandora2Showercalo:ShowerCalorimetry           11.1933       19.7723       37.3978       17.2779       7.44623        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      9.86568       17.202        32.3866       14.846        6.75197        10     
[art]:TriggerResults:TriggerResultInserter         4.455e-05    6.67359e-05   0.000168549   5.3932e-05    3.57501e-05      10     
end_path:out1:RootOutput                           5.839e-06    7.8258e-06    1.3788e-05    7.0505e-06    2.11103e-06      10     
end_path:out1:RootOutput(write)                     2.32471       3.52754       6.08572       3.16676       1.14904        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5289.47 MB
  Peak resident set size usage (VmHWM): 4483.22 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.25 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.02 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11488916
-rw-r--r-- 1 pildune22 pildune 2655935306 Jan 18 08:33 detsim_stage2.root
-rw-r--r-- 1 pildune22 pildune 2518594611 Jan 18 08:30 detsim_stage1.root
-rw-r--r-- 1 pildune22 pildune 2399364000 Jan 18 10:20 reco_stage1.root
-rw-r--r-- 1 pildune22 pildune 2100742126 Jan 18 06:34 g4_stage2.root
-rw-r--r-- 1 pildune22 pildune 2001062285 Jan 18 06:13 g4_stage1.root
-rw-r--r-- 1 pildune22 pildune   88337720 Jan 18 10:13 Pandora_Events.pndr
-rw-r--r-- 1 pildune22 pildune     243071 Jan 18 10:20 jobscript.log
-rw-r--r-- 1 pildune22 pildune     217088 Jan 18 10:20 mem.db
-rw-r--r-- 1 pildune22 pildune      62376 Jan 18 10:20 Pandora_Geometry.xml
-rw-r--r-- 1 pildune22 pildune      36864 Jan 18 10:20 time.db
-rw-r--r-- 1 pildune22 pildune        519 Jan 18 08:33 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Jan 18 06:34 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Jan 18 10:20 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune22 pildune         52 Jan 18 06:06 all-input-dids.txt
-rw-r--r-- 1 pildune22 pildune          7 Jan 18 10:20 justin-processed-pfns.txt
-rw-r--r-- 1 pildune22 pildune          0 Jan 18 08:43 ProtoDUNE.log
-rw-r--r-- 1 pildune22 pildune          0 Jan 18 08:33 debugprod.log
justIN time: 2026-02-05 19:48:07 UTC       justIN version: 01.06.00