justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 272043.1@dunegpschedd02.fnal.gov

Jobsub ID272043.1@dunegpschedd02.fnal.gov
Workflow ID11900
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-14 14:28:39
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2026-01-14 18:56:36
From worker nodeHostnamecn501.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-14 14:29:34
Input filesmonte-carlo-011900-000001
JobscriptExit code0
Real time4h (15808s)
CPU time4h (15564s = 98%)
Max RSS bytes4528857088 (4319 MiB)
Outputting started2026-01-14 18:53:03
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11900/1/001/reco_stage1.root
Finished2026-01-14 18:56:36
Saved logsjustin-logs:272043.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

lBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 139057 traces, 0 summary
Retagger: tagging trace set: gauss with 102860 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 102860 traces tagged "gauss"
FrameSaver: q=3.57394e+06 n=1765126 tag=gauss
wclsFrameSaver: saving 139057 traces tagged "wiener"
FrameSaver: q=3.81844e+06 n=1549629 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 32614 XUs and 43896 XVs -> 483 XUVs
C:0 T:1 72976 XUs and 102958 XVs -> 8169 XUVs
C:0 T:2 36386 XUs and 62734 XVs -> 2207 XUVs
C:0 T:3 17583 XUs and 28096 XVs -> 97 XUVs
C:0 T:4 10316 XUs and 10985 XVs -> 113 XUVs
C:0 T:5 32845 XUs and 48110 XVs -> 6716 XUVs
C:0 T:6 198166 XUs and 190720 XVs -> 11409 XUVs
C:0 T:7 65876 XUs and 61405 XVs -> 966 XUVs
C:0 T:8 8736 XUs and 4798 XVs -> 76 XUVs
C:0 T:9 113884 XUs and 55945 XVs -> 14375 XUVs
C:0 T:10 126482 XUs and 43479 XVs -> 9877 XUVs
C:0 T:11 46 XUs and 38 XVs -> 7 XUVs
54495 XUVs total
12311 collection wire objects
49866 potential space points
Neighbour search...
8375592 tests to find 3698228 neighbours
Iterating with no regularization...
Begin: 6.67261e+07
0 4.85812e+07
1 4.7472e+07
2 4.72806e+07
3 4.72161e+07
4 4.71865e+07
Now with regularization...
Begin: 3.78998e+07
0 3.78453e+07
1 3.78105e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411737785
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411737785
fLifetime: 30000 [us]
14-Jan-2026 18:52:59 GMT  Closed output file "reco_stage1.root"
14-Jan-2026 18:52:59 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          444.257       651.99        925.54        655.677       142.885        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00170965    0.00405798    0.00642037    0.00427501    0.00141892       10     
reco:ophit:OpHitFinder                             0.288619      0.390084      0.608192      0.375633      0.0898425       10     
reco:opflash:OpFlashFinder                         0.190083       0.32871      0.507198      0.319201      0.0856642       10     
reco:opslicer:OpSlicer                              0.66295      0.963479       1.60498      0.902751      0.275326        10     
reco:caldata:DataPrepModule                         6.16503       7.49803       11.0362       7.07012       1.5037         10     
reco:wclsdatasp:WireCellToolkit                     53.0617       60.6925       76.626        59.7713       7.45886        10     
reco:gaushit:GausHitFinder                          1.39274       2.03389       3.4387        1.79726      0.632781        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000696155   0.00096836    0.00175554    0.000903252   0.000299929      10     
reco:reco3d:SpacePointSolver                        13.8953       27.3774       54.5129       23.1273       12.2781        10     
reco:hitpdune:DisambigFromSpacePoints               2.42316       3.97952       8.59951       3.50263       1.77346        10     
reco:pandora:StandardPandora                        34.4855       76.8564       185.282       64.5596       43.8178        10     
reco:pandoraWriter:StandardPandora                 0.518171      0.761559       1.04275      0.757933      0.176097        10     
reco:pandoraTrack:LArPandoraTrackCreation           8.36252       13.6281       18.7666       13.4134       3.12509        10     
reco:pandoraShower:LArPandoraShowerCreation         8.9948        15.5617       22.3341       15.9825       4.06373        10     
reco:pandoracalo:Calorimetry                        3.39698       6.01688       8.14289       6.24715       1.24464        10     
reco:pandoracalonosce:Calorimetry                   3.2329        5.82288       8.36076       6.10812       1.3113         10     
reco:pandorapid:Chi2ParticleID                     0.0059845    0.00840635     0.0111904    0.00824064    0.00159324       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0709917     0.091547      0.112549       0.09318      0.0133045       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00573859     0.0085883     0.011682     0.00850141    0.00167133       10     
reco:pandoraShowercalo:ShowerCalorimetry            8.05495       13.9207       20.9042       14.6612       3.99112        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       7.59948       13.2992       19.0904       13.9568       3.70064        10     
reco:emtrkmichelid:EmTrackMichelId                  210.283       281.997       378.371       278.947       48.2413        10     
reco:crttag:SingleCRTMatchingProducer              0.0243443     0.0347483     0.0473925     0.035536     0.00737928       10     
reco:crtreco:TwoCRTMatchingProducer                0.0157466     0.0270859     0.039431      0.0272378    0.00679965       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.09837       1.8243        3.32831       1.80734      0.588074        10     
reco:pandora2Track:LArPandoraTrackCreation          23.6402       42.715        73.6118       43.8144       14.3001        10     
reco:pandora2calo:Calorimetry                       11.3686        18.5         34.6085       17.9842       6.55024        10     
reco:pandora2calonosce:Calorimetry                  10.4418       16.7187       25.9062       17.1965       4.99399        10     
reco:pandora2pid:Chi2ParticleID                    0.0144844     0.0196627     0.0274359     0.0200894    0.00361444       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0856293     0.108305      0.148878      0.109316      0.0187036       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0142271     0.0191759     0.0267619     0.0193937    0.00353726       10     
reco:pandora2Shower:LArPandoraShowerCreation        6.67522       11.4343       18.4476       11.1942       3.64865        10     
reco:pandora2Showercalo:ShowerCalorimetry           8.72767       13.7378       19.0491       14.2642       3.64527        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      6.88593       11.7137       18.0445       11.8307       3.59828        10     
[art]:TriggerResults:TriggerResultInserter        4.7752e-05    7.19135e-05   0.000135202   6.28125e-05   2.62321e-05      10     
end_path:out1:RootOutput                           7.763e-06    9.6307e-06    1.5137e-05     8.396e-06    2.4099e-06       10     
end_path:out1:RootOutput(write)                     2.92985       3.70364       4.98262       3.79907      0.663865        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5338.18 MB
  Peak resident set size usage (VmHWM): 4528.86 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.28 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.75 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.26 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11158388
-rw-r--r-- 1 pildune32 pildune 2589629378 Jan 14 17:01 detsim_stage2.root
-rw-r--r-- 1 pildune32 pildune 2453785365 Jan 14 16:58 detsim_stage1.root
-rw-r--r-- 1 pildune32 pildune 2322517583 Jan 14 18:52 reco_stage1.root
-rw-r--r-- 1 pildune32 pildune 2031922242 Jan 14 14:59 g4_stage2.root
-rw-r--r-- 1 pildune32 pildune 1941540153 Jan 14 14:37 g4_stage1.root
-rw-r--r-- 1 pildune32 pildune   86185020 Jan 14 18:47 Pandora_Events.pndr
-rw-r--r-- 1 pildune32 pildune     242884 Jan 14 18:53 jobscript.log
-rw-r--r-- 1 pildune32 pildune     217088 Jan 14 18:53 mem.db
-rw-r--r-- 1 pildune32 pildune      62376 Jan 14 18:53 Pandora_Geometry.xml
-rw-r--r-- 1 pildune32 pildune      36864 Jan 14 18:52 time.db
-rw-r--r-- 1 pildune32 pildune        519 Jan 14 17:01 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Jan 14 14:59 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Jan 14 18:53 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune32 pildune         52 Jan 14 14:29 all-input-dids.txt
-rw-r--r-- 1 pildune32 pildune          7 Jan 14 18:53 justin-processed-pfns.txt
-rw-r--r-- 1 pildune32 pildune          0 Jan 14 17:11 ProtoDUNE.log
-rw-r--r-- 1 pildune32 pildune          0 Jan 14 17:01 debugprod.log
justIN time: 2026-02-06 03:23:58 UTC       justIN version: 01.06.00