justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 274095.0@dunegpschedd01.fnal.gov

Jobsub ID274095.0@dunegpschedd01.fnal.gov
Workflow ID11713
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-11 05:48:26
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce02
Last heartbeat2026-01-11 09:21:01
From worker nodeHostnamecn547.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-11 05:49:55
Input filesmonte-carlo-011713-000001
JobscriptExit code0
Real time3h (12450s)
CPU time3h (12267s = 98%)
Max RSS bytes4296896512 (4097 MiB)
Outputting started2026-01-11 09:17:26
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11713/1/001/reco_stage1.root
Finished2026-01-11 09:21:01
Saved logsjustin-logs:274095.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

repService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 107733 traces, 0 summary
Retagger: tagging trace set: gauss with 88865 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 88865 traces tagged "gauss"
FrameSaver: q=3.03198e+06 n=1683083 tag=gauss
wclsFrameSaver: saving 107733 traces tagged "wiener"
FrameSaver: q=3.17459e+06 n=1529114 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 21008 XUs and 23844 XVs -> 128 XUVs
C:0 T:1 32036 XUs and 32086 XVs -> 1737 XUVs
C:0 T:2 149816 XUs and 262689 XVs -> 11710 XUVs
C:0 T:3 63946 XUs and 80493 XVs -> 765 XUVs
C:0 T:4 9204 XUs and 7921 XVs -> 46 XUVs
C:0 T:5 25880 XUs and 16965 XVs -> 1784 XUVs
C:0 T:6 123879 XUs and 108718 XVs -> 5184 XUVs
C:0 T:7 46607 XUs and 42007 XVs -> 381 XUVs
C:0 T:8 5872 XUs and 4693 XVs -> 84 XUVs
C:0 T:9 22966 XUs and 26401 XVs -> 1396 XUVs
C:0 T:10 51137 XUs and 16502 XVs -> 3636 XUVs
C:0 T:11 180 XUs and 116 XVs -> 17 XUVs
26868 XUVs total
10130 collection wire objects
24186 potential space points
Neighbour search...
1412246 tests to find 694962 neighbours
Iterating with no regularization...
Begin: 7.45719e+07
0 5.66959e+07
1 5.59337e+07
2 5.58448e+07
3 5.58234e+07
Now with regularization...
Begin: 4.80675e+07
0 4.80551e+07
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
run: 1 ; subrun: 0 ; event: 10
evttime: 411670204
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411670204
fLifetime: 30000 [us]
11-Jan-2026 09:17:23 GMT  Closed output file "reco_stage1.root"
11-Jan-2026 09:17:23 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          373.224       493.199       672.129       443.094       105.64         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00145101    0.00380675    0.00536146    0.00417419    0.00134555       10     
reco:ophit:OpHitFinder                             0.253795      0.354422      0.556189      0.326327      0.0879773       10     
reco:opflash:OpFlashFinder                         0.173779      0.266487      0.374505       0.25464      0.0527812       10     
reco:opslicer:OpSlicer                             0.481724      0.865728       1.41742       0.80863      0.264897        10     
reco:caldata:DataPrepModule                         6.27743       7.2965        10.793        6.44617       1.71933        10     
reco:wclsdatasp:WireCellToolkit                     51.608        58.5823       70.0341       57.3882       4.9704         10     
reco:gaushit:GausHitFinder                          1.26264       1.82734       3.00865       1.68392       0.48235        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000596458   0.000792678   0.00126731    0.000761927   0.000194357      10     
reco:reco3d:SpacePointSolver                        9.59401       16.6623       26.4411       15.6051       5.72371        10     
reco:hitpdune:DisambigFromSpacePoints               1.65831       2.6948        4.42537       2.32409      0.889809        10     
reco:pandora:StandardPandora                        23.1503       47.1473       73.1858       43.4622       16.9517        10     
reco:pandoraWriter:StandardPandora                 0.420732      0.521866      0.645173      0.491971      0.0783956       10     
reco:pandoraTrack:LArPandoraTrackCreation           4.72623       9.22834       15.3757       9.44445       2.79996        10     
reco:pandoraShower:LArPandoraShowerCreation         5.00761       9.91355       17.9316       9.59568       3.8093         10     
reco:pandoracalo:Calorimetry                        2.03066       4.18768       7.28475        4.253        1.42129        10     
reco:pandoracalonosce:Calorimetry                   1.71443       3.90868       6.49167       4.04405       1.29322        10     
reco:pandorapid:Chi2ParticleID                     0.0046009    0.00672565    0.00847065    0.00675501    0.000994248      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0550906     0.071378      0.0920558     0.0681662     0.0121771       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0046106     0.0065084    0.00775457    0.00659875    0.000754299      10     
reco:pandoraShowercalo:ShowerCalorimetry            3.80943       9.25618       14.4098       8.98058       3.90418        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.83918       8.52172       13.9388       8.35923       3.26968        10     
reco:emtrkmichelid:EmTrackMichelId                  178.555       220.365       271.765       208.118       32.3117        10     
reco:crttag:SingleCRTMatchingProducer              0.0162483     0.0269005     0.038697      0.0253817    0.00647458       10     
reco:crtreco:TwoCRTMatchingProducer                0.0102958     0.0217707     0.038903      0.0208578    0.00728896       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.10612       1.58069       2.14506       1.54299      0.350763        10     
reco:pandora2Track:LArPandoraTrackCreation           16.61        33.1873       56.7124       29.457        13.8224        10     
reco:pandora2calo:Calorimetry                       6.2979        13.7212       23.9649       12.1887       6.16801        10     
reco:pandora2calonosce:Calorimetry                  5.84708       12.2578       23.3593       11.4476       5.27966        10     
reco:pandora2pid:Chi2ParticleID                    0.0127371     0.0178041     0.0310255     0.0161038     0.0050116       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0710308     0.0973452     0.156404      0.0925339     0.0248906       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.012798      0.017741      0.0305972     0.0160651    0.00484875       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.86908       8.52405       15.8174       7.77553       3.38313        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.73243       9.74872       17.7819       8.88318       3.94361        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.25565       8.73595       16.3316       7.92209       3.54934        10     
[art]:TriggerResults:TriggerResultInserter        4.2382e-05    6.48408e-05   0.000121214   5.37285e-05   2.43626e-05      10     
end_path:out1:RootOutput                           6.662e-06     8.576e-06    1.2856e-05    7.5965e-06    2.14169e-06      10     
end_path:out1:RootOutput(write)                     2.48006       3.42837       5.48541       3.25629      0.887636        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5101.7 MB
  Peak resident set size usage (VmHWM): 4296.9 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.27 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.63 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.25 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10313696
-rw-r--r-- 1 pildune22 pildune 2420863015 Jan 11 07:52 detsim_stage2.root
-rw-r--r-- 1 pildune22 pildune 2298605447 Jan 11 07:50 detsim_stage1.root
-rw-r--r-- 1 pildune22 pildune 2108046135 Jan 11 09:17 reco_stage1.root
-rw-r--r-- 1 pildune22 pildune 1867428608 Jan 11 06:13 g4_stage2.root
-rw-r--r-- 1 pildune22 pildune 1790130038 Jan 11 05:56 g4_stage1.root
-rw-r--r-- 1 pildune22 pildune   75471020 Jan 11 09:12 Pandora_Events.pndr
-rw-r--r-- 1 pildune22 pildune     242742 Jan 11 09:17 jobscript.log
-rw-r--r-- 1 pildune22 pildune     217088 Jan 11 09:17 mem.db
-rw-r--r-- 1 pildune22 pildune      62376 Jan 11 09:17 Pandora_Geometry.xml
-rw-r--r-- 1 pildune22 pildune      36864 Jan 11 09:17 time.db
-rw-r--r-- 1 pildune22 pildune        519 Jan 11 07:52 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Jan 11 06:13 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Jan 11 09:17 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune22 pildune         52 Jan 11 05:49 all-input-dids.txt
-rw-r--r-- 1 pildune22 pildune          7 Jan 11 09:17 justin-processed-pfns.txt
-rw-r--r-- 1 pildune22 pildune          0 Jan 11 08:00 ProtoDUNE.log
-rw-r--r-- 1 pildune22 pildune          0 Jan 11 07:52 debugprod.log
justIN time: 2026-02-06 11:42:53 UTC       justIN version: 01.06.00