justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 272560.1@dunegpschedd02.fnal.gov

Jobsub ID272560.1@dunegpschedd02.fnal.gov
Workflow ID11929
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-14 22:13:04
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2026-01-15 02:15:42
From worker nodeHostnamecn064.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-14 22:14:38
Input filesmonte-carlo-011929-000001
JobscriptExit code0
Real time3h (14254s)
CPU time3h (14000s = 98%)
Max RSS bytes4459638784 (4253 MiB)
Outputting started2026-01-15 02:12:13
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11929/1/001/reco_stage1.root
Finished2026-01-15 02:15:42
Saved logsjustin-logs:272560.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 170423 traces, 0 summary
Retagger: tagging trace set: gauss with 123424 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 123424 traces tagged "gauss"
FrameSaver: q=4.47234e+06 n=2101098 tag=gauss
wclsFrameSaver: saving 170423 traces tagged "wiener"
FrameSaver: q=4.78787e+06 n=1826437 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 45596 XUs and 39629 XVs -> 1146 XUVs
C:0 T:1 144863 XUs and 124794 XVs -> 18637 XUVs
C:0 T:2 147246 XUs and 163136 XVs -> 8302 XUVs
C:0 T:3 61266 XUs and 57418 XVs -> 634 XUVs
C:0 T:4 14306 XUs and 20309 XVs -> 242 XUVs
C:0 T:5 97575 XUs and 177820 XVs -> 22451 XUVs
C:0 T:6 108658 XUs and 82719 XVs -> 5370 XUVs
C:0 T:7 40344 XUs and 29762 XVs -> 302 XUVs
C:0 T:8 9001 XUs and 10941 XVs -> 195 XUVs
C:0 T:9 76931 XUs and 131885 XVs -> 11864 XUVs
C:0 T:10 65141 XUs and 29770 XVs -> 7131 XUVs
C:0 T:11 138 XUs and 113 XVs -> 21 XUVs
76295 XUVs total
16260 collection wire objects
70375 potential space points
Neighbour search...
11227859 tests to find 5119410 neighbours
Iterating with no regularization...
Begin: 1.18765e+08
0 9.8183e+07
1 9.68081e+07
2 9.66124e+07
3 9.65505e+07
Now with regularization...
Begin: 8.12843e+07
0 8.11822e+07
1 8.11183e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411744327
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411744327
fLifetime: 30000 [us]
15-Jan-2026 02:12:10 GMT  Closed output file "reco_stage1.root"
15-Jan-2026 02:12:10 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          436.568       609.878       971.062       608.436       133.619        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00104115    0.00322807    0.00463715    0.00325148    0.00117056       10     
reco:ophit:OpHitFinder                             0.263729      0.353778      0.495631      0.354765      0.0746282       10     
reco:opflash:OpFlashFinder                         0.201706      0.278013      0.410007      0.265083      0.053522        10     
reco:opslicer:OpSlicer                             0.541437      0.860699       1.21914       0.85834      0.207707        10     
reco:caldata:DataPrepModule                         5.06316       7.00802       8.62636       7.68802       1.40716        10     
reco:wclsdatasp:WireCellToolkit                     43.7936       55.1852       64.5485       56.7792       6.78451        10     
reco:gaushit:GausHitFinder                          1.24529       1.8607        2.42832       1.96207      0.389724        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000738696   0.000915162    0.0011964    0.000935829   0.000136675      10     
reco:reco3d:SpacePointSolver                        15.4959       22.4888       28.3999       22.4331       4.59147        10     
reco:hitpdune:DisambigFromSpacePoints               2.08403       3.19281       4.60582       3.02615      0.885292        10     
reco:pandora:StandardPandora                        28.1391       57.873        114.164       55.026        22.4195        10     
reco:pandoraWriter:StandardPandora                 0.463692      0.672772      0.804917      0.697806      0.109764        10     
reco:pandoraTrack:LArPandoraTrackCreation           9.37609       13.1716       18.9691       11.9918       3.20571        10     
reco:pandoraShower:LArPandoraShowerCreation         6.56491       14.901        29.3652       15.2391       5.96864        10     
reco:pandoracalo:Calorimetry                        3.99127       6.21193       8.91088       5.82403       1.5044         10     
reco:pandoracalonosce:Calorimetry                   3.57434       5.93153       9.34681       5.41241       1.60065        10     
reco:pandorapid:Chi2ParticleID                    0.00827209     0.0109734     0.0242052    0.00904661    0.00459721       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.073681      0.0971209     0.174212      0.0917123     0.0267666       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0081022    0.00991631     0.0154684    0.00934841    0.00213022       10     
reco:pandoraShowercalo:ShowerCalorimetry            6.7213        13.9443       26.5465       14.1251       5.07321        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       7.06954       13.4922       27.0883       13.3755       5.36492        10     
reco:emtrkmichelid:EmTrackMichelId                  192.006       285.537       448.266       276.898       68.1055        10     
reco:crttag:SingleCRTMatchingProducer              0.026455      0.0347023     0.0586511     0.0312463    0.00923834       10     
reco:crtreco:TwoCRTMatchingProducer                0.0194504     0.0254067     0.0355831     0.023806     0.00524183       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.23996       1.53309       2.0084        1.47416       0.21608        10     
reco:pandora2Track:LArPandoraTrackCreation          26.7843       40.2129       77.5574       36.3964       13.899         10     
reco:pandora2calo:Calorimetry                       11.3599       16.0867       32.845        14.1311       5.92899        10     
reco:pandora2calonosce:Calorimetry                  9.78167       13.4674       24.0515       12.3187       4.0802         10     
reco:pandora2pid:Chi2ParticleID                    0.0134743     0.0186938     0.0339494     0.0174873    0.00564164       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0775758     0.0990446     0.159211      0.0965746     0.0228713       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0131545     0.0188557     0.0345321     0.0183577    0.00576131       10     
reco:pandora2Shower:LArPandoraShowerCreation        6.57843       9.62857       17.4222       8.17476       3.19899        10     
reco:pandora2Showercalo:ShowerCalorimetry           7.88081       11.9195       21.8254       10.8341       3.84303        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      7.30839       10.0463       20.2687       9.02076       3.66104        10     
[art]:TriggerResults:TriggerResultInserter        3.9494e-05    6.67545e-05   0.000153329   5.5533e-05    3.14999e-05      10     
end_path:out1:RootOutput                           7.224e-06    3.13399e-05   0.000233905    9.076e-06    6.7532e-05       10     
end_path:out1:RootOutput(write)                     2.55166       3.53226       5.91691       3.31481      0.965949        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5399.64 MB
  Peak resident set size usage (VmHWM): 4459.64 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.27 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.43 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.25 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11240920
-rw-r--r-- 1 pildune32 pildune 2595252709 Jan 15 00:28 detsim_stage2.root
-rw-r--r-- 1 pildune32 pildune 2458568863 Jan 15 00:25 detsim_stage1.root
-rw-r--r-- 1 pildune32 pildune 2320947009 Jan 15 02:12 reco_stage1.root
-rw-r--r-- 1 pildune32 pildune 2069429347 Jan 14 22:40 g4_stage2.root
-rw-r--r-- 1 pildune32 pildune 1981618679 Jan 14 22:22 g4_stage1.root
-rw-r--r-- 1 pildune32 pildune   84273990 Jan 15 02:04 Pandora_Events.pndr
-rw-r--r-- 1 pildune32 pildune     242934 Jan 15 02:12 jobscript.log
-rw-r--r-- 1 pildune32 pildune     217088 Jan 15 02:12 mem.db
-rw-r--r-- 1 pildune32 pildune      62376 Jan 15 02:12 Pandora_Geometry.xml
-rw-r--r-- 1 pildune32 pildune      36864 Jan 15 02:12 time.db
-rw-r--r-- 1 pildune32 pildune        519 Jan 15 00:28 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Jan 14 22:40 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Jan 15 02:12 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune32 pildune         52 Jan 14 22:14 all-input-dids.txt
-rw-r--r-- 1 pildune32 pildune          7 Jan 15 02:12 justin-processed-pfns.txt
-rw-r--r-- 1 pildune32 pildune          0 Jan 15 00:41 ProtoDUNE.log
-rw-r--r-- 1 pildune32 pildune          0 Jan 15 00:28 debugprod.log
justIN time: 2026-02-06 02:07:14 UTC       justIN version: 01.06.00