justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 269085.1@dunegpschedd01.fnal.gov

Jobsub ID269085.1@dunegpschedd01.fnal.gov
Workflow ID11259
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-18 16:44:58
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2025-12-18 20:15:11
From worker nodeHostnamecn076.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-18 16:45:47
Input filesmonte-carlo-011259-000001
JobscriptExit code0
Real time3h (12339s)
CPU time3h (12136s = 98%)
Max RSS bytes4411711488 (4207 MiB)
Outputting started2025-12-18 20:11:27
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11259/1/001/reco_stage1.root
Finished2025-12-18 20:15:11
Saved logsjustin-logs:269085.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 138454 traces, 0 summary
Retagger: tagging trace set: gauss with 106388 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 106388 traces tagged "gauss"
FrameSaver: q=3.68686e+06 n=1878757 tag=gauss
wclsFrameSaver: saving 138454 traces tagged "wiener"
FrameSaver: q=3.90338e+06 n=1669203 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 41627 XUs and 38607 XVs -> 719 XUVs
C:0 T:1 99325 XUs and 79135 XVs -> 8264 XUVs
C:0 T:2 94213 XUs and 167273 XVs -> 7548 XUVs
C:0 T:3 44661 XUs and 59237 XVs -> 601 XUVs
C:0 T:4 17120 XUs and 14529 XVs -> 212 XUVs
C:0 T:5 100352 XUs and 62263 XVs -> 11134 XUVs
C:0 T:6 113819 XUs and 154402 XVs -> 8233 XUVs
C:0 T:7 41559 XUs and 50610 XVs -> 562 XUVs
C:0 T:8 6589 XUs and 4468 XVs -> 81 XUVs
C:0 T:9 79026 XUs and 42613 XVs -> 5912 XUVs
C:0 T:10 30722 XUs and 19982 XVs -> 2814 XUVs
C:0 T:11 268 XUs and 194 XVs -> 27 XUVs
46107 XUVs total
12149 collection wire objects
42385 potential space points
Neighbour search...
5762971 tests to find 2670426 neighbours
Iterating with no regularization...
Begin: 7.00827e+07
0 5.61244e+07
1 5.50431e+07
2 5.49019e+07
3 5.48636e+07
Now with regularization...
Begin: 4.45839e+07
0 4.45472e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411196519
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411196519
fLifetime: 30000 [us]
18-Dec-2025 20:11:24 GMT  Closed output file "reco_stage1.root"
18-Dec-2025 20:11:24 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          327.487       474.684         643         444.885       84.9543        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000761485   0.00310071    0.00465676    0.00330976    0.00116374       10     
reco:ophit:OpHitFinder                             0.255208      0.306619      0.429315       0.28052      0.0526619       10     
reco:opflash:OpFlashFinder                         0.214883      0.252557      0.301956      0.249307      0.0264012       10     
reco:opslicer:OpSlicer                             0.565661      0.680403      0.747348      0.695832      0.0546143       10     
reco:caldata:DataPrepModule                         5.19048       5.36589       5.64479       5.35577      0.135815        10     
reco:wclsdatasp:WireCellToolkit                     43.3256       46.9923       49.4363       47.7448       2.00708        10     
reco:gaushit:GausHitFinder                          1.15566       1.55857       2.49664       1.44891      0.371631        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000678572   0.000949812   0.00153765    0.00082691    0.000246697      10     
reco:reco3d:SpacePointSolver                        9.75318       15.7121       24.8438       14.9056       4.36249        10     
reco:hitpdune:DisambigFromSpacePoints               1.55717       2.40371       3.30591       2.26699      0.510858        10     
reco:pandora:StandardPandora                        30.4621       52.5718       138.628       37.0003       34.1104        10     
reco:pandoraWriter:StandardPandora                 0.425664      0.603164       1.22157      0.505508      0.223609        10     
reco:pandoraTrack:LArPandoraTrackCreation           5.18062       10.7523       16.1715       11.1901       2.94247        10     
reco:pandoraShower:LArPandoraShowerCreation         4.66779       12.2914       19.1391       12.1612       4.11074        10     
reco:pandoracalo:Calorimetry                        2.38411       4.72644       6.94447       4.46101       1.20646        10     
reco:pandoracalonosce:Calorimetry                   2.28162       4.40172       6.49535       4.25628       1.09155        10     
reco:pandorapid:Chi2ParticleID                    0.00480214    0.00646143     0.0100152    0.00607298     0.001528        10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0506408     0.0658465     0.0865388     0.0634899     0.0104874       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00469622    0.00592928    0.00724958    0.00597617    0.000637808      10     
reco:pandoraShowercalo:ShowerCalorimetry            5.00117       11.0374       17.5314       10.8511       3.25228        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.69206       10.5662       16.2808       10.1356       3.09664        10     
reco:emtrkmichelid:EmTrackMichelId                  157.183       201.096       248.238       192.813       27.0186        10     
reco:crttag:SingleCRTMatchingProducer              0.0184292     0.0282918     0.0388812     0.0283127    0.00558032       10     
reco:crtreco:TwoCRTMatchingProducer                0.0117474     0.0209625     0.0275038     0.0218407    0.00522612       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.943681       1.21499       1.6028        1.23356      0.187189        10     
reco:pandora2Track:LArPandoraTrackCreation          16.5394       33.9947       49.5701       35.1125       9.18312        10     
reco:pandora2calo:Calorimetry                       7.01095       13.7904       21.1706       14.0173       3.81085        10     
reco:pandora2calonosce:Calorimetry                  6.58953       12.7297       19.2022       13.4072       3.40571        10     
reco:pandora2pid:Chi2ParticleID                    0.0114196     0.0148183     0.0198653     0.0146363    0.00219093       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0750384     0.100469      0.250081      0.0799271     0.0508498       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0123313     0.0148136     0.017973      0.014625     0.00163619       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.81269       8.85517       12.1797       8.36717       2.31675        10     
reco:pandora2Showercalo:ShowerCalorimetry           5.89518       10.4817       14.3545       10.3004       2.42681        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.21957       9.13401       12.7887       8.61315       2.03389        10     
[art]:TriggerResults:TriggerResultInserter        4.3351e-05    5.33236e-05   0.000104952   4.8743e-05    1.74869e-05      10     
end_path:out1:RootOutput                           6.218e-06    7.2808e-06    1.1348e-05     6.67e-06     1.44913e-06      10     
end_path:out1:RootOutput(write)                     2.1346        2.74224       3.75073       2.61109      0.430381        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5220.72 MB
  Peak resident set size usage (VmHWM): 4411.71 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.20 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.28 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.18 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10907476
-rw-r--r-- 1 pildune32 pildune 2535574126 Dec 18 18:50 detsim_stage2.root
-rw-r--r-- 1 pildune32 pildune 2399298140 Dec 18 18:48 detsim_stage1.root
-rw-r--r-- 1 pildune32 pildune 2259042885 Dec 18 20:11 reco_stage1.root
-rw-r--r-- 1 pildune32 pildune 1981816215 Dec 18 17:10 g4_stage2.root
-rw-r--r-- 1 pildune32 pildune 1909441810 Dec 18 16:52 g4_stage1.root
-rw-r--r-- 1 pildune32 pildune   83462080 Dec 18 20:06 Pandora_Events.pndr
-rw-r--r-- 1 pildune32 pildune     243303 Dec 18 20:11 jobscript.log
-rw-r--r-- 1 pildune32 pildune     217088 Dec 18 20:11 mem.db
-rw-r--r-- 1 pildune32 pildune      62376 Dec 18 20:11 Pandora_Geometry.xml
-rw-r--r-- 1 pildune32 pildune      36864 Dec 18 20:11 time.db
-rw-r--r-- 1 pildune32 pildune        519 Dec 18 18:50 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Dec 18 17:10 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Dec 18 20:11 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune32 pildune         52 Dec 18 16:45 all-input-dids.txt
-rw-r--r-- 1 pildune32 pildune          7 Dec 18 20:11 justin-processed-pfns.txt
-rw-r--r-- 1 pildune32 pildune          0 Dec 18 18:59 ProtoDUNE.log
-rw-r--r-- 1 pildune32 pildune          0 Dec 18 18:50 debugprod.log
justIN time: 2025-12-19 05:14:17 UTC       justIN version: 01.05.03