justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 269580.1@dunegpschedd01.fnal.gov

Jobsub ID269580.1@dunegpschedd01.fnal.gov
Workflow ID11304
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-21 21:31:47
SiteUK_Durham
EntryDUNE_UK_SGridDurham_ce3
Last heartbeat2025-12-22 01:14:53
From worker nodeHostnamen174.dur.scotgrid.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 5220 CPU @ 2.20GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-21 21:32:40
Input filesmonte-carlo-011304-000001
JobscriptExit code0
Real time3h (13231s)
CPU time3h (13010s = 98%)
Max RSS bytes4507000832 (4298 MiB)
Outputting started2025-12-22 01:13:12
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11304/1/001/reco_stage1.root
Finished2025-12-22 01:14:53
Saved logsjustin-logs:269580.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

asedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 168486 traces, 0 summary
Retagger: tagging trace set: gauss with 123720 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 123720 traces tagged "gauss"
FrameSaver: q=4.39104e+06 n=2131177 tag=gauss
wclsFrameSaver: saving 168486 traces tagged "wiener"
FrameSaver: q=4.68354e+06 n=1860499 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 36243 XUs and 37453 XVs -> 421 XUVs
C:0 T:1 96635 XUs and 93125 XVs -> 7649 XUVs
C:0 T:2 119301 XUs and 128231 XVs -> 5062 XUVs
C:0 T:3 60662 XUs and 49821 XVs -> 345 XUVs
C:0 T:4 21427 XUs and 16178 XVs -> 264 XUVs
C:0 T:5 111406 XUs and 66303 XVs -> 10538 XUVs
C:0 T:6 75351 XUs and 102154 XVs -> 3339 XUVs
C:0 T:7 28138 XUs and 38626 XVs -> 254 XUVs
C:0 T:8 11616 XUs and 7253 XVs -> 147 XUVs
C:0 T:9 160563 XUs and 91791 XVs -> 14828 XUVs
C:0 T:10 64635 XUs and 70933 XVs -> 16040 XUVs
C:0 T:11 49 XUs and 39 XVs -> 3 XUVs
58890 XUVs total
15424 collection wire objects
54736 potential space points
Neighbour search...
7084322 tests to find 3390876 neighbours
Iterating with no regularization...
Begin: 9.71353e+07
0 7.4018e+07
1 7.24717e+07
2 7.22339e+07
3 7.21607e+07
4 7.21324e+07
Now with regularization...
Begin: 5.85996e+07
0 5.85398e+07
1 5.85014e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411260979
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411260979
fLifetime: 30000 [us]
22-Dec-2025 01:13:07 GMT  Closed output file "reco_stage1.root"
22-Dec-2025 01:13:07 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          407.436       540.931       688.146       552.296       88.2938        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00099144    0.00367587     0.0056211    0.00404388    0.00147649       10     
reco:ophit:OpHitFinder                             0.299875      0.361281      0.441582      0.354134      0.0394232       10     
reco:opflash:OpFlashFinder                         0.242917      0.327567      0.495461      0.317682      0.0711652       10     
reco:opslicer:OpSlicer                             0.607657      0.913497       1.50496      0.854505      0.224065        10     
reco:caldata:DataPrepModule                         6.32911       6.75528       9.14458       6.53198      0.802728        10     
reco:wclsdatasp:WireCellToolkit                     48.3875       53.5849       60.8467       53.2249       3.41863        10     
reco:gaushit:GausHitFinder                          1.44393       1.77495       2.17339       1.8334       0.218933        10     
reco:nhitsfilter:NumberOfHitsFilter                0.0006321    0.000744981   0.000882535    0.0007559    7.4309e-05       10     
reco:reco3d:SpacePointSolver                        12.4053       19.5313       30.9703       18.6202       5.28318        10     
reco:hitpdune:DisambigFromSpacePoints               2.0205        3.00284       3.91913       3.08421      0.648433        10     
reco:pandora:StandardPandora                        33.8648       53.9579       73.1642       50.2326        12.78         10     
reco:pandoraWriter:StandardPandora                  0.49111      0.584217      0.680635      0.601556      0.0648542       10     
reco:pandoraTrack:LArPandoraTrackCreation           7.96696       11.9404       16.6378       10.9493       3.04084        10     
reco:pandoraShower:LArPandoraShowerCreation         7.0987        13.0426       19.7461       13.5978       3.29335        10     
reco:pandoracalo:Calorimetry                        3.4557        5.13467       7.21286       4.7956        1.21559        10     
reco:pandoracalonosce:Calorimetry                   3.33424       4.80402       6.68396       4.54385       1.16287        10     
reco:pandorapid:Chi2ParticleID                    0.00584349     0.0072523    0.00871335    0.00722139    0.000938399      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0644421     0.0813133     0.101004      0.0803888     0.0112806       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0058128    0.00703375    0.00859736    0.00656807    0.000943728      10     
reco:pandoraShowercalo:ShowerCalorimetry            6.7336        11.2128       17.0442       11.6243       2.77756        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       6.41047       10.8778       15.8595       11.5064       2.65719        10     
reco:emtrkmichelid:EmTrackMichelId                  192.929       240.089       279.137       248.348       29.6169        10     
reco:crttag:SingleCRTMatchingProducer              0.0234545     0.0289706     0.0353304     0.0293601    0.00335875       10     
reco:crtreco:TwoCRTMatchingProducer                 0.01297      0.022906      0.0284513     0.022837      0.0041298       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.23238       1.58778       2.04657       1.50268       0.28268        10     
reco:pandora2Track:LArPandoraTrackCreation          20.2972       37.3211       53.6613       38.5561       10.2879        10     
reco:pandora2calo:Calorimetry                       8.43543       14.7456       20.7632       15.1628       3.94544        10     
reco:pandora2calonosce:Calorimetry                  7.52925       13.865        21.0311       14.2687       4.09959        10     
reco:pandora2pid:Chi2ParticleID                    0.013757      0.017774      0.0217993     0.0175989    0.00230952       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0802799     0.147067      0.273246      0.104551      0.075698        10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0139335     0.017662      0.0217505     0.0175452    0.00236382       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.43741       10.3208       15.5103       10.5728       2.89966        10     
reco:pandora2Showercalo:ShowerCalorimetry           6.50772       11.4083       15.467        12.0831       2.69967        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.65274       9.80451       12.7272       10.454         2.266         10     
[art]:TriggerResults:TriggerResultInserter        5.5766e-05    7.3912e-05    0.000136974   6.6013e-05    2.32288e-05      10     
end_path:out1:RootOutput                           8.368e-06    1.12247e-05   1.6181e-05    1.05165e-05   2.27714e-06      10     
end_path:out1:RootOutput(write)                     2.8092        3.49245       4.25761       3.63233      0.449251        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5313.16 MB
  Peak resident set size usage (VmHWM): 4507 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.24 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.25 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.23 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11677452
-rw-r--r--. 1 dune004 dune 2680522056 Dec 21 23:40 detsim_stage2.root
-rw-r--r--. 1 dune004 dune 2540828866 Dec 21 23:38 detsim_stage1.root
-rw-r--r--. 1 dune004 dune 2416612273 Dec 22 01:13 reco_stage1.root
-rw-r--r--. 1 dune004 dune 2158448868 Dec 21 21:57 g4_stage2.root
-rw-r--r--. 1 dune004 dune 2073737326 Dec 21 21:40 g4_stage1.root
-rw-r--r--. 1 dune004 dune   86928070 Dec 22 01:06 Pandora_Events.pndr
-rw-r--r--. 1 dune004 dune     242896 Dec 22 01:13 jobscript.log
-rw-r--r--. 1 dune004 dune     217088 Dec 22 01:13 mem.db
-rw-r--r--. 1 dune004 dune      62376 Dec 22 01:13 Pandora_Geometry.xml
-rw-r--r--. 1 dune004 dune      36864 Dec 22 01:13 time.db
-rw-r--r--. 1 dune004 dune        519 Dec 21 23:40 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune004 dune        519 Dec 21 21:57 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune004 dune        519 Dec 22 01:13 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune004 dune         52 Dec 21 21:32 all-input-dids.txt
-rw-r--r--. 1 dune004 dune          7 Dec 22 01:13 justin-processed-pfns.txt
-rw-r--r--. 1 dune004 dune          0 Dec 21 23:47 ProtoDUNE.log
-rw-r--r--. 1 dune004 dune          0 Dec 21 23:40 debugprod.log
justIN time: 2026-02-07 06:22:02 UTC       justIN version: 01.06.00