justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 269485.0@dunegpschedd02.fnal.gov

Jobsub ID269485.0@dunegpschedd02.fnal.gov
Workflow ID11734
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-11 22:53:21
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2026-01-12 03:23:29
From worker nodeHostnamewn2206291.tier2.hep.manchester.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-11 22:54:39
Input filesmonte-carlo-011734-000001
JobscriptExit code0
Real time4h (16037s)
CPU time4h (15803s = 98%)
Max RSS bytes4515819520 (4306 MiB)
Outputting started2026-01-12 03:21:57
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11734/1/001/reco_stage1.root
Finished2026-01-12 03:23:29
Saved logsjustin-logs:269485.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

eepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 157341 traces, 0 summary
Retagger: tagging trace set: gauss with 123286 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 123286 traces tagged "gauss"
FrameSaver: q=4.32246e+06 n=2225650 tag=gauss
wclsFrameSaver: saving 157341 traces tagged "wiener"
FrameSaver: q=4.56036e+06 n=1987838 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 21952 XUs and 14658 XVs -> 144 XUVs
C:0 T:1 50613 XUs and 31045 XVs -> 4383 XUVs
C:0 T:2 178929 XUs and 150558 XVs -> 7411 XUVs
C:0 T:3 80676 XUs and 62383 XVs -> 555 XUVs
C:0 T:4 14595 XUs and 16297 XVs -> 407 XUVs
C:0 T:5 122705 XUs and 90726 XVs -> 28362 XUVs
C:0 T:6 100145 XUs and 97290 XVs -> 3030 XUVs
C:0 T:7 40865 XUs and 41374 XVs -> 384 XUVs
C:0 T:8 9334 XUs and 11902 XVs -> 148 XUVs
C:0 T:9 64113 XUs and 96677 XVs -> 6409 XUVs
C:0 T:10 84515 XUs and 60095 XVs -> 6888 XUVs
C:0 T:11 154 XUs and 155 XVs -> 25 XUVs
58146 XUVs total
14150 collection wire objects
54222 potential space points
Neighbour search...
11538092 tests to find 4618314 neighbours
Iterating with no regularization...
Begin: 1.10126e+08
0 9.18477e+07
1 9.06882e+07
2 9.05108e+07
3 9.0455e+07
Now with regularization...
Begin: 7.73663e+07
0 7.73193e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411684522
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411684522
fLifetime: 30000 [us]
12-Jan-2026 03:21:54 GMT  Closed output file "reco_stage1.root"
12-Jan-2026 03:21:54 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          342.528       498.821       621.784       522.718       80.1291        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000871895   0.00300431    0.00462913    0.00302959    0.00110159       10     
reco:ophit:OpHitFinder                             0.277023      0.350249       0.40425      0.362099      0.037975        10     
reco:opflash:OpFlashFinder                         0.196953      0.296889      0.377431      0.317323      0.0585324       10     
reco:opslicer:OpSlicer                             0.560698       0.85041       1.04898      0.891996       0.14569        10     
reco:caldata:DataPrepModule                         8.19591       8.69671       9.17862       8.69543      0.306911        10     
reco:wclsdatasp:WireCellToolkit                     49.0467       52.7299       54.7815       53.2217       1.83827        10     
reco:gaushit:GausHitFinder                          1.38403       1.87597       2.16844       1.9228       0.245799        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000539845   0.000735545   0.000936215   0.000729107   0.000109119      10     
reco:reco3d:SpacePointSolver                        10.8528       21.3012       35.0888       21.2492       6.49937        10     
reco:hitpdune:DisambigFromSpacePoints               1.77126       3.07724       4.03616       3.2747       0.691883        10     
reco:pandora:StandardPandora                        33.4163       62.7623       89.8954       70.2997       17.3532        10     
reco:pandoraWriter:StandardPandora                 0.417818      0.579436      0.693513      0.594238      0.0863019       10     
reco:pandoraTrack:LArPandoraTrackCreation           4.19219       7.30493       10.1653       7.30129       1.96105        10     
reco:pandoraShower:LArPandoraShowerCreation         3.85175       7.14487       10.7582       7.39173       1.95664        10     
reco:pandoracalo:Calorimetry                        1.92058       3.36893       4.48751       3.35942      0.796075        10     
reco:pandoracalonosce:Calorimetry                   1.73752       3.09756       4.16285       3.0959       0.758236        10     
reco:pandorapid:Chi2ParticleID                    0.00573922    0.00752983     0.0100452    0.00744572    0.00121395       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0598119     0.0797244     0.0918407     0.0840622    0.00982879       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00555699    0.00715537    0.00832909    0.00728539    0.000944979      10     
reco:pandoraShowercalo:ShowerCalorimetry            3.3253        6.92438       10.3078       7.34196       1.98517        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.23807       6.8039        10.086        7.22508       1.94901        10     
reco:emtrkmichelid:EmTrackMichelId                  175.674       239.42        277.457       247.64        30.2318        10     
reco:crttag:SingleCRTMatchingProducer              0.0159755     0.0241703     0.029521      0.0244991    0.00407516       10     
reco:crtreco:TwoCRTMatchingProducer               0.00895379     0.0140811     0.0279392     0.0130725    0.00489138       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.11626       1.46454       1.91027       1.44261      0.247821        10     
reco:pandora2Track:LArPandoraTrackCreation          12.2232       22.9826       34.0249       23.6587       6.24269        10     
reco:pandora2calo:Calorimetry                       5.02327       9.80453       14.0601       10.1096       2.53627        10     
reco:pandora2calonosce:Calorimetry                  4.71741       9.35032       13.3975       9.65452       2.42932        10     
reco:pandora2pid:Chi2ParticleID                    0.0131663     0.0189642     0.0238881     0.0198632    0.00317692       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.073903      0.109855      0.185461      0.108147      0.0287617       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0129405     0.0183159     0.0229083     0.0194805    0.00288744       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.72887       8.01675       11.2854       8.21827       1.90932        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.43381       8.68337       12.1933       8.92596       2.17905        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.08317       8.08774       11.3245       8.34735       2.04037        10     
[art]:TriggerResults:TriggerResultInserter        3.2291e-05    5.23342e-05   0.000129425   4.37205e-05   2.69917e-05      10     
end_path:out1:RootOutput                           7.854e-06    1.01031e-05   2.0659e-05     8.605e-06     3.805e-06       10     
end_path:out1:RootOutput(write)                     2.44728       3.44564       4.28077       3.5159       0.506476        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5318.52 MB
  Peak resident set size usage (VmHWM): 4515.82 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   6.27 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11364672
-rw-r--r-- 1 dune01 dune01 2646903618 Jan 12 01:56 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2511848978 Jan 12 01:54 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2382666886 Jan 12 03:21 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 2044757646 Jan 11 23:17 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1963229373 Jan 11 23:01 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   87391720 Jan 12 03:15 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242820 Jan 12 03:21 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Jan 12 03:21 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Jan 12 03:21 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Jan 12 03:21 time.db
-rw-r--r-- 1 dune01 dune01        519 Jan 12 01:56 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Jan 11 23:17 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Jan 12 03:21 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Jan 11 22:54 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Jan 12 03:21 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Jan 12 02:04 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Jan 12 01:56 debugprod.log
justIN time: 2026-02-06 10:22:30 UTC       justIN version: 01.06.00