justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 266396.0@dunegpschedd01.fnal.gov

Jobsub ID266396.0@dunegpschedd01.fnal.gov
Workflow ID11146
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-10 23:21:29
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2025-12-11 04:40:41
From worker nodeHostnamewn1208091.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-10 23:22:50
Input filesmonte-carlo-011146-000001
JobscriptExit code0
Real time5h (18966s)
CPU time5h (18293s = 96%)
Max RSS bytes4237799424 (4041 MiB)
Outputting started2025-12-11 04:38:57
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11146/1/001/reco_stage1.root
Finished2025-12-11 04:40:41
Saved logsjustin-logs:266396.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

RawDigitPrepService:prepare:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 130285 traces, 0 summary
Retagger: tagging trace set: gauss with 104696 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 104696 traces tagged "gauss"
FrameSaver: q=3.55093e+06 n=1917771 tag=gauss
wclsFrameSaver: saving 130285 traces tagged "wiener"
FrameSaver: q=3.71315e+06 n=1715107 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 27754 XUs and 26465 XVs -> 200 XUVs
C:0 T:1 51681 XUs and 46216 XVs -> 2850 XUVs
C:0 T:2 94309 XUs and 130880 XVs -> 5284 XUVs
C:0 T:3 49094 XUs and 52562 XVs -> 263 XUVs
C:0 T:5 43428 XUs and 30819 XVs -> 2906 XUVs
C:0 T:6 135560 XUs and 114142 XVs -> 6045 XUVs
C:0 T:7 51297 XUs and 38445 XVs -> 271 XUVs
C:0 T:8 11061 XUs and 11430 XVs -> 131 XUVs
C:0 T:9 67469 XUs and 84075 XVs -> 5286 XUVs
C:0 T:10 30113 XUs and 22027 XVs -> 3729 XUVs
C:0 T:11 110 XUs and 73 XVs -> 17 XUVs
26982 XUVs total
12314 collection wire objects
24836 potential space points
Neighbour search...
1310290 tests to find 686526 neighbours
Iterating with no regularization...
Begin: 6.99974e+07
0 6.00539e+07
1 5.92817e+07
2 5.91606e+07
3 5.91313e+07
Now with regularization...
Begin: 4.94336e+07
0 4.94104e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411041175
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411041175
fLifetime: 30000 [us]
11-Dec-2025 04:38:46 GMT  Closed output file "reco_stage1.root"
11-Dec-2025 04:38:46 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          526.545       708.621       892.133       705.656       116.469        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00167206    0.00509473    0.00958339    0.00465068    0.00254204       10     
reco:ophit:OpHitFinder                             0.287519      0.392653      0.514018      0.383082      0.0654698       10     
reco:opflash:OpFlashFinder                         0.176419      0.252661      0.338472      0.246701      0.0472038       10     
reco:opslicer:OpSlicer                             0.571249       1.05282       1.4798        1.06096      0.264281        10     
reco:caldata:DataPrepModule                         6.11319       7.89696       8.92473       8.28121      0.979835        10     
reco:wclsdatasp:WireCellToolkit                     44.193        54.4044       60.3221       54.8658       4.72918        10     
reco:gaushit:GausHitFinder                          1.36343       1.93255       2.86997       1.90527      0.419022        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000829353   0.00122424    0.00185631    0.00116888    0.000342853      10     
reco:reco3d:SpacePointSolver                        13.2704       29.9194       70.4301       23.4198       17.395         10     
reco:hitpdune:DisambigFromSpacePoints               1.92655       2.98984       4.47705       2.57222      0.947126        10     
reco:pandora:StandardPandora                        35.3868       70.2501       95.9791       74.2001       21.6369        10     
reco:pandoraWriter:StandardPandora                 0.540411       1.02634       2.55782      0.897446      0.548808        10     
reco:pandoraTrack:LArPandoraTrackCreation           7.41402       10.1896       15.0772       9.72195       2.26669        10     
reco:pandoraShower:LArPandoraShowerCreation         7.00973       11.7719       20.5347       12.2678       3.84463        10     
reco:pandoracalo:Calorimetry                        3.75614       5.48618       8.91084       5.3468        1.4906         10     
reco:pandoracalonosce:Calorimetry                   3.81491       4.72745       7.77702       4.4288        1.07965        10     
reco:pandorapid:Chi2ParticleID                     0.007186     0.00957417     0.016011     0.00848055    0.00264709       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0601077     0.090043      0.118361      0.0930184     0.018593        10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00698128     0.0094454     0.0127803     0.009307     0.00185747       10     
reco:pandoraShowercalo:ShowerCalorimetry            6.56064       12.4677       21.803        11.3773       4.69101        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       6.60915       12.7053       20.4742       13.2873       4.66198        10     
reco:emtrkmichelid:EmTrackMichelId                  297.924       373.358       441.444       380.472       42.8717        10     
reco:crttag:SingleCRTMatchingProducer              0.0205229     0.0865388     0.489906      0.0272335     0.140442        10     
reco:crtreco:TwoCRTMatchingProducer                0.0170236     0.0228909     0.0320095     0.0227631    0.00398166       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.0831        1.52036       1.79333       1.52947      0.203934        10     
reco:pandora2Track:LArPandoraTrackCreation          20.6423       31.9132       46.9439       30.7801       9.3504         10     
reco:pandora2calo:Calorimetry                       9.55723       16.062        25.1758       16.3859       5.77931        10     
reco:pandora2calonosce:Calorimetry                  8.91763       15.0901       25.407        15.3891       4.76113        10     
reco:pandora2pid:Chi2ParticleID                    0.0167316     0.0305403      0.09255      0.0229481     0.0212408       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0732696     0.120777      0.183806      0.107155      0.0396492       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0166062     0.0235411     0.0327966     0.022387     0.00502081       10     
reco:pandora2Shower:LArPandoraShowerCreation        6.57966       11.2852       16.2405       11.4803       3.3051         10     
reco:pandora2Showercalo:ShowerCalorimetry           7.61095       14.1932       25.9339       13.9754       5.23117        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      6.68795       12.1037       17.2791       12.8324       3.64054        10     
[art]:TriggerResults:TriggerResultInserter        3.5467e-05    5.37873e-05   0.000117752   4.53055e-05   2.4797e-05       10     
end_path:out1:RootOutput                           6.693e-06    0.00269091     0.0268382    7.6645e-06    0.00804911       10     
end_path:out1:RootOutput(write)                     3.34542       4.98431       6.1685        5.03888       0.92597        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5079.63 MB
  Peak resident set size usage (VmHWM): 4237.8 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.33 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.08 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.20 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10744936
-rw-r--r-- 1 dune01 dune01 2502017109 Dec 11 02:37 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2375800990 Dec 11 02:32 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2216913539 Dec 11 04:38 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1948302658 Dec 10 23:51 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1877993649 Dec 10 23:32 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   81156150 Dec 11 04:30 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242969 Dec 11 04:38 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Dec 11 04:38 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Dec 11 04:38 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Dec 11 04:38 time.db
-rw-r--r-- 1 dune01 dune01        519 Dec 11 02:37 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Dec 10 23:51 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Dec 11 04:38 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Dec 10 23:22 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Dec 11 04:38 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Dec 11 02:48 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Dec 11 02:37 debugprod.log
justIN time: 2026-02-07 13:30:28 UTC       justIN version: 01.06.00