justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 266051.0@dunegpschedd01.fnal.gov

Jobsub ID266051.0@dunegpschedd01.fnal.gov
Workflow ID11136
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-10 13:59:17
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2025-12-10 22:02:19
From worker nodeHostnamewn2209250.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7513 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-10 18:27:43
Input filesmonte-carlo-011136-000001
JobscriptExit code0
Real time3h (12784s)
CPU time3h (12421s = 97%)
Max RSS bytes4246904832 (4050 MiB)
Outputting started2025-12-10 22:00:48
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11136/1/001/reco_stage1.root
Finished2025-12-10 22:02:19
Saved logsjustin-logs:266051.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

eToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 180403 traces, 0 summary
Retagger: tagging trace set: gauss with 133426 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 133426 traces tagged "gauss"
FrameSaver: q=4.51378e+06 n=2317163 tag=gauss
wclsFrameSaver: saving 180403 traces tagged "wiener"
FrameSaver: q=4.81221e+06 n=2038110 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 37319 XUs and 30738 XVs -> 509 XUVs
C:0 T:1 94787 XUs and 75828 XVs -> 14124 XUVs
C:0 T:2 89439 XUs and 104601 XVs -> 3412 XUVs
C:0 T:3 49497 XUs and 46405 XVs -> 426 XUVs
C:0 T:4 15871 XUs and 17694 XVs -> 140 XUVs
C:0 T:5 81135 XUs and 73273 XVs -> 11395 XUVs
C:0 T:6 95735 XUs and 120030 XVs -> 4134 XUVs
C:0 T:7 37442 XUs and 50165 XVs -> 312 XUVs
C:0 T:8 15455 XUs and 18499 XVs -> 431 XUVs
C:0 T:9 147174 XUs and 203783 XVs -> 20651 XUVs
C:0 T:10 48524 XUs and 42027 XVs -> 4057 XUVs
C:0 T:11 192 XUs and 170 XVs -> 26 XUVs
59617 XUVs total
16877 collection wire objects
54883 potential space points
Neighbour search...
6316439 tests to find 3319438 neighbours
Iterating with no regularization...
Begin: 7.75856e+07
0 6.42961e+07
1 6.33985e+07
2 6.32746e+07
3 6.32396e+07
Now with regularization...
Begin: 5.07602e+07
0 5.07096e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411032589
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411032589
fLifetime: 30000 [us]
10-Dec-2025 22:00:46 GMT  Closed output file "reco_stage1.root"
10-Dec-2025 22:00:46 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          305.674       434.836       679.859       416.275       92.365         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000933734   0.00414894    0.00784437     0.0038484    0.00222863       10     
reco:ophit:OpHitFinder                             0.266322      0.360568      0.423436      0.363757      0.0420773       10     
reco:opflash:OpFlashFinder                         0.157712      0.226665      0.293687      0.230052      0.0374102       10     
reco:opslicer:OpSlicer                             0.565599      0.739998      0.994906      0.710973       0.12749        10     
reco:caldata:DataPrepModule                         4.82106       6.19627       7.75164       5.78331       1.14698        10     
reco:wclsdatasp:WireCellToolkit                     30.0515       36.3914       41.1973       35.7964       3.66356        10     
reco:gaushit:GausHitFinder                         0.904425       1.39918       1.80103       1.42566      0.271611        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000437888   0.000602579   0.000749836   0.000591283   8.83289e-05      10     
reco:reco3d:SpacePointSolver                        10.3812       20.661        36.5389       20.3347       8.38058        10     
reco:hitpdune:DisambigFromSpacePoints               1.33676       1.94891       2.88147       1.76921      0.562594        10     
reco:pandora:StandardPandora                        18.502        46.8767       97.0734       43.7021       20.1759        10     
reco:pandoraWriter:StandardPandora                 0.346541      0.522713      0.809962      0.490971      0.146885        10     
reco:pandoraTrack:LArPandoraTrackCreation           3.4354        5.24674       9.93757       5.05072       1.79486        10     
reco:pandoraShower:LArPandoraShowerCreation         3.19891       5.80792       13.1017       5.19195       2.73409        10     
reco:pandoracalo:Calorimetry                        2.01978       3.17502       5.91208       2.9199        1.08984        10     
reco:pandoracalonosce:Calorimetry                   1.75684       2.83006       5.27312       2.57273      0.920251        10     
reco:pandorapid:Chi2ParticleID                    0.00444896    0.00699695     0.0112309     0.0074596     0.0020281       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0399963     0.0687558     0.134718      0.068372      0.0277666       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00442149    0.00679347     0.0111598    0.00709972     0.0020286       10     
reco:pandoraShowercalo:ShowerCalorimetry            3.86657       6.56408       13.5991       6.09274       2.64934        10     
reco:pandoraShowercalonosce:ShowerCalorimetry        3.735        6.37875       14.2265       5.26139       2.95565        10     
reco:emtrkmichelid:EmTrackMichelId                  171.778       225.171       331.39        213.951       40.4131        10     
reco:crttag:SingleCRTMatchingProducer              0.0135984     0.0203597     0.0290679     0.0194392    0.00481797       10     
reco:crtreco:TwoCRTMatchingProducer               0.00963571     0.0118979     0.0149524     0.0112521    0.00176302       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.03668       1.20103       1.50435       1.11885      0.147923        10     
reco:pandora2Track:LArPandoraTrackCreation          13.3477       19.1449       34.2462       17.5922       5.95435        10     
reco:pandora2calo:Calorimetry                       5.62453       9.61562       19.0528       8.87735       3.61305        10     
reco:pandora2calonosce:Calorimetry                  4.54819       8.82505       18.855        7.51286       3.85262        10     
reco:pandora2pid:Chi2ParticleID                    0.0117061     0.0179666     0.0232409     0.0185206    0.00353579       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.050258      0.0896081     0.142621      0.0904344     0.0257412       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0102154     0.0173678     0.0231566     0.0190626    0.00436351       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.19395       6.37303       13.2034       5.88042       2.74125        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.69667       8.22498       17.5541       7.46929       3.55624        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.87257       7.37881       14.2888       6.45176       2.79991        10     
[art]:TriggerResults:TriggerResultInserter        6.0124e-05    9.65076e-05   0.000256936   8.2972e-05    5.44773e-05      10     
end_path:out1:RootOutput                          1.1562e-05    1.64872e-05   3.2351e-05    1.44975e-05   5.61806e-06      10     
end_path:out1:RootOutput(write)                     2.52921       3.19096       4.47043       3.06191       0.57861        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5078.31 MB
  Peak resident set size usage (VmHWM): 4246.9 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.29 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.87 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10660340
-rw-r--r-- 1 dune01 dune01 2488811463 Dec 10 20:46 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2362976251 Dec 10 20:43 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2196577170 Dec 10 22:00 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1935048662 Dec 10 18:47 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1851401786 Dec 10 18:35 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   80745300 Dec 10 21:52 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     243293 Dec 10 22:00 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Dec 10 22:00 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Dec 10 22:00 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Dec 10 22:00 time.db
-rw-r--r-- 1 dune01 dune01        519 Dec 10 20:46 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Dec 10 18:47 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Dec 10 22:00 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Dec 10 18:27 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Dec 10 22:00 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Dec 10 20:52 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Dec 10 20:46 debugprod.log
justIN time: 2026-02-07 13:29:59 UTC       justIN version: 01.06.00