justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 295858.1@dunegpschedd02.fnal.gov

Jobsub ID295858.1@dunegpschedd02.fnal.gov
Workflow ID12425
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-27 08:23:09
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2026-01-27 11:29:56
From worker nodeHostnamewn1207091.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-27 08:25:02
Input filesmonte-carlo-012425-000001
JobscriptExit code0
Real time3h (11000s)
CPU time2h (10736s = 97%)
Max RSS bytes4181204992 (3987 MiB)
Outputting started2026-01-27 11:28:23
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12425/1/001/reco_stage1.root
Finished2026-01-27 11:29:56
Saved logsjustin-logs:295858.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ssing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 103949 traces, 0 summary
Retagger: tagging trace set: gauss with 75992 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 75992 traces tagged "gauss"
FrameSaver: q=2.77187e+06 n=1288553 tag=gauss
wclsFrameSaver: saving 103949 traces tagged "wiener"
FrameSaver: q=2.9748e+06 n=1121699 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 23160 XUs and 27335 XVs -> 289 XUVs
C:0 T:1 34532 XUs and 38462 XVs -> 2699 XUVs
C:0 T:2 52088 XUs and 50216 XVs -> 3087 XUVs
C:0 T:3 24007 XUs and 23077 XVs -> 124 XUVs
C:0 T:4 11116 XUs and 10201 XVs -> 74 XUVs
C:0 T:5 36314 XUs and 29810 XVs -> 3992 XUVs
C:0 T:6 186568 XUs and 124528 XVs -> 9136 XUVs
C:0 T:7 50355 XUs and 36412 XVs -> 457 XUVs
C:0 T:8 3850 XUs and 5104 XVs -> 49 XUVs
C:0 T:9 17675 XUs and 28913 XVs -> 1621 XUVs
C:0 T:10 51075 XUs and 17181 XVs -> 5026 XUVs
C:0 T:11 109 XUs and 62 XVs -> 9 XUVs
26563 XUVs total
9339 collection wire objects
24022 potential space points
Neighbour search...
1981928 tests to find 846108 neighbours
Iterating with no regularization...
Begin: 6.26654e+07
0 4.46298e+07
1 4.32522e+07
2 4.299e+07
3 4.28936e+07
4 4.28438e+07
5 4.28126e+07
Now with regularization...
Begin: 3.27345e+07
0 3.26069e+07
1 3.25457e+07
2 3.25127e+07
3 3.24922e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411994268
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411994268
fLifetime: 30000 [us]
27-Jan-2026 11:28:21 GMT  Closed output file "reco_stage1.root"
27-Jan-2026 11:28:21 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          217.867       313.513       443.676       304.654       66.9617        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000827471    0.0030026    0.00536416    0.00279636    0.00148399       10     
reco:ophit:OpHitFinder                             0.210988      0.267623      0.310959      0.268997      0.0263547       10     
reco:opflash:OpFlashFinder                         0.107543      0.159097      0.201697      0.161987      0.0242332       10     
reco:opslicer:OpSlicer                             0.471655      0.649861      0.715447      0.669631      0.0732252       10     
reco:caldata:DataPrepModule                         4.7993        5.02261       5.38021       4.98147       0.15584        10     
reco:wclsdatasp:WireCellToolkit                     31.8837        36.64        39.3239       38.3017       2.88721        10     
reco:gaushit:GausHitFinder                         0.714483      0.978554       1.33632      0.991647      0.175964        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000340382   0.000511003   0.000677919   0.000523198   8.50271e-05      10     
reco:reco3d:SpacePointSolver                        5.96095       10.8243       21.2642       9.36206       4.11035        10     
reco:hitpdune:DisambigFromSpacePoints              0.755462       1.25014       2.13264       1.26877      0.368811        10     
reco:pandora:StandardPandora                        13.6838       33.339        86.6892       26.7524       20.5756        10     
reco:pandoraWriter:StandardPandora                 0.278561      0.379302      0.542397       0.37302      0.074316        10     
reco:pandoraTrack:LArPandoraTrackCreation           1.95774       3.61224       7.77508       3.08138       1.73598        10     
reco:pandoraShower:LArPandoraShowerCreation         1.27186       3.11782       6.55239       3.21883       1.38627        10     
reco:pandoracalo:Calorimetry                       0.996032         1.8         3.49594       1.60336      0.760244        10     
reco:pandoracalonosce:Calorimetry                   0.85563       1.63873       3.25969       1.44104      0.728269        10     
reco:pandorapid:Chi2ParticleID                    0.00431661    0.00560968    0.00814483    0.00520909    0.00123528       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0308983     0.0431854     0.0538299     0.0450516    0.00700636       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00428345    0.00536379     0.008084     0.00485764    0.00123072       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.25658       2.91327       5.49738       3.02452       1.15896        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.14496       2.85855       5.18739       3.02826       1.11321        10     
reco:emtrkmichelid:EmTrackMichelId                  128.706       175.021       229.524       177.24        27.4438        10     
reco:crttag:SingleCRTMatchingProducer             0.00578995    0.00920933     0.0125968    0.00881672    0.00198508       10     
reco:crtreco:TwoCRTMatchingProducer               0.00477761    0.00722245    0.00916648    0.00757773    0.00131234       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.515886      0.764389      0.868401      0.812341      0.103777        10     
reco:pandora2Track:LArPandoraTrackCreation          4.54411       9.81564       18.6479       9.82218       3.82389        10     
reco:pandora2calo:Calorimetry                       2.08234       4.40104       7.94888       4.49212       1.65294        10     
reco:pandora2calonosce:Calorimetry                  1.91468       4.17595       7.52663       4.26114       1.58215        10     
reco:pandora2pid:Chi2ParticleID                   0.00904143     0.0127943     0.0191842     0.0130016    0.00272968       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.040374      0.0554166     0.0717705     0.0564989    0.00918211       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00922805     0.0127949     0.0193829     0.0129135    0.00274886       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.76733       3.6469        6.36869       3.83392       1.26553        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.78111       3.84558       6.48367       3.96523       1.35095        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.65943       3.48002       5.94777       3.59822       1.18867        10     
[art]:TriggerResults:TriggerResultInserter        2.5198e-05    3.43609e-05   9.6412e-05    2.7522e-05    2.07497e-05      10     
end_path:out1:RootOutput                           4.589e-06    5.5994e-06     9.127e-06    5.2245e-06    1.21432e-06      10     
end_path:out1:RootOutput(write)                     1.98721       2.64963       3.47894       2.66498      0.405266        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4992.93 MB
  Peak resident set size usage (VmHWM): 4181.2 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.21 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.15 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10050688
-rw-r--r-- 1 dune01 dune01 2369440022 Jan 27 10:34 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2256765088 Jan 27 10:32 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2020782148 Jan 27 11:28 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1821535103 Jan 27 08:41 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1755368028 Jan 27 08:30 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   67381730 Jan 27 11:25 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     243027 Jan 27 11:28 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Jan 27 11:28 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Jan 27 11:28 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Jan 27 11:28 time.db
-rw-r--r-- 1 dune01 dune01        519 Jan 27 10:34 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Jan 27 08:41 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Jan 27 11:28 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Jan 27 08:25 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Jan 27 11:28 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Jan 27 10:38 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Jan 27 10:34 debugprod.log
justIN time: 2026-02-04 19:29:20 UTC       justIN version: 01.06.00