justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 264651.0@dunegpschedd02.fnal.gov

Jobsub ID264651.0@dunegpschedd02.fnal.gov
Workflow ID11371
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-28 08:47:45
SiteIT_CNAF
EntryCMSHTPC_T1_IT_CNAF_condor_ce02
Last heartbeat2025-12-28 11:19:11
From worker nodeHostnamewn-s04-05-09-02.cr.cnaf.infn.it
cpuinfoAMD EPYC 7313 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-28 08:50:06
Input filesmonte-carlo-011371-000001
JobscriptExit code0
Real time2h (8625s)
CPU time2h (8325s = 96%)
Max RSS bytes4518764544 (4309 MiB)
Outputting started2025-12-28 11:13:52
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11371/1/001/reco_stage1.root
Finished2025-12-28 11:19:11
Saved logsjustin-logs:264651.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Service:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 120093 traces, 0 summary
Retagger: tagging trace set: gauss with 93317 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 93317 traces tagged "gauss"
FrameSaver: q=3.18755e+06 n=1654802 tag=gauss
wclsFrameSaver: saving 120093 traces tagged "wiener"
FrameSaver: q=3.33983e+06 n=1464061 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 29328 XUs and 30739 XVs -> 250 XUVs
C:0 T:1 64513 XUs and 62786 XVs -> 4053 XUVs
C:0 T:2 67328 XUs and 103266 XVs -> 3461 XUVs
C:0 T:3 33694 XUs and 43917 XVs -> 120 XUVs
C:0 T:4 11516 XUs and 10215 XVs -> 92 XUVs
C:0 T:5 47271 XUs and 28348 XVs -> 3712 XUVs
C:0 T:6 56532 XUs and 68860 XVs -> 2509 XUVs
C:0 T:7 26772 XUs and 29452 XVs -> 177 XUVs
C:0 T:8 8917 XUs and 5762 XVs -> 84 XUVs
C:0 T:9 46043 XUs and 42547 XVs -> 2525 XUVs
C:0 T:10 40244 XUs and 32246 XVs -> 5013 XUVs
C:0 T:11 340 XUs and 196 XVs -> 41 XUVs
22037 XUVs total
10475 collection wire objects
20494 potential space points
Neighbour search...
796272 tests to find 435756 neighbours
Iterating with no regularization...
Begin: 6.94953e+07
0 6.079e+07
1 6.01661e+07
2 6.00659e+07
3 6.00346e+07
Now with regularization...
Begin: 5.08414e+07
0 5.08159e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411391147
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411391147
fLifetime: 30000 [us]
28-Dec-2025 12:13:51 CET  Closed output file "reco_stage1.root"
28-Dec-2025 12:13:51 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          179.737       245.976       317.077       260.373       40.594         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000617101   0.00279801    0.00483379    0.00308326    0.00128816       10     
reco:ophit:OpHitFinder                             0.201395      0.230409      0.272357      0.228462      0.0182575       10     
reco:opflash:OpFlashFinder                         0.142143      0.168754      0.206044      0.166996      0.0203998       10     
reco:opslicer:OpSlicer                             0.388134      0.480768      0.551044      0.487995      0.0522139       10     
reco:caldata:DataPrepModule                         3.30918       3.59485       4.05175       3.56082      0.183693        10     
reco:wclsdatasp:WireCellToolkit                     23.6639       24.3549       25.0365       24.3809      0.455809        10     
reco:gaushit:GausHitFinder                         0.716574      0.872313       1.08274      0.858402      0.112152        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000311186   0.000412797   0.000532231   0.000402672   6.82986e-05      10     
reco:reco3d:SpacePointSolver                        4.02121       13.1858       21.7452       13.5626       5.36045        10     
reco:hitpdune:DisambigFromSpacePoints              0.894037       1.30494       1.88789       1.31529       0.29183        10     
reco:pandora:StandardPandora                        15.429        26.3942       42.0765       27.9609       8.29235        10     
reco:pandoraWriter:StandardPandora                 0.219761       0.30695      0.423437      0.307367      0.0570892       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.57863       2.6643        3.55501       2.72851      0.644828        10     
reco:pandoraShower:LArPandoraShowerCreation         1.83067       3.05807       4.67161       3.17103      0.798775        10     
reco:pandoracalo:Calorimetry                        1.00619       1.60635       2.04504       1.64447      0.343662        10     
reco:pandoracalonosce:Calorimetry                  0.918734       1.49498       1.91766       1.54986      0.329662        10     
reco:pandorapid:Chi2ParticleID                    0.00280915    0.00382036     0.0050408    0.00377798    0.000596201      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0283498     0.0362016     0.0459103     0.0371535    0.00514638       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00271385    0.00350029    0.00423891     0.0034838    0.000449976      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.11492       3.51623       5.34143       3.62141      0.919001        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.06825       3.42107       5.17886       3.5517       0.887102        10     
reco:emtrkmichelid:EmTrackMichelId                  99.7668       126.833       155.595       132.417       16.5429        10     
reco:crttag:SingleCRTMatchingProducer              0.0075644     0.0101337     0.0154194    0.00980543    0.00229108       10     
reco:crtreco:TwoCRTMatchingProducer               0.00459296    0.00569108    0.00666626    0.00579565    0.000621856      10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.546242      0.670422       0.81281      0.669721      0.086727        10     
reco:pandora2Track:LArPandoraTrackCreation          5.3007        8.82745       12.9934       9.4259        2.22344        10     
reco:pandora2calo:Calorimetry                       2.99157       4.82788       6.80893       5.14415       1.13511        10     
reco:pandora2calonosce:Calorimetry                  2.84748       4.60047       6.4772        4.9154        1.07111        10     
reco:pandora2pid:Chi2ParticleID                   0.00685627     0.0096557     0.0147679    0.00960962    0.00214539       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0341391     0.0481221     0.0825637     0.0474926     0.0126952       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00685125    0.00958664     0.0163089    0.00942509    0.00251805       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.97182       3.18208       4.6046        3.39603      0.739781        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.67751       4.18824       5.94537       4.48827      0.935425        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.46869       3.83558       5.38223        4.109       0.832234        10     
[art]:TriggerResults:TriggerResultInserter         3.179e-05    5.12995e-05   0.000105228   4.2425e-05    2.08344e-05      10     
end_path:out1:RootOutput                           5.951e-06    7.2054e-06    1.2844e-05    6.5525e-06    1.93759e-06      10     
end_path:out1:RootOutput(write)                     1.64838       2.15751       2.69531       2.23404      0.307462        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5325.44 MB
  Peak resident set size usage (VmHWM): 4518.76 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.23 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11005108
-rw-r--r-- 1 neutrino002 neutrino 2556338590 Dec 28 11:31 detsim_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 2424515670 Dec 28 11:30 detsim_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 2290153954 Dec 28 12:13 reco_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 1994124614 Dec 28 10:06 g4_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 1917549035 Dec 28 09:57 g4_stage1.root
-rw-r--r-- 1 neutrino002 neutrino   85919480 Dec 28 12:11 Pandora_Events.pndr
-rw-r--r-- 1 neutrino002 neutrino     243147 Dec 28 12:13 jobscript.log
-rw-r--r-- 1 neutrino002 neutrino     217088 Dec 28 12:13 mem.db
-rw-r--r-- 1 neutrino002 neutrino      62376 Dec 28 12:13 Pandora_Geometry.xml
-rw-r--r-- 1 neutrino002 neutrino      36864 Dec 28 12:13 time.db
-rw-r--r-- 1 neutrino002 neutrino        519 Dec 28 11:31 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Dec 28 10:06 g4_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Dec 28 12:13 protoDUNE_reco1_hist.root
-rw-r--r-- 1 neutrino002 neutrino         52 Dec 28 09:50 all-input-dids.txt
-rw-r--r-- 1 neutrino002 neutrino          7 Dec 28 12:13 justin-processed-pfns.txt
-rw-r--r-- 1 neutrino002 neutrino          0 Dec 28 11:35 ProtoDUNE.log
-rw-r--r-- 1 neutrino002 neutrino          0 Dec 28 11:31 debugprod.log
justIN time: 2026-02-07 03:35:08 UTC       justIN version: 01.06.00