justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 253731.0@dunegpschedd01.fnal.gov

Jobsub ID253731.0@dunegpschedd01.fnal.gov
Workflow ID10334
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-18 15:27:52
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2025-11-18 21:10:28
From worker nodeHostnamewn1207080.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-18 15:28:51
Input filesmonte-carlo-010334-000001
JobscriptExit code0
Real time5h (20396s)
CPU time5h (19714s = 96%)
Max RSS bytes4046725120 (3859 MiB)
Outputting started2025-11-18 21:08:48
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10334/1/001/reco_stage1.root
Finished2025-11-18 21:10:28
Saved logsjustin-logs:253731.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 138436 traces, 0 summary
Retagger: tagging trace set: gauss with 106433 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 106433 traces tagged "gauss"
FrameSaver: q=3.39167e+06 n=1886058 tag=gauss
wclsFrameSaver: saving 138436 traces tagged "wiener"
FrameSaver: q=3.57076e+06 n=1662872 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 27830 XUs and 33392 XVs -> 268 XUVs
C:0 T:1 56553 XUs and 61124 XVs -> 6089 XUVs
C:0 T:2 100730 XUs and 142926 XVs -> 6346 XUVs
C:0 T:3 44855 XUs and 50698 XVs -> 345 XUVs
C:0 T:4 7522 XUs and 7025 XVs -> 29 XUVs
C:0 T:5 25133 XUs and 17229 XVs -> 2099 XUVs
C:0 T:6 162027 XUs and 142940 XVs -> 6839 XUVs
C:0 T:7 57440 XUs and 50170 XVs -> 427 XUVs
C:0 T:8 6862 XUs and 9692 XVs -> 121 XUVs
C:0 T:9 44985 XUs and 76520 XVs -> 3957 XUVs
C:0 T:10 52930 XUs and 28400 XVs -> 4791 XUVs
C:0 T:11 24 XUs and 43 XVs -> 8 XUVs
31319 XUVs total
13604 collection wire objects
28950 potential space points
Neighbour search...
1924102 tests to find 1016228 neighbours
Iterating with no regularization...
Begin: 5.93576e+07
0 4.81236e+07
1 4.75129e+07
2 4.74494e+07
3 4.74328e+07
Now with regularization...
Begin: 3.90989e+07
0 3.90793e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410591934
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410591934
fLifetime: 30000 [us]
18-Nov-2025 21:08:45 GMT  Closed output file "reco_stage1.root"
18-Nov-2025 21:08:45 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          435.077       705.825       951.353       730.971       168.086        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00178618    0.00514622    0.00836065    0.00570808    0.00206264       10     
reco:ophit:OpHitFinder                             0.316523      0.419193      0.484745      0.435419      0.0532825       10     
reco:opflash:OpFlashFinder                         0.177736       0.23606       0.3006       0.224294      0.0344985       10     
reco:opslicer:OpSlicer                             0.658614       1.07618       1.38047       1.11756      0.215499        10     
reco:caldata:DataPrepModule                         6.23429       8.42977       9.94964       8.87659       1.24097        10     
reco:wclsdatasp:WireCellToolkit                     44.4856       67.2287       86.4122       69.8736       11.911         10     
reco:gaushit:GausHitFinder                          1.08299       1.9405        2.75243       1.81559      0.475191        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00107977    0.00153239    0.00212257    0.00153533    0.000363307      10     
reco:reco3d:SpacePointSolver                        12.4706       24.3912       51.3545       23.9557       10.2386        10     
reco:hitpdune:DisambigFromSpacePoints               1.31805       2.60737       3.98126       2.64513      0.852461        10     
reco:pandora:StandardPandora                        26.2445       58.2365       115.256       53.1636       23.335         10     
reco:pandoraWriter:StandardPandora                 0.676382       0.8407        1.02286      0.852366      0.120955        10     
reco:pandoraTrack:LArPandoraTrackCreation           5.06266       10.8678       15.9201       11.0533       3.25827        10     
reco:pandoraShower:LArPandoraShowerCreation         6.75124       12.3175       17.7945       11.5173       3.55886        10     
reco:pandoracalo:Calorimetry                        3.35197       6.30024       10.7654       6.51458       2.32099        10     
reco:pandoracalonosce:Calorimetry                   2.32321       5.64151       7.95498       5.94982       1.97163        10     
reco:pandorapid:Chi2ParticleID                    0.00609711    0.00886915     0.0126333    0.00848201    0.00191771       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0455168     0.0853389     0.130191      0.082034      0.0238947       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00573362    0.00883829     0.0140328    0.00810515    0.00223087       10     
reco:pandoraShowercalo:ShowerCalorimetry            4.79371       13.0664       23.948        13.841        5.38606        10     
reco:pandoraShowercalonosce:ShowerCalorimetry        4.059        12.729        23.6294       13.5133       5.42933        10     
reco:emtrkmichelid:EmTrackMichelId                  248.772       361.197       473.11        368.526       71.5782        10     
reco:crttag:SingleCRTMatchingProducer              0.0154344     0.0356315     0.0501386     0.0375772    0.00989026       10     
reco:crtreco:TwoCRTMatchingProducer                0.0106272     0.0288076     0.0392089     0.034238      0.0104833       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.911246       1.45909       1.86266       1.60679      0.324312        10     
reco:pandora2Track:LArPandoraTrackCreation          11.1418       33.2282       52.9436       37.0234       13.1402        10     
reco:pandora2calo:Calorimetry                       5.29088       18.0294       32.0908       18.7648       8.96954        10     
reco:pandora2calonosce:Calorimetry                  4.37624       17.913        36.0051       19.0474       9.69096        10     
reco:pandora2pid:Chi2ParticleID                    0.0121479     0.0219489     0.0310084     0.0221199     0.0056773       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0560772     0.112263      0.166521       0.11439      0.0378947       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0119536     0.0223542     0.0338257     0.0221967    0.00660117       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.03333       12.7598       22.4573       13.5762       5.44862        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.54722       15.9542       27.3064       17.3123       8.28013        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.54727       14.0351       23.9956       15.326        7.31012        10     
[art]:TriggerResults:TriggerResultInserter        3.4425e-05    6.13163e-05   0.000143801   5.5189e-05    2.94907e-05      10     
end_path:out1:RootOutput                           7.004e-06    9.3308e-06    1.2784e-05    8.5315e-06    1.93583e-06      10     
end_path:out1:RootOutput(write)                     2.99776       4.28715       5.00382       4.6172       0.680817        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4891.85 MB
  Peak resident set size usage (VmHWM): 4046.73 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.38 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.18 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.20 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9348572
-rw-r--r-- 1 dune01 dune01 2224705393 Nov 18 19:07 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2113820314 Nov 18 19:04 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 1888549199 Nov 18 21:08 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1666010195 Nov 18 15:58 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1609888232 Nov 18 15:38 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   69333350 Nov 18 20:55 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242767 Nov 18 21:08 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Nov 18 21:08 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Nov 18 21:08 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Nov 18 21:08 time.db
-rw-r--r-- 1 dune01 dune01        519 Nov 18 19:07 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 18 15:58 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 18 21:08 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Nov 18 15:28 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Nov 18 21:08 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Nov 18 19:15 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Nov 18 19:07 debugprod.log
justIN time: 2026-02-09 06:20:25 UTC       justIN version: 01.06.00