justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 242132.0@dunegpschedd02.fnal.gov

Jobsub ID242132.0@dunegpschedd02.fnal.gov
Workflow ID9990
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-12 12:22:26
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2025-11-12 17:55:17
From worker nodeHostnamewn1205080.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-12 12:23:55
Input filesmonte-carlo-009990-000001
JobscriptExit code0
Real time5h (19412s)
CPU time5h (18735s = 96%)
Max RSS bytes4010176512 (3824 MiB)
Outputting started2025-11-12 17:47:28
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09990/1/001/reco_stage1.root
Finished2025-11-12 17:55:17
Saved logsjustin-logs:242132.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

unning tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 173673 traces, 0 summary
Retagger: tagging trace set: gauss with 131280 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 131280 traces tagged "gauss"
FrameSaver: q=4.52755e+06 n=2360717 tag=gauss
wclsFrameSaver: saving 173673 traces tagged "wiener"
FrameSaver: q=4.80335e+06 n=2080679 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 28523 XUs and 29393 XVs -> 471 XUVs
C:0 T:1 87310 XUs and 89550 XVs -> 10136 XUVs
C:0 T:2 139014 XUs and 169040 XVs -> 14310 XUVs
C:0 T:3 55055 XUs and 51577 XVs -> 480 XUVs
C:0 T:4 19203 XUs and 16776 XVs -> 288 XUVs
C:0 T:5 101788 XUs and 70913 XVs -> 13540 XUVs
C:0 T:6 237237 XUs and 255667 XVs -> 20553 XUVs
C:0 T:7 77727 XUs and 73144 XVs -> 1165 XUVs
C:0 T:8 12116 XUs and 10487 XVs -> 162 XUVs
C:0 T:9 79564 XUs and 88266 XVs -> 8273 XUVs
C:0 T:10 57606 XUs and 90433 XVs -> 14174 XUVs
C:0 T:11 931 XUs and 740 XVs -> 120 XUVs
83672 XUVs total
16488 collection wire objects
76383 potential space points
Neighbour search...
12638909 tests to find 6069288 neighbours
Iterating with no regularization...
Begin: 9.56474e+07
0 7.61839e+07
1 7.48357e+07
2 7.46239e+07
3 7.45496e+07
Now with regularization...
Begin: 6.20783e+07
0 6.19434e+07
1 6.18571e+07
2 6.17921e+07
3 6.17402e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410468620
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410468620
fLifetime: 30000 [us]
12-Nov-2025 17:47:24 GMT  Closed output file "reco_stage1.root"
12-Nov-2025 17:47:24 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          484.354       680.563       1186.35       608.883       197.905        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00184013    0.00490057    0.00640755    0.00541843    0.00142197       10     
reco:ophit:OpHitFinder                             0.368122      0.423064      0.508082      0.425321      0.0404045       10     
reco:opflash:OpFlashFinder                         0.167916      0.221228      0.322079      0.212964      0.0390782       10     
reco:opslicer:OpSlicer                             0.852827       1.12498       1.3472        1.18202      0.156887        10     
reco:caldata:DataPrepModule                         7.3051        9.15728       9.73214       9.4237       0.752315        10     
reco:wclsdatasp:WireCellToolkit                     52.0234       64.3356       73.539        65.2882       7.1369         10     
reco:gaushit:GausHitFinder                          1.55874       2.05188       3.11407       1.95555      0.424166        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000559885   0.00119734    0.00204275    0.00120397    0.000373681      10     
reco:reco3d:SpacePointSolver                         14.81        22.5894       52.5383       17.9367       11.2051        10     
reco:hitpdune:DisambigFromSpacePoints               1.63001       2.70864       5.67382       2.41713       1.14173        10     
reco:pandora:StandardPandora                        29.3343       55.4454       129.287       50.5232       28.1819        10     
reco:pandoraWriter:StandardPandora                 0.664436      0.856662       1.20892      0.838908      0.152853        10     
reco:pandoraTrack:LArPandoraTrackCreation           4.48023       8.23269       14.2396       8.39501       2.6836         10     
reco:pandoraShower:LArPandoraShowerCreation         5.26817       10.3918       21.6118       9.42062       4.73572        10     
reco:pandoracalo:Calorimetry                        2.50315       4.67072       8.25168       4.39826       1.62378        10     
reco:pandoracalonosce:Calorimetry                   2.43234       4.41342        8.166        4.09066       1.68299        10     
reco:pandorapid:Chi2ParticleID                    0.00705311    0.00990594     0.0127761    0.00957347     0.0017049       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0694725     0.0953786     0.120959      0.0957884     0.0182102       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00752486    0.00957082     0.0127665     0.009398     0.00149821       10     
reco:pandoraShowercalo:ShowerCalorimetry            5.3632        11.2335       23.2837       9.34809       5.48601        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.82627       11.2425       20.3279       9.28474       5.51239        10     
reco:emtrkmichelid:EmTrackMichelId                  281.156       371.852       598.25        352.834       87.9468        10     
reco:crttag:SingleCRTMatchingProducer              0.0191928     0.032829      0.0453014     0.0326407    0.00945135       10     
reco:crtreco:TwoCRTMatchingProducer                0.0158084     0.0254312     0.0415945     0.0243486     0.0077569       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.06671       1.4859        2.41078       1.38612      0.394837        10     
reco:pandora2Track:LArPandoraTrackCreation          17.6472       28.8869       67.177        22.7821       14.3616        10     
reco:pandora2calo:Calorimetry                       8.52647       15.1841       39.4719       10.7559       8.9584         10     
reco:pandora2calonosce:Calorimetry                  7.68357       14.759        38.0129       9.36641       9.18591        10     
reco:pandora2pid:Chi2ParticleID                    0.0164668     0.0241004     0.0358563     0.021781     0.00557246       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0749566      0.12417      0.172269      0.120415      0.0287131       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0160098     0.0244378     0.0345823     0.0232068    0.00527729       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.97231       10.0135       21.1828       7.81325       4.3968         10     
reco:pandora2Showercalo:ShowerCalorimetry           7.04851       12.5838       29.4798       8.63812       6.78256        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      6.04827       11.4747       26.999        7.84162       6.24378        10     
[art]:TriggerResults:TriggerResultInserter        4.6748e-05    6.50186e-05   0.000117091   5.6722e-05    1.95608e-05      10     
end_path:out1:RootOutput                           7.705e-06    8.8329e-06    1.2945e-05     8.471e-06    1.43717e-06      10     
end_path:out1:RootOutput(write)                     3.7473        4.63391       7.59541       4.17982       1.09459        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4855.39 MB
  Peak resident set size usage (VmHWM): 4010.18 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.39 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.79 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.24 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9200044
-rw-r--r-- 1 dune01 dune01 2200875698 Nov 12 15:50 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2095190956 Nov 12 15:47 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 1856145817 Nov 12 17:47 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1634302044 Nov 12 12:51 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1566107295 Nov 12 12:32 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   67593480 Nov 12 17:31 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242635 Nov 12 17:47 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Nov 12 17:47 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Nov 12 17:47 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Nov 12 17:47 time.db
-rw-r--r-- 1 dune01 dune01        519 Nov 12 15:50 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 12 12:51 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 12 17:47 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Nov 12 12:23 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Nov 12 17:47 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Nov 12 16:01 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Nov 12 15:50 debugprod.log
justIN time: 2026-02-10 01:28:23 UTC       justIN version: 01.06.00