justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 263483.1@dunegpschedd01.fnal.gov

Jobsub ID263483.1@dunegpschedd01.fnal.gov
Workflow ID10996
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-04 16:45:43
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc02
Last heartbeat2025-12-04 20:45:00
From worker nodeHostnamewn-lb-10.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-04 16:47:13
Input filesmonte-carlo-010996-000001
JobscriptExit code0
Real time3h (14168s)
CPU time3h (14036s = 99%)
Max RSS bytes4665028608 (4448 MiB)
Outputting started2025-12-04 20:43:22
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10996/1/001/reco_stage1.root
Finished2025-12-04 20:45:00
Saved logsjustin-logs:263483.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 96544 traces, 0 summary
Retagger: tagging trace set: gauss with 74342 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 74342 traces tagged "gauss"
FrameSaver: q=2.36906e+06 n=1297671 tag=gauss
wclsFrameSaver: saving 96544 traces tagged "wiener"
FrameSaver: q=2.5039e+06 n=1143775 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 23323 XUs and 22296 XVs -> 120 XUVs
C:0 T:1 42454 XUs and 38472 XVs -> 2958 XUVs
C:0 T:2 100202 XUs and 132317 XVs -> 8002 XUVs
C:0 T:3 48282 XUs and 52985 XVs -> 477 XUVs
C:0 T:4 6751 XUs and 5719 XVs -> 32 XUVs
C:0 T:5 28671 XUs and 18055 XVs -> 2731 XUVs
C:0 T:6 62763 XUs and 66303 XVs -> 2273 XUVs
C:0 T:7 27177 XUs and 26997 XVs -> 217 XUVs
C:0 T:8 5806 XUs and 4907 XVs -> 54 XUVs
C:0 T:9 33986 XUs and 54994 XVs -> 3583 XUVs
C:0 T:10 25219 XUs and 7645 XVs -> 2191 XUVs
C:0 T:11 58 XUs and 24 XVs -> 8 XUVs
22646 XUVs total
9306 collection wire objects
21009 potential space points
Neighbour search...
1394495 tests to find 734960 neighbours
Iterating with no regularization...
Begin: 3.83192e+07
0 2.84123e+07
1 2.79457e+07
2 2.78714e+07
3 2.78467e+07
Now with regularization...
Begin: 2.19439e+07
0 2.19373e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410913670
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410913670
fLifetime: 30000 [us]
04-Dec-2025 21:43:20 CET  Closed output file "reco_stage1.root"
04-Dec-2025 21:43:20 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          234.794       384.63        509.011       402.962       93.8087        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00100792    0.00564205    0.00920604    0.00601417    0.00289891       10     
reco:ophit:OpHitFinder                             0.267082      0.361477      0.467353      0.369146      0.0571078       10     
reco:opflash:OpFlashFinder                         0.163237      0.255657      0.342487      0.258218      0.0490931       10     
reco:opslicer:OpSlicer                             0.457628      0.727209      0.979618      0.768634      0.133916        10     
reco:caldata:DataPrepModule                         4.73975       5.54328       6.46107       5.18962      0.732175        10     
reco:wclsdatasp:WireCellToolkit                     33.0051       52.6551       67.5258       50.4568       11.7239        10     
reco:gaushit:GausHitFinder                         0.761405       1.16055       1.53406       1.17996      0.250875        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000325369   0.000500377   0.000932703   0.000454058   0.000169093      10     
reco:reco3d:SpacePointSolver                        8.73282       16.3558       27.2066       17.6557       5.94236        10     
reco:hitpdune:DisambigFromSpacePoints              0.707681       1.54595       2.2053        1.61026      0.505731        10     
reco:pandora:StandardPandora                        14.0251       32.5947       50.902        33.1257       11.9644        10     
reco:pandoraWriter:StandardPandora                 0.298684      0.450481      0.572876       0.48368      0.0985977       10     
reco:pandoraTrack:LArPandoraTrackCreation           2.64097       5.88364       11.1509       5.8037        2.40195        10     
reco:pandoraShower:LArPandoraShowerCreation         2.18606       6.85152       12.2805       6.59695       3.12454        10     
reco:pandoracalo:Calorimetry                        1.33843       3.18381       6.43674       3.01061       1.4057         10     
reco:pandoracalonosce:Calorimetry                   1.20522       2.80112       4.84297       2.84053       1.06669        10     
reco:pandorapid:Chi2ParticleID                    0.00395965    0.00535003    0.00679496    0.00508425    0.000904867      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0310933     0.0474341     0.0609322     0.0513931     0.0107503       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00401962    0.00518493    0.00686822    0.00504818    0.000835529      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.16766       6.3995        11.4345       6.62089       2.92558        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.10872       6.14001       10.1707       6.45035       2.68962        10     
reco:emtrkmichelid:EmTrackMichelId                  121.741       184.965       237.496       198.958       39.5752        10     
reco:crttag:SingleCRTMatchingProducer             0.00993178     0.0162739     0.0217071     0.017118     0.00381044       10     
reco:crtreco:TwoCRTMatchingProducer                0.0068445     0.0122389     0.0171603     0.0130031    0.00364294       10     
reco:anodepiercerst0:T0RecoAnodePiercers            0.53614      0.907933       1.2213       0.943865      0.223356        10     
reco:pandora2Track:LArPandoraTrackCreation          8.65402       17.178        26.3189       16.4533       6.21696        10     
reco:pandora2calo:Calorimetry                       3.75654       8.39342       13.2281       8.5027        3.31632        10     
reco:pandora2calonosce:Calorimetry                  3.55427       7.88107       12.1883       8.15735       3.07408        10     
reco:pandora2pid:Chi2ParticleID                   0.00981297     0.0120104     0.0154805     0.0114995    0.00189234       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0448963     0.0603609     0.0781138     0.0602373     0.0108436       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.009749      0.0120464     0.0156206     0.0114833    0.00193911       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.93579       5.96179       9.01804       6.08653       1.98158        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.28658       7.09378       11.1661       7.25965       2.83374        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.77454       5.97762       9.3154        6.58196       2.25554        10     
[art]:TriggerResults:TriggerResultInserter        3.2508e-05    5.06271e-05   0.000127822   4.28645e-05   2.63612e-05      10     
end_path:out1:RootOutput                           6.71e-06     8.2104e-06    1.1538e-05     7.942e-06    1.28221e-06      10     
end_path:out1:RootOutput(write)                     1.90325       2.9712        4.07296       2.96483      0.670365        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5468.63 MB
  Peak resident set size usage (VmHWM): 4665.03 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.24 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.45 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10844716
-rw-r--r--. 1 dune009 dune 2527036834 Dec  4 20:37 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2400054575 Dec  4 20:35 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2216114977 Dec  4 21:43 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1983966925 Dec  4 18:05 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1901311001 Dec  4 17:53 g4_stage1.root
-rw-r--r--. 1 dune009 dune   75893530 Dec  4 21:40 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     242575 Dec  4 21:43 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Dec  4 21:43 mem.db
-rw-r--r--. 1 dune009 dune      62376 Dec  4 21:43 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Dec  4 21:43 time.db
-rw-r--r--. 1 dune009 dune        519 Dec  4 20:37 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Dec  4 18:05 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Dec  4 21:43 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Dec  4 17:47 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Dec  4 21:43 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Dec  4 20:43 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Dec  4 20:37 debugprod.log
justIN time: 2025-12-18 18:07:35 UTC       justIN version: 01.05.03