justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 233776.0@dunegpschedd02.fnal.gov

Jobsub ID233776.0@dunegpschedd02.fnal.gov
Workflow ID9415
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-28 15:34:19
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc02
Last heartbeat2025-10-28 19:58:52
From worker nodeHostnamewn-da-13.gina.surf.nl
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-28 15:35:41
Input filesmonte-carlo-009415-000001
JobscriptExit code0
Real time4h (15696s)
CPU time4h (15495s = 98%)
Max RSS bytes4313968640 (4114 MiB)
Outputting started2025-10-28 19:57:18
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09415/1/001/reco_stage1.root
Finished2025-10-28 19:58:52
Saved logsjustin-logs:233776.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ing tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 128911 traces, 0 summary
Retagger: tagging trace set: gauss with 99692 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 99692 traces tagged "gauss"
FrameSaver: q=3.4427e+06 n=1759616 tag=gauss
wclsFrameSaver: saving 128911 traces tagged "wiener"
FrameSaver: q=3.6301e+06 n=1559807 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 20517 XUs and 17147 XVs -> 120 XUVs
C:0 T:1 33393 XUs and 27597 XVs -> 1691 XUVs
C:0 T:2 116445 XUs and 169295 XVs -> 6593 XUVs
C:0 T:3 53918 XUs and 65117 XVs -> 546 XUVs
C:0 T:4 7331 XUs and 5194 XVs -> 82 XUVs
C:0 T:5 30359 XUs and 12401 XVs -> 1564 XUVs
C:0 T:6 149489 XUs and 136460 XVs -> 7219 XUVs
C:0 T:7 57049 XUs and 48120 XVs -> 627 XUVs
C:0 T:8 7719 XUs and 8460 XVs -> 84 XUVs
C:0 T:9 45337 XUs and 60478 XVs -> 5415 XUVs
C:0 T:10 42126 XUs and 21725 XVs -> 4777 XUVs
C:0 T:11 123 XUs and 85 XVs -> 9 XUVs
28727 XUVs total
11139 collection wire objects
26649 potential space points
Neighbour search...
1446557 tests to find 797686 neighbours
Iterating with no regularization...
Begin: 8.18056e+07
0 7.15642e+07
1 7.09589e+07
2 7.08759e+07
3 7.08514e+07
Now with regularization...
Begin: 6.06945e+07
0 6.06741e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410169636
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410169636
fLifetime: 30000 [us]
28-Oct-2025 20:57:16 CET  Closed output file "reco_stage1.root"
28-Oct-2025 20:57:16 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          382.37        487.191       607.687       502.105       71.5352        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00100745    0.00554735    0.00995986    0.00573637    0.00283668       10     
reco:ophit:OpHitFinder                             0.285946      0.336021      0.373201      0.341349      0.0258582       10     
reco:opflash:OpFlashFinder                         0.179639      0.238051      0.275171      0.243754      0.0286031       10     
reco:opslicer:OpSlicer                             0.713687      0.843432       1.01948      0.831179      0.0839661       10     
reco:caldata:DataPrepModule                         5.16883       5.3542        5.62307       5.29708      0.153298        10     
reco:wclsdatasp:WireCellToolkit                     39.2057       54.4658       64.6307       58.2992       8.24691        10     
reco:gaushit:GausHitFinder                          1.20474       1.50821       1.84098       1.52503      0.207616        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000547512   0.000810866   0.00099819    0.00080962    0.000121566      10     
reco:reco3d:SpacePointSolver                        11.5102       17.3464       23.4965       16.7322       4.07845        10     
reco:hitpdune:DisambigFromSpacePoints               1.38088       1.99904       2.6762        2.0264       0.409768        10     
reco:pandora:StandardPandora                        25.1753       48.3088       83.8734       47.7736       15.2268        10     
reco:pandoraWriter:StandardPandora                  0.40818       0.50069      0.598919      0.500672      0.0604147       10     
reco:pandoraTrack:LArPandoraTrackCreation           4.12774       6.32369       8.16226       6.60558       1.45308        10     
reco:pandoraShower:LArPandoraShowerCreation         5.26819       7.71883       11.7519       7.3156        2.16838        10     
reco:pandoracalo:Calorimetry                        2.27993       3.35448       4.3835         3.442       0.609889        10     
reco:pandoracalonosce:Calorimetry                   2.14439       3.18034       4.12769       3.36382      0.659225        10     
reco:pandorapid:Chi2ParticleID                    0.00578401    0.00707478    0.00763959    0.00734446    0.000642133      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP          0.04859      0.0607176     0.0741824     0.0604696    0.00730844       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00567235    0.00667383    0.00743861    0.00687421    0.000569834      10     
reco:pandoraShowercalo:ShowerCalorimetry            4.9389        7.54048       11.2881       7.46709       1.98122        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.87375       7.21329       10.9572       7.12918       1.92548        10     
reco:emtrkmichelid:EmTrackMichelId                  207.541       252.168       305.649       255.379       31.7684        10     
reco:crttag:SingleCRTMatchingProducer              0.0105146     0.0191843     0.0279798     0.0193996    0.00608197       10     
reco:crtreco:TwoCRTMatchingProducer                0.0091513     0.0133805     0.0161121     0.0140153    0.00214369       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.844044       1.04886       1.24548       1.06164      0.140757        10     
reco:pandora2Track:LArPandoraTrackCreation          12.6706       21.0941       31.4344       20.9563       5.61709        10     
reco:pandora2calo:Calorimetry                       6.28245       10.1803       14.5069       10.0547       2.45905        10     
reco:pandora2calonosce:Calorimetry                  5.90358       9.59731       13.6038       9.51715       2.28791        10     
reco:pandora2pid:Chi2ParticleID                    0.013413      0.0176944     0.0221036     0.0174282    0.00280632       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0605243     0.077301      0.0935408     0.0781718     0.0116879       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0140427     0.0173837     0.0206108     0.0176061     0.0024754       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.00681       7.56184       11.2386       7.18004       1.84061        10     
reco:pandora2Showercalo:ShowerCalorimetry           5.5527        8.25989       11.1332       8.22526       1.78478        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.21279       7.09739       9.43889       7.1127        1.40617        10     
[art]:TriggerResults:TriggerResultInserter        3.3243e-05    4.48403e-05   0.00010529    3.8679e-05    2.04054e-05      10     
end_path:out1:RootOutput                           6.452e-06     7.401e-06    1.1493e-05     7.023e-06    1.39549e-06      10     
end_path:out1:RootOutput(write)                     2.85503       3.5472        4.32492       3.55238      0.468059        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5116.37 MB
  Peak resident set size usage (VmHWM): 4313.97 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.36 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11020308
-rw-r--r--. 1 dune009 dune 2547466321 Oct 28 19:33 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2408594149 Oct 28 19:31 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2282353370 Oct 28 20:57 reco_stage1.root
-rw-r--r--. 1 dune009 dune 2026903140 Oct 28 17:01 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1932454651 Oct 28 16:43 g4_stage1.root
-rw-r--r--. 1 dune009 dune   86340560 Oct 28 20:52 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     243012 Oct 28 20:57 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Oct 28 20:57 mem.db
-rw-r--r--. 1 dune009 dune      62376 Oct 28 20:57 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Oct 28 20:57 time.db
-rw-r--r--. 1 dune009 dune        519 Oct 28 19:33 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Oct 28 17:01 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Oct 28 20:57 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Oct 28 16:35 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Oct 28 20:57 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Oct 28 19:40 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Oct 28 19:33 debugprod.log
justIN time: 2025-11-05 17:30:03 UTC       justIN version: 01.05.01