justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 255111.0@dunegpschedd01.fnal.gov

Jobsub ID255111.0@dunegpschedd01.fnal.gov
Workflow ID10491
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-21 21:05:48
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc03
Last heartbeat2025-11-22 00:14:19
From worker nodeHostnamewn-da-18.gina.surf.nl
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-21 21:06:27
Input filesmonte-carlo-010491-000001
JobscriptExit code0
Real time3h (11133s)
CPU time3h (10994s = 98%)
Max RSS bytes4193468416 (3999 MiB)
Outputting started2025-11-22 00:12:01
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10491/1/001/reco_stage1.root
Finished2025-11-22 00:14:19
Saved logsjustin-logs:255111.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

AllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 135663 traces, 0 summary
Retagger: tagging trace set: gauss with 100084 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 100084 traces tagged "gauss"
FrameSaver: q=3.25766e+06 n=1710143 tag=gauss
wclsFrameSaver: saving 135663 traces tagged "wiener"
FrameSaver: q=3.45755e+06 n=1482770 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 28519 XUs and 32038 XVs -> 265 XUVs
C:0 T:1 65304 XUs and 62931 XVs -> 5057 XUVs
C:0 T:2 112282 XUs and 95928 XVs -> 5804 XUVs
C:0 T:3 50880 XUs and 38228 XVs -> 336 XUVs
C:0 T:4 11093 XUs and 14807 XVs -> 88 XUVs
C:0 T:5 51562 XUs and 58745 XVs -> 5514 XUVs
C:0 T:6 109321 XUs and 105395 XVs -> 4551 XUVs
C:0 T:7 41055 XUs and 37780 XVs -> 350 XUVs
C:0 T:8 7746 XUs and 6826 XVs -> 60 XUVs
C:0 T:9 60574 XUs and 72880 XVs -> 4859 XUVs
C:0 T:10 39035 XUs and 16365 XVs -> 3582 XUVs
C:0 T:11 45 XUs and 24 XVs -> 4 XUVs
30470 XUVs total
13508 collection wire objects
27916 potential space points
Neighbour search...
1260876 tests to find 683502 neighbours
Iterating with no regularization...
Begin: 4.61597e+07
0 3.7043e+07
1 3.63083e+07
2 3.61937e+07
3 3.61595e+07
Now with regularization...
Begin: 2.74784e+07
0 2.74479e+07
1 2.74337e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410657104
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410657104
fLifetime: 30000 [us]
22-Nov-2025 01:11:59 CET  Closed output file "reco_stage1.root"
22-Nov-2025 01:11:59 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          270.49        344.972       436.314       363.384       57.2031        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000808275   0.00386486    0.00648344     0.0041246    0.00173862       10     
reco:ophit:OpHitFinder                             0.199194      0.249944      0.285786      0.254628      0.0255552       10     
reco:opflash:OpFlashFinder                          0.10099      0.149109      0.187902      0.147641      0.0252415       10     
reco:opslicer:OpSlicer                             0.454575      0.604308      0.764964      0.626426      0.0904521       10     
reco:caldata:DataPrepModule                         4.46334       4.70914       5.0771        4.67228      0.159828        10     
reco:wclsdatasp:WireCellToolkit                     35.0352       59.0326       72.8478       67.5981       15.2972        10     
reco:gaushit:GausHitFinder                         0.776539       1.07502       1.34408       1.09899      0.202257        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000505168   0.000685961   0.000814486   0.000723257   0.000100812      10     
reco:reco3d:SpacePointSolver                        5.97173       11.041        23.3953       10.2599       4.73723        10     
reco:hitpdune:DisambigFromSpacePoints              0.828115       1.2306        1.70845       1.22011       0.28978        10     
reco:pandora:StandardPandora                        14.1451       26.5521       39.5245       26.3202       7.92501        10     
reco:pandoraWriter:StandardPandora                 0.284655      0.365896      0.437074      0.380073      0.0577479       10     
reco:pandoraTrack:LArPandoraTrackCreation           2.10647       3.42846       5.70201       3.60742       1.03032        10     
reco:pandoraShower:LArPandoraShowerCreation         2.11332       4.0523        6.96286       4.06738       1.61429        10     
reco:pandoracalo:Calorimetry                        1.18887       1.94157       3.15058       2.09623      0.589566        10     
reco:pandoracalonosce:Calorimetry                   1.0715        1.79829       2.93653       1.93462      0.565488        10     
reco:pandorapid:Chi2ParticleID                    0.00393863    0.00508189    0.00624725    0.00496585    0.00068741       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.033202      0.0435998     0.0501798     0.0455436    0.00572728       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0039874    0.00487318    0.00636477    0.00475484     0.0006203       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.01829       4.16041       7.01836       4.34968       1.68547        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.90845       3.97791       6.60621       4.19666       1.62166        10     
reco:emtrkmichelid:EmTrackMichelId                  140.677       181.93        218.825       186.726       26.8592        10     
reco:crttag:SingleCRTMatchingProducer             0.00760779     0.010618      0.0151499     0.0105115    0.00232345       10     
reco:crtreco:TwoCRTMatchingProducer               0.00668603    0.00926773     0.014471     0.00915432    0.00219728       10     
reco:anodepiercerst0:T0RecoAnodePiercers            0.61314      0.758957       0.9169       0.730091      0.0922914       10     
reco:pandora2Track:LArPandoraTrackCreation          6.12205       11.3008       18.5681       11.7342       3.98725        10     
reco:pandora2calo:Calorimetry                       2.96075       5.58948       9.10391       6.04904       2.06594        10     
reco:pandora2calonosce:Calorimetry                  2.7994        5.26865       8.52956       5.69214       1.95094        10     
reco:pandora2pid:Chi2ParticleID                   0.00922909     0.0123952     0.0165229     0.012457     0.00212555       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0425208     0.0551237     0.066936      0.0557704    0.00780992       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0090534     0.0122931     0.0155742     0.0124288    0.00202595       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.34508       4.1154        6.21231       4.28233       1.33568        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.57742       4.67335       6.90419       5.12006       1.64533        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.28557       4.14296       6.1061        4.56399       1.40554        10     
[art]:TriggerResults:TriggerResultInserter        2.9316e-05    3.91735e-05   9.8408e-05    3.25265e-05   1.99455e-05      10     
end_path:out1:RootOutput                           5.861e-06    8.6055e-06    2.3535e-05     6.623e-06    5.20934e-06      10     
end_path:out1:RootOutput(write)                     1.95718       2.54072       3.07968       2.54603      0.372112        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4998.65 MB
  Peak resident set size usage (VmHWM): 4193.47 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.20 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.90 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9658040
-rw-r--r--. 1 dune009 dune 2290017459 Nov 22 00:12 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2177608922 Nov 22 00:10 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 1954120689 Nov 22 01:11 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1734755720 Nov 21 22:23 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1663063320 Nov 21 22:12 g4_stage1.root
-rw-r--r--. 1 dune009 dune   69650040 Nov 22 01:07 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     242611 Nov 22 01:12 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Nov 22 01:11 mem.db
-rw-r--r--. 1 dune009 dune      62376 Nov 22 01:12 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Nov 22 01:11 time.db
-rw-r--r--. 1 dune009 dune        519 Nov 22 00:12 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov 21 22:23 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov 22 01:12 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Nov 21 22:06 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Nov 22 01:12 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Nov 22 00:17 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Nov 22 00:12 debugprod.log
justIN time: 2026-02-08 22:58:18 UTC       justIN version: 01.06.00