justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 239647.1@dunegpschedd01.fnal.gov

Jobsub ID239647.1@dunegpschedd01.fnal.gov
Workflow ID9429
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-28 23:31:44
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc02
Last heartbeat2025-10-29 03:01:51
From worker nodeHostnamewn-la-14.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-28 23:32:47
Input filesmonte-carlo-009429-000001
JobscriptExit code0
Real time3h (12431s)
CPU time3h (12255s = 98%)
Max RSS bytes4351770624 (4150 MiB)
Outputting started2025-10-29 02:59:59
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09429/1/001/reco_stage1.root
Finished2025-10-29 03:01:51
Saved logsjustin-logs:239647.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 148466 traces, 0 summary
Retagger: tagging trace set: gauss with 112404 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 112404 traces tagged "gauss"
FrameSaver: q=3.92741e+06 n=1986948 tag=gauss
wclsFrameSaver: saving 148466 traces tagged "wiener"
FrameSaver: q=4.13541e+06 n=1750363 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 46722 XUs and 40859 XVs -> 888 XUVs
C:0 T:1 121204 XUs and 111899 XVs -> 18977 XUVs
C:0 T:2 89022 XUs and 147547 XVs -> 4187 XUVs
C:0 T:3 45047 XUs and 62646 XVs -> 350 XUVs
C:0 T:4 14112 XUs and 13344 XVs -> 149 XUVs
C:0 T:5 61068 XUs and 49019 XVs -> 5674 XUVs
C:0 T:6 91140 XUs and 115348 XVs -> 4471 XUVs
C:0 T:7 35221 XUs and 45627 XVs -> 354 XUVs
C:0 T:8 9055 XUs and 8629 XVs -> 66 XUVs
C:0 T:9 51365 XUs and 65171 XVs -> 3620 XUVs
C:0 T:10 27083 XUs and 20807 XVs -> 3601 XUVs
C:0 T:11 8 XUs and 1 XVs -> 0 XUVs
42337 XUVs total
13476 collection wire objects
38760 potential space points
Neighbour search...
4791792 tests to find 2219010 neighbours
Iterating with no regularization...
Begin: 7.18422e+07
0 6.06943e+07
1 5.99417e+07
2 5.98296e+07
3 5.9798e+07
Now with regularization...
Begin: 4.88614e+07
0 4.88282e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410176334
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410176334
fLifetime: 30000 [us]
29-Oct-2025 03:59:57 CET  Closed output file "reco_stage1.root"
29-Oct-2025 03:59:57 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          188.206       356.48        821.65        324.27        173.609        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000942502   0.00483144    0.00808234    0.00478482    0.00243076       10     
reco:ophit:OpHitFinder                             0.262358      0.310208      0.350754      0.313393      0.0254259       10     
reco:opflash:OpFlashFinder                         0.155517      0.206905      0.246903      0.210891      0.0235713       10     
reco:opslicer:OpSlicer                             0.472269      0.595503      0.673724      0.611297      0.0591883       10     
reco:caldata:DataPrepModule                         4.65834       4.88633       5.32493       4.8295       0.191759        10     
reco:wclsdatasp:WireCellToolkit                     31.7884       37.9293       43.6439       38.3704       3.24067        10     
reco:gaushit:GausHitFinder                          0.63161      0.984551       1.48009        0.975       0.243035        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000251646   0.000390838   0.00067406    0.000397295   0.000117532      10     
reco:reco3d:SpacePointSolver                        6.00323       21.1036       56.6312       15.3467       16.1535        10     
reco:hitpdune:DisambigFromSpacePoints              0.589266       1.33633       2.91666       1.23239      0.629062        10     
reco:pandora:StandardPandora                        9.65724       53.0506       309.336       27.126        85.9366        10     
reco:pandoraWriter:StandardPandora                 0.248423      0.393328      0.605991       0.38931      0.110141        10     
reco:pandoraTrack:LArPandoraTrackCreation           2.49818       5.86649       16.3766       5.25484       3.90168        10     
reco:pandoraShower:LArPandoraShowerCreation         1.88792       6.07563       14.2633       5.52504       3.78778        10     
reco:pandoracalo:Calorimetry                        1.26145       3.03134       7.39387       2.87061       1.76209        10     
reco:pandoracalonosce:Calorimetry                   1.22383       2.90472       7.28458       2.64301       1.74791        10     
reco:pandorapid:Chi2ParticleID                    0.00390555    0.00517466    0.00797643    0.00486181    0.00117955       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0277886     0.0433525     0.0650481     0.0425032     0.0110855       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00351258    0.00492187     0.0076795    0.00455832     0.0011664       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.7294        5.72811       11.7858       5.87914       3.23164        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.61752       5.58237       12.3806       5.52589       3.36002        10     
reco:emtrkmichelid:EmTrackMichelId                  98.4283       153.831       227.767       152.766       41.9442        10     
reco:crttag:SingleCRTMatchingProducer             0.00762915     0.0154631     0.0232971     0.0164413     0.0048558       10     
reco:crtreco:TwoCRTMatchingProducer               0.00622215     0.0106837     0.0151989     0.0108851    0.00286936       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.559047      0.794867       1.16653      0.755818      0.190996        10     
reco:pandora2Track:LArPandoraTrackCreation          6.05116       16.1351       39.0799       14.9609       9.80001        10     
reco:pandora2calo:Calorimetry                       2.72765       8.04442       16.7485       7.80278       4.61801        10     
reco:pandora2calonosce:Calorimetry                  2.61482       7.5011        15.5605       7.16334       4.33989        10     
reco:pandora2pid:Chi2ParticleID                   0.00650289     0.0110121     0.0173725     0.0104247    0.00325974       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0351791     0.0549997     0.0842544     0.0526036     0.0159006       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0065848     0.0109425     0.0172796     0.0103652     0.0030907       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.23458       5.55187       10.6899       5.57933       2.86507        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.27542       6.23811       11.1181       6.44228       3.24939        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.03296       5.42667       9.95065       5.64148       2.76655        10     
[art]:TriggerResults:TriggerResultInserter        3.2056e-05    4.90773e-05   0.000101192   4.40605e-05   1.93768e-05      10     
end_path:out1:RootOutput                           6.33e-06     8.3001e-06    1.2829e-05     8.006e-06    1.63839e-06      10     
end_path:out1:RootOutput(write)                     1.70747       2.64347       3.97109       2.62879      0.690593        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5152.43 MB
  Peak resident set size usage (VmHWM): 4351.77 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.04 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.13 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10365744
-rw-r--r--. 1 dune009 dune 2437059830 Oct 29 02:58 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2314458996 Oct 29 02:57 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2121158799 Oct 29 03:59 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1874713924 Oct 29 00:50 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1792068352 Oct 29 00:38 g4_stage1.root
-rw-r--r--. 1 dune009 dune   74446920 Oct 29 03:54 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     242894 Oct 29 03:59 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Oct 29 03:59 mem.db
-rw-r--r--. 1 dune009 dune      62376 Oct 29 03:59 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Oct 29 03:59 time.db
-rw-r--r--. 1 dune009 dune        519 Oct 29 02:58 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Oct 29 00:50 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Oct 29 03:59 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Oct 29 00:32 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Oct 29 03:59 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Oct 29 03:03 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Oct 29 02:58 debugprod.log
justIN time: 2025-11-03 18:41:13 UTC       justIN version: 01.05.01