justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 238953.0@dunegpschedd01.fnal.gov

Jobsub ID238953.0@dunegpschedd01.fnal.gov
Workflow ID9391
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-27 21:19:20
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc01
Last heartbeat2025-10-28 00:45:26
From worker nodeHostnamewn-lb-13.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-27 21:20:42
Input filesmonte-carlo-009391-000001
JobscriptExit code0
Real time3h (12185s)
CPU time3h (12005s = 98%)
Max RSS bytes4067377152 (3878 MiB)
Outputting started2025-10-28 00:43:48
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09391/1/001/reco_stage1.root
Finished2025-10-28 00:45:26
Saved logsjustin-logs:238953.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

gitPrepService:prepare:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 112671 traces, 0 summary
Retagger: tagging trace set: gauss with 84793 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 84793 traces tagged "gauss"
FrameSaver: q=2.97203e+06 n=1503908 tag=gauss
wclsFrameSaver: saving 112671 traces tagged "wiener"
FrameSaver: q=3.145e+06 n=1330749 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 32316 XUs and 45621 XVs -> 417 XUVs
C:0 T:1 89220 XUs and 102529 XVs -> 7905 XUVs
C:0 T:2 46432 XUs and 56769 XVs -> 1437 XUVs
C:0 T:3 27537 XUs and 26427 XVs -> 159 XUVs
C:0 T:4 7824 XUs and 6218 XVs -> 74 XUVs
C:0 T:5 29328 XUs and 18545 XVs -> 2400 XUVs
C:0 T:6 69159 XUs and 82328 XVs -> 5402 XUVs
C:0 T:7 26424 XUs and 31311 XVs -> 326 XUVs
C:0 T:9 63171 XUs and 72631 XVs -> 4951 XUVs
C:0 T:10 40577 XUs and 24665 XVs -> 7361 XUVs
C:0 T:11 159 XUs and 110 XVs -> 25 XUVs
30457 XUVs total
9979 collection wire objects
28061 potential space points
Neighbour search...
2688059 tests to find 1298860 neighbours
Iterating with no regularization...
Begin: 6.69274e+07
0 5.4419e+07
1 5.34374e+07
2 5.32787e+07
3 5.32276e+07
Now with regularization...
Begin: 4.42072e+07
0 4.41605e+07
1 4.41335e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410154364
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410154364
fLifetime: 30000 [us]
28-Oct-2025 01:43:46 CET  Closed output file "reco_stage1.root"
28-Oct-2025 01:43:46 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          247.963        368.1        493.881       378.998       80.552         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00148033    0.00570748    0.00989785    0.00530851    0.00281529       10     
reco:ophit:OpHitFinder                             0.252648      0.304108      0.447399      0.283706      0.0562585       10     
reco:opflash:OpFlashFinder                         0.143008      0.191395      0.297774      0.185298      0.0423933       10     
reco:opslicer:OpSlicer                             0.442561      0.595883      0.895254      0.561776      0.127212        10     
reco:caldata:DataPrepModule                         4.78496       5.24924       7.18283       4.8992        0.78342        10     
reco:wclsdatasp:WireCellToolkit                     46.3251       50.3466       55.0704       50.5416       3.18682        10     
reco:gaushit:GausHitFinder                         0.754903       1.09404       1.41331       1.11565      0.224807        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00053431    0.000762206   0.00118875    0.000696917   0.000173111      10     
reco:reco3d:SpacePointSolver                        10.6153       14.8722       22.743        13.2928       4.00402        10     
reco:hitpdune:DisambigFromSpacePoints              0.820539       1.37318       1.99714       1.39384      0.375178        10     
reco:pandora:StandardPandora                        14.6895       27.8684       48.6592       27.0911       11.1114        10     
reco:pandoraWriter:StandardPandora                 0.343978      0.441864      0.550268      0.441433      0.0634181       10     
reco:pandoraTrack:LArPandoraTrackCreation           3.60431       6.87508       11.4616       7.00239       2.16309        10     
reco:pandoraShower:LArPandoraShowerCreation         3.1736        9.29029       14.9195       8.08372       3.7601         10     
reco:pandoracalo:Calorimetry                        1.6154        3.46114       4.96398       3.77692       1.04541        10     
reco:pandoracalonosce:Calorimetry                   1.54973       3.31276       4.74165       3.66588       1.02031        10     
reco:pandorapid:Chi2ParticleID                    0.00359598    0.00504195    0.00802122    0.00451845    0.00130285       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0305871     0.0436483     0.0588943     0.0435579    0.00833553       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00365956    0.00478396     0.0061655    0.00455062    0.000791126      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.82405       7.81531       12.5646       7.98835       3.13732        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.67898       7.54946       12.3656       7.37667       3.3056         10     
reco:emtrkmichelid:EmTrackMichelId                  117.252       156.935       214.953       157.86        30.661         10     
reco:crttag:SingleCRTMatchingProducer             0.00969799     0.0192844     0.0321035     0.0179119    0.00655022       10     
reco:crtreco:TwoCRTMatchingProducer                0.0081958     0.0156803     0.0295511     0.014654     0.00583577       10     
reco:anodepiercerst0:T0RecoAnodePiercers            0.60631      0.810641       1.23633      0.734132      0.193993        10     
reco:pandora2Track:LArPandoraTrackCreation          8.89401       22.3481       41.3247       22.0227       8.92564        10     
reco:pandora2calo:Calorimetry                       3.84897       10.6492       22.6119       10.9231       5.33545        10     
reco:pandora2calonosce:Calorimetry                  3.47517       9.92701       20.895        9.41264       4.92746        10     
reco:pandora2pid:Chi2ParticleID                   0.00793273     0.0112728     0.0163354     0.0108141    0.00239388       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0385099     0.0557736     0.0760035     0.0541993     0.0111973       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00825231     0.0111092     0.0163884     0.0107162    0.00233227       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.44005       8.48132       15.8705       8.39431       3.42345        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.36378       8.45347       17.5777       7.91735       3.90212        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.91198       6.81689       14.3487       6.47353       3.05827        10     
[art]:TriggerResults:TriggerResultInserter        4.1362e-05    6.01742e-05   0.000124296    5.35e-05     2.22584e-05      10     
end_path:out1:RootOutput                           7.631e-06    8.5838e-06     1.313e-05     8.047e-06    1.55361e-06      10     
end_path:out1:RootOutput(write)                     2.01461       2.58818       4.07281       2.51136      0.583333        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4869.27 MB
  Peak resident set size usage (VmHWM): 4067.38 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.25 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.20 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.13 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9579404
-rw-r--r--. 1 dune009 dune 2277861165 Oct 28 00:40 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2165482762 Oct 28 00:38 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 1947835663 Oct 28 01:43 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1711530493 Oct 27 22:38 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1634959584 Oct 27 22:27 g4_stage1.root
-rw-r--r--. 1 dune009 dune   71022070 Oct 28 01:39 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     242947 Oct 28 01:43 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Oct 28 01:43 mem.db
-rw-r--r--. 1 dune009 dune      62376 Oct 28 01:43 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Oct 28 01:43 time.db
-rw-r--r--. 1 dune009 dune        519 Oct 28 00:40 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Oct 27 22:38 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Oct 28 01:43 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Oct 27 22:20 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Oct 28 01:43 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Oct 28 00:48 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Oct 28 00:40 debugprod.log
justIN time: 2025-11-05 19:16:08 UTC       justIN version: 01.05.01