justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 238299.0@dunegpschedd01.fnal.gov

Jobsub ID238299.0@dunegpschedd01.fnal.gov
Workflow ID9321
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-24 18:47:25
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc03
Last heartbeat2025-10-24 22:58:06
From worker nodeHostnamewn-la-22.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-24 18:48:49
Input filesmonte-carlo-009321-000001
JobscriptExit code0
Real time4h (14846s)
CPU time4h (14605s = 98%)
Max RSS bytes4426313728 (4221 MiB)
Outputting started2025-10-24 22:56:16
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09321/1/001/reco_stage1.root
Finished2025-10-24 22:58:06
Saved logsjustin-logs:238299.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

g tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 105802 traces, 0 summary
Retagger: tagging trace set: gauss with 80718 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 80718 traces tagged "gauss"
FrameSaver: q=2.53102e+06 n=1376562 tag=gauss
wclsFrameSaver: saving 105802 traces tagged "wiener"
FrameSaver: q=2.67787e+06 n=1202659 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 26353 XUs and 18414 XVs -> 157 XUVs
C:0 T:1 46937 XUs and 39933 XVs -> 3098 XUVs
C:0 T:2 68668 XUs and 116545 XVs -> 4520 XUVs
C:0 T:3 31144 XUs and 47671 XVs -> 255 XUVs
C:0 T:4 6812 XUs and 9905 XVs -> 52 XUVs
C:0 T:5 27132 XUs and 29775 XVs -> 4159 XUVs
C:0 T:6 84846 XUs and 86005 XVs -> 5835 XUVs
C:0 T:7 31599 XUs and 27787 XVs -> 220 XUVs
C:0 T:8 6628 XUs and 4655 XVs -> 83 XUVs
C:0 T:9 31030 XUs and 31564 XVs -> 3199 XUVs
C:0 T:10 46433 XUs and 25755 XVs -> 3875 XUVs
C:0 T:11 138 XUs and 125 XVs -> 19 XUVs
25472 XUVs total
9946 collection wire objects
23699 potential space points
Neighbour search...
1590421 tests to find 893238 neighbours
Iterating with no regularization...
Begin: 3.38344e+07
0 2.67101e+07
1 2.62656e+07
2 2.61987e+07
3 2.61816e+07
Now with regularization...
Begin: 1.98367e+07
0 1.98197e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410091938
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410091938
fLifetime: 30000 [us]
25-Oct-2025 00:56:14 CEST  Closed output file "reco_stage1.root"
25-Oct-2025 00:56:14 CEST  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          300.608       468.853       752.295       450.071       138.185        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00136683     0.0105434     0.0463145    0.00665905     0.0122396       10     
reco:ophit:OpHitFinder                              0.2651        0.33148      0.412174       0.32276      0.0391211       10     
reco:opflash:OpFlashFinder                         0.148642      0.241203      0.343925      0.238473      0.0508579       10     
reco:opslicer:OpSlicer                             0.462402      0.649293      0.940751      0.626425      0.114819        10     
reco:caldata:DataPrepModule                         4.85603       5.07108       5.67194       5.00515      0.229003        10     
reco:wclsdatasp:WireCellToolkit                     36.4024       44.6059       50.5865       46.3568       4.91842        10     
reco:gaushit:GausHitFinder                          0.85102       1.15867       1.65866       1.05136      0.230079        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000872574   0.00120661    0.00150513    0.00123141    0.000245143      10     
reco:reco3d:SpacePointSolver                        8.09186       19.5138       42.5929       16.5024       9.89452        10     
reco:hitpdune:DisambigFromSpacePoints              0.824619       1.54681       2.63061       1.46089      0.558315        10     
reco:pandora:StandardPandora                        16.2225       40.3744       116.832       31.0879       27.2528        10     
reco:pandoraWriter:StandardPandora                 0.387266      0.537351      0.752707      0.544544      0.110584        10     
reco:pandoraTrack:LArPandoraTrackCreation            7.349        12.507        20.0505       13.1982       3.75883        10     
reco:pandoraShower:LArPandoraShowerCreation         5.99485       15.3904       25.7936       15.9632       6.03577        10     
reco:pandoracalo:Calorimetry                        2.81481       6.04038       9.85216       6.69948       2.20074        10     
reco:pandoracalonosce:Calorimetry                   2.67972       5.72801       9.73961       5.91145       2.21343        10     
reco:pandorapid:Chi2ParticleID                    0.00421818    0.00607592    0.00849212    0.00587544    0.00126538       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0345684     0.0500079     0.0638073     0.0510985     0.0100923       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0044978    0.00583087    0.00811377    0.00554298    0.00114762       10     
reco:pandoraShowercalo:ShowerCalorimetry            5.5471        14.1282       24.5065       13.392        6.40115        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       5.38187       13.3784       23.4625       11.8997       6.35156        10     
reco:emtrkmichelid:EmTrackMichelId                  123.665       166.152       229.557       161.018       34.3052        10     
reco:crttag:SingleCRTMatchingProducer              0.0144034     0.0295824     0.0469311     0.0325215     0.0104814       10     
reco:crtreco:TwoCRTMatchingProducer                0.013365      0.0258079     0.0444445     0.0240115     0.0100829       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.594565      0.891783       1.29058      0.884225      0.199639        10     
reco:pandora2Track:LArPandoraTrackCreation          21.6482       40.6406       74.0678       36.9956       16.512         10     
reco:pandora2calo:Calorimetry                       7.77718       17.4878       29.5663       17.1729       7.99014        10     
reco:pandora2calonosce:Calorimetry                  7.73807       16.2335       25.9072       16.9272       6.79334        10     
reco:pandora2pid:Chi2ParticleID                   0.00885483     0.011873      0.0160439     0.0112422    0.00209163       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0455763     0.0610511     0.0799594     0.0598356     0.0108815       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00872241     0.0117173     0.0155592     0.0113807    0.00207821       10     
reco:pandora2Shower:LArPandoraShowerCreation        6.90416       15.4668       23.4802       16.1106       5.46041        10     
reco:pandora2Showercalo:ShowerCalorimetry           6.80215       14.6211       21.5245       14.6488       5.84465        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.94902       12.7513       19.6084       13.6985       4.96693        10     
[art]:TriggerResults:TriggerResultInserter        4.4337e-05    6.26397e-05   0.000123254   5.36755e-05   2.31675e-05      10     
end_path:out1:RootOutput                           7.341e-06    1.04839e-05   1.6014e-05    1.03005e-05   2.51339e-06      10     
end_path:out1:RootOutput(write)                     2.04618       2.86003       3.89854       2.83364      0.595047        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5229.34 MB
  Peak resident set size usage (VmHWM): 4426.31 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.25 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.08 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.13 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10838512
-rw-r--r--. 1 dune009 dune 2509555775 Oct 24 23:36 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2379091904 Oct 24 23:33 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2210866613 Oct 25 00:56 reco_stage1.root
-rw-r--r--. 1 dune009 dune 2006958989 Oct 24 21:12 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1913635869 Oct 24 20:56 g4_stage1.root
-rw-r--r--. 1 dune009 dune   77913900 Oct 25 00:52 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     243421 Oct 25 00:56 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Oct 25 00:56 mem.db
-rw-r--r--. 1 dune009 dune      62376 Oct 25 00:56 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Oct 25 00:56 time.db
-rw-r--r--. 1 dune009 dune        519 Oct 24 23:36 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Oct 24 21:12 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Oct 25 00:56 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Oct 24 20:48 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Oct 25 00:56 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Oct 24 23:43 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Oct 24 23:36 debugprod.log
justIN time: 2025-11-06 01:54:12 UTC       justIN version: 01.05.01