justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 268866.0@dunegpschedd02.fnal.gov

Jobsub ID268866.0@dunegpschedd02.fnal.gov
Workflow ID11727
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-11 13:06:50
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc01
Last heartbeat2026-01-11 18:35:57
From worker nodeHostnamewn-la-22.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-11 13:07:48
Input filesmonte-carlo-011727-000001
JobscriptExit code0
Real time5h (19590s)
CPU time5h (19453s = 99%)
Max RSS bytes4426944512 (4221 MiB)
Outputting started2026-01-11 18:34:19
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11727/1/001/reco_stage1.root
Finished2026-01-11 18:35:57
Saved logsjustin-logs:268866.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

SignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 129849 traces, 0 summary
Retagger: tagging trace set: gauss with 95171 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 95171 traces tagged "gauss"
FrameSaver: q=3.20146e+06 n=1635321 tag=gauss
wclsFrameSaver: saving 129849 traces tagged "wiener"
FrameSaver: q=3.40076e+06 n=1417330 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 43468 XUs and 26707 XVs -> 452 XUVs
C:0 T:1 94024 XUs and 68689 XVs -> 6698 XUVs
C:0 T:2 135201 XUs and 201394 XVs -> 10016 XUVs
C:0 T:3 57276 XUs and 68024 XVs -> 728 XUVs
C:0 T:4 12679 XUs and 9789 XVs -> 93 XUVs
C:0 T:5 49790 XUs and 27616 XVs -> 5626 XUVs
C:0 T:6 70676 XUs and 79960 XVs -> 3379 XUVs
C:0 T:7 28337 XUs and 30690 XVs -> 311 XUVs
C:0 T:8 3886 XUs and 5479 XVs -> 149 XUVs
C:0 T:9 30229 XUs and 44130 XVs -> 9127 XUVs
C:0 T:10 25817 XUs and 14374 XVs -> 2298 XUVs
C:0 T:11 186 XUs and 87 XVs -> 16 XUVs
38893 XUVs total
12408 collection wire objects
35827 potential space points
Neighbour search...
3583221 tests to find 1874936 neighbours
Iterating with no regularization...
Begin: 5.02618e+07
0 4.02771e+07
1 3.95365e+07
2 3.94382e+07
3 3.94103e+07
Now with regularization...
Begin: 3.09745e+07
0 3.09328e+07
1 3.09042e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411676397
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411676397
fLifetime: 30000 [us]
11-Jan-2026 19:34:16 CET  Closed output file "reco_stage1.root"
11-Jan-2026 19:34:16 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          466.926       615.947       985.767       560.157       163.309        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00121689    0.00695933     0.0131618    0.00645795     0.0038563       10     
reco:ophit:OpHitFinder                             0.388266      0.471787      0.514363      0.475114      0.0337594       10     
reco:opflash:OpFlashFinder                         0.249019      0.359154      0.435047      0.359335      0.0549008       10     
reco:opslicer:OpSlicer                             0.781049       1.00032       1.15496      0.997518      0.0971636       10     
reco:caldata:DataPrepModule                         6.4148        6.57323       7.0422         6.508       0.183615        10     
reco:wclsdatasp:WireCellToolkit                     44.9032       59.3293       67.5622       62.9811       8.01484        10     
reco:gaushit:GausHitFinder                          1.30976       1.58664       2.24583       1.48605      0.305051        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000499576   0.000692511   0.00130277    0.000630826   0.000220492      10     
reco:reco3d:SpacePointSolver                        16.7021       25.9275       42.9377       22.0851       9.06798        10     
reco:hitpdune:DisambigFromSpacePoints               1.52036       2.46194       4.40489       2.15349      0.906287        10     
reco:pandora:StandardPandora                        30.8659       64.1248       144.623       50.3967       37.2033        10     
reco:pandoraWriter:StandardPandora                  0.49226      0.617942      0.876175      0.586211      0.117177        10     
reco:pandoraTrack:LArPandoraTrackCreation           7.34481       11.8967       22.2349       11.041        4.16993        10     
reco:pandoraShower:LArPandoraShowerCreation         8.19806       15.8327       37.8376       13.165        8.69809        10     
reco:pandoracalo:Calorimetry                        4.02886       6.8093        13.4258       5.74429       2.81339        10     
reco:pandoracalonosce:Calorimetry                   3.83129       6.22996       11.9711       5.51956       2.61997        10     
reco:pandorapid:Chi2ParticleID                    0.00618164    0.00741182    0.00882617    0.00744639    0.000840222      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0518062     0.0665675     0.0891431     0.0637282     0.0113188       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00602079    0.00715144    0.00866686    0.00711123    0.000909486      10     
reco:pandoraShowercalo:ShowerCalorimetry            8.78307       16.0576       36.0917       12.684        8.47163        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       7.98532       14.3811       33.1232       11.5304       7.00856        10     
reco:emtrkmichelid:EmTrackMichelId                  206.899       261.542       361.403       249.665       46.1738        10     
reco:crttag:SingleCRTMatchingProducer              0.0171744     0.0256636     0.0386099     0.0236161    0.00677995       10     
reco:crtreco:TwoCRTMatchingProducer                0.0140478     0.0248557     0.0380717     0.0239764    0.00798096       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.910559       1.21993       1.76443       1.18468      0.235627        10     
reco:pandora2Track:LArPandoraTrackCreation          20.8921       38.3411       64.668        38.3703       12.9833        10     
reco:pandora2calo:Calorimetry                       11.7701       18.9708       26.8122       18.3712       5.14925        10     
reco:pandora2calonosce:Calorimetry                  11.3741       19.2497       31.039        16.6128       6.58963        10     
reco:pandora2pid:Chi2ParticleID                    0.012923      0.0171111     0.0228143     0.0167693    0.00318915       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0663817     0.0851001     0.115484      0.0803177     0.0155784       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0135613     0.0169924     0.0222667     0.0165565    0.00283776       10     
reco:pandora2Shower:LArPandoraShowerCreation        8.41345       12.564        21.552        11.7615       3.96934        10     
reco:pandora2Showercalo:ShowerCalorimetry           8.94357       14.1454       27.5147       13.2495       5.43087        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      7.85171       11.6026       23.4344       10.4964       4.50187        10     
[art]:TriggerResults:TriggerResultInserter        3.7607e-05    6.00878e-05   0.000133328   4.91485e-05   2.63962e-05      10     
end_path:out1:RootOutput                           7.201e-06    8.7974e-06    1.3681e-05    8.2475e-06    1.78409e-06      10     
end_path:out1:RootOutput(write)                     3.2142        4.07739       5.65031       3.83271      0.736467        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5227.04 MB
  Peak resident set size usage (VmHWM): 4426.94 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.29 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.10 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11423468
-rw-r--r--. 1 dune009 dune 2627064329 Jan 11 17:49 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2486524280 Jan 11 17:47 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2374707611 Jan 11 19:34 reco_stage1.root
-rw-r--r--. 1 dune009 dune 2099283936 Jan 11 14:34 g4_stage2.root
-rw-r--r--. 1 dune009 dune 2020175020 Jan 11 14:15 g4_stage1.root
-rw-r--r--. 1 dune009 dune   89260950 Jan 11 19:28 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     242878 Jan 11 19:34 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Jan 11 19:34 mem.db
-rw-r--r--. 1 dune009 dune      62376 Jan 11 19:34 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Jan 11 19:34 time.db
-rw-r--r--. 1 dune009 dune        519 Jan 11 17:49 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Jan 11 14:34 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Jan 11 19:34 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Jan 11 14:07 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Jan 11 19:34 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Jan 11 17:56 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Jan 11 17:49 debugprod.log
justIN time: 2026-02-06 10:21:21 UTC       justIN version: 01.06.00