justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 254430.0@dunegpschedd01.fnal.gov

Jobsub ID254430.0@dunegpschedd01.fnal.gov
Workflow ID10397
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-20 10:00:58
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce15-multicore
Last heartbeat2025-11-20 12:16:38
From worker nodeHostnamehnode79.pic.es
cpuinfoAMD EPYC 7443P 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-20 10:01:34
Input filesmonte-carlo-010397-000001
JobscriptExit code0
Real time2h (8048s)
CPU time1h (6637s = 82%)
Max RSS bytes4097425408 (3907 MiB)
Outputting started2025-11-20 12:15:42
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10397/1/001/reco_stage1.root
Finished2025-11-20 12:16:38
Saved logsjustin-logs:254430.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 115395 traces, 0 summary
Retagger: tagging trace set: gauss with 89707 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 89707 traces tagged "gauss"
FrameSaver: q=3.05791e+06 n=1577547 tag=gauss
wclsFrameSaver: saving 115395 traces tagged "wiener"
FrameSaver: q=3.20677e+06 n=1386943 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 30564 XUs and 34996 XVs -> 487 XUVs
C:0 T:1 53983 XUs and 58459 XVs -> 3413 XUVs
C:0 T:2 109112 XUs and 152866 XVs -> 5330 XUVs
C:0 T:3 54557 XUs and 63789 XVs -> 430 XUVs
C:0 T:4 4899 XUs and 4584 XVs -> 21 XUVs
C:0 T:5 15852 XUs and 10865 XVs -> 1647 XUVs
C:0 T:6 124955 XUs and 88810 XVs -> 6900 XUVs
C:0 T:7 49806 XUs and 31798 XVs -> 342 XUVs
C:0 T:8 7010 XUs and 6248 XVs -> 59 XUVs
C:0 T:9 35692 XUs and 48066 XVs -> 3483 XUVs
C:0 T:10 38429 XUs and 10717 XVs -> 2848 XUVs
C:0 T:11 43 XUs and 21 XVs -> 9 XUVs
24969 XUVs total
10658 collection wire objects
22743 potential space points
Neighbour search...
1375269 tests to find 676204 neighbours
Iterating with no regularization...
Begin: 6.05857e+07
0 5.16443e+07
1 5.10285e+07
2 5.0957e+07
3 5.09371e+07
Now with regularization...
Begin: 4.21823e+07
0 4.21718e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410627676
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410627676
fLifetime: 30000 [us]
20-Nov-2025 13:15:41 CET  Closed output file "reco_stage1.root"
20-Nov-2025 13:15:41 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          137.943       281.205       684.007       230.322       166.608        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000648581   0.00263823    0.00401019    0.00284557    0.00102616       10     
reco:ophit:OpHitFinder                             0.151645      0.194237      0.235291      0.188166      0.028123        10     
reco:opflash:OpFlashFinder                         0.080417      0.114467      0.143002      0.118641      0.0198955       10     
reco:opslicer:OpSlicer                             0.290474      0.416195      0.558563      0.384757      0.096489        10     
reco:caldata:DataPrepModule                         3.15322       3.73564       5.08185       3.39376      0.695225        10     
reco:wclsdatasp:WireCellToolkit                     21.2342       60.3421       390.556       23.7976       110.092        10     
reco:gaushit:GausHitFinder                         0.478272      0.664195      0.939088      0.645538      0.147427        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000192174   0.000299043   0.000406047   0.000304256   6.58742e-05      10     
reco:reco3d:SpacePointSolver                        3.21449       7.68019       13.0337       6.97202       2.97772        10     
reco:hitpdune:DisambigFromSpacePoints              0.518415       0.87248       1.51482      0.811953       0.32917        10     
reco:pandora:StandardPandora                        7.86332       19.2477       50.4613       13.9607       13.7292        10     
reco:pandoraWriter:StandardPandora                 0.165489      0.226473      0.381698      0.197804      0.0668828       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.08592       1.81048       3.73976       1.48908      0.771594        10     
reco:pandoraShower:LArPandoraShowerCreation        0.998022       1.81001       3.36802       1.54028      0.822354        10     
reco:pandoracalo:Calorimetry                        0.6185        1.02078       1.57761       0.95501      0.324968        10     
reco:pandoracalonosce:Calorimetry                  0.550964      0.931211       1.52675      0.847333      0.308444        10     
reco:pandorapid:Chi2ParticleID                    0.00244496    0.00303319     0.0039965    0.00290007    0.000529499      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0212639     0.0260287     0.0377082     0.0254687    0.00470627       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00226586    0.00282496    0.00367416    0.00280736    0.000416683      10     
reco:pandoraShowercalo:ShowerCalorimetry            1.01631       2.02332       3.68584       1.79047      0.961149        10     
reco:pandoraShowercalonosce:ShowerCalorimetry      0.970586       1.98256       3.79139       1.7171       0.966714        10     
reco:emtrkmichelid:EmTrackMichelId                  80.6768       131.296       234.105       100.467       56.3361        10     
reco:crttag:SingleCRTMatchingProducer             0.00414684    0.00976733     0.0392875    0.00564116     0.0101691       10     
reco:crtreco:TwoCRTMatchingProducer               0.00339142     0.0052681     0.0134978    0.00376059    0.00305755       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.441954      0.606158      0.901983      0.554466      0.141285        10     
reco:pandora2Track:LArPandoraTrackCreation          3.13253       18.6669       135.492       4.78844       39.0307        10     
reco:pandora2calo:Calorimetry                       1.56907       3.16299       6.02374       2.80551       1.35161        10     
reco:pandora2calonosce:Calorimetry                  1.47971       15.0549       124.678       2.97083       36.5572        10     
reco:pandora2pid:Chi2ParticleID                   0.00550344    0.00704188    0.00931448    0.00673273    0.00121635       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0263118     0.0345495     0.0457633     0.0318207     0.0068391       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.005455      0.0071129    0.00942816    0.00695146     0.0013961       10     
reco:pandora2Shower:LArPandoraShowerCreation         1.251        2.24775       3.76415       1.84092      0.849905        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.40734       2.61706       4.59325       2.38736       1.04397        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.30636       2.66512       5.1133        2.21552       1.17968        10     
[art]:TriggerResults:TriggerResultInserter         2.049e-05    3.13975e-05   9.0412e-05    2.4345e-05    1.99633e-05      10     
end_path:out1:RootOutput                           3.82e-06      5.081e-06     8.84e-06      4.65e-06     1.33871e-06      10     
end_path:out1:RootOutput(write)                     1.27367       1.66176       2.42165       1.50832      0.368984        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4911.67 MB
  Peak resident set size usage (VmHWM): 4097.43 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.40 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9418060
-rw-r--r-- 1 dune001 dune 2239480317 Nov 20 12:25 detsim_stage2.root
-rw-r--r-- 1 dune001 dune 2128932782 Nov 20 12:24 detsim_stage1.root
-rw-r--r-- 1 dune001 dune 1889858630 Nov 20 13:15 reco_stage1.root
-rw-r--r-- 1 dune001 dune 1696158611 Nov 20 11:14 g4_stage2.root
-rw-r--r-- 1 dune001 dune 1622165104 Nov 20 11:07 g4_stage1.root
-rw-r--r-- 1 dune001 dune   66866820 Nov 20 13:13 Pandora_Events.pndr
-rw-r--r-- 1 dune001 dune     242896 Nov 20 13:15 jobscript.log
-rw-r--r-- 1 dune001 dune     217088 Nov 20 13:15 mem.db
-rw-r--r-- 1 dune001 dune      62376 Nov 20 13:15 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 dune      36864 Nov 20 13:15 time.db
-rw-r--r-- 1 dune001 dune        519 Nov 20 12:25 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Nov 20 11:14 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Nov 20 13:15 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 dune         52 Nov 20 11:01 all-input-dids.txt
-rw-r--r-- 1 dune001 dune          7 Nov 20 13:15 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune          0 Nov 20 12:31 ProtoDUNE.log
-rw-r--r-- 1 dune001 dune          0 Nov 20 12:25 debugprod.log
justIN time: 2026-02-09 03:24:07 UTC       justIN version: 01.06.00