justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 240830.1@dunegpschedd01.fnal.gov

Jobsub ID240830.1@dunegpschedd01.fnal.gov
Workflow ID9461
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-30 07:25:24
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce16-multicore
Last heartbeat2025-10-30 12:19:02
From worker nodeHostnametds516.pic.es
cpuinfoAMD EPYC 7513 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-30 07:26:16
Input filesmonte-carlo-009461-000001
JobscriptExit code0
Real time4h (17402s)
CPU time4h (16583s = 95%)
Max RSS bytes4343099392 (4141 MiB)
Outputting started2025-10-30 12:16:20
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09461/1/001/reco_stage1.root
Finished2025-10-30 12:19:02
Saved logsjustin-logs:240830.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ng tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 189621 traces, 0 summary
Retagger: tagging trace set: gauss with 142195 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 142195 traces tagged "gauss"
FrameSaver: q=4.87777e+06 n=2507992 tag=gauss
wclsFrameSaver: saving 189621 traces tagged "wiener"
FrameSaver: q=5.15484e+06 n=2197217 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 54797 XUs and 56942 XVs -> 1013 XUVs
C:0 T:1 148541 XUs and 143384 XVs -> 13120 XUVs
C:0 T:2 80592 XUs and 99754 XVs -> 4333 XUVs
C:0 T:3 37932 XUs and 44090 XVs -> 224 XUVs
C:0 T:4 17565 XUs and 16778 XVs -> 271 XUVs
C:0 T:5 114438 XUs and 86947 XVs -> 11590 XUVs
C:0 T:6 137283 XUs and 125925 XVs -> 5759 XUVs
C:0 T:7 50050 XUs and 46347 XVs -> 661 XUVs
C:0 T:8 14172 XUs and 13265 XVs -> 351 XUVs
C:0 T:9 106561 XUs and 126994 XVs -> 7461 XUVs
C:0 T:10 73209 XUs and 31874 XVs -> 4550 XUVs
C:0 T:11 739 XUs and 385 XVs -> 42 XUVs
49375 XUVs total
18862 collection wire objects
44742 potential space points
Neighbour search...
2987878 tests to find 1653246 neighbours
Iterating with no regularization...
Begin: 9.72273e+07
0 7.94801e+07
1 7.83221e+07
2 7.81782e+07
3 7.81385e+07
Now with regularization...
Begin: 6.50524e+07
0 6.50135e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410202985
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410202985
fLifetime: 30000 [us]
30-Oct-2025 13:16:14 CET  Closed output file "reco_stage1.root"
30-Oct-2025 13:16:14 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          434.518       591.928       845.788       564.425       117.968        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00110713    0.00594865     0.0122312    0.00651513    0.00339728       10     
reco:ophit:OpHitFinder                             0.388425      0.432877      0.518038      0.421097      0.0421499       10     
reco:opflash:OpFlashFinder                          0.20526      0.262295      0.355513      0.238832      0.0534938       10     
reco:opslicer:OpSlicer                             0.859593       1.03466       1.26714       1.02136      0.129743        10     
reco:caldata:DataPrepModule                         6.49114       7.32321       8.34248       7.36391      0.488151        10     
reco:wclsdatasp:WireCellToolkit                     36.6672       42.3085       48.7713       42.6009       3.57649        10     
reco:gaushit:GausHitFinder                          1.74422       2.02306       2.51804       1.97384      0.224104        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000539045   0.000814961   0.00100786    0.000854568   0.000131521      10     
reco:reco3d:SpacePointSolver                        23.7894       33.0253       47.0452       32.1778       6.95541        10     
reco:hitpdune:DisambigFromSpacePoints               2.21152       3.04474       3.94018       3.09981      0.475005        10     
reco:pandora:StandardPandora                        48.347        83.0914       223.988       69.0475       48.5629        10     
reco:pandoraWriter:StandardPandora                  0.44176      0.737394       0.9167       0.763348      0.162888        10     
reco:pandoraTrack:LArPandoraTrackCreation           4.09237       8.40208       12.7376       7.91512       2.83851        10     
reco:pandoraShower:LArPandoraShowerCreation         5.07554       10.3072       17.4041       8.73311       4.11786        10     
reco:pandoracalo:Calorimetry                        2.23108       4.78913       6.61417       4.93098       1.43546        10     
reco:pandoracalonosce:Calorimetry                   2.01596       4.36741       6.11252       4.29242       1.43228        10     
reco:pandorapid:Chi2ParticleID                    0.00407068    0.00888582     0.0113544     0.0095274    0.00228702       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0430924     0.091738      0.116135      0.100124      0.024485        10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00384784    0.00852877      0.0106      0.00945886    0.00228725       10     
reco:pandoraShowercalo:ShowerCalorimetry            5.27083       11.0406       18.1039       9.87695       4.14864        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.82287       9.36331        13.7         8.47911       3.02524        10     
reco:emtrkmichelid:EmTrackMichelId                  213.39        277.678       333.06        277.089       36.4729        10     
reco:crttag:SingleCRTMatchingProducer              0.0133857     0.0263106     0.056982      0.0238403     0.0123947       10     
reco:crtreco:TwoCRTMatchingProducer               0.00743185     0.0156629     0.0219382     0.0160064    0.00432119       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.731048       1.24782       1.6345        1.35333      0.280533        10     
reco:pandora2Track:LArPandoraTrackCreation          13.6234       27.0744       46.3098       24.6555       9.88687        10     
reco:pandora2calo:Calorimetry                       7.60718       13.7627       23.1664       12.2209       4.78049        10     
reco:pandora2calonosce:Calorimetry                  6.96321       12.9519       21.3058       11.4345       4.26777        10     
reco:pandora2pid:Chi2ParticleID                    0.0119204      0.02315      0.0322263     0.0221767    0.00582774       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0602683     0.127016      0.179563      0.123575      0.0317737       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0118198     0.024394      0.041494      0.0226874    0.00785336       10     
reco:pandora2Shower:LArPandoraShowerCreation        6.32754       9.89707       17.4211       8.4453        3.45094        10     
reco:pandora2Showercalo:ShowerCalorimetry           8.03402       11.8996       19.3666       10.0785       3.73431        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      7.22982       10.8016       16.9505       9.29923       3.28116        10     
[art]:TriggerResults:TriggerResultInserter        6.1316e-05    8.66492e-05   0.000148058   7.94345e-05   2.58393e-05      10     
end_path:out1:RootOutput                           1.06e-05     1.32588e-05   1.8454e-05    1.2924e-05    1.95118e-06      10     
end_path:out1:RootOutput(write)                     3.6516        4.53057       5.60242       4.35735      0.652122        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5169.27 MB
  Peak resident set size usage (VmHWM): 4343.1 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.33 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.75 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.20 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11198712
-rw-r--r-- 1 dune001 dune 2602010592 Oct 30 11:34 detsim_stage2.root
-rw-r--r-- 1 dune001 dune 2463969570 Oct 30 11:32 detsim_stage1.root
-rw-r--r-- 1 dune001 dune 2358928622 Oct 30 13:16 reco_stage1.root
-rw-r--r-- 1 dune001 dune 2025106039 Oct 30 08:55 g4_stage2.root
-rw-r--r-- 1 dune001 dune 1924294979 Oct 30 08:36 g4_stage1.root
-rw-r--r-- 1 dune001 dune   92539830 Oct 30 13:07 Pandora_Events.pndr
-rw-r--r-- 1 dune001 dune     243154 Oct 30 13:16 jobscript.log
-rw-r--r-- 1 dune001 dune     217088 Oct 30 13:16 mem.db
-rw-r--r-- 1 dune001 dune      62376 Oct 30 13:16 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 dune      36864 Oct 30 13:16 time.db
-rw-r--r-- 1 dune001 dune        519 Oct 30 11:34 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Oct 30 08:55 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Oct 30 13:16 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 dune         52 Oct 30 08:26 all-input-dids.txt
-rw-r--r-- 1 dune001 dune          7 Oct 30 13:16 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune          0 Oct 30 11:44 ProtoDUNE.log
-rw-r--r-- 1 dune001 dune          0 Oct 30 11:34 debugprod.log
justIN time: 2025-11-05 04:51:41 UTC       justIN version: 01.05.01