justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 240828.1@dunegpschedd01.fnal.gov

Jobsub ID240828.1@dunegpschedd01.fnal.gov
Workflow ID9459
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-30 07:25:24
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce15-multicore
Last heartbeat2025-10-30 12:12:30
From worker nodeHostnametds410.pic.es
cpuinfoAMD EPYC 7502 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-30 07:26:16
Input filesmonte-carlo-009459-000001
JobscriptExit code0
Real time4h (17123s)
CPU time4h (15985s = 93%)
Max RSS bytes4298694656 (4099 MiB)
Outputting started2025-10-30 12:11:40
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09459/1/001/reco_stage1.root
Finished2025-10-30 12:12:30
Saved logsjustin-logs:240828.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

SignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 134352 traces, 0 summary
Retagger: tagging trace set: gauss with 98368 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 98368 traces tagged "gauss"
FrameSaver: q=3.45845e+06 n=1689025 tag=gauss
wclsFrameSaver: saving 134352 traces tagged "wiener"
FrameSaver: q=3.68429e+06 n=1466709 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 44298 XUs and 33878 XVs -> 508 XUVs
C:0 T:1 85047 XUs and 91289 XVs -> 9385 XUVs
C:0 T:2 125619 XUs and 165472 XVs -> 22130 XUVs
C:0 T:3 46065 XUs and 38725 XVs -> 430 XUVs
C:0 T:4 15077 XUs and 13570 XVs -> 105 XUVs
C:0 T:5 81277 XUs and 61504 XVs -> 8930 XUVs
C:0 T:6 91049 XUs and 79252 XVs -> 7967 XUVs
C:0 T:7 36498 XUs and 22343 XVs -> 205 XUVs
C:0 T:8 11964 XUs and 9217 XVs -> 157 XUVs
C:0 T:9 108102 XUs and 101195 XVs -> 10123 XUVs
C:0 T:10 9996 XUs and 5787 XVs -> 1667 XUVs
C:0 T:11 123 XUs and 74 XVs -> 41 XUVs
61648 XUVs total
12892 collection wire objects
56020 potential space points
Neighbour search...
10312940 tests to find 4556488 neighbours
Iterating with no regularization...
Begin: 7.39298e+07
0 5.8938e+07
1 5.78647e+07
2 5.77108e+07
3 5.76582e+07
Now with regularization...
Begin: 4.66701e+07
0 4.6585e+07
1 4.65263e+07
2 4.64825e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410203009
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410203009
fLifetime: 30000 [us]
30-Oct-2025 13:11:36 CET  Closed output file "reco_stage1.root"
30-Oct-2025 13:11:36 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          353.425       547.853       874.876       543.982       131.21         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00118184    0.00460231    0.00804761     0.0049465    0.00200162       10     
reco:ophit:OpHitFinder                             0.263934      0.383081      0.420326      0.393758      0.0412993       10     
reco:opflash:OpFlashFinder                         0.140907      0.222928      0.276285      0.226579      0.0354499       10     
reco:opslicer:OpSlicer                             0.565188      0.953851       1.13435       1.02458       0.18852        10     
reco:caldata:DataPrepModule                         6.46838       7.4904        8.1283        7.67504      0.487223        10     
reco:wclsdatasp:WireCellToolkit                     44.2284       48.7854       51.5142       50.0818       2.73652        10     
reco:gaushit:GausHitFinder                          1.05956       1.79795       2.89373       1.74141      0.465147        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000531325   0.000813077   0.00117543    0.000784408   0.000171033      10     
reco:reco3d:SpacePointSolver                        9.04078       19.8045       34.2707       17.5773       8.01794        10     
reco:hitpdune:DisambigFromSpacePoints               1.16351       2.25345       4.23853       2.10403      0.786779        10     
reco:pandora:StandardPandora                        17.838        52.1711       139.164       44.3433       31.2299        10     
reco:pandoraWriter:StandardPandora                 0.430542      0.669482      0.975385      0.660481      0.141704        10     
reco:pandoraTrack:LArPandoraTrackCreation           2.64793       6.5691        9.98238       6.56298       2.01229        10     
reco:pandoraShower:LArPandoraShowerCreation         1.8143        6.91995       13.5879       7.43161       3.00376        10     
reco:pandoracalo:Calorimetry                        1.47983       3.56714       5.5475        3.60144       1.09604        10     
reco:pandoracalonosce:Calorimetry                   1.32906       3.26522       4.89072       3.42299      0.995282        10     
reco:pandorapid:Chi2ParticleID                    0.00638396    0.00917016     0.0114433     0.0090955    0.00148255       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0584629     0.0898952     0.113446      0.0874175     0.0141457       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00682923    0.00897777     0.0111835    0.00864599    0.00128876       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.85285       6.7345        10.7114       7.48044       2.43872        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.84651       6.49896       10.9618       6.91259       2.52162        10     
reco:emtrkmichelid:EmTrackMichelId                  227.071       315.05        464.264       307.168       61.2166        10     
reco:crttag:SingleCRTMatchingProducer              0.0136849     0.020415      0.0294961     0.0194651    0.00579331       10     
reco:crtreco:TwoCRTMatchingProducer                0.0100839     0.0141893     0.020563      0.0132077    0.00383839       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.949771       1.1753        1.52455       1.14149      0.176408        10     
reco:pandora2Track:LArPandoraTrackCreation          8.05847       19.2604       36.9902       18.7323       7.52639        10     
reco:pandora2calo:Calorimetry                       3.85273       9.17916        17.43        9.05964       3.60706        10     
reco:pandora2calonosce:Calorimetry                  3.4445         8.975        16.8293       8.78629       3.37073        10     
reco:pandora2pid:Chi2ParticleID                    0.014026      0.0206013     0.0313441     0.0209818    0.00489167       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0717405     0.105109      0.153904      0.103833      0.0235879       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0140769     0.0202079     0.0293125     0.0207403    0.00452775       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.93475       7.17896       13.1092       6.52322       2.70434        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.83476       7.77832       14.326        7.26903       2.99357        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.68656       6.60431       10.7615       6.18651       2.18437        10     
[art]:TriggerResults:TriggerResultInserter        6.1361e-05    7.88587e-05   0.000167072   6.98005e-05   2.98039e-05      10     
end_path:out1:RootOutput                           1.084e-05    1.35742e-05   1.8401e-05     1.309e-05    1.93887e-06      10     
end_path:out1:RootOutput(write)                     2.90942       4.10212       5.87653       3.87236      0.806863        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5133.76 MB
  Peak resident set size usage (VmHWM): 4298.69 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.32 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.73 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.19 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10399796
-rw-r--r-- 1 dune001 dune 2432525428 Oct 30 11:36 detsim_stage2.root
-rw-r--r-- 1 dune001 dune 2312518037 Oct 30 11:32 detsim_stage1.root
-rw-r--r-- 1 dune001 dune 2108479082 Oct 30 13:11 reco_stage1.root
-rw-r--r-- 1 dune001 dune 1911806755 Oct 30 08:59 g4_stage2.root
-rw-r--r-- 1 dune001 dune 1810952896 Oct 30 08:37 g4_stage1.root
-rw-r--r-- 1 dune001 dune   72477700 Oct 30 13:04 Pandora_Events.pndr
-rw-r--r-- 1 dune001 dune     243227 Oct 30 13:11 jobscript.log
-rw-r--r-- 1 dune001 dune     217088 Oct 30 13:11 mem.db
-rw-r--r-- 1 dune001 dune      62376 Oct 30 13:11 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 dune      36864 Oct 30 13:11 time.db
-rw-r--r-- 1 dune001 dune        519 Oct 30 11:36 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Oct 30 08:59 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Oct 30 13:11 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 dune         52 Oct 30 08:26 all-input-dids.txt
-rw-r--r-- 1 dune001 dune          7 Oct 30 13:11 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune          0 Oct 30 11:44 ProtoDUNE.log
-rw-r--r-- 1 dune001 dune          0 Oct 30 11:36 debugprod.log
justIN time: 2025-11-05 04:49:59 UTC       justIN version: 01.05.01