justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 271354.0@dunegpschedd01.fnal.gov

Jobsub ID271354.0@dunegpschedd01.fnal.gov
Workflow ID11475
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-02 19:14:47
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce15-multicore
Last heartbeat2026-01-02 21:52:33
From worker nodeHostnametdm003.pic.es
cpuinfoAMD EPYC 7662 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-02 19:15:30
Input filesmonte-carlo-011475-000001
JobscriptExit code0
Real time2h (9385s)
CPU time2h (9053s = 96%)
Max RSS bytes4319404032 (4119 MiB)
Outputting started2026-01-02 21:51:56
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11475/1/001/reco_stage1.root
Finished2026-01-02 21:52:33
Saved logsjustin-logs:271354.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

unning tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 164245 traces, 0 summary
Retagger: tagging trace set: gauss with 125217 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 125217 traces tagged "gauss"
FrameSaver: q=4.37708e+06 n=2175827 tag=gauss
wclsFrameSaver: saving 164245 traces tagged "wiener"
FrameSaver: q=4.63817e+06 n=1920351 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 36141 XUs and 57307 XVs -> 642 XUVs
C:0 T:1 94664 XUs and 101776 XVs -> 7220 XUVs
C:0 T:2 75887 XUs and 78559 XVs -> 4242 XUVs
C:0 T:3 37421 XUs and 33524 XVs -> 325 XUVs
C:0 T:4 16604 XUs and 25694 XVs -> 276 XUVs
C:0 T:5 90030 XUs and 118078 XVs -> 16209 XUVs
C:0 T:6 135123 XUs and 137668 XVs -> 12129 XUVs
C:0 T:7 51956 XUs and 47926 XVs -> 770 XUVs
C:0 T:8 4699 XUs and 8697 XVs -> 71 XUVs
C:0 T:9 30583 XUs and 68207 XVs -> 4058 XUVs
C:0 T:10 69031 XUs and 28863 XVs -> 4361 XUVs
C:0 T:11 403 XUs and 290 XVs -> 33 XUVs
50336 XUVs total
13893 collection wire objects
46138 potential space points
Neighbour search...
5840896 tests to find 2828936 neighbours
Iterating with no regularization...
Begin: 8.74513e+07
0 7.18763e+07
1 7.08608e+07
2 7.07309e+07
3 7.06929e+07
Now with regularization...
Begin: 5.92574e+07
0 5.92073e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411500449
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411500449
fLifetime: 30000 [us]
02-Jan-2026 22:51:54 CET  Closed output file "reco_stage1.root"
02-Jan-2026 22:51:54 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          278.099       333.447       414.97        321.555       42.854         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000825534   0.00347441    0.00556923    0.00362549    0.00156272       10     
reco:ophit:OpHitFinder                             0.237447      0.264946      0.324464      0.260032      0.0245408       10     
reco:opflash:OpFlashFinder                         0.148399       0.17178      0.221285      0.162499      0.0219185       10     
reco:opslicer:OpSlicer                             0.518824      0.637539      0.820599      0.648497      0.0868323       10     
reco:caldata:DataPrepModule                         4.31199       4.43319       4.50907       4.44429      0.0552139       10     
reco:wclsdatasp:WireCellToolkit                     29.429        32.3402       34.6637       32.9846       1.89676        10     
reco:gaushit:GausHitFinder                         0.903126       1.10175       1.45525       1.07982      0.157119        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000561079   0.000769202   0.00101274    0.000740917   0.000135193      10     
reco:reco3d:SpacePointSolver                        7.27226       11.9287       18.7507       10.8911       3.45323        10     
reco:hitpdune:DisambigFromSpacePoints               1.05949       1.45465       2.28115       1.32509      0.376239        10     
reco:pandora:StandardPandora                        19.6046       28.8377       40.6213       27.1367       6.40002        10     
reco:pandoraWriter:StandardPandora                 0.360843      0.424696      0.517634      0.409626      0.0472801       10     
reco:pandoraTrack:LArPandoraTrackCreation           2.44265       3.50976       4.97643       3.32006      0.889789        10     
reco:pandoraShower:LArPandoraShowerCreation         2.99773       3.85108       5.62765       3.27087      0.957089        10     
reco:pandoracalo:Calorimetry                        1.36244       1.77567       2.33258       1.68184      0.364434        10     
reco:pandoracalonosce:Calorimetry                   1.22575       1.61339       2.12099       1.52332      0.348541        10     
reco:pandorapid:Chi2ParticleID                    0.00422711    0.00526024    0.00620197    0.00527274    0.000622053      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.042496      0.047904      0.0569637     0.0482276    0.00492609       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00395989    0.00485834    0.00576138    0.00476243    0.000582314      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.97282       3.7294        5.44274       3.21053      0.850404        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.86489       3.65046       5.34131       3.13969      0.845062        10     
reco:emtrkmichelid:EmTrackMichelId                  165.956       194.067       232.909       187.754       21.8075        10     
reco:crttag:SingleCRTMatchingProducer             0.00906691     0.015951      0.0457853     0.0129188     0.0101284       10     
reco:crtreco:TwoCRTMatchingProducer               0.00657108     0.0080836     0.010515     0.00778278    0.00130891       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.688574      0.804833       1.02379      0.724093      0.127627        10     
reco:pandora2Track:LArPandoraTrackCreation          9.03217       12.1494       17.4186       10.552        3.04835        10     
reco:pandora2calo:Calorimetry                       4.08101       5.2772        7.28425       4.66661       1.12953        10     
reco:pandora2calonosce:Calorimetry                  3.83291       5.0077        6.87901       4.45171       1.06988        10     
reco:pandora2pid:Chi2ParticleID                    0.0112554     0.0131907     0.0157205     0.0128436     0.0012923       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0517072     0.0621137     0.0774454     0.0610992    0.00761083       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0110016     0.0130365     0.0153996     0.0126089    0.00137149       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.3301        4.42874       6.06247       3.94411      0.992023        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.51163       4.5987        6.12423       4.08868      0.923414        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.22238       4.23457       5.59746       3.78017       0.83539        10     
[art]:TriggerResults:TriggerResultInserter         3.497e-05    4.94569e-05   0.000116662   4.2971e-05    2.28374e-05      10     
end_path:out1:RootOutput                           9.13e-06     1.10441e-05    1.629e-05     1.057e-05    1.83425e-06      10     
end_path:out1:RootOutput(write)                     2.41243       2.8567        3.47954       2.72078       0.37478        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5138.58 MB
  Peak resident set size usage (VmHWM): 4319.4 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.19 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.74 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10771000
-rw-r--r-- 1 dune001 dune 2497551183 Jan  2 21:54 detsim_stage2.root
-rw-r--r-- 1 dune001 dune 2370077799 Jan  2 21:52 detsim_stage1.root
-rw-r--r-- 1 dune001 dune 2210848218 Jan  2 22:51 reco_stage1.root
-rw-r--r-- 1 dune001 dune 1963900228 Jan  2 20:34 g4_stage2.root
-rw-r--r-- 1 dune001 dune 1905236523 Jan  2 20:22 g4_stage1.root
-rw-r--r-- 1 dune001 dune   81261530 Jan  2 22:46 Pandora_Events.pndr
-rw-r--r-- 1 dune001 dune     242872 Jan  2 22:51 jobscript.log
-rw-r--r-- 1 dune001 dune     217088 Jan  2 22:51 mem.db
-rw-r--r-- 1 dune001 dune      62376 Jan  2 22:51 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 dune      36864 Jan  2 22:51 time.db
-rw-r--r-- 1 dune001 dune        519 Jan  2 21:54 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Jan  2 20:34 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Jan  2 22:51 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 dune         52 Jan  2 20:15 all-input-dids.txt
-rw-r--r-- 1 dune001 dune          7 Jan  2 22:51 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune          0 Jan  2 21:59 ProtoDUNE.log
-rw-r--r-- 1 dune001 dune          0 Jan  2 21:54 debugprod.log
justIN time: 2026-02-06 22:22:28 UTC       justIN version: 01.06.00