justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 20184.1@dunegpschedd01.fnal.gov

Jobsub ID20184.1@dunegpschedd01.fnal.gov
Workflow ID206
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-08-01 11:28:01
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-08-01 14:09:35
From worker nodeHostnamedunegli-6428343-0-fnpc19119.fnal.gov
cpuinfoAMD EPYC 7502 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-08-01 11:29:31
Input filesmonte-carlo-000206-000001
JobscriptExit code0
Real time2h (9587s)
CPU time2h (9495s = 99%)
Max RSS bytes4167163904 (3974 MiB)
Outputting started2025-08-01 14:09:19
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/00206/1/001/reco_stage1.root
Finished2025-08-01 14:09:35
Saved logsjustin-logs:20184.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

epare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 138692 traces, 0 summary
Retagger: tagging trace set: gauss with 106818 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 106818 traces tagged "gauss"
FrameSaver: q=3.78669e+06 n=1924080 tag=gauss
wclsFrameSaver: saving 138692 traces tagged "wiener"
FrameSaver: q=4.01152e+06 n=1720411 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 36586 XUs and 43728 XVs -> 600 XUVs
C:0 T:1 88133 XUs and 89218 XVs -> 7013 XUVs
C:0 T:2 47831 XUs and 57893 XVs -> 1577 XUVs
C:0 T:3 28905 XUs and 29611 XVs -> 129 XUVs
C:0 T:4 16299 XUs and 12921 XVs -> 105 XUVs
C:0 T:5 87947 XUs and 38644 XVs -> 5222 XUVs
C:0 T:6 83977 XUs and 115116 XVs -> 3520 XUVs
C:0 T:7 36466 XUs and 49606 XVs -> 473 XUVs
C:0 T:8 11344 XUs and 11512 XVs -> 378 XUVs
C:0 T:9 108074 XUs and 109776 XVs -> 17111 XUVs
C:0 T:10 36210 XUs and 27767 XVs -> 4210 XUVs
C:0 T:11 266 XUs and 210 XVs -> 19 XUVs
40357 XUVs total
12020 collection wire objects
36700 potential space points
Neighbour search...
4295624 tests to find 1996460 neighbours
Iterating with no regularization...
Begin: 7.09766e+07
0 5.62383e+07
1 5.52541e+07
2 5.50955e+07
3 5.50442e+07
Now with regularization...
Begin: 4.44463e+07
0 4.44149e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 408381189
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 408381189
fLifetime: 30000 [us]
01-Aug-2025 14:09:18 UTC  Closed output file "reco_stage1.root"
01-Aug-2025 14:09:18 UTC  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          191.315       294.117       483.906       266.594       86.6218        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000667875   0.00305514    0.00464655    0.00326801    0.00129619       10     
reco:ophit:OpHitFinder                             0.213037      0.243715      0.298505      0.234499      0.0273724       10     
reco:opflash:OpFlashFinder                         0.112726      0.146651      0.210063      0.140754      0.0292706       10     
reco:opslicer:OpSlicer                             0.457093      0.566508      0.818603       0.52834      0.102961        10     
reco:caldata:DataPrepModule                         4.2949        4.4347        4.69273       4.40507       0.11377        10     
reco:wclsdatasp:WireCellToolkit                     28.0772       30.7052       33.4168       31.1708       1.62881        10     
reco:gaushit:GausHitFinder                         0.696937      0.981948       1.74972      0.866585      0.298048        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000459573   0.000701925   0.000991247   0.00068619    0.00016774       10     
reco:reco3d:SpacePointSolver                        4.59754       10.2646       18.7403       8.81801       4.42353        10     
reco:hitpdune:DisambigFromSpacePoints              0.737656       1.31612       2.78424       1.06442      0.587291        10     
reco:pandora:StandardPandora                        10.1983       29.169        82.2537       21.1017       20.4714        10     
reco:pandoraWriter:StandardPandora                 0.243397      0.381802      0.575257      0.353912      0.102098        10     
reco:pandoraTrack:LArPandoraTrackCreation           1.58432       3.22481       7.23407       2.40134       1.76401        10     
reco:pandoraShower:LArPandoraShowerCreation         1.22014       3.88266       9.00244       2.56967       2.66497        10     
reco:pandoracalo:Calorimetry                       0.850741       1.62579       3.43091       1.34539      0.772059        10     
reco:pandoracalonosce:Calorimetry                  0.750493       1.5315        3.81131       1.20825      0.882993        10     
reco:pandorapid:Chi2ParticleID                    0.00376685    0.00475188    0.00725059    0.00415767    0.00107767       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0319351     0.0446479     0.0607687     0.0417993    0.00959527       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00349357    0.00440514    0.00566467     0.0040407    0.000814839      10     
reco:pandoraShowercalo:ShowerCalorimetry            1.18208       3.63863       8.71353       2.58703       2.4492         10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.15853       3.48449       7.9029        2.56368       2.25966        10     
reco:emtrkmichelid:EmTrackMichelId                  117.183       165.565       246.329       156.87        38.5398        10     
reco:crttag:SingleCRTMatchingProducer             0.00806771     0.011812      0.0192025     0.0103978    0.00374107       10     
reco:crtreco:TwoCRTMatchingProducer               0.00469525    0.00685444     0.0099907    0.00670966    0.00171212       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.395035      0.683039       1.03943      0.640498      0.179235        10     
reco:pandora2Track:LArPandoraTrackCreation          4.05984       9.99198       21.9668       8.4062        5.25713        10     
reco:pandora2calo:Calorimetry                       1.90749       4.35886       8.84232       4.04478       2.05332        10     
reco:pandora2calonosce:Calorimetry                  1.75647       4.08184       8.12629       3.75192       1.89489        10     
reco:pandora2pid:Chi2ParticleID                    0.0076113     0.0117227     0.0184269     0.0109406    0.00304917       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0378925      0.05535      0.082679      0.0540785     0.0131662       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00733349     0.0116397     0.0182482     0.0110882    0.00306332       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.57952       3.71769       7.37956       3.33204       1.73468        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.65688       3.76438       7.19939       3.40206       1.6643         10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.5334        3.47258       6.58483       3.22317       1.51523        10     
[art]:TriggerResults:TriggerResultInserter         3.678e-05    5.20591e-05   0.000136601    4.372e-05    2.84162e-05      10     
end_path:out1:RootOutput                           7.55e-06     1.1047e-05     1.618e-05    1.0835e-05    2.06832e-06      10     
end_path:out1:RootOutput(write)                     1.87653       2.62244       3.80698       2.43695      0.631845        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4977.15 MB
  Peak resident set size usage (VmHWM): 4167.16 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.19 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.72 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9899464
-rw-r--r-- 1 dunegli fnalgrid 2335208316 Aug  1 13:18 detsim_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2217422774 Aug  1 13:16 detsim_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 2022165368 Aug  1 14:09 reco_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1777790891 Aug  1 11:43 g4_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 1709272599 Aug  1 11:33 g4_stage1.root
-rw-r--r-- 1 dunegli fnalgrid   74563850 Aug  1 14:04 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid     242813 Aug  1 14:09 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid     217088 Aug  1 14:09 mem.db
-rw-r--r-- 1 dunegli fnalgrid      62376 Aug  1 14:09 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid      36864 Aug  1 14:09 time.db
-rw-r--r-- 1 dunegli fnalgrid        519 Aug  1 13:18 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Aug  1 11:43 g4_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Aug  1 14:09 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dunegli fnalgrid         52 Aug  1 11:29 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid          7 Aug  1 14:09 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid          0 Aug  1 13:24 ProtoDUNE.log
-rw-r--r-- 1 dunegli fnalgrid          0 Aug  1 13:18 debugprod.log
justIN time: 2025-08-04 19:41:43 UTC       justIN version: 01.04.00