justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 231480.1@dunegpschedd02.fnal.gov

Jobsub ID231480.1@dunegpschedd02.fnal.gov
Workflow ID9223
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-20 23:10:23
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce04
Last heartbeat2025-10-21 02:44:53
From worker nodeHostnameccwcondor0053
cpuinfoAMD EPYC 9334 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-20 23:11:12
Input filesmonte-carlo-009223-000001
JobscriptExit code0
Real time3h (12725s)
CPU time3h (12361s = 97%)
Max RSS bytes4322758656 (4122 MiB)
Outputting started2025-10-21 02:43:18
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09223/1/001/reco_stage1.root
Finished2025-10-21 02:44:53
Saved logsjustin-logs:231480.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

e:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 121504 traces, 0 summary
Retagger: tagging trace set: gauss with 93365 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 93365 traces tagged "gauss"
FrameSaver: q=3.26851e+06 n=1679599 tag=gauss
wclsFrameSaver: saving 121504 traces tagged "wiener"
FrameSaver: q=3.46061e+06 n=1493906 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 29075 XUs and 36527 XVs -> 423 XUVs
C:0 T:1 69039 XUs and 126786 XVs -> 10591 XUVs
C:0 T:2 99051 XUs and 101324 XVs -> 5496 XUVs
C:0 T:3 55094 XUs and 44721 XVs -> 353 XUVs
C:0 T:4 9153 XUs and 13525 XVs -> 147 XUVs
C:0 T:5 55684 XUs and 60078 XVs -> 4404 XUVs
C:0 T:6 98787 XUs and 89928 XVs -> 3242 XUVs
C:0 T:7 43482 XUs and 37795 XVs -> 273 XUVs
C:0 T:8 6521 XUs and 7306 XVs -> 102 XUVs
C:0 T:9 62881 XUs and 67054 XVs -> 7110 XUVs
C:0 T:10 32062 XUs and 20277 XVs -> 2349 XUVs
C:0 T:11 118 XUs and 81 XVs -> 10 XUVs
34500 XUVs total
10904 collection wire objects
30981 potential space points
Neighbour search...
2652555 tests to find 1356330 neighbours
Iterating with no regularization...
Begin: 6.82631e+07
0 5.51753e+07
1 5.43168e+07
2 5.41778e+07
3 5.41383e+07
Now with regularization...
Begin: 4.47053e+07
0 4.46773e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410015117
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410015117
fLifetime: 30000 [us]
21-Oct-2025 04:43:15 CEST  Closed output file "reco_stage1.root"
21-Oct-2025 04:43:15 CEST  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          288.157       410.351       640.947       397.527       92.695         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00114805    0.00444995    0.00727409    0.00487547    0.00189729       10     
reco:ophit:OpHitFinder                             0.230802      0.296002      0.359023      0.302172      0.0351667       10     
reco:opflash:OpFlashFinder                         0.137888      0.181374      0.253058      0.177024      0.0307964       10     
reco:opslicer:OpSlicer                             0.506077      0.684458      0.971223      0.673272      0.128225        10     
reco:caldata:DataPrepModule                         5.07099       5.59698       6.09452       5.58936       0.3153         10     
reco:wclsdatasp:WireCellToolkit                     32.9645       34.9167       38.2834       34.6439       1.40192        10     
reco:gaushit:GausHitFinder                         0.997343       1.28067       1.72831       1.28439      0.217439        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000305945   0.000609453   0.00103532    0.000552565   0.000189839      10     
reco:reco3d:SpacePointSolver                        8.90265       20.1642       33.7416       19.4332       7.61014        10     
reco:hitpdune:DisambigFromSpacePoints               1.04652       1.7738        3.1118        1.70214      0.542621        10     
reco:pandora:StandardPandora                        22.9295       43.3494       102.581       32.1995       24.9032        10     
reco:pandoraWriter:StandardPandora                 0.357852       0.54057       0.82918      0.510778      0.116657        10     
reco:pandoraTrack:LArPandoraTrackCreation           3.46422       6.97232       14.7067       6.44966       2.9019         10     
reco:pandoraShower:LArPandoraShowerCreation         3.54607       8.13433       16.7975       8.18706       3.40297        10     
reco:pandoracalo:Calorimetry                        2.25256       3.68868       8.4666        3.0624        1.75514        10     
reco:pandoracalonosce:Calorimetry                   2.03293       3.34683       7.50828       2.89639       1.51258        10     
reco:pandorapid:Chi2ParticleID                    0.00478523    0.00641061     0.0085137    0.00608853    0.00108421       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.052524      0.0626629     0.0797887     0.0600667    0.00848756       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00430648    0.00684068     0.0141362    0.00591546    0.00267692       10     
reco:pandoraShowercalo:ShowerCalorimetry            4.41598       7.73823       15.155        7.04048       3.02119        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.52178       7.63812       14.7478       7.37745       2.9378         10     
reco:emtrkmichelid:EmTrackMichelId                  141.716       195.546       285.659       184.842       36.4703        10     
reco:crttag:SingleCRTMatchingProducer              0.0139835     0.020741      0.0285735     0.0215331    0.00435787       10     
reco:crtreco:TwoCRTMatchingProducer               0.00883851     0.0139562     0.0191345     0.0138098    0.00349793       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.747409       1.03981       1.53796      0.947997      0.256896        10     
reco:pandora2Track:LArPandoraTrackCreation          15.361        21.4566       39.5459       19.4452       6.79533        10     
reco:pandora2calo:Calorimetry                       5.96782       10.6625       19.463        10.1142       3.59949        10     
reco:pandora2calonosce:Calorimetry                  4.94321       10.1976       20.2449       9.39238       3.89098        10     
reco:pandora2pid:Chi2ParticleID                   0.00763999     0.014388      0.0254221     0.0150922    0.00504449       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0425435     0.0762204      0.10935      0.0771071     0.0231849       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0074312     0.0136069     0.0210104     0.0137825    0.00377401       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.36683       7.11288       11.8634       6.91108       2.00144        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.75988       7.75245       12.379        7.8867        2.03501        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.86202       6.83007       10.8538       6.49698       1.66886        10     
[art]:TriggerResults:TriggerResultInserter        6.0522e-05    8.40506e-05   0.000169007   7.3326e-05    3.11635e-05      10     
end_path:out1:RootOutput                           9.074e-06    1.16689e-05   1.6716e-05    1.09315e-05   2.03219e-06      10     
end_path:out1:RootOutput(write)                     2.32071       3.01706       4.21105       2.89926      0.518017        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5166.08 MB
  Peak resident set size usage (VmHWM): 4322.76 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.29 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.43 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10077124
-rw-r--r-- 1 dune001 lbno 2379654722 Oct 21 03:32 detsim_stage2.root
-rw-r--r-- 1 dune001 lbno 2257870889 Oct 21 03:30 detsim_stage1.root
-rw-r--r-- 1 dune001 lbno 2078446160 Oct 21 04:43 reco_stage1.root
-rw-r--r-- 1 dune001 lbno 1793477718 Oct 21 01:29 g4_stage2.root
-rw-r--r-- 1 dune001 lbno 1730698313 Oct 21 01:18 g4_stage1.root
-rw-r--r-- 1 dune001 lbno   78212990 Oct 21 04:38 Pandora_Events.pndr
-rw-r--r-- 1 dune001 lbno     243374 Oct 21 04:43 jobscript.log
-rw-r--r-- 1 dune001 lbno     217088 Oct 21 04:43 mem.db
-rw-r--r-- 1 dune001 lbno      62376 Oct 21 04:43 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 lbno      36864 Oct 21 04:43 time.db
-rw-r--r-- 1 dune001 lbno        519 Oct 21 03:32 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Oct 21 01:29 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Oct 21 04:43 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 lbno         52 Oct 21 01:11 all-input-dids.txt
-rw-r--r-- 1 dune001 lbno          7 Oct 21 04:43 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 lbno          0 Oct 21 03:38 ProtoDUNE.log
-rw-r--r-- 1 dune001 lbno          0 Oct 21 03:32 debugprod.log
justIN time: 2025-11-04 07:36:33 UTC       justIN version: 01.05.01