justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 265608.0@dunegpschedd02.fnal.gov

Jobsub ID265608.0@dunegpschedd02.fnal.gov
Workflow ID11479
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-03 04:01:15
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce04
Last heartbeat2026-01-03 08:00:21
From worker nodeHostnameccwcondor0406
cpuinfoAMD EPYC 9334 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-03 04:03:53
Input filesmonte-carlo-011479-000001
JobscriptExit code0
Real time3h (14095s)
CPU time3h (13671s = 96%)
Max RSS bytes4180127744 (3986 MiB)
Outputting started2026-01-03 07:58:50
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11479/1/001/reco_stage1.root
Finished2026-01-03 08:00:21
Saved logsjustin-logs:265608.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 129003 traces, 0 summary
Retagger: tagging trace set: gauss with 95731 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 95731 traces tagged "gauss"
FrameSaver: q=3.32855e+06 n=1658139 tag=gauss
wclsFrameSaver: saving 129003 traces tagged "wiener"
FrameSaver: q=3.53574e+06 n=1447648 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 11125 XUs and 18525 XVs -> 77 XUVs
C:0 T:1 24266 XUs and 28611 XVs -> 1113 XUVs
C:0 T:2 146434 XUs and 166277 XVs -> 17260 XUVs
C:0 T:3 62801 XUs and 47853 XVs -> 420 XUVs
C:0 T:4 5578 XUs and 4853 XVs -> 12 XUVs
C:0 T:5 21583 XUs and 12114 XVs -> 2620 XUVs
C:0 T:6 147290 XUs and 151396 XVs -> 9536 XUVs
C:0 T:7 53081 XUs and 49557 XVs -> 535 XUVs
C:0 T:8 10987 XUs and 13152 XVs -> 155 XUVs
C:0 T:9 86791 XUs and 119912 XVs -> 7587 XUVs
C:0 T:10 30302 XUs and 16480 XVs -> 4336 XUVs
C:0 T:11 226 XUs and 113 XVs -> 34 XUVs
43685 XUVs total
11830 collection wire objects
39978 potential space points
Neighbour search...
7002850 tests to find 2971350 neighbours
Iterating with no regularization...
Begin: 6.13751e+07
0 4.87714e+07
1 4.78016e+07
2 4.76634e+07
3 4.76294e+07
Now with regularization...
Begin: 3.71028e+07
0 3.70609e+07
1 3.7034e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411507858
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411507858
fLifetime: 30000 [us]
03-Jan-2026 08:58:47 CET  Closed output file "reco_stage1.root"
03-Jan-2026 08:58:47 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          294.896       463.95        672.245       425.743       102.498        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00145837    0.00481498     0.008933     0.00442712    0.00216491       10     
reco:ophit:OpHitFinder                             0.299359      0.341248      0.416723      0.335524      0.0322205       10     
reco:opflash:OpFlashFinder                         0.146356       0.1973       0.277717      0.186398      0.0377712       10     
reco:opslicer:OpSlicer                             0.582491      0.803086       1.02913      0.800099      0.123112        10     
reco:caldata:DataPrepModule                         5.96516       6.41321       6.8041        6.53334      0.255731        10     
reco:wclsdatasp:WireCellToolkit                     37.9136       41.2645       45.2837       41.4701       2.31802        10     
reco:gaushit:GausHitFinder                          1.10425       1.50611       1.93991       1.47853      0.215628        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000551015   0.000718859   0.00101671    0.000666525   0.000149935      10     
reco:reco3d:SpacePointSolver                        7.4735        22.1364       40.3108       20.202        11.1571        10     
reco:hitpdune:DisambigFromSpacePoints              0.979546       1.82033       2.72152       1.71551      0.494601        10     
reco:pandora:StandardPandora                        23.9515       41.7312       83.445        36.6038       17.6361        10     
reco:pandoraWriter:StandardPandora                 0.437228      0.582492      0.755061      0.553537      0.0940078       10     
reco:pandoraTrack:LArPandoraTrackCreation           3.71627       6.40466       10.244         6.094        1.89648        10     
reco:pandoraShower:LArPandoraShowerCreation         3.5861        9.12564       18.5404       8.36203       4.00625        10     
reco:pandoracalo:Calorimetry                        1.81543       3.6552        5.57482       3.32722       1.00658        10     
reco:pandoracalonosce:Calorimetry                   1.76887       3.41286       5.32874       3.15234       1.03422        10     
reco:pandorapid:Chi2ParticleID                    0.00540677    0.00618804    0.00777707    0.00575879    0.000851075      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0596726     0.0757817     0.0947931     0.0740727     0.0130629       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00511749    0.00601477    0.00780607    0.00575619    0.00080272       10     
reco:pandoraShowercalo:ShowerCalorimetry            3.22131       9.12408       19.2733       7.47782       4.42206        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.2132        8.77754       19.2139       7.01382       4.33959        10     
reco:emtrkmichelid:EmTrackMichelId                  165.145       228.19        295.144       219.054       34.5878        10     
reco:crttag:SingleCRTMatchingProducer              0.0149686     0.0287644     0.0497658     0.0247931     0.0108117       10     
reco:crtreco:TwoCRTMatchingProducer                0.0108903     0.0181403     0.0249783     0.019265     0.00386035       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.783723       1.0769        1.48444       1.07665      0.218376        10     
reco:pandora2Track:LArPandoraTrackCreation          12.0131       24.5015       45.2059       20.3814       9.62154        10     
reco:pandora2calo:Calorimetry                       4.41878       12.1869       21.8325       10.4649       5.30182        10     
reco:pandora2calonosce:Calorimetry                  4.05391       11.5532       21.1867       9.94888       5.14329        10     
reco:pandora2pid:Chi2ParticleID                    0.0108274     0.0157406     0.0223755     0.0144421    0.00346021       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0732616     0.100694      0.147002      0.0920672     0.0226438       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0107039     0.0147197     0.0202674     0.0140776    0.00288757       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.79007       8.37054       15.5379       7.29307       3.87894        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.86845       9.05718       17.5306       8.33264       4.10394        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.72016       8.05243       16.7476       7.45951       3.90606        10     
[art]:TriggerResults:TriggerResultInserter        6.4728e-05    0.000160599   0.000789136   8.6221e-05    0.000211399      10     
end_path:out1:RootOutput                          1.1958e-05    1.52332e-05   2.7522e-05    1.30995e-05   4.64875e-06      10     
end_path:out1:RootOutput(write)                     2.19206       3.17707       4.44031       2.93958      0.654492        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5025.72 MB
  Peak resident set size usage (VmHWM): 4180.13 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.34 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.92 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.18 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10132500
-rw-r--r-- 1 dune001 lbno 2370796646 Jan  3 07:39 detsim_stage2.root
-rw-r--r-- 1 dune001 lbno 2247334452 Jan  3 07:36 detsim_stage1.root
-rw-r--r-- 1 dune001 lbno 2057520160 Jan  3 08:58 reco_stage1.root
-rw-r--r-- 1 dune001 lbno 1845194828 Jan  3 05:26 g4_stage2.root
-rw-r--r-- 1 dune001 lbno 1779915855 Jan  3 05:11 g4_stage1.root
-rw-r--r-- 1 dune001 lbno   74303480 Jan  3 08:52 Pandora_Events.pndr
-rw-r--r-- 1 dune001 lbno     242720 Jan  3 08:58 jobscript.log
-rw-r--r-- 1 dune001 lbno     217088 Jan  3 08:58 mem.db
-rw-r--r-- 1 dune001 lbno      62376 Jan  3 08:58 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 lbno      36864 Jan  3 08:58 time.db
-rw-r--r-- 1 dune001 lbno        519 Jan  3 07:39 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan  3 05:26 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan  3 08:58 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 lbno         52 Jan  3 05:04 all-input-dids.txt
-rw-r--r-- 1 dune001 lbno          7 Jan  3 08:58 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 lbno          0 Jan  3 07:45 ProtoDUNE.log
-rw-r--r-- 1 dune001 lbno          0 Jan  3 07:39 debugprod.log
justIN time: 2026-02-06 22:18:14 UTC       justIN version: 01.06.00