justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 294672.0@dunegpschedd02.fnal.gov

Jobsub ID294672.0@dunegpschedd02.fnal.gov
Workflow ID12380
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-24 21:42:02
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce504
Last heartbeat2026-01-25 01:18:41
From worker nodeHostnameb9p06p0505.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-24 21:42:44
Input filesmonte-carlo-012380-000001
JobscriptExit code0
Real time3h (12864s)
CPU time3h (12225s = 95%)
Max RSS bytes4142841856 (3950 MiB)
Outputting started2026-01-25 01:17:09
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12380/1/001/reco_stage1.root
Finished2026-01-25 01:18:41
Saved logsjustin-logs:294672.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

aleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 150641 traces, 0 summary
Retagger: tagging trace set: gauss with 113112 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 113112 traces tagged "gauss"
FrameSaver: q=4.09931e+06 n=1973149 tag=gauss
wclsFrameSaver: saving 150641 traces tagged "wiener"
FrameSaver: q=4.36491e+06 n=1740144 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 34950 XUs and 36877 XVs -> 632 XUVs
C:0 T:1 83995 XUs and 95353 XVs -> 9689 XUVs
C:0 T:2 96115 XUs and 125088 XVs -> 6863 XUVs
C:0 T:3 42411 XUs and 50778 XVs -> 848 XUVs
C:0 T:4 10643 XUs and 15517 XVs -> 141 XUVs
C:0 T:5 44861 XUs and 55497 XVs -> 9872 XUVs
C:0 T:6 76429 XUs and 79871 XVs -> 4260 XUVs
C:0 T:7 31202 XUs and 30279 XVs -> 373 XUVs
C:0 T:8 11724 XUs and 14081 XVs -> 248 XUVs
C:0 T:9 75142 XUs and 111938 XVs -> 7916 XUVs
C:0 T:10 54084 XUs and 58589 XVs -> 6861 XUVs
C:0 T:11 187 XUs and 126 XVs -> 24 XUVs
47727 XUVs total
12724 collection wire objects
43077 potential space points
Neighbour search...
5478241 tests to find 2450250 neighbours
Iterating with no regularization...
Begin: 8.70492e+07
0 6.54317e+07
1 6.35837e+07
2 6.33126e+07
3 6.32343e+07
4 6.32025e+07
Now with regularization...
Begin: 5.00917e+07
0 5.00205e+07
1 4.99875e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411945047
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411945047
fLifetime: 30000 [us]
25-Jan-2026 02:17:07 CET  Closed output file "reco_stage1.root"
25-Jan-2026 02:17:07 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          266.519       326.42        418.904       314.436       45.8451        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000860506   0.00339484    0.00506741    0.00389368    0.00133885       10     
reco:ophit:OpHitFinder                             0.241324      0.306232       0.37843      0.311315      0.0366414       10     
reco:opflash:OpFlashFinder                         0.137319      0.186613      0.238943       0.18252      0.0273074       10     
reco:opslicer:OpSlicer                             0.609947      0.724075      0.894041      0.700897      0.0773928       10     
reco:caldata:DataPrepModule                         5.08567       5.52345       6.06457       5.42737      0.289077        10     
reco:wclsdatasp:WireCellToolkit                     30.2293       32.1565       34.6641       31.9963       1.40346        10     
reco:gaushit:GausHitFinder                         0.967746       1.14074       1.40938       1.13645      0.132186        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000315512   0.000508095   0.000676615   0.000506589   0.000103173      10     
reco:reco3d:SpacePointSolver                        8.11821       16.6409       28.2115       16.9737       5.44162        10     
reco:hitpdune:DisambigFromSpacePoints               1.2187        1.67699       2.43369       1.65483       0.36544        10     
reco:pandora:StandardPandora                        20.2189       30.3955       53.1651       27.6707       8.88093        10     
reco:pandoraWriter:StandardPandora                 0.366503      0.438244      0.537138       0.42859      0.0561899       10     
reco:pandoraTrack:LArPandoraTrackCreation           2.75603       3.82594       6.36563       3.7692       0.966092        10     
reco:pandoraShower:LArPandoraShowerCreation         2.66286       3.72752       5.06252       3.44916      0.731497        10     
reco:pandoracalo:Calorimetry                        1.62688       2.13756       3.25993       2.04264       0.43096        10     
reco:pandoracalonosce:Calorimetry                   1.3215        1.89072       2.97213       1.76055      0.427703        10     
reco:pandorapid:Chi2ParticleID                    0.00315193    0.00564304    0.00762012    0.00549917    0.00134564       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0296456     0.0517858     0.062876      0.0527864    0.00858401       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00298564    0.00554362     0.0080985    0.00560721    0.00129985       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.87802       3.89205       5.49944       3.83178      0.833748        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.87408       3.78448       5.60527       3.78777      0.852946        10     
reco:emtrkmichelid:EmTrackMichelId                  149.873       177.379       215.526       170.902       20.5737        10     
reco:crttag:SingleCRTMatchingProducer             0.00734292     0.0144395     0.0219643     0.0141151    0.00391959       10     
reco:crtreco:TwoCRTMatchingProducer               0.00463269     0.0074976    0.00915774    0.00744388    0.00115801       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.558932      0.904057       1.16738      0.888108      0.161884        10     
reco:pandora2Track:LArPandoraTrackCreation          8.08905       11.4508       15.1743       10.8563       2.4273         10     
reco:pandora2calo:Calorimetry                       4.24919       5.84722       8.43527       5.50383       1.35581        10     
reco:pandora2calonosce:Calorimetry                  3.62842       5.53768       7.65646       5.29138       1.20478        10     
reco:pandora2pid:Chi2ParticleID                   0.00972981     0.0139214     0.0166404     0.0144179    0.00204319       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0565568     0.0675358     0.0809105     0.065635     0.00897857       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0119715     0.0138592     0.0170497     0.0131881    0.00179799       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.02122       4.0836        5.32174       3.8789       0.813405        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.75812       5.02907       6.57636       4.80092      0.983943        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.40518       4.68101       6.24267       4.42848      0.958388        10     
[art]:TriggerResults:TriggerResultInserter        3.8381e-05    6.15407e-05   0.000131702   5.5666e-05    2.52953e-05      10     
end_path:out1:RootOutput                           6.78e-06     9.1361e-06     1.339e-05     8.87e-06     1.82954e-06      10     
end_path:out1:RootOutput(write)                     2.13098       2.79559       3.69793       2.61567      0.480391        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4989.56 MB
  Peak resident set size usage (VmHWM): 4142.84 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.25 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.50 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10729828
-rw-r--r--. 1 duneprd np-comp 2496065905 Jan 25 01:20 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2368867312 Jan 25 01:18 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2189483942 Jan 25 02:17 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1974645673 Jan 24 23:06 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1880809084 Jan 24 22:49 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   76599070 Jan 25 02:12 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     243007 Jan 25 02:17 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan 25 02:17 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan 25 02:17 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan 25 02:17 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan 25 01:20 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 24 23:06 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 25 02:17 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan 24 22:42 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan 25 02:17 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan 25 01:25 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan 25 01:20 debugprod.log
justIN time: 2026-02-05 01:19:47 UTC       justIN version: 01.06.00