justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 230073.1@dunegpschedd02.fnal.gov

Jobsub ID230073.1@dunegpschedd02.fnal.gov
Workflow ID9114
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-17 19:35:23
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce508
Last heartbeat2025-10-18 00:40:28
From worker nodeHostnameb9p02p9176.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-17 19:36:38
Input filesmonte-carlo-009114-000001
JobscriptExit code0
Real time5h (18128s)
CPU time4h (17297s = 95%)
Max RSS bytes4836192256 (4612 MiB)
Outputting started2025-10-18 00:38:47
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09114/1/001/reco_stage1.root
Finished2025-10-18 00:40:28
Saved logsjustin-logs:230073.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 224321 traces, 0 summary
Retagger: tagging trace set: gauss with 167066 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 167066 traces tagged "gauss"
FrameSaver: q=6.01145e+06 n=2984287 tag=gauss
wclsFrameSaver: saving 224321 traces tagged "wiener"
FrameSaver: q=6.38834e+06 n=2637183 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 86710 XUs and 73855 XVs -> 1970 XUVs
C:0 T:1 241441 XUs and 234991 XVs -> 23800 XUVs
C:0 T:2 101810 XUs and 145862 XVs -> 4884 XUVs
C:0 T:3 44662 XUs and 54894 XVs -> 377 XUVs
C:0 T:4 23629 XUs and 22472 XVs -> 495 XUVs
C:0 T:5 113556 XUs and 99227 XVs -> 12198 XUVs
C:0 T:6 148935 XUs and 243839 XVs -> 8963 XUVs
C:0 T:7 48932 XUs and 86301 XVs -> 850 XUVs
C:0 T:8 11283 XUs and 11316 XVs -> 157 XUVs
C:0 T:9 84589 XUs and 113276 XVs -> 12484 XUVs
C:0 T:10 61675 XUs and 52818 XVs -> 5563 XUVs
C:0 T:11 138 XUs and 95 XVs -> 10 XUVs
71751 XUVs total
21846 collection wire objects
63616 potential space points
Neighbour search...
7421556 tests to find 3475296 neighbours
Iterating with no regularization...
Begin: 1.15829e+08
0 9.32044e+07
1 9.16448e+07
2 9.14564e+07
3 9.14078e+07
Now with regularization...
Begin: 7.42888e+07
0 7.42334e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 409951762
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 409951762
fLifetime: 30000 [us]
18-Oct-2025 02:38:45 CEST  Closed output file "reco_stage1.root"
18-Oct-2025 02:38:45 CEST  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          373.832       483.727       638.576       449.067       96.1271        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00104153    0.00423734    0.00755752    0.00379433    0.00208156       10     
reco:ophit:OpHitFinder                             0.278941      0.341115      0.470443      0.333266      0.0514207       10     
reco:opflash:OpFlashFinder                          0.22479      0.265365      0.352802      0.261697      0.0392115       10     
reco:opslicer:OpSlicer                              0.5888       0.840716       1.3405       0.851466      0.212786        10     
reco:caldata:DataPrepModule                         4.23691       5.84822       7.49374       5.69262       1.06952        10     
reco:wclsdatasp:WireCellToolkit                     31.592        35.6781       40.6424       34.1934       3.25452        10     
reco:gaushit:GausHitFinder                          1.20809       1.63512       2.4664        1.54671      0.356493        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000385424   0.000676715   0.000966099   0.000694956   0.000170856      10     
reco:reco3d:SpacePointSolver                        16.1359       29.3828       45.9905       27.4979       9.22009        10     
reco:hitpdune:DisambigFromSpacePoints               1.68087       2.97955       5.22936       2.84343      0.950039        10     
reco:pandora:StandardPandora                        35.5354       61.4622       85.5976       61.3238       17.8954        10     
reco:pandoraWriter:StandardPandora                 0.393867      0.553452      0.690681      0.546588      0.106595        10     
reco:pandoraTrack:LArPandoraTrackCreation           3.22849       5.64229       8.75192       5.45065       1.59069        10     
reco:pandoraShower:LArPandoraShowerCreation         3.56484       6.3387        11.3226       5.86246       2.38066        10     
reco:pandoracalo:Calorimetry                        1.97893       3.0658        5.06409       2.96935       0.83072        10     
reco:pandoracalonosce:Calorimetry                   1.70871       2.91961       4.67674       2.77932      0.813872        10     
reco:pandorapid:Chi2ParticleID                    0.00449316    0.00685117    0.00903816    0.00649003    0.00159674       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0473501     0.0657488     0.0974096     0.0596302     0.0157455       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00480151    0.00684785    0.00939117     0.0069948    0.00152403       10     
reco:pandoraShowercalo:ShowerCalorimetry            3.92495       6.46786       11.6242       6.22611       2.09196        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.47343       6.57372       11.3285       6.13919       1.94007        10     
reco:emtrkmichelid:EmTrackMichelId                  198.619       247.42        315.005       229.706       40.5205        10     
reco:crttag:SingleCRTMatchingProducer              0.0109818     0.0184447     0.0268062     0.0180606    0.00567518       10     
reco:crtreco:TwoCRTMatchingProducer               0.00675131     0.0105806     0.014967      0.0102553    0.00297109       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.869976       1.17103       1.53445       1.07258      0.216944        10     
reco:pandora2Track:LArPandoraTrackCreation          11.6225       19.6058       33.5788       15.4882       8.08087        10     
reco:pandora2calo:Calorimetry                       5.32381       9.67808       19.8815       8.09067       4.23929        10     
reco:pandora2calonosce:Calorimetry                   5.506        9.37012       18.3766       8.19539       3.90973        10     
reco:pandora2pid:Chi2ParticleID                    0.0104251     0.0176089     0.0303145     0.016308     0.00652403       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.052994      0.0874331     0.143536      0.0830447     0.0297822       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0111589     0.0178164     0.0299349     0.0158339     0.0062237       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.93069       6.64562       10.4954       6.03486       2.09935        10     
reco:pandora2Showercalo:ShowerCalorimetry           6.03863       8.32807       12.5344       7.0829        2.29807        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.55384       7.41668       11.4078       6.3556        2.34438        10     
[art]:TriggerResults:TriggerResultInserter        3.7381e-05    7.0022e-05    0.000205802   5.3091e-05    4.72734e-05      10     
end_path:out1:RootOutput                           9.58e-06     1.22591e-05    2.145e-05    1.0785e-05    3.45219e-06      10     
end_path:out1:RootOutput(write)                     2.71278       3.74709       5.3002        3.50413      0.964225        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5674.66 MB
  Peak resident set size usage (VmHWM): 4836.19 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.25 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.76 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 12772600
-rw-r--r--. 1 duneprd np-comp 2913952214 Oct 18 01:14 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2761364036 Oct 18 01:12 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2708183800 Oct 18 02:38 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 2345242657 Oct 17 22:05 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 2248061061 Oct 17 21:47 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp  101715700 Oct 18 02:31 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     243330 Oct 18 02:38 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Oct 18 02:38 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Oct 18 02:38 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Oct 18 02:38 time.db
-rw-r--r--. 1 duneprd np-comp        519 Oct 18 01:14 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Oct 17 22:05 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Oct 18 02:38 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Oct 17 21:36 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Oct 18 02:38 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Oct 18 01:24 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Oct 18 01:14 debugprod.log
justIN time: 2025-11-05 04:53:14 UTC       justIN version: 01.05.01