justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 267283.0@dunegpschedd01.fnal.gov

Jobsub ID267283.0@dunegpschedd01.fnal.gov
Workflow ID11181
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-12 08:07:11
SiteIT_CNAF
EntryCMSHTPC_T1_IT_CNAF_condor_ce02
Last heartbeat2025-12-12 17:25:54
From worker nodeHostnamewn-f01-13-05-05.cr.cnaf.infn.it
cpuinfoIntel(R) Xeon(R) CPU E5-2697 v4 @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-12 11:35:59
Input filesmonte-carlo-011181-000001
JobscriptExit code0
Real time5h (20925s)
CPU time5h (20423s = 97%)
Max RSS bytes4290719744 (4091 MiB)
Outputting started2025-12-12 17:24:45
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11181/1/001/reco_stage1.root
Finished2025-12-12 17:25:54
Saved logsjustin-logs:267283.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ndEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 146024 traces, 0 summary
Retagger: tagging trace set: gauss with 110406 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 110406 traces tagged "gauss"
FrameSaver: q=3.93229e+06 n=1910016 tag=gauss
wclsFrameSaver: saving 146024 traces tagged "wiener"
FrameSaver: q=4.16073e+06 n=1684292 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 23001 XUs and 28086 XVs -> 105 XUVs
C:0 T:1 31332 XUs and 32544 XVs -> 2648 XUVs
C:0 T:2 120055 XUs and 119926 XVs -> 4563 XUVs
C:0 T:3 63012 XUs and 51649 XVs -> 351 XUVs
C:0 T:4 12013 XUs and 16360 XVs -> 81 XUVs
C:0 T:5 47298 XUs and 47932 XVs -> 3957 XUVs
C:0 T:6 109154 XUs and 113996 XVs -> 8233 XUVs
C:0 T:7 46489 XUs and 46252 XVs -> 746 XUVs
C:0 T:8 9247 XUs and 8866 XVs -> 138 XUVs
C:0 T:9 52712 XUs and 60608 XVs -> 4144 XUVs
C:0 T:10 46665 XUs and 36220 XVs -> 7513 XUVs
C:0 T:11 436 XUs and 354 XVs -> 66 XUVs
32545 XUVs total
11420 collection wire objects
29859 potential space points
Neighbour search...
1747001 tests to find 977634 neighbours
Iterating with no regularization...
Begin: 7.86967e+07
0 6.90344e+07
1 6.83266e+07
2 6.82351e+07
3 6.82122e+07
Now with regularization...
Begin: 5.64771e+07
0 5.64645e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411058653
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411058653
fLifetime: 30000 [us]
12-Dec-2025 18:24:41 CET  Closed output file "reco_stage1.root"
12-Dec-2025 18:24:41 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          573.519       839.517       1262.27       765.335       210.216        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00118925     0.0062526     0.0116792    0.00620241     0.0031282       10     
reco:ophit:OpHitFinder                             0.405573       0.49388      0.553084      0.509697      0.0426534       10     
reco:opflash:OpFlashFinder                         0.211835      0.277345      0.347328      0.280539      0.0417776       10     
reco:opslicer:OpSlicer                             0.926989       1.32808       1.4952        1.39118       0.17691        10     
reco:caldata:DataPrepModule                         8.81126       9.34036       9.69623       9.38312       0.21792        10     
reco:wclsdatasp:WireCellToolkit                     61.131        66.6326       71.4919       67.1957       3.21375        10     
reco:gaushit:GausHitFinder                          1.87388       2.51355       3.35212       2.37755      0.460894        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000676588   0.00096517    0.00126202    0.000932064   0.000182039      10     
reco:reco3d:SpacePointSolver                        15.388        34.9681       67.3669       30.6998       16.9101        10     
reco:hitpdune:DisambigFromSpacePoints               3.43398       5.3877        8.33882       4.9416        1.63686        10     
reco:pandora:StandardPandora                        51.075        91.3871       215.291       70.1326       50.582         10     
reco:pandoraWriter:StandardPandora                 0.640863       0.88589       1.18591      0.843856      0.165404        10     
reco:pandoraTrack:LArPandoraTrackCreation           8.96277       15.4887       30.6353       13.1704       6.19441        10     
reco:pandoraShower:LArPandoraShowerCreation         7.62124       17.0625       28.7519       14.6946       6.67337        10     
reco:pandoracalo:Calorimetry                        3.95016       7.05238       12.7425       6.19716       2.50945        10     
reco:pandoracalonosce:Calorimetry                   3.88726       6.63517       12.0456       5.82741       2.35932        10     
reco:pandorapid:Chi2ParticleID                     0.010397      0.0127677     0.0180835     0.0118378     0.0022342       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0919044      0.1297        0.23773      0.114931      0.0401469       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0102469     0.012044      0.0170785     0.0110975    0.00205722       10     
reco:pandoraShowercalo:ShowerCalorimetry            6.6127        16.0521       26.5452       14.0569       6.18389        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       6.48868       16.0617       29.2808       13.8387       6.61812        10     
reco:emtrkmichelid:EmTrackMichelId                  292.984       389.802       498.999       375.046       64.8941        10     
reco:crttag:SingleCRTMatchingProducer              0.0257385     0.0354392     0.0453482     0.0363411    0.00688359       10     
reco:crtreco:TwoCRTMatchingProducer                0.021505      0.0282887     0.036599      0.0272573    0.00538915       10     
reco:anodepiercerst0:T0RecoAnodePiercers            2.12128       2.54817       3.41474       2.44806      0.413722        10     
reco:pandora2Track:LArPandoraTrackCreation          30.6921       52.5473       86.5844       45.0177       20.2975        10     
reco:pandora2calo:Calorimetry                       12.4021       22.8359       38.6916       19.5283       9.00046        10     
reco:pandora2calonosce:Calorimetry                  11.8417       21.5189       35.6179       18.6848       8.4469         10     
reco:pandora2pid:Chi2ParticleID                    0.0256477     0.0357615     0.051972      0.0326028    0.00899828       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.123359      0.158948       0.21973      0.154238      0.0291347       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0255929     0.0326328     0.0492245     0.0297673    0.00723939       10     
reco:pandora2Shower:LArPandoraShowerCreation        9.95862       16.027        23.5364       14.2199       4.94703        10     
reco:pandora2Showercalo:ShowerCalorimetry           11.2324       19.0205       28.8656       17.1159       6.42979        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      10.6654       17.6388       29.1609       15.4487       6.28298        10     
[art]:TriggerResults:TriggerResultInserter        8.5239e-05    0.000104942   0.000177885   9.83845e-05   2.52624e-05      10     
end_path:out1:RootOutput                          1.2756e-05    0.000230058   0.00184801    1.44765e-05   0.000547703      10     
end_path:out1:RootOutput(write)                     4.14931       5.38043       7.41323       5.12504       1.03031        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5266.13 MB
  Peak resident set size usage (VmHWM): 4290.72 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.37 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.94 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.31 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11251320
-rw-r--r-- 1 neutrino002 neutrino 2607326379 Dec 12 16:01 detsim_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 2472335288 Dec 12 15:57 detsim_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 2325137332 Dec 12 18:24 reco_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 2061242911 Dec 12 13:14 g4_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 1972757914 Dec 12 12:49 g4_stage1.root
-rw-r--r-- 1 neutrino002 neutrino   81929670 Dec 12 18:14 Pandora_Events.pndr
-rw-r--r-- 1 neutrino002 neutrino     242969 Dec 12 18:24 jobscript.log
-rw-r--r-- 1 neutrino002 neutrino     217088 Dec 12 18:24 mem.db
-rw-r--r-- 1 neutrino002 neutrino      62376 Dec 12 18:24 Pandora_Geometry.xml
-rw-r--r-- 1 neutrino002 neutrino      36864 Dec 12 18:24 time.db
-rw-r--r-- 1 neutrino002 neutrino        519 Dec 12 16:01 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Dec 12 13:14 g4_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Dec 12 18:24 protoDUNE_reco1_hist.root
-rw-r--r-- 1 neutrino002 neutrino         52 Dec 12 12:36 all-input-dids.txt
-rw-r--r-- 1 neutrino002 neutrino          7 Dec 12 18:24 justin-processed-pfns.txt
-rw-r--r-- 1 neutrino002 neutrino          0 Dec 12 16:13 ProtoDUNE.log
-rw-r--r-- 1 neutrino002 neutrino          0 Dec 12 16:01 debugprod.log
justIN time: 2025-12-20 00:16:38 UTC       justIN version: 01.05.03