justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 264623.0@dunegpschedd02.fnal.gov

Jobsub ID264623.0@dunegpschedd02.fnal.gov
Workflow ID11369
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-27 22:55:14
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep208
Last heartbeat2025-12-28 02:46:14
From worker nodeHostnameheplnc155.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-27 23:04:41
Input filesmonte-carlo-011369-000001
JobscriptExit code0
Real time3h (13195s)
CPU time3h (12712s = 96%)
Max RSS bytes4446310400 (4240 MiB)
Outputting started2025-12-28 02:44:38
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11369/1/001/reco_stage1.root
Finished2025-12-28 02:46:14
Saved logsjustin-logs:264623.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

gitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 153818 traces, 0 summary
Retagger: tagging trace set: gauss with 115949 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 115949 traces tagged "gauss"
FrameSaver: q=4.43694e+06 n=2061068 tag=gauss
wclsFrameSaver: saving 153818 traces tagged "wiener"
FrameSaver: q=4.79048e+06 n=1842210 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 27680 XUs and 21240 XVs -> 192 XUVs
C:0 T:1 50279 XUs and 49177 XVs -> 3304 XUVs
C:0 T:2 103507 XUs and 117063 XVs -> 5726 XUVs
C:0 T:3 53990 XUs and 51228 XVs -> 447 XUVs
C:0 T:4 19488 XUs and 17238 XVs -> 166 XUVs
C:0 T:5 100223 XUs and 65502 XVs -> 12847 XUVs
C:0 T:6 101781 XUs and 92471 XVs -> 4022 XUVs
C:0 T:7 43841 XUs and 37711 XVs -> 330 XUVs
C:0 T:8 9025 XUs and 8687 XVs -> 173 XUVs
C:0 T:9 80405 XUs and 85366 XVs -> 10008 XUVs
C:0 T:10 136439 XUs and 119085 XVs -> 14274 XUVs
C:0 T:11 202 XUs and 136 XVs -> 16 XUVs
51505 XUVs total
14028 collection wire objects
48503 potential space points
Neighbour search...
5095591 tests to find 2295842 neighbours
Iterating with no regularization...
Begin: 9.82716e+07
0 6.92348e+07
1 6.66651e+07
2 6.62458e+07
3 6.61286e+07
4 6.60813e+07
Now with regularization...
Begin: 5.24522e+07
0 5.23535e+07
1 5.23025e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411382681
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411382681
fLifetime: 30000 [us]
28-Dec-2025 02:44:35 GMT  Closed output file "reco_stage1.root"
28-Dec-2025 02:44:35 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          267.471       428.787       584.289       436.62        82.3462        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00134089     0.0039062    0.00568719    0.00427201    0.00156204       10     
reco:ophit:OpHitFinder                             0.271222      0.316164      0.443225      0.302075      0.0472881       10     
reco:opflash:OpFlashFinder                         0.177409      0.224539      0.269147      0.227811      0.028051        10     
reco:opslicer:OpSlicer                              0.46176      0.712579       1.08507      0.672351      0.164878        10     
reco:caldata:DataPrepModule                         4.58524       5.36995       7.51267       4.99605      0.970088        10     
reco:wclsdatasp:WireCellToolkit                     29.624        35.5439       42.9566       35.2854       4.20874        10     
reco:gaushit:GausHitFinder                         0.726865       1.44498       2.15973       1.29672      0.459253        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00031513    0.000922381   0.00194047    0.000802317   0.000513834      10     
reco:reco3d:SpacePointSolver                        7.65072       19.4135       40.5148       17.1325       9.34183        10     
reco:hitpdune:DisambigFromSpacePoints               0.78293       2.15047       3.69175        2.024       0.683487        10     
reco:pandora:StandardPandora                        13.1066       47.4262       118.052       40.6484       28.3583        10     
reco:pandoraWriter:StandardPandora                 0.270238      0.527856      0.767355      0.556031      0.156294        10     
reco:pandoraTrack:LArPandoraTrackCreation           2.11672       5.3923        6.83938       5.85005       1.43697        10     
reco:pandoraShower:LArPandoraShowerCreation          1.959        5.79719       8.1713        5.84539       1.90003        10     
reco:pandoracalo:Calorimetry                        1.06206       2.8096        3.54931       2.99746      0.659411        10     
reco:pandoracalonosce:Calorimetry                  0.960426       2.47393       3.19226       2.54702      0.587333        10     
reco:pandorapid:Chi2ParticleID                    0.00383406    0.00597802     0.0103087     0.0052801    0.00182047       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0299511     0.0533453     0.0834051     0.0494894     0.0157451       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00400785    0.00594562    0.00827161    0.00572893    0.00123373       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.90898       6.10099       10.0418       5.68205       2.35931        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.81097       6.02724       8.8129        5.68555       2.28174        10     
reco:emtrkmichelid:EmTrackMichelId                  168.067        231.3        272.543       238.039       28.6052        10     
reco:crttag:SingleCRTMatchingProducer              0.0125079     0.0184577     0.0237463     0.0187399    0.00315589       10     
reco:crtreco:TwoCRTMatchingProducer               0.00801142     0.011931      0.0164247     0.0107806    0.00242155       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.814981       1.09944       1.42848       1.06842      0.177211        10     
reco:pandora2Track:LArPandoraTrackCreation          9.40142       18.6407       27.3996       19.4824       4.84575        10     
reco:pandora2calo:Calorimetry                       4.34085       8.08544       10.6186       8.29877       1.64335        10     
reco:pandora2calonosce:Calorimetry                  4.01709       7.4543        10.8943       7.40161       1.8885         10     
reco:pandora2pid:Chi2ParticleID                    0.0101694     0.0130974     0.0174375     0.0131158     0.0019307       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0519651     0.0627088     0.0747811     0.0636048    0.00641372       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00995244     0.0130911     0.0178984     0.0129766    0.00195862       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.07236       4.97845       7.09902       5.02364       1.33235        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.65574       6.21979       10.3669       6.38878       1.80636        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.80477       5.70592       9.33161       5.81393       1.80533        10     
[art]:TriggerResults:TriggerResultInserter        5.4772e-05    8.30765e-05   0.000163095   6.8593e-05    3.33406e-05      10     
end_path:out1:RootOutput                          1.1441e-05    1.57655e-05   1.9175e-05    1.7022e-05    2.89379e-06      10     
end_path:out1:RootOutput(write)                     2.47961       3.2324        4.58932       3.06999      0.619638        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5287.72 MB
  Peak resident set size usage (VmHWM): 4446.31 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.24 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.32 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.13 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11375308
-rw-r--r-- 1 pltdune04 pltdune 2634585990 Dec 28 01:31 detsim_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2501918736 Dec 28 01:29 detsim_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2361573145 Dec 28 02:44 reco_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2070182549 Dec 27 23:24 g4_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 1994575588 Dec 27 23:11 g4_stage1.root
-rw-r--r-- 1 pltdune04 pltdune   84868430 Dec 28 02:38 Pandora_Events.pndr
-rw-r--r-- 1 pltdune04 pltdune     243157 Dec 28 02:44 jobscript.log
-rw-r--r-- 1 pltdune04 pltdune     217088 Dec 28 02:44 mem.db
-rw-r--r-- 1 pltdune04 pltdune      62376 Dec 28 02:44 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune04 pltdune      36864 Dec 28 02:44 time.db
-rw-r--r-- 1 pltdune04 pltdune        519 Dec 28 01:31 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Dec 27 23:24 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Dec 28 02:44 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune04 pltdune         52 Dec 27 23:04 all-input-dids.txt
-rw-r--r-- 1 pltdune04 pltdune          7 Dec 28 02:44 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune04 pltdune          0 Dec 28 01:37 ProtoDUNE.log
-rw-r--r-- 1 pltdune04 pltdune          0 Dec 28 01:31 debugprod.log
justIN time: 2026-02-07 03:35:45 UTC       justIN version: 01.06.00