justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 243309.0@dunegpschedd01.fnal.gov

Jobsub ID243309.0@dunegpschedd01.fnal.gov
Workflow ID9605
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-05 04:30:55
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce02
Last heartbeat2025-11-05 08:47:13
From worker nodeHostnamecn012.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-05 04:33:00
Input filesmonte-carlo-009605-000001
JobscriptExit code0
Real time4h (15040s)
CPU time4h (14770s = 98%)
Max RSS bytes4399828992 (4196 MiB)
Outputting started2025-11-05 08:43:40
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09605/1/001/reco_stage1.root
Finished2025-11-05 08:47:13
Saved logsjustin-logs:243309.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ol pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 138510 traces, 0 summary
Retagger: tagging trace set: gauss with 107302 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 107302 traces tagged "gauss"
FrameSaver: q=3.77432e+06 n=1941938 tag=gauss
wclsFrameSaver: saving 138510 traces tagged "wiener"
FrameSaver: q=3.9874e+06 n=1736338 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 40611 XUs and 36845 XVs -> 600 XUVs
C:0 T:1 85629 XUs and 74893 XVs -> 9103 XUVs
C:0 T:2 99364 XUs and 90760 XVs -> 5176 XUVs
C:0 T:3 49513 XUs and 39194 XVs -> 481 XUVs
C:0 T:4 16778 XUs and 17202 XVs -> 138 XUVs
C:0 T:5 90068 XUs and 73364 XVs -> 16804 XUVs
C:0 T:6 119344 XUs and 124350 XVs -> 3181 XUVs
C:0 T:7 52504 XUs and 53527 XVs -> 451 XUVs
C:0 T:8 8201 XUs and 8388 XVs -> 74 XUVs
C:0 T:9 65142 XUs and 75025 XVs -> 7440 XUVs
C:0 T:10 22407 XUs and 11435 XVs -> 2342 XUVs
C:0 T:11 147 XUs and 54 XVs -> 14 XUVs
45804 XUVs total
12054 collection wire objects
42724 potential space points
Neighbour search...
6088088 tests to find 2818232 neighbours
Iterating with no regularization...
Begin: 7.4704e+07
0 6.50826e+07
1 6.43152e+07
2 6.4185e+07
3 6.41412e+07
Now with regularization...
Begin: 5.3484e+07
0 5.34394e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410321246
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410321246
fLifetime: 30000 [us]
05-Nov-2025 08:43:37 GMT  Closed output file "reco_stage1.root"
05-Nov-2025 08:43:37 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          484.446       699.12        890.847       699.34        113.162        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00181873    0.00494472    0.00846932    0.00482761    0.00198533       10     
reco:ophit:OpHitFinder                             0.244999      0.388033      0.702927      0.313397      0.157241        10     
reco:opflash:OpFlashFinder                         0.195107      0.275438      0.370026      0.270821      0.0524817       10     
reco:opslicer:OpSlicer                             0.581919      0.783314       1.28721      0.725853      0.217144        10     
reco:caldata:DataPrepModule                         6.78025       8.12827       10.136        7.74316       1.13365        10     
reco:wclsdatasp:WireCellToolkit                     47.1775       57.7306       74.874        55.9279       7.54282        10     
reco:gaushit:GausHitFinder                          1.17271       1.8223        2.76613       1.59695      0.524013        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000821793   0.00115791    0.00151537    0.00120738    0.000236452      10     
reco:reco3d:SpacePointSolver                        9.67639       23.3282       45.1248       22.3565       9.7099         10     
reco:hitpdune:DisambigFromSpacePoints               1.65317       3.21461       7.15537       2.70632       1.56482        10     
reco:pandora:StandardPandora                        35.0801       67.7359       125.993       59.3013       26.9336        10     
reco:pandoraWriter:StandardPandora                  0.53248      0.659718      0.834172      0.653222      0.103372        10     
reco:pandoraTrack:LArPandoraTrackCreation           13.5964       17.2813       24.6437       16.4673       3.30062        10     
reco:pandoraShower:LArPandoraShowerCreation         13.4078       21.4909       30.7799       20.8839       5.67661        10     
reco:pandoracalo:Calorimetry                        6.41212       8.19703       11.8491       7.87763       1.68303        10     
reco:pandoracalonosce:Calorimetry                   6.16913       8.01539       11.5924       7.66476       1.6785         10     
reco:pandorapid:Chi2ParticleID                    0.00572084     0.0068081    0.00802468    0.00661283    0.000859607      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0565513     0.0732402     0.102165      0.0671337     0.0147261       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00561928    0.00637739    0.00748146    0.00621641    0.00058047       10     
reco:pandoraShowercalo:ShowerCalorimetry            11.9425       21.038        31.6927       20.3571       6.17868        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       12.4308       20.8443       29.8891       20.652        5.76319        10     
reco:emtrkmichelid:EmTrackMichelId                  171.933       240.406       288.995       241.863       32.1589        10     
reco:crttag:SingleCRTMatchingProducer               0.03476      0.0484092     0.0641365     0.0486656    0.00876835       10     
reco:crtreco:TwoCRTMatchingProducer                0.0279385     0.0403215     0.0530277     0.0406545    0.00726276       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.08155       1.67366       2.76277       1.55308       0.44995        10     
reco:pandora2Track:LArPandoraTrackCreation          40.9563       63.4312       83.4863       64.6329       12.2132        10     
reco:pandora2calo:Calorimetry                       19.2213       30.972        40.0477       31.4206       6.01809        10     
reco:pandora2calonosce:Calorimetry                  18.2765       29.5247       38.0151       31.1042       5.99571        10     
reco:pandora2pid:Chi2ParticleID                    0.0133271     0.0191773     0.0229109     0.0207623    0.00336211       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0690034     0.107625      0.146577      0.111621      0.0208803       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0128579     0.0182837     0.0224872     0.0190392     0.0032775       10     
reco:pandora2Shower:LArPandoraShowerCreation        12.6179       19.8422       24.5331       20.4584       3.60852        10     
reco:pandora2Showercalo:ShowerCalorimetry           15.4059       25.5133       32.0327       26.5819       4.83765        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      12.4389       22.6225       30.5081       23.278        5.0595         10     
[art]:TriggerResults:TriggerResultInserter        5.4969e-05    7.23684e-05   0.000128485   6.60625e-05   1.9919e-05       10     
end_path:out1:RootOutput                           9.78e-06     1.1362e-05    1.4752e-05    1.11215e-05   1.32594e-06      10     
end_path:out1:RootOutput(write)                     2.42326       3.62353       5.31966       3.62848      0.791526        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5204.49 MB
  Peak resident set size usage (VmHWM): 4399.83 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.23 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.43 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.18 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11225420
-rw-r--r-- 1 pildune22 pildune 2594191641 Nov  5 06:44 detsim_stage2.root
-rw-r--r-- 1 pildune22 pildune 2457104795 Nov  5 06:41 detsim_stage1.root
-rw-r--r-- 1 pildune22 pildune 2333406693 Nov  5 08:43 reco_stage1.root
-rw-r--r-- 1 pildune22 pildune 2053070229 Nov  5 05:03 g4_stage2.root
-rw-r--r-- 1 pildune22 pildune 1968424101 Nov  5 04:41 g4_stage1.root
-rw-r--r-- 1 pildune22 pildune   88016740 Nov  5 08:33 Pandora_Events.pndr
-rw-r--r-- 1 pildune22 pildune     243162 Nov  5 08:43 jobscript.log
-rw-r--r-- 1 pildune22 pildune     217088 Nov  5 08:43 mem.db
-rw-r--r-- 1 pildune22 pildune      62376 Nov  5 08:43 Pandora_Geometry.xml
-rw-r--r-- 1 pildune22 pildune      36864 Nov  5 08:43 time.db
-rw-r--r-- 1 pildune22 pildune        519 Nov  5 06:44 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Nov  5 05:03 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Nov  5 08:43 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune22 pildune         52 Nov  5 04:33 all-input-dids.txt
-rw-r--r-- 1 pildune22 pildune          7 Nov  5 08:43 justin-processed-pfns.txt
-rw-r--r-- 1 pildune22 pildune          0 Nov  5 06:53 ProtoDUNE.log
-rw-r--r-- 1 pildune22 pildune          0 Nov  5 06:44 debugprod.log
justIN time: 2026-02-11 07:48:01 UTC       justIN version: 01.06.00