justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 250833.0@dunegpschedd02.fnal.gov

Jobsub ID250833.0@dunegpschedd02.fnal.gov
Workflow ID10618
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-24 20:39:32
SiteUK_Brunel
EntryCMSHTPC_T2_UK_London_Brunel_dc2_22
Last heartbeat2025-11-25 01:10:09
From worker nodeHostnamewn-b8-28-00.brunel.ac.uk
cpuinfoIntel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-24 21:13:45
Input filesmonte-carlo-010618-000001
JobscriptExit code0
Real time3h (14080s)
CPU time3h (13599s = 96%)
Max RSS bytes4422348800 (4217 MiB)
Outputting started2025-11-25 01:08:25
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10618/1/001/reco_stage1.root
Finished2025-11-25 01:10:09
Saved logsjustin-logs:250833.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     (HTCondor job logs unavailable)

Jobscript log (last 10,000 characters)

pService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 135890 traces, 0 summary
Retagger: tagging trace set: gauss with 100843 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 100843 traces tagged "gauss"
FrameSaver: q=3.53218e+06 n=1752810 tag=gauss
wclsFrameSaver: saving 135890 traces tagged "wiener"
FrameSaver: q=3.75597e+06 n=1538251 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 40471 XUs and 36587 XVs -> 664 XUVs
C:0 T:1 84501 XUs and 124963 XVs -> 11080 XUVs
C:0 T:2 82339 XUs and 96441 XVs -> 5463 XUVs
C:0 T:3 36654 XUs and 40313 XVs -> 237 XUVs
C:0 T:4 11288 XUs and 11616 XVs -> 93 XUVs
C:0 T:5 55741 XUs and 31811 XVs -> 6595 XUVs
C:0 T:6 134129 XUs and 141364 XVs -> 6665 XUVs
C:0 T:7 51820 XUs and 55997 XVs -> 562 XUVs
C:0 T:8 4382 XUs and 4005 XVs -> 39 XUVs
C:0 T:9 24773 XUs and 28317 XVs -> 1848 XUVs
C:0 T:10 35914 XUs and 42137 XVs -> 5412 XUVs
C:0 T:11 43 XUs and 49 XVs -> 5 XUVs
38663 XUVs total
11686 collection wire objects
35785 potential space points
Neighbour search...
3263031 tests to find 1638250 neighbours
Iterating with no regularization...
Begin: 5.84702e+07
0 4.75169e+07
1 4.66325e+07
2 4.65076e+07
3 4.64682e+07
Now with regularization...
Begin: 3.65237e+07
0 3.64869e+07
1 3.6461e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410717119
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410717119
fLifetime: 30000 [us]
25-Nov-2025 01:08:23 GMT  Closed output file "reco_stage1.root"
25-Nov-2025 01:08:23 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          324.902       447.304       743.97        402.879       125.956        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000863147   0.00373323    0.00627228    0.00407957    0.00159981       10     
reco:ophit:OpHitFinder                             0.252119       0.28624      0.320295      0.287279      0.0236702       10     
reco:opflash:OpFlashFinder                         0.151842      0.193381      0.243379       0.19155      0.029984        10     
reco:opslicer:OpSlicer                             0.607478      0.745446      0.939813      0.743925      0.0994292       10     
reco:caldata:DataPrepModule                         5.78467       5.94668       6.34334       5.8993       0.158713        10     
reco:wclsdatasp:WireCellToolkit                     43.8454       54.4722       58.6246       56.4117       5.07112        10     
reco:gaushit:GausHitFinder                          1.25403       1.58606       2.26706       1.48141      0.309144        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000389906   0.00062545    0.000923374   0.000607873   0.000149406      10     
reco:reco3d:SpacePointSolver                        9.02478       19.6212       46.5363       15.8977       10.146         10     
reco:hitpdune:DisambigFromSpacePoints               1.66245       2.74967       4.2613        2.4535       0.843779        10     
reco:pandora:StandardPandora                        25.8103       68.3116       276.774       40.0722       71.7573        10     
reco:pandoraWriter:StandardPandora                  0.36863      0.509613       0.80719      0.460391      0.130464        10     
reco:pandoraTrack:LArPandoraTrackCreation           3.79861       5.83569       9.50262       5.32711       1.95017        10     
reco:pandoraShower:LArPandoraShowerCreation         3.49305       5.48539       8.27339       4.93613       1.77345        10     
reco:pandoracalo:Calorimetry                        2.04768        2.85         4.3144        2.59801      0.814609        10     
reco:pandoracalonosce:Calorimetry                   1.81866       2.54924       4.00912       2.34484       0.74966        10     
reco:pandorapid:Chi2ParticleID                    0.00550119    0.00682175     0.0104302    0.00648319    0.00138705       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0612221     0.0795285     0.100016      0.0793551     0.0107071       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00532712    0.00645381    0.00854232    0.00625442    0.000939696      10     
reco:pandoraShowercalo:ShowerCalorimetry            3.1962        5.48636       8.42854       5.24725       1.77155        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.10804       5.42948       8.34262       5.43822       1.76319        10     
reco:emtrkmichelid:EmTrackMichelId                  165.746       205.632       263.467       197.57        32.8294        10     
reco:crttag:SingleCRTMatchingProducer              0.0136352     0.020645      0.0268051     0.0206382     0.003943        10     
reco:crtreco:TwoCRTMatchingProducer               0.00827815     0.0134652     0.0196954     0.0140179    0.00304981       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.23207       1.88374       2.67221       1.87726      0.357782        10     
reco:pandora2Track:LArPandoraTrackCreation          12.0348       18.4451       28.0272       17.5717       5.37275        10     
reco:pandora2calo:Calorimetry                       5.01316       8.00586       11.9107       7.5943        2.27531        10     
reco:pandora2calonosce:Calorimetry                  4.72348       7.68276       11.9143       7.20576       2.27744        10     
reco:pandora2pid:Chi2ParticleID                    0.0125918     0.0165146     0.0221347     0.015904     0.00284369       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.078418      0.0992348     0.125769      0.0969385     0.0151345       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0125551     0.0162894     0.0212078     0.0156536    0.00264693       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.38898       6.50113       9.52964       5.90401       1.76553        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.44297       7.14411       11.3886       6.68069       2.09855        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.08937       6.5506        10.0634       6.17823       1.8886         10     
[art]:TriggerResults:TriggerResultInserter         5.128e-05    6.24565e-05   0.000115175   5.58495e-05   1.78922e-05      10     
end_path:out1:RootOutput                           8.701e-06    1.24502e-05   2.4353e-05    1.1033e-05    4.23757e-06      10     
end_path:out1:RootOutput(write)                     2.33608       3.03924       4.09829       2.87085       0.53939        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5226.73 MB
  Peak resident set size usage (VmHWM): 4422.35 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.84 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10947412
-rw-r--r--. 1 pildune003 dunepil 2544275315 Nov 24 23:50 detsim_stage2.root
-rw-r--r--. 1 pildune003 dunepil 2418367198 Nov 24 23:48 detsim_stage1.root
-rw-r--r--. 1 pildune003 dunepil 2246502883 Nov 25 01:08 reco_stage1.root
-rw-r--r--. 1 pildune003 dunepil 1998730260 Nov 24 21:36 g4_stage2.root
-rw-r--r--. 1 pildune003 dunepil 1922650151 Nov 24 21:22 g4_stage1.root
-rw-r--r--. 1 pildune003 dunepil   79009720 Nov 25 01:03 Pandora_Events.pndr
-rw-r--r--. 1 pildune003 dunepil     242813 Nov 25 01:08 jobscript.log
-rw-r--r--. 1 pildune003 dunepil     217088 Nov 25 01:08 mem.db
-rw-r--r--. 1 pildune003 dunepil      62376 Nov 25 01:08 Pandora_Geometry.xml
-rw-r--r--. 1 pildune003 dunepil      36864 Nov 25 01:08 time.db
-rw-r--r--. 1 pildune003 dunepil        519 Nov 24 23:51 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 pildune003 dunepil        519 Nov 24 21:36 g4_protoDUNE_hist.root
-rw-r--r--. 1 pildune003 dunepil        519 Nov 25 01:08 protoDUNE_reco1_hist.root
-rw-r--r--. 1 pildune003 dunepil         52 Nov 24 21:13 all-input-dids.txt
-rw-r--r--. 1 pildune003 dunepil          7 Nov 25 01:08 justin-processed-pfns.txt
-rw-r--r--. 1 pildune003 dunepil          0 Nov 25 00:03 ProtoDUNE.log
-rw-r--r--. 1 pildune003 dunepil          0 Nov 24 23:51 debugprod.log
justIN time: 2026-02-08 16:49:39 UTC       justIN version: 01.06.00