justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 301207.0@dunegpschedd01.fnal.gov

Jobsub ID301207.0@dunegpschedd01.fnal.gov
Workflow ID12410
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-26 18:54:27
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2026-01-27 01:08:30
From worker nodeHostnamewn1907301.tier2.hep.manchester.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-26 18:55:42
Input filesmonte-carlo-012410-000001
JobscriptExit code0
Real time6h (22270s)
CPU time6h (21756s = 97%)
Max RSS bytes4169609216 (3976 MiB)
Outputting started2026-01-27 01:06:52
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12410/1/001/reco_stage1.root
Finished2026-01-27 01:08:30
Saved logsjustin-logs:301207.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

RawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 144134 traces, 0 summary
Retagger: tagging trace set: gauss with 106303 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 106303 traces tagged "gauss"
FrameSaver: q=3.7547e+06 n=1783324 tag=gauss
wclsFrameSaver: saving 144134 traces tagged "wiener"
FrameSaver: q=4.01558e+06 n=1558972 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 26165 XUs and 38505 XVs -> 349 XUVs
C:0 T:1 53309 XUs and 67401 XVs -> 3894 XUVs
C:0 T:2 121577 XUs and 102107 XVs -> 15637 XUVs
C:0 T:3 47519 XUs and 34146 XVs -> 2206 XUVs
C:0 T:4 6684 XUs and 9788 XVs -> 64 XUVs
C:0 T:5 26483 XUs and 25465 XVs -> 2808 XUVs
C:0 T:6 177781 XUs and 122397 XVs -> 9282 XUVs
C:0 T:7 67967 XUs and 44586 XVs -> 779 XUVs
C:0 T:8 32812 XUs and 33218 XVs -> 3573 XUVs
C:0 T:9 88280 XUs and 137261 XVs -> 10330 XUVs
C:0 T:10 49343 XUs and 34723 XVs -> 5387 XUVs
C:0 T:11 194 XUs and 154 XVs -> 22 XUVs
54331 XUVs total
12780 collection wire objects
46448 potential space points
Neighbour search...
4785966 tests to find 2053174 neighbours
Iterating with no regularization...
Begin: 8.69559e+07
0 6.42038e+07
1 6.2701e+07
2 6.24374e+07
3 6.23466e+07
4 6.23005e+07
Now with regularization...
Begin: 5.10714e+07
0 5.10049e+07
1 5.09645e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411982876
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411982876
fLifetime: 30000 [us]
27-Jan-2026 01:06:48 GMT  Closed output file "reco_stage1.root"
27-Jan-2026 01:06:48 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          688.188       817.591       921.261       835.983       77.5208        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000962719   0.00462407    0.00736329    0.00471659    0.00182347       10     
reco:ophit:OpHitFinder                             0.300131      0.424365      0.497694      0.449254      0.0636766       10     
reco:opflash:OpFlashFinder                         0.206311      0.255348       0.30938      0.252655      0.0348438       10     
reco:opslicer:OpSlicer                             0.791198       1.06889       1.40269       1.0929       0.175452        10     
reco:caldata:DataPrepModule                         8.99546       9.99838       10.8305       10.0656      0.578168        10     
reco:wclsdatasp:WireCellToolkit                     67.0249       69.2228       72.898        69.3778       1.83773        10     
reco:gaushit:GausHitFinder                          2.13514       2.6054        3.28569       2.53294      0.372754        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000753009   0.000974418   0.00123444    0.00095953    0.000150206      10     
reco:reco3d:SpacePointSolver                        15.4445       34.8123       67.5971       35.3338       14.0582        10     
reco:hitpdune:DisambigFromSpacePoints               3.03683       4.22964       5.02027       4.39084      0.571065        10     
reco:pandora:StandardPandora                        57.2086       85.4689       107.631       86.8896       16.9212        10     
reco:pandoraWriter:StandardPandora                 0.808831      0.902566       1.0231       0.899608      0.056675        10     
reco:pandoraTrack:LArPandoraTrackCreation           12.4797       16.1211       21.958        15.2659       3.22537        10     
reco:pandoraShower:LArPandoraShowerCreation         13.0042       16.5789       21.574        16.2827       2.72625        10     
reco:pandoracalo:Calorimetry                        5.81968       7.55245       10.7375       7.27596       1.46006        10     
reco:pandoracalonosce:Calorimetry                   5.42771       7.12386       9.90807       6.83366       1.37369        10     
reco:pandorapid:Chi2ParticleID                     0.0074757     0.0111491     0.0160647     0.0105044    0.00304172       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP          0.08421      0.117018      0.157854      0.116838      0.0203006       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00717377     0.0110204     0.0163424     0.0109511    0.00265376       10     
reco:pandoraShowercalo:ShowerCalorimetry            12.902        16.3724       19.8284       16.6824       2.42159        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       12.4334       16.1816       22.836        15.7097       3.02747        10     
reco:emtrkmichelid:EmTrackMichelId                  342.553       378.499       429.057       377.074       26.9629        10     
reco:crttag:SingleCRTMatchingProducer              0.0292766     0.0404679     0.0556937     0.0400832    0.00696773       10     
reco:crtreco:TwoCRTMatchingProducer                0.0221055     0.0304464     0.0347953     0.0326103    0.00421524       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.44649       1.80704       2.24171       1.74698      0.262416        10     
reco:pandora2Track:LArPandoraTrackCreation          38.7074       50.121        62.498        51.0779       7.53988        10     
reco:pandora2calo:Calorimetry                       16.3734       22.1216       27.0892       22.9179       3.40138        10     
reco:pandora2calonosce:Calorimetry                  14.9372       20.9639       25.7132       21.7492       3.09937        10     
reco:pandora2pid:Chi2ParticleID                    0.0154411     0.0280302     0.0348989     0.0294316    0.00550662       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0838305     0.141098       0.16183       0.14765      0.0232113       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0151413     0.0268272     0.0325259     0.0283906    0.00511919       10     
reco:pandora2Shower:LArPandoraShowerCreation        12.0047       15.0642       18.7468       14.8225       1.95654        10     
reco:pandora2Showercalo:ShowerCalorimetry           14.1177       18.3559       22.7549       18.4688       2.89101        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      12.5561       16.3162       19.8973       16.8292       2.42875        10     
[art]:TriggerResults:TriggerResultInserter        7.0623e-05    0.000107118   0.000184548   0.000102885   3.13875e-05      10     
end_path:out1:RootOutput                           9.376e-06    1.35411e-05   2.0962e-05    1.32625e-05   2.84267e-06      10     
end_path:out1:RootOutput(write)                     3.54339       4.80372       5.89854       4.87044      0.661306        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4982.92 MB
  Peak resident set size usage (VmHWM): 4169.61 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.38 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   6.33 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.35 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10326944
-rw-r--r-- 1 dune01 dune01 2413583306 Jan 26 22:47 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2297398301 Jan 26 22:43 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2123296993 Jan 27 01:06 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1866477753 Jan 26 19:25 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1792586576 Jan 26 19:04 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   80813720 Jan 27 00:55 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242981 Jan 27 01:06 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Jan 27 01:06 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Jan 27 01:06 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Jan 27 01:06 time.db
-rw-r--r-- 1 dune01 dune01        519 Jan 26 22:47 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Jan 26 19:25 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Jan 27 01:06 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Jan 26 18:55 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Jan 27 01:06 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Jan 26 23:00 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Jan 26 22:47 debugprod.log
justIN time: 2026-02-04 21:54:41 UTC       justIN version: 01.06.00