justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 265272.1@dunegpschedd02.fnal.gov

Jobsub ID265272.1@dunegpschedd02.fnal.gov
Workflow ID11444
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-31 16:14:03
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2025-12-31 19:50:08
From worker nodeHostnameheplnc157.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-31 16:32:56
Input filesmonte-carlo-011444-000001
JobscriptExit code0
Real time3h (11750s)
CPU time3h (11338s = 96%)
Max RSS bytes4035608576 (3848 MiB)
Outputting started2025-12-31 19:48:47
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11444/1/001/reco_stage1.root
Finished2025-12-31 19:50:08
Saved logsjustin-logs:265272.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

g tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 142980 traces, 0 summary
Retagger: tagging trace set: gauss with 106159 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 106159 traces tagged "gauss"
FrameSaver: q=3.75707e+06 n=1863430 tag=gauss
wclsFrameSaver: saving 142980 traces tagged "wiener"
FrameSaver: q=3.99357e+06 n=1644706 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 14749 XUs and 16646 XVs -> 82 XUVs
C:0 T:1 19736 XUs and 18743 XVs -> 1485 XUVs
C:0 T:2 113199 XUs and 164303 XVs -> 7371 XUVs
C:0 T:3 55994 XUs and 67322 XVs -> 527 XUVs
C:0 T:4 10799 XUs and 10931 XVs -> 177 XUVs
C:0 T:5 40833 XUs and 36204 XVs -> 8899 XUVs
C:0 T:6 112536 XUs and 131808 XVs -> 6917 XUVs
C:0 T:7 42834 XUs and 50734 XVs -> 536 XUVs
C:0 T:8 10919 XUs and 11754 XVs -> 138 XUVs
C:0 T:9 75919 XUs and 100276 XVs -> 5877 XUVs
C:0 T:10 41186 XUs and 34940 XVs -> 4690 XUVs
C:0 T:11 370 XUs and 515 XVs -> 43 XUVs
36742 XUVs total
11435 collection wire objects
33860 potential space points
Neighbour search...
2249844 tests to find 1192708 neighbours
Iterating with no regularization...
Begin: 6.65268e+07
0 5.5533e+07
1 5.4515e+07
2 5.43721e+07
3 5.43317e+07
Now with regularization...
Begin: 4.34785e+07
0 4.34597e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411457690
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411457690
fLifetime: 30000 [us]
31-Dec-2025 19:48:45 GMT  Closed output file "reco_stage1.root"
31-Dec-2025 19:48:45 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          247.216       350.918       427.745       364.246       59.5698        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00122781    0.00365589    0.00583862     0.0040572    0.00146377       10     
reco:ophit:OpHitFinder                             0.245924      0.303353       0.38255      0.290862      0.0436286       10     
reco:opflash:OpFlashFinder                         0.144796      0.174936      0.215058      0.171033      0.020984        10     
reco:opslicer:OpSlicer                             0.539705      0.675766      0.901822      0.666893      0.107817        10     
reco:caldata:DataPrepModule                         4.39185       5.29235       6.94876       5.20752      0.784284        10     
reco:wclsdatasp:WireCellToolkit                     30.3631       32.8938       35.0989       32.7916       1.3653         10     
reco:gaushit:GausHitFinder                         0.980227       1.36218       1.83363       1.25075      0.306127        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000406403   0.00059961    0.000790234   0.000587548   0.000106868      10     
reco:reco3d:SpacePointSolver                        9.57819       15.6236       24.6081       15.4738       4.53983        10     
reco:hitpdune:DisambigFromSpacePoints              0.875813       1.67063       2.68784       1.66617      0.505698        10     
reco:pandora:StandardPandora                        17.2526       35.5942       51.4003       36.1965       10.3227        10     
reco:pandoraWriter:StandardPandora                 0.334393      0.532475      0.821138      0.494763      0.164081        10     
reco:pandoraTrack:LArPandoraTrackCreation           3.0259        4.22374       5.94048       4.23374      0.954084        10     
reco:pandoraShower:LArPandoraShowerCreation         2.6122        4.59788       7.30249       4.44689       1.33317        10     
reco:pandoracalo:Calorimetry                        1.63993       2.29544       3.49774       2.23239      0.561438        10     
reco:pandoracalonosce:Calorimetry                   1.00311       2.07711       3.93274       2.07043      0.722664        10     
reco:pandorapid:Chi2ParticleID                    0.00408952    0.00486647    0.00553733    0.00489042    0.000448288      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.034583      0.0445226     0.0547362     0.0451674     0.0057495       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00423627     0.005048      0.0065501    0.00490757    0.00067075       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.74234       5.10966       9.58787       5.07872       1.92666        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.70472       4.69341       7.04818       4.8229        1.39251        10     
reco:emtrkmichelid:EmTrackMichelId                  131.916       187.246       219.395       195.912        29.36         10     
reco:crttag:SingleCRTMatchingProducer             0.00990577     0.0167713     0.0218922     0.0182776    0.00388053       10     
reco:crtreco:TwoCRTMatchingProducer               0.00744797     0.0106661     0.0145352     0.0105267    0.00257143       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.769297      0.968431       1.34012      0.906879      0.184351        10     
reco:pandora2Track:LArPandoraTrackCreation          7.85248       14.0881       22.0171       14.5957       4.15112        10     
reco:pandora2calo:Calorimetry                       3.11589       6.90585       10.5559       7.34834       1.84662        10     
reco:pandora2calonosce:Calorimetry                  2.42701       5.86024       7.95074       5.93939       1.54665        10     
reco:pandora2pid:Chi2ParticleID                   0.00793677     0.0110485     0.0133775     0.0113515    0.00141842       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0404677     0.0569925      0.06722      0.0564694     0.0079616       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00808604     0.0111053     0.0131349     0.0112779    0.00136508       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.06149       4.29285       5.95648       4.50117       1.11081        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.23334       5.84454       8.39392       5.89788       1.66648        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.04241       5.51655       9.43319       5.19697       2.01559        10     
[art]:TriggerResults:TriggerResultInserter        4.5996e-05    8.4238e-05    0.00035141    5.40215e-05   8.92782e-05      10     
end_path:out1:RootOutput                          1.0339e-05    1.26198e-05   1.5349e-05    1.2308e-05    1.82638e-06      10     
end_path:out1:RootOutput(write)                     1.94082       2.77982       4.28522       2.63194      0.663976        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4892.38 MB
  Peak resident set size usage (VmHWM): 4035.61 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.24 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.30 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.13 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9999428
-rw-r--r-- 1 pltdune04 pltdune 2354944345 Dec 31 18:48 detsim_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2230856930 Dec 31 18:46 detsim_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2062384013 Dec 31 19:48 reco_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 1790041721 Dec 31 16:51 g4_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 1720859135 Dec 31 16:38 g4_stage1.root
-rw-r--r-- 1 pltdune04 pltdune   79718890 Dec 31 19:43 Pandora_Events.pndr
-rw-r--r-- 1 pltdune04 pltdune     242774 Dec 31 19:48 jobscript.log
-rw-r--r-- 1 pltdune04 pltdune     217088 Dec 31 19:48 mem.db
-rw-r--r-- 1 pltdune04 pltdune      62376 Dec 31 19:48 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune04 pltdune      36864 Dec 31 19:48 time.db
-rw-r--r-- 1 pltdune04 pltdune        519 Dec 31 18:48 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Dec 31 16:51 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Dec 31 19:48 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune04 pltdune         52 Dec 31 16:32 all-input-dids.txt
-rw-r--r-- 1 pltdune04 pltdune          7 Dec 31 19:48 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune04 pltdune          0 Dec 31 18:53 ProtoDUNE.log
-rw-r--r-- 1 pltdune04 pltdune          0 Dec 31 18:48 debugprod.log
justIN time: 2026-02-06 23:31:08 UTC       justIN version: 01.06.00