justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 279849.1@dunegpschedd01.fnal.gov

Jobsub ID279849.1@dunegpschedd01.fnal.gov
Workflow ID12003
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-17 15:54:35
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep208
Last heartbeat2026-01-17 21:22:18
From worker nodeHostnameheplnc143.pp.rl.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-17 15:55:16
Input filesmonte-carlo-012003-000001
JobscriptExit code0
Real time5h (19539s)
CPU time5h (19014s = 97%)
Max RSS bytes4278976512 (4080 MiB)
Outputting started2026-01-17 21:20:56
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12003/1/001/reco_stage1.root
Finished2026-01-17 21:22:18
Saved logsjustin-logs:279849.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

asedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 133711 traces, 0 summary
Retagger: tagging trace set: gauss with 102990 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 102990 traces tagged "gauss"
FrameSaver: q=3.71362e+06 n=1800966 tag=gauss
wclsFrameSaver: saving 133711 traces tagged "wiener"
FrameSaver: q=3.94044e+06 n=1600684 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 27222 XUs and 20668 XVs -> 135 XUVs
C:0 T:1 46402 XUs and 39987 XVs -> 2707 XUVs
C:0 T:2 111909 XUs and 182805 XVs -> 7210 XUVs
C:0 T:3 54830 XUs and 67646 XVs -> 657 XUVs
C:0 T:4 14042 XUs and 13061 XVs -> 149 XUVs
C:0 T:5 51561 XUs and 50297 XVs -> 5682 XUVs
C:0 T:6 116059 XUs and 213413 XVs -> 9092 XUVs
C:0 T:7 42096 XUs and 50816 XVs -> 556 XUVs
C:0 T:8 5577 XUs and 5440 XVs -> 162 XUVs
C:0 T:9 29941 XUs and 40308 XVs -> 2676 XUVs
C:0 T:10 34753 XUs and 29035 XVs -> 4448 XUVs
C:0 T:11 721 XUs and 603 XVs -> 234 XUVs
33708 XUVs total
11129 collection wire objects
30334 potential space points
Neighbour search...
1867370 tests to find 878656 neighbours
Iterating with no regularization...
Begin: 1.10395e+08
0 7.67163e+07
1 7.49837e+07
2 7.4688e+07
3 7.45987e+07
4 7.45576e+07
Now with regularization...
Begin: 6.20209e+07
0 6.19337e+07
1 6.19004e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411799362
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411799362
fLifetime: 30000 [us]
17-Jan-2026 21:20:51 GMT  Closed output file "reco_stage1.root"
17-Jan-2026 21:20:51 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          515.41        663.586       942.824       658.319       110.968        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00160977    0.00442641    0.00650807    0.00479463    0.00149021       10     
reco:ophit:OpHitFinder                             0.301118       0.42538      0.549776      0.430525      0.0748553       10     
reco:opflash:OpFlashFinder                         0.218134      0.293844      0.407576      0.296697      0.0569514       10     
reco:opslicer:OpSlicer                             0.769971       1.03349       1.36686       0.96332      0.222015        10     
reco:caldata:DataPrepModule                         6.37198       8.98134       11.1759       9.62902       1.52147        10     
reco:wclsdatasp:WireCellToolkit                     53.1931       68.9316       85.0371       69.3073       7.75037        10     
reco:gaushit:GausHitFinder                          1.53585       2.16166       2.88762       2.1582       0.482914        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000618048   0.000842401   0.00106927    0.000866105   0.000148141      10     
reco:reco3d:SpacePointSolver                        8.15806       20.7844       41.935        18.3014       10.3316        10     
reco:hitpdune:DisambigFromSpacePoints               1.72254       3.29502       7.15453       3.02717       1.52373        10     
reco:pandora:StandardPandora                        30.3922       50.2173       94.5814       43.6053       18.1271        10     
reco:pandoraWriter:StandardPandora                 0.443577      0.760012       1.03809      0.807489       0.1635         10     
reco:pandoraTrack:LArPandoraTrackCreation           8.14803       12.9371       20.096        12.6898       3.21067        10     
reco:pandoraShower:LArPandoraShowerCreation         9.25228       14.0821       21.197        13.2637       3.49203        10     
reco:pandoracalo:Calorimetry                        3.41736       5.89241       9.63209       6.02859       1.72817        10     
reco:pandoracalonosce:Calorimetry                   2.91432       5.60717       9.23487       5.69451       1.67028        10     
reco:pandorapid:Chi2ParticleID                    0.00660801     0.0100837     0.0137805     0.0104072    0.00250751       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0588778     0.0942651     0.138666      0.104247      0.025447        10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00647383    0.00959928     0.0132442    0.00988597    0.00222585       10     
reco:pandoraShowercalo:ShowerCalorimetry            5.94995       13.261        20.9213       12.9505       4.25692        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       5.46672       12.8112       20.8448       12.328        4.25345        10     
reco:emtrkmichelid:EmTrackMichelId                  271.838       319.863       415.684       311.213       39.2626        10     
reco:crttag:SingleCRTMatchingProducer              0.0302296     0.038739       0.05084      0.039388     0.00568216       10     
reco:crtreco:TwoCRTMatchingProducer                0.022998      0.029885      0.0397233     0.0308935    0.00472669       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.22063       1.7215        2.46741       1.71307      0.322191        10     
reco:pandora2Track:LArPandoraTrackCreation          32.4935       42.739        55.1918       44.0847       6.53226        10     
reco:pandora2calo:Calorimetry                       10.9362       17.7625       29.1985       17.6213       5.1241         10     
reco:pandora2calonosce:Calorimetry                  9.42131       17.0997       26.8858       16.6739       4.55415        10     
reco:pandora2pid:Chi2ParticleID                    0.0161469     0.0238144     0.0366465     0.0248836    0.00595143       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0796638     0.126867      0.178343      0.132113      0.0258875       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0161419     0.0241166     0.0364233     0.0249147    0.00549848       10     
reco:pandora2Shower:LArPandoraShowerCreation        6.48701       11.9095       19.0419       11.3284       3.31962        10     
reco:pandora2Showercalo:ShowerCalorimetry           8.45929       13.9208       20.3136       14.7277       3.95871        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      6.69028        12.45        20.6334       10.8248       3.90002        10     
[art]:TriggerResults:TriggerResultInserter         6.86e-05     0.000101456   0.000194696   9.3063e-05    3.54966e-05      10     
end_path:out1:RootOutput                           9.684e-06    1.49588e-05   2.4519e-05    1.43965e-05   3.81494e-06      10     
end_path:out1:RootOutput(write)                     2.35625       4.10236       6.09924       4.32969       1.01191        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5097.11 MB
  Peak resident set size usage (VmHWM): 4278.98 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.35 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.59 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.31 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10410328
-rw-r--r-- 1 pltdune04 pltdune 2445689597 Jan 17 19:26 detsim_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2319013697 Jan 17 19:22 detsim_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2130639036 Jan 17 21:20 reco_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 1892472869 Jan 17 16:25 g4_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 1797452889 Jan 17 16:06 g4_stage1.root
-rw-r--r-- 1 pltdune04 pltdune   74289400 Jan 17 21:12 Pandora_Events.pndr
-rw-r--r-- 1 pltdune04 pltdune     242549 Jan 17 21:20 jobscript.log
-rw-r--r-- 1 pltdune04 pltdune     217088 Jan 17 21:20 mem.db
-rw-r--r-- 1 pltdune04 pltdune      62376 Jan 17 21:20 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune04 pltdune      36864 Jan 17 21:20 time.db
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 17 19:26 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 17 16:25 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 17 21:20 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune04 pltdune         52 Jan 17 15:55 all-input-dids.txt
-rw-r--r-- 1 pltdune04 pltdune          7 Jan 17 21:20 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune04 pltdune          0 Jan 17 19:36 ProtoDUNE.log
-rw-r--r-- 1 pltdune04 pltdune          0 Jan 17 19:26 debugprod.log
justIN time: 2026-02-05 22:02:09 UTC       justIN version: 01.06.00