justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 294539.0@dunegpschedd02.fnal.gov

Jobsub ID294539.0@dunegpschedd02.fnal.gov
Workflow ID12367
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-24 11:01:36
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce506
Last heartbeat2026-01-24 14:39:40
From worker nodeHostnameb9p05p8732.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-24 11:01:55
Input filesmonte-carlo-012367-000001
JobscriptExit code0
Real time3h (12974s)
CPU time3h (11966s = 92%)
Max RSS bytes4241936384 (4045 MiB)
Outputting started2026-01-24 14:38:09
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12367/1/001/reco_stage1.root
Finished2026-01-24 14:39:40
Saved logsjustin-logs:294539.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

eKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 123014 traces, 0 summary
Retagger: tagging trace set: gauss with 94193 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 94193 traces tagged "gauss"
FrameSaver: q=3.23279e+06 n=1684635 tag=gauss
wclsFrameSaver: saving 123014 traces tagged "wiener"
FrameSaver: q=3.42956e+06 n=1492899 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 22602 XUs and 20063 XVs -> 369 XUVs
C:0 T:1 51200 XUs and 34417 XVs -> 5095 XUVs
C:0 T:2 55446 XUs and 122572 XVs -> 5064 XUVs
C:0 T:3 24792 XUs and 48494 XVs -> 276 XUVs
C:0 T:4 17558 XUs and 14162 XVs -> 225 XUVs
C:0 T:5 132918 XUs and 61520 XVs -> 16010 XUVs
C:0 T:6 115408 XUs and 209170 XVs -> 20039 XUVs
C:0 T:7 35308 XUs and 43922 XVs -> 550 XUVs
C:0 T:8 7348 XUs and 4819 XVs -> 47 XUVs
C:0 T:9 106805 XUs and 50059 XVs -> 12273 XUVs
C:0 T:10 116245 XUs and 58781 XVs -> 12517 XUVs
C:0 T:11 96 XUs and 53 XVs -> 15 XUVs
72480 XUVs total
11916 collection wire objects
68335 potential space points
Neighbour search...
16944793 tests to find 7065844 neighbours
Iterating with no regularization...
Begin: 6.53625e+07
0 5.51807e+07
1 5.44712e+07
2 5.43517e+07
3 5.43123e+07
Now with regularization...
Begin: 4.58036e+07
0 4.57374e+07
1 4.56837e+07
2 4.56431e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411936095
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411936095
fLifetime: 30000 [us]
24-Jan-2026 15:38:07 CET  Closed output file "reco_stage1.root"
24-Jan-2026 15:38:07 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          240.179       337.495       449.04        323.589       57.6121        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000976128   0.00389898    0.00707266    0.00398489     0.001871        10     
reco:ophit:OpHitFinder                             0.257859      0.298777      0.341061      0.298159      0.0289378       10     
reco:opflash:OpFlashFinder                         0.141556      0.182572       0.2493       0.183109      0.0337742       10     
reco:opslicer:OpSlicer                             0.460433      0.624418       0.79052      0.639739      0.0939489       10     
reco:caldata:DataPrepModule                         4.49783       5.30505       5.64366       5.37772      0.315244        10     
reco:wclsdatasp:WireCellToolkit                     29.5152       34.4294       41.1615       33.0648       3.74338        10     
reco:gaushit:GausHitFinder                         0.897237       1.22293       1.81253       1.16923      0.301656        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000328919   0.000514672   0.000967378   0.000456528   0.000183741      10     
reco:reco3d:SpacePointSolver                        8.41047       16.1876       25.3148       16.5263       4.70585        10     
reco:hitpdune:DisambigFromSpacePoints               1.17823       1.80971       2.89934       1.61883      0.582769        10     
reco:pandora:StandardPandora                        16.9366       32.4153       54.7148       29.7305       11.8797        10     
reco:pandoraWriter:StandardPandora                 0.329743       0.40959      0.503676       0.42061      0.0575823       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.96345       3.29936       4.54761       3.01297      0.915291        10     
reco:pandoraShower:LArPandoraShowerCreation         1.96718       3.86849       6.09641       3.74076       1.13547        10     
reco:pandoracalo:Calorimetry                        1.04896       1.9748        2.68497       1.94827      0.508075        10     
reco:pandoracalonosce:Calorimetry                   0.92572       1.81753       2.49679       1.82028      0.503125        10     
reco:pandorapid:Chi2ParticleID                    0.00351792    0.00526782    0.00687682    0.00508224    0.00102604       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0341414     0.0486306     0.0677569     0.0481845    0.00877608       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00379202    0.00523447    0.00679977    0.00520481    0.00111686       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.90607       4.36635       6.48444       4.33075       1.35858        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.86316       4.13694       6.55255       4.05931       1.3028         10     
reco:emtrkmichelid:EmTrackMichelId                  141.218       183.032       238.926       177.112       26.8043        10     
reco:crttag:SingleCRTMatchingProducer             0.00989621     0.0144247     0.0226362     0.0129113    0.00368502       10     
reco:crtreco:TwoCRTMatchingProducer               0.00607062    0.00831682     0.0116554    0.00811158    0.00152568       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.792406       0.93838       1.11782      0.951086      0.0995371       10     
reco:pandora2Track:LArPandoraTrackCreation          5.80539       11.5973       17.0147       11.7071       3.1161         10     
reco:pandora2calo:Calorimetry                       2.97674       6.14566       8.47271       6.18273       1.64849        10     
reco:pandora2calonosce:Calorimetry                  2.7046        5.79573       8.89781       5.56017       1.85004        10     
reco:pandora2pid:Chi2ParticleID                   0.00834155     0.0147188     0.0217014     0.0152565    0.00387557       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0434684     0.0705787     0.104775      0.0722684     0.0188764       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00767464     0.0144496     0.0214885     0.0152386    0.00445097       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.30663        4.155        5.69127       4.03459       1.06368        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.53605       5.43194       9.06459       5.25226       1.73623        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.50516       4.96036       8.47718       4.81804       1.59648        10     
[art]:TriggerResults:TriggerResultInserter        3.7471e-05    5.58178e-05   0.000120484   4.9737e-05    2.31829e-05      10     
end_path:out1:RootOutput                           8.951e-06    1.12904e-05   1.5101e-05    1.12955e-05    1.723e-06       10     
end_path:out1:RootOutput(write)                     2.05503       2.79412       4.01141       2.69432      0.548993        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5103.19 MB
  Peak resident set size usage (VmHWM): 4241.94 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.23 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.31 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.13 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10736072
-rw-r--r--. 1 duneprd np-comp 2495498339 Jan 24 14:40 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2370524989 Jan 24 14:38 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2195848265 Jan 24 15:38 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1962720470 Jan 24 12:30 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1889712987 Jan 24 12:07 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   78822720 Jan 24 15:34 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     243038 Jan 24 15:38 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan 24 15:38 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan 24 15:38 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan 24 15:38 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan 24 14:40 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 24 12:30 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 24 15:38 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan 24 12:01 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan 24 15:38 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan 24 14:45 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan 24 14:40 debugprod.log
justIN time: 2026-02-04 23:53:06 UTC       justIN version: 01.06.00