justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 271430.0@dunegpschedd01.fnal.gov

Jobsub ID271430.0@dunegpschedd01.fnal.gov
Workflow ID11490
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-03 13:51:51
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce505
Last heartbeat2026-01-03 17:33:15
From worker nodeHostnameb9p06p2399.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-03 13:52:45
Input filesmonte-carlo-011490-000001
JobscriptExit code0
Real time3h (13142s)
CPU time3h (12010s = 91%)
Max RSS bytes4329586688 (4129 MiB)
Outputting started2026-01-03 17:31:48
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11490/1/001/reco_stage1.root
Finished2026-01-03 17:33:15
Saved logsjustin-logs:271430.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

re:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 124522 traces, 0 summary
Retagger: tagging trace set: gauss with 96596 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 96596 traces tagged "gauss"
FrameSaver: q=3.47562e+06 n=1755279 tag=gauss
wclsFrameSaver: saving 124522 traces tagged "wiener"
FrameSaver: q=3.66167e+06 n=1561429 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 37185 XUs and 38244 XVs -> 487 XUVs
C:0 T:1 78886 XUs and 112475 XVs -> 16411 XUVs
C:0 T:2 85839 XUs and 98147 XVs -> 5828 XUVs
C:0 T:3 41247 XUs and 40569 XVs -> 204 XUVs
C:0 T:4 10624 XUs and 10167 XVs -> 73 XUVs
C:0 T:5 45444 XUs and 30839 XVs -> 3019 XUVs
C:0 T:6 100588 XUs and 97766 XVs -> 7123 XUVs
C:0 T:7 37385 XUs and 37401 XVs -> 442 XUVs
C:0 T:8 7863 XUs and 7404 XVs -> 120 XUVs
C:0 T:9 48975 XUs and 67257 XVs -> 4741 XUVs
C:0 T:10 50690 XUs and 24956 XVs -> 5518 XUVs
C:0 T:11 157 XUs and 92 XVs -> 13 XUVs
43979 XUVs total
11234 collection wire objects
40793 potential space points
Neighbour search...
5887647 tests to find 2767144 neighbours
Iterating with no regularization...
Begin: 7.16963e+07
0 6.04231e+07
1 5.96771e+07
2 5.95596e+07
3 5.9525e+07
Now with regularization...
Begin: 4.91027e+07
0 4.90559e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411515875
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411515875
fLifetime: 30000 [us]
03-Jan-2026 18:31:46 CET  Closed output file "reco_stage1.root"
03-Jan-2026 18:31:46 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          220.957       329.436       531.507       325.089       87.3344        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00104447    0.00351991    0.00692328    0.00332196    0.00175694       10     
reco:ophit:OpHitFinder                             0.224203      0.268306      0.324566      0.259096      0.0330101       10     
reco:opflash:OpFlashFinder                         0.132285      0.174527      0.241103       0.15944      0.034089        10     
reco:opslicer:OpSlicer                             0.489912      0.663312      0.918079      0.647832      0.137831        10     
reco:caldata:DataPrepModule                         4.45962       5.12288       6.22396       4.8211       0.616658        10     
reco:wclsdatasp:WireCellToolkit                     28.2679       31.1012       32.4848       31.5547       1.4047         10     
reco:gaushit:GausHitFinder                         0.693733       1.18494       1.96497       1.05528      0.393912        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000315881   0.00048522    0.000970456   0.000401217   0.000203767      10     
reco:reco3d:SpacePointSolver                        7.21364       17.216        35.8841       14.9541       8.87634        10     
reco:hitpdune:DisambigFromSpacePoints              0.848498       1.54292       3.1674        1.40927      0.675526        10     
reco:pandora:StandardPandora                        13.1242       30.4145       79.5066       27.9642       18.0945        10     
reco:pandoraWriter:StandardPandora                 0.272454      0.422537      0.685808       0.38165      0.131965        10     
reco:pandoraTrack:LArPandoraTrackCreation           1.57114       3.16554       5.55358       2.93609       1.21819        10     
reco:pandoraShower:LArPandoraShowerCreation         1.9465        3.74329       8.70404       3.31959       1.82898        10     
reco:pandoracalo:Calorimetry                       0.917111       1.80274       2.96826       1.7384        0.63475        10     
reco:pandoracalonosce:Calorimetry                  0.796165       1.71662       2.93607       1.66384      0.670217        10     
reco:pandorapid:Chi2ParticleID                    0.00294076    0.00469843     0.0077985     0.004575     0.00137195       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0305858     0.048248      0.0838196     0.0456234     0.015518        10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00296363    0.00452527    0.00822639    0.00399377    0.00152125       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.87944       4.10931       8.82695       3.84534       1.93253        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.79094       3.91284       7.83263       3.64954       1.6441         10     
reco:emtrkmichelid:EmTrackMichelId                  130.726       181.642       265.375       184.412       38.6215        10     
reco:crttag:SingleCRTMatchingProducer              0.0102533     0.0134913     0.0231392     0.0120353    0.00385463       10     
reco:crtreco:TwoCRTMatchingProducer               0.00640536    0.00862831     0.0146778    0.00759969    0.00239258       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.655849      0.899731       1.56223      0.830661      0.242852        10     
reco:pandora2Track:LArPandoraTrackCreation          5.57679       11.3563       23.1234       11.0792       4.66662        10     
reco:pandora2calo:Calorimetry                       2.73766       5.70089       10.3112       5.25437       2.33543        10     
reco:pandora2calonosce:Calorimetry                  2.59843       5.52416       10.1227       5.32839       2.26746        10     
reco:pandora2pid:Chi2ParticleID                   0.00736505     0.0131785     0.0231073    0.00975848    0.00581836       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0392069     0.0644286     0.114025      0.0513781     0.0259695       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00737271     0.0118293     0.021588     0.00964347    0.00438601       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.15982       4.14956       7.81834       3.47105       1.73965        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.57674       5.32742       10.1296       5.01867       2.18863        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.62869       5.09545       10.9468       4.5761        2.35169        10     
[art]:TriggerResults:TriggerResultInserter         4.346e-05    6.49924e-05   0.00011865    6.21755e-05   2.15653e-05      10     
end_path:out1:RootOutput                           1.044e-05    1.52571e-05    3.656e-05     1.331e-05    7.23943e-06      10     
end_path:out1:RootOutput(write)                     1.89048       2.91006       3.94574       2.96369      0.718628        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5154.96 MB
  Peak resident set size usage (VmHWM): 4329.59 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.23 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.25 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.13 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10318064
-rw-r--r--. 1 duneprd np-comp 2418121601 Jan  3 17:35 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2293949854 Jan  3 17:33 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2129066574 Jan  3 18:31 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1861951709 Jan  3 15:21 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1781565749 Jan  3 15:00 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   80428500 Jan  3 18:27 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     243144 Jan  3 18:31 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan  3 18:31 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan  3 18:31 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan  3 18:31 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan  3 17:35 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan  3 15:21 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan  3 18:31 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan  3 14:52 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan  3 18:31 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan  3 17:40 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan  3 17:35 debugprod.log
justIN time: 2026-02-06 22:18:28 UTC       justIN version: 01.06.00