justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 250014.1@dunegpschedd02.fnal.gov

Jobsub ID250014.1@dunegpschedd02.fnal.gov
Workflow ID10536
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-23 12:38:51
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce508
Last heartbeat2025-11-23 15:07:00
From worker nodeHostnameb9p06p6630.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-23 12:39:48
Input filesmonte-carlo-010536-000001
JobscriptExit code0
Real time2h (8751s)
CPU time2h (8171s = 93%)
Max RSS bytes3992502272 (3807 MiB)
Outputting started2025-11-23 15:05:40
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10536/1/001/reco_stage1.root
Finished2025-11-23 15:07:00
Saved logsjustin-logs:250014.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

repare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 119303 traces, 0 summary
Retagger: tagging trace set: gauss with 93575 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 93575 traces tagged "gauss"
FrameSaver: q=3.41058e+06 n=1670935 tag=gauss
wclsFrameSaver: saving 119303 traces tagged "wiener"
FrameSaver: q=3.62141e+06 n=1496387 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 24160 XUs and 22468 XVs -> 259 XUVs
C:0 T:1 37774 XUs and 32219 XVs -> 2499 XUVs
C:0 T:2 76308 XUs and 81231 XVs -> 4550 XUVs
C:0 T:3 39562 XUs and 38523 XVs -> 641 XUVs
C:0 T:4 9198 XUs and 7336 XVs -> 45 XUVs
C:0 T:5 30317 XUs and 18991 XVs -> 2122 XUVs
C:0 T:6 55451 XUs and 72467 XVs -> 6505 XUVs
C:0 T:7 22305 XUs and 25271 XVs -> 531 XUVs
C:0 T:8 10653 XUs and 9620 XVs -> 118 XUVs
C:0 T:9 60765 XUs and 72000 XVs -> 3127 XUVs
C:0 T:10 77927 XUs and 66994 XVs -> 8266 XUVs
C:0 T:11 471 XUs and 451 XVs -> 49 XUVs
28712 XUVs total
9983 collection wire objects
26373 potential space points
Neighbour search...
2131463 tests to find 951954 neighbours
Iterating with no regularization...
Begin: 9.25667e+07
0 7.3524e+07
1 7.17833e+07
2 7.15978e+07
3 7.15573e+07
Now with regularization...
Begin: 5.9504e+07
0 5.94367e+07
1 5.94172e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410690283
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410690283
fLifetime: 30000 [us]
23-Nov-2025 16:05:38 CET  Closed output file "reco_stage1.root"
23-Nov-2025 16:05:38 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          170.245       208.266       235.782       210.447       19.3011        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000781283   0.00278672    0.00490791    0.00280812    0.00113566       10     
reco:ophit:OpHitFinder                             0.199728      0.224583      0.241427      0.225709      0.0139941       10     
reco:opflash:OpFlashFinder                          0.09879      0.131344      0.156782      0.129559      0.0164046       10     
reco:opslicer:OpSlicer                              0.36566      0.475529      0.589072      0.480417      0.0580404       10     
reco:caldata:DataPrepModule                         3.76218       3.89846       4.08221       3.88933      0.0899602       10     
reco:wclsdatasp:WireCellToolkit                     25.1626       25.9094       28.0624       25.7408      0.800997        10     
reco:gaushit:GausHitFinder                         0.615417      0.748434      0.866714      0.749749      0.0765556       10     
reco:nhitsfilter:NumberOfHitsFilter               0.000314635   0.00036707    0.000459898   0.000349196   4.90122e-05      10     
reco:reco3d:SpacePointSolver                        4.81016       8.91968       12.978        9.34081       2.23589        10     
reco:hitpdune:DisambigFromSpacePoints              0.708038      0.907272       1.18988      0.896548       0.13965        10     
reco:pandora:StandardPandora                        11.7456       18.7398       23.1106       18.1716       3.26725        10     
reco:pandoraWriter:StandardPandora                 0.224559      0.262687      0.357289      0.256128      0.0365638       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.24132       2.13003       2.61343       2.21733      0.454331        10     
reco:pandoraShower:LArPandoraShowerCreation         1.36357       2.16378       3.05338       2.16851      0.484996        10     
reco:pandoracalo:Calorimetry                       0.862976       1.32187       1.6895        1.37547      0.304691        10     
reco:pandoracalonosce:Calorimetry                  0.722252       1.19551       1.63354       1.27632      0.294277        10     
reco:pandorapid:Chi2ParticleID                    0.00308455    0.00386739    0.00561261    0.00378158    0.000676492      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0278212     0.0318508     0.0363869     0.0324202    0.00244165       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00289802    0.00355876    0.00432824    0.00347339    0.000395417      10     
reco:pandoraShowercalo:ShowerCalorimetry            1.28439       2.38221       3.52679       2.38775      0.664089        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.26105       2.33835       3.3994        2.37962      0.649926        10     
reco:emtrkmichelid:EmTrackMichelId                  97.2699       111.839       123.056       112.514       8.97244        10     
reco:crttag:SingleCRTMatchingProducer             0.00637714    0.00946743      0.01803      0.0075977     0.003541        10     
reco:crtreco:TwoCRTMatchingProducer                0.0039983    0.00544765     0.0104973    0.00486268    0.00176517       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.465083       0.56007      0.641248      0.579761      0.0557618       10     
reco:pandora2Track:LArPandoraTrackCreation           4.038        6.76249       10.8769       6.78056       1.96234        10     
reco:pandora2calo:Calorimetry                       2.09166       3.57589       5.27161       3.72236      0.999117        10     
reco:pandora2calonosce:Calorimetry                  1.92846       3.42329       5.15663       3.46128       1.00562        10     
reco:pandora2pid:Chi2ParticleID                   0.00695404     0.0084383     0.0101583    0.00859051    0.000987498      10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0319601     0.0397949     0.045582      0.0401444    0.00433904       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00654066    0.00871039     0.0125386    0.00838758    0.00177101       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.53441       2.42749       3.14068       2.56534       0.51972        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.75995       3.06274       4.07713       3.23615       0.79955        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.65681       2.82952       3.61361       3.05717      0.702517        10     
[art]:TriggerResults:TriggerResultInserter         3.595e-05     6.131e-05    0.000198223   4.49205e-05   4.64025e-05      10     
end_path:out1:RootOutput                           7.92e-06     9.8021e-06     1.579e-05     9.345e-06    2.11229e-06      10     
end_path:out1:RootOutput(write)                     1.55139       1.85214       2.0444        1.88236      0.154811        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4819.53 MB
  Peak resident set size usage (VmHWM): 3992.5 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.42 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9491824
-rw-r--r--. 1 duneprd np-comp 2245650249 Nov 23 15:27 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2132850938 Nov 23 15:26 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 1895571638 Nov 23 16:05 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1720510272 Nov 23 13:54 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1658121596 Nov 23 13:46 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   66309780 Nov 23 16:02 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     242624 Nov 23 16:05 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Nov 23 16:05 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Nov 23 16:05 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Nov 23 16:05 time.db
-rw-r--r--. 1 duneprd np-comp        519 Nov 23 15:27 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Nov 23 13:54 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Nov 23 16:05 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Nov 23 13:39 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Nov 23 16:05 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Nov 23 15:32 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Nov 23 15:27 debugprod.log
justIN time: 2026-02-08 21:29:32 UTC       justIN version: 01.06.00