justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 271426.1@dunegpschedd01.fnal.gov

Jobsub ID271426.1@dunegpschedd01.fnal.gov
Workflow ID11485
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-03 13:49:50
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce505
Last heartbeat2026-01-03 17:23:46
From worker nodeHostnameb9p06p2399.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-03 13:51:39
Input filesmonte-carlo-011485-000001
JobscriptExit code0
Real time3h (12641s)
CPU time3h (11331s = 89%)
Max RSS bytes4055764992 (3867 MiB)
Outputting started2026-01-03 17:22:20
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11485/1/001/reco_stage1.root
Finished2026-01-03 17:23:46
Saved logsjustin-logs:271426.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 109214 traces, 0 summary
Retagger: tagging trace set: gauss with 83089 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 83089 traces tagged "gauss"
FrameSaver: q=2.96519e+06 n=1472262 tag=gauss
wclsFrameSaver: saving 109214 traces tagged "wiener"
FrameSaver: q=3.14535e+06 n=1312330 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 22611 XUs and 18192 XVs -> 286 XUVs
C:0 T:1 43395 XUs and 35236 XVs -> 3049 XUVs
C:0 T:2 94037 XUs and 152960 XVs -> 5328 XUVs
C:0 T:3 46164 XUs and 67189 XVs -> 470 XUVs
C:0 T:4 14432 XUs and 13826 XVs -> 137 XUVs
C:0 T:5 47430 XUs and 35576 XVs -> 3599 XUVs
C:0 T:6 59548 XUs and 113557 XVs -> 7108 XUVs
C:0 T:7 22798 XUs and 35944 XVs -> 378 XUVs
C:0 T:8 6122 XUs and 7666 XVs -> 83 XUVs
C:0 T:9 36105 XUs and 57537 XVs -> 5729 XUVs
C:0 T:10 14132 XUs and 11547 XVs -> 3131 XUVs
C:0 T:11 107 XUs and 103 XVs -> 20 XUVs
29318 XUVs total
8676 collection wire objects
26935 potential space points
Neighbour search...
2508347 tests to find 1222304 neighbours
Iterating with no regularization...
Begin: 5.73245e+07
0 4.7767e+07
1 4.70184e+07
2 4.69158e+07
3 4.68877e+07
Now with regularization...
Begin: 3.80816e+07
0 3.80384e+07
1 3.8019e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411515933
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411515933
fLifetime: 30000 [us]
03-Jan-2026 18:22:18 CET  Closed output file "reco_stage1.root"
03-Jan-2026 18:22:18 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          201.997       301.546       376.586       310.942       52.1517        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00115773    0.00360372    0.00621413    0.00376085    0.00136609       10     
reco:ophit:OpHitFinder                             0.227166       0.27881      0.348051      0.270678      0.0424258       10     
reco:opflash:OpFlashFinder                         0.120489      0.159433      0.193039      0.156508      0.0202782       10     
reco:opslicer:OpSlicer                             0.446983      0.600112      0.849789       0.57124      0.119119        10     
reco:caldata:DataPrepModule                         4.14203       5.10633       6.67998       5.11743      0.726559        10     
reco:wclsdatasp:WireCellToolkit                     27.8091       30.9245       33.742        31.161        1.85808        10     
reco:gaushit:GausHitFinder                         0.744332       1.13091       1.45805       1.15492      0.262645        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000385262   0.000491677   0.000632013   0.000504733   7.57786e-05      10     
reco:reco3d:SpacePointSolver                        10.1125       15.2268       26.7996       13.6272       5.24697        10     
reco:hitpdune:DisambigFromSpacePoints              0.623499       1.30627       2.20757       1.16723      0.450633        10     
reco:pandora:StandardPandora                        15.0015       26.2096       40.3444       26.0872       8.38263        10     
reco:pandoraWriter:StandardPandora                 0.207087       0.3736       0.577732      0.359047      0.107043        10     
reco:pandoraTrack:LArPandoraTrackCreation           1.25249       2.70501       3.75634       3.03938       0.67896        10     
reco:pandoraShower:LArPandoraShowerCreation         1.11493       3.20509       4.47996       3.36871      0.973646        10     
reco:pandoracalo:Calorimetry                        0.79145       1.60827       2.19521       1.65542      0.383351        10     
reco:pandoracalonosce:Calorimetry                  0.659461       1.50207       2.33375       1.49468      0.431514        10     
reco:pandorapid:Chi2ParticleID                    0.00261445    0.00480647    0.00696066    0.00482172    0.00147664       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0269587     0.0455937     0.0647675     0.0419607     0.0126271       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00288901     0.0045914    0.00682314    0.00401452    0.00139489       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.16974       3.62047       5.11338       3.92134       1.1301         10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.37489       3.58523       4.91465       3.6648       0.945348        10     
reco:emtrkmichelid:EmTrackMichelId                  112.803       167.427       213.775       166.382       28.2575        10     
reco:crttag:SingleCRTMatchingProducer             0.00837673     0.0116358     0.0169092     0.0108306    0.00250719       10     
reco:crtreco:TwoCRTMatchingProducer               0.00511593    0.00741752     0.010497     0.00684621    0.00159092       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.567671      0.814271       1.14061      0.786811      0.173338        10     
reco:pandora2Track:LArPandoraTrackCreation          5.62627       10.0273       16.186        9.55021       3.46847        10     
reco:pandora2calo:Calorimetry                       2.91413       5.10377       7.68053       5.30538       1.53512        10     
reco:pandora2calonosce:Calorimetry                  2.60459       5.16466       7.72107       5.27479       1.55419        10     
reco:pandora2pid:Chi2ParticleID                   0.00832266     0.0124607     0.018388      0.0113131    0.00337703       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0370939     0.060616      0.0850111     0.057577      0.0167503       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00865166     0.0123369     0.0194609     0.010957     0.00332074       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.7046        3.7258        5.76178       3.46139       1.23561        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.12671       4.64418       7.29795       4.70865       1.44122        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.17879       4.17066       6.71708       4.01701       1.34193        10     
[art]:TriggerResults:TriggerResultInserter         4.235e-05    6.73793e-05   0.000162191    5.621e-05    3.31881e-05      10     
end_path:out1:RootOutput                           9.08e-06     1.16941e-05    2.064e-05    1.1025e-05    3.15139e-06      10     
end_path:out1:RootOutput(write)                     2.04864       2.66155       3.96736       2.25473      0.651344        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4907.97 MB
  Peak resident set size usage (VmHWM): 4055.76 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.23 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.20 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.13 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10033476
-rw-r--r--. 1 duneprd np-comp 2356038680 Jan  3 17:28 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2240092636 Jan  3 17:26 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2032539624 Jan  3 18:22 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1823578540 Jan  3 15:22 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1749055244 Jan  3 15:00 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   72361430 Jan  3 18:19 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     242697 Jan  3 18:22 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan  3 18:22 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan  3 18:22 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan  3 18:22 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan  3 17:28 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan  3 15:22 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan  3 18:22 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan  3 14:51 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan  3 18:22 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan  3 17:36 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan  3 17:28 debugprod.log
justIN time: 2026-02-06 22:22:29 UTC       justIN version: 01.06.00