justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 271429.1@dunegpschedd01.fnal.gov

Jobsub ID271429.1@dunegpschedd01.fnal.gov
Workflow ID11488
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-03 13:51:50
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce511
Last heartbeat2026-01-03 17:47:08
From worker nodeHostnameb9p17p1249.cern.ch
cpuinfoIntel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-03 13:52:34
Input filesmonte-carlo-011488-000001
JobscriptExit code0
Real time3h (13990s)
CPU time3h (12879s = 92%)
Max RSS bytes4441006080 (4235 MiB)
Outputting started2026-01-03 17:45:45
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11488/1/001/reco_stage1.root
Finished2026-01-03 17:47:08
Saved logsjustin-logs:271429.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

g tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 135017 traces, 0 summary
Retagger: tagging trace set: gauss with 101848 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 101848 traces tagged "gauss"
FrameSaver: q=3.38405e+06 n=1810993 tag=gauss
wclsFrameSaver: saving 135017 traces tagged "wiener"
FrameSaver: q=3.57864e+06 n=1592714 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 34668 XUs and 28983 XVs -> 280 XUVs
C:0 T:1 81036 XUs and 56587 XVs -> 5881 XUVs
C:0 T:2 64974 XUs and 77403 XVs -> 3317 XUVs
C:0 T:3 34323 XUs and 32193 XVs -> 166 XUVs
C:0 T:4 17750 XUs and 12421 XVs -> 186 XUVs
C:0 T:5 164081 XUs and 76303 XVs -> 16693 XUVs
C:0 T:6 103164 XUs and 111487 XVs -> 3631 XUVs
C:0 T:7 44891 XUs and 47898 XVs -> 304 XUVs
C:0 T:8 10525 XUs and 7897 XVs -> 119 XUVs
C:0 T:9 110262 XUs and 84891 XVs -> 11757 XUVs
C:0 T:10 29993 XUs and 22421 XVs -> 2308 XUVs
C:0 T:11 143 XUs and 116 XVs -> 9 XUVs
44651 XUVs total
13240 collection wire objects
41896 potential space points
Neighbour search...
6129194 tests to find 2678946 neighbours
Iterating with no regularization...
Begin: 5.15583e+07
0 4.24011e+07
1 4.16992e+07
2 4.15813e+07
3 4.15424e+07
Now with regularization...
Begin: 3.27892e+07
0 3.27564e+07
1 3.27296e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411515899
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411515899
fLifetime: 30000 [us]
03-Jan-2026 18:45:43 CET  Closed output file "reco_stage1.root"
03-Jan-2026 18:45:43 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          416.503       509.042       606.861       513.682       57.1093        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00104056    0.00347099    0.00564812    0.00332914    0.00141184       10     
reco:ophit:OpHitFinder                             0.293935       0.34427      0.394942      0.347828      0.0312308       10     
reco:opflash:OpFlashFinder                         0.201172      0.287051      0.349135       0.29328      0.0431196       10     
reco:opslicer:OpSlicer                             0.707477      0.862771      0.985122      0.876183      0.0968426       10     
reco:caldata:DataPrepModule                         6.24515       6.62816       7.20745       6.5812       0.272145        10     
reco:wclsdatasp:WireCellToolkit                     49.9146       54.0803       56.0936       55.2364       2.15949        10     
reco:gaushit:GausHitFinder                          1.46473       1.74599       2.06987       1.74842      0.176546        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000617807   0.000734753   0.000887496   0.000732828   8.42169e-05      10     
reco:reco3d:SpacePointSolver                        11.4142       18.9345       29.1685       17.1301       5.13779        10     
reco:hitpdune:DisambigFromSpacePoints               2.02506       2.8608        3.6049        2.94931      0.505321        10     
reco:pandora:StandardPandora                        33.9575       47.869        58.2934       48.8746       7.31179        10     
reco:pandoraWriter:StandardPandora                 0.445866       0.56742      0.644058      0.576771      0.0664563       10     
reco:pandoraTrack:LArPandoraTrackCreation           6.04259       9.20566       12.1135       8.92982       1.67051        10     
reco:pandoraShower:LArPandoraShowerCreation         7.02848       10.7263       14.8342       10.1181       2.55499        10     
reco:pandoracalo:Calorimetry                        2.79169       4.39671       5.24779       4.51047      0.756573        10     
reco:pandoracalonosce:Calorimetry                   2.5701        4.05801       5.05811       4.17383      0.777661        10     
reco:pandorapid:Chi2ParticleID                    0.00514225    0.00662712    0.00803083    0.00651312    0.000766524      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0650347     0.0814208     0.0973895     0.0800867    0.00857526       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00506168    0.00639939    0.00755198    0.00643722    0.000632834      10     
reco:pandoraShowercalo:ShowerCalorimetry            6.02037       9.99209       13.222        9.94271       2.20869        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       6.18733       9.46254       12.1745       9.68774       1.88759        10     
reco:emtrkmichelid:EmTrackMichelId                  202.868       237.276       278.22        238.602       22.5982        10     
reco:crttag:SingleCRTMatchingProducer               0.01986      0.0266262     0.0327884     0.0271848    0.00367804       10     
reco:crtreco:TwoCRTMatchingProducer                0.0145749     0.0207861     0.0300421     0.0200621    0.00457512       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.41373       1.56781       1.75257       1.54944      0.107372        10     
reco:pandora2Track:LArPandoraTrackCreation          22.8669       31.2422       40.6036       30.2411       5.9145         10     
reco:pandora2calo:Calorimetry                       8.84524       13.0821       17.5344       13.0932       2.47563        10     
reco:pandora2calonosce:Calorimetry                  8.54939       11.9448       15.613        11.8501       2.20621        10     
reco:pandora2pid:Chi2ParticleID                    0.0149855     0.0176745     0.0218984     0.0175234    0.00191067       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0931076     0.104586      0.123004      0.102615     0.00996428       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0151555     0.0176159     0.021553      0.0175521     0.0018366       10     
reco:pandora2Shower:LArPandoraShowerCreation        6.54816       8.64443       11.6359       8.36529       1.69946        10     
reco:pandora2Showercalo:ShowerCalorimetry           7.10449       10.5269       14.4143       10.4639       1.98668        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      6.60248       8.97066       12.0679       8.66488       1.59713        10     
[art]:TriggerResults:TriggerResultInserter        4.7013e-05    5.80426e-05   0.000107813   5.2613e-05    1.68579e-05      10     
end_path:out1:RootOutput                           7.33e-06     8.7152e-06    1.0584e-05    8.6035e-06    8.06518e-07      10     
end_path:out1:RootOutput(write)                     2.77086       3.33011       3.97582       3.32519      0.357507        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5261.03 MB
  Peak resident set size usage (VmHWM): 4441.01 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.24 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.17 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.23 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10627596
-rw-r--r--. 1 duneprd np-comp 2479150684 Jan  3 17:17 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2345894159 Jan  3 17:14 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2201058826 Jan  3 18:45 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1936355800 Jan  3 15:27 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1836728017 Jan  3 15:01 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   82857850 Jan  3 18:39 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     242791 Jan  3 18:45 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan  3 18:45 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan  3 18:45 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan  3 18:45 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan  3 17:17 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan  3 15:27 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan  3 18:45 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan  3 14:52 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan  3 18:45 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan  3 17:26 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan  3 17:17 debugprod.log
justIN time: 2026-02-06 22:18:30 UTC       justIN version: 01.06.00