Jobsub ID 236336.1@dunegpschedd02.fnal.gov
| Jobsub ID | 236336.1@dunegpschedd02.fnal.gov | 
| Workflow ID | 9526 | 
| Stage ID | 1 | 
| User name | pmelas@fnal.gov | 
| HTCondor Group | group_dune | 
| Requested | Processors | 1 | 
| GPU | No | 
| RSS bytes | 8388608000 (8000 MiB) | 
| Wall seconds limit | 86400 (24 hours) | 
| Submitted time | 2025-11-02 14:19:37 | 
| Site | NL_SURFsara | 
| Entry | DUNE_SurfSARA_arc03 | 
| Last heartbeat | 2025-11-02 18:21:06 | 
| From worker node | Hostname | wn-da-20.gina.surf.nl | 
| cpuinfo | AMD EPYC 7702P 64-Core Processor | 
| OS release | Scientific Linux release 7.9 (Nitrogen) | 
| Processors | 1 | 
| RSS bytes | 10485760000 (10000 MiB) | 
| Wall seconds limit | 129600 (36 hours) | 
| GPU |  | 
| Inner Apptainer? | True | 
| Job state | finished | 
| Started | 2025-11-02 14:20:23 | 
| Input files | monte-carlo-009526-000001
  | 
| Jobscript | Exit code | 0 | 
| Real time | 3h (14351s) | 
| CPU time | 3h (14218s = 99%) | 
| Max RSS bytes | 4556361728 (4345 MiB) | 
| Outputting started | 2025-11-02 18:19:35 | 
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09526/1/001/reco_stage1.root
  | 
| Finished | 2025-11-02 18:21:06 | 
| Saved logs | justin-logs:236336.1-dunegpschedd02.fnal.gov.logs.tgz | 
| List job events     Cached HTCondor job logs | 
Jobscript log (last 10,000 characters)
tagger: tagging trace set: wiener with 178080 traces, 0 summary
Retagger: tagging trace set: gauss with 129797 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 129797 traces tagged "gauss"
FrameSaver: q=4.44478e+06 n=2247405 tag=gauss
wclsFrameSaver: saving 178080 traces tagged "wiener"
FrameSaver: q=4.73575e+06 n=1952429 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 57011 XUs and 45862 XVs -> 962 XUVs
C:0 T:1 134753 XUs and 111432 XVs -> 9119 XUVs
C:0 T:2 114092 XUs and 91232 XVs -> 4813 XUVs
C:0 T:3 48096 XUs and 36765 XVs -> 331 XUVs
C:0 T:4 13132 XUs and 11333 XVs -> 105 XUVs
C:0 T:5 65573 XUs and 38362 XVs -> 5235 XUVs
C:0 T:6 196675 XUs and 187479 XVs -> 18329 XUVs
C:0 T:7 63715 XUs and 59870 XVs -> 915 XUVs
C:0 T:8 12173 XUs and 10284 XVs -> 122 XUVs
C:0 T:9 85668 XUs and 90626 XVs -> 6543 XUVs
C:0 T:10 56226 XUs and 60555 XVs -> 12040 XUVs
C:0 T:11 40 XUs and 46 XVs -> 10 XUVs
58524 XUVs total
17889 collection wire objects
52591 potential space points
Neighbour search...
5189805 tests to find 2626726 neighbours
Iterating with no regularization...
Begin: 9.75391e+07
0 7.35354e+07
1 7.13691e+07
2 7.11101e+07
3 7.10412e+07
Now with regularization...
Begin: 5.72429e+07
0 5.71495e+07
1 5.71008e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410269054
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410269054
fLifetime: 30000 [us]
02-Nov-2025 19:19:33 CET  Closed output file "reco_stage1.root"
02-Nov-2025 19:19:33 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          355.126       466.324       619.227       415.854       88.1482        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000951665   0.00548314    0.00968406    0.00547715     0.0029249       10     
reco:ophit:OpHitFinder                             0.270331      0.310941       0.36098      0.305033      0.0267012       10     
reco:opflash:OpFlashFinder                         0.158429      0.216697       0.27807      0.214009      0.0366631       10     
reco:opslicer:OpSlicer                             0.552362      0.732173      0.961864      0.740114      0.108991        10     
reco:caldata:DataPrepModule                         5.22537       5.39234       5.79407       5.3698        0.15755        10     
reco:wclsdatasp:WireCellToolkit                     38.3536       58.3145       68.4399       66.3081       12.7842        10     
reco:gaushit:GausHitFinder                          1.17452       1.44854       1.71347       1.46758      0.165327        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000617348   0.000734352   0.00114072    0.000682802   0.000146558      10     
reco:reco3d:SpacePointSolver                        9.01174       17.7773       24.5789       18.1685       4.84475        10     
reco:hitpdune:DisambigFromSpacePoints               1.30425       1.79528       2.3058        1.69214      0.380855        10     
reco:pandora:StandardPandora                         29.38        43.1292       76.1159       38.9878       13.6475        10     
reco:pandoraWriter:StandardPandora                 0.384883      0.486996      0.615249      0.457594      0.0670373       10     
reco:pandoraTrack:LArPandoraTrackCreation           5.0514        7.19819       12.2822       6.81385       2.14746        10     
reco:pandoraShower:LArPandoraShowerCreation         5.08442       7.38883       12.3931       6.62589       2.3555         10     
reco:pandoracalo:Calorimetry                        2.46606       3.79438       6.22739       3.8538        1.00743        10     
reco:pandoracalonosce:Calorimetry                   2.26673       3.32983       5.86493       3.15612      0.987687        10     
reco:pandorapid:Chi2ParticleID                     0.0065137    0.00747889    0.00889653    0.00725631    0.000858785      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0471333     0.0567835     0.0714479     0.0554299    0.00751059       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00585606    0.00721086    0.00912189    0.00694676    0.000890223      10     
reco:pandoraShowercalo:ShowerCalorimetry            4.07139       6.73104       11.6208       5.99145       2.13944        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.83767       6.50715       11.4479       5.9017        2.12688        10     
reco:emtrkmichelid:EmTrackMichelId                  176.006       236.758       292.258       228.347       35.5997        10     
reco:crttag:SingleCRTMatchingProducer              0.0101563     0.013598      0.0190852     0.0132062    0.00292298       10     
reco:crtreco:TwoCRTMatchingProducer               0.00803353     0.0112428     0.0151076     0.0105535    0.00230128       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.752715      0.940048       1.22309      0.895886      0.153979        10     
reco:pandora2Track:LArPandoraTrackCreation          9.37254       20.0524       31.8385       16.5868       7.72008        10     
reco:pandora2calo:Calorimetry                       4.17693       9.40008       16.0536       8.00915       3.79578        10     
reco:pandora2calonosce:Calorimetry                  4.17041       8.94167       15.4084       7.76063       3.59807        10     
reco:pandora2pid:Chi2ParticleID                    0.0123488      0.0175       0.0215877     0.0168822    0.00267117       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0559783     0.074753      0.0922712     0.0731093     0.0121866       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0125494     0.0175386     0.0210963     0.0173309     0.0025212       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.58099       7.47068       12.3531       6.48772       3.00639        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.72272       7.7248        13.3305       6.52326       3.15298        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.42471       6.74893       11.4612       5.35195       2.74269        10     
[art]:TriggerResults:TriggerResultInserter        3.2743e-05    5.01189e-05   0.000119939   4.1079e-05    2.47912e-05      10     
end_path:out1:RootOutput                           5.881e-06    7.2458e-06    1.0601e-05    7.0135e-06    1.32637e-06      10     
end_path:out1:RootOutput(write)                     2.53843       3.34805       4.16307       3.2426       0.535116        10     
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
  Peak virtual memory usage (VmPeak)  : 5357.84 MB
  Peak resident set size usage (VmHWM): 4556.36 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.38 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.13 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11020876
-rw-r--r--. 1 dune009 dune 2569164511 Nov  2 17:59 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2446493908 Nov  2 17:56 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2273508859 Nov  2 19:19 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1999478314 Nov  2 15:41 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1916708103 Nov  2 15:27 g4_stage1.root
-rw-r--r--. 1 dune009 dune   79411880 Nov  2 19:11 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     243167 Nov  2 19:19 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Nov  2 19:19 mem.db
-rw-r--r--. 1 dune009 dune      62376 Nov  2 19:19 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Nov  2 19:19 time.db
-rw-r--r--. 1 dune009 dune        519 Nov  2 17:59 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov  2 15:41 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov  2 19:19 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Nov  2 15:20 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Nov  2 19:19 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Nov  2 18:05 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Nov  2 17:59 debugprod.log