Jobsub ID 257420.1@dunegpschedd01.fnal.gov
| Jobsub ID | 257420.1@dunegpschedd01.fnal.gov |
| Workflow ID | 10688 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| HTCondor Group | group_dune |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2025-11-25 15:08:27 |
| Site | NL_SURFsara |
| Entry | DUNE_SurfSARA_arc01 |
| Last heartbeat | 2025-11-25 19:47:07 |
| From worker node | Hostname | wn-la-17.gina.surf.nl |
| cpuinfo | AMD EPYC 9754 128-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 129600 (36 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2025-11-25 15:09:12 |
| Input files | monte-carlo-010688-000001
|
| Jobscript | Exit code | 0 |
| Real time | 4h (16623s) |
| CPU time | 4h (16421s = 98%) |
| Max RSS bytes | 4197744640 (4003 MiB) |
| Outputting started | 2025-11-25 19:46:16 |
| Output files | |
| Finished | 2025-11-25 19:47:07 |
| List job events (HTCondor job logs unavailable) |
Jobscript log (last 10,000 characters)
c
ToolBasedRawDigitPrepService:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 180701 traces, 0 summary
Retagger: tagging trace set: gauss with 139551 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 139551 traces tagged "gauss"
FrameSaver: q=4.87428e+06 n=2542380 tag=gauss
wclsFrameSaver: saving 180701 traces tagged "wiener"
FrameSaver: q=5.16612e+06 n=2267799 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 36447 XUs and 38154 XVs -> 552 XUVs
C:0 T:1 83096 XUs and 79827 XVs -> 4914 XUVs
C:0 T:2 235285 XUs and 288886 XVs -> 13116 XUVs
C:0 T:3 100256 XUs and 107325 XVs -> 1495 XUVs
C:0 T:4 18243 XUs and 19826 XVs -> 378 XUVs
C:0 T:5 119704 XUs and 100460 XVs -> 10144 XUVs
C:0 T:6 128660 XUs and 137352 XVs -> 8621 XUVs
C:0 T:7 45850 XUs and 54874 XVs -> 513 XUVs
C:0 T:8 11579 XUs and 12995 XVs -> 214 XUVs
C:0 T:9 156420 XUs and 148502 XVs -> 19600 XUVs
C:0 T:10 55186 XUs and 23167 XVs -> 5098 XUVs
C:0 T:11 108 XUs and 51 XVs -> 8 XUVs
64653 XUVs total
17844 collection wire objects
58580 potential space points
Neighbour search...
7451162 tests to find 3449292 neighbours
Iterating with no regularization...
Begin: 1.0521e+08
0 8.46402e+07
1 8.32297e+07
2 8.30242e+07
3 8.29628e+07
Now with regularization...
Begin: 6.87149e+07
0 6.86831e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410732528
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410732528
fLifetime: 30000 [us]
25-Nov-2025 20:46:14 CET Closed output file "reco_stage1.root"
25-Nov-2025 20:46:14 CET Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 320.209 466.996 699.372 463.909 110.957 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00107638 0.00504245 0.00832981 0.00512628 0.00228045 10
reco:ophit:OpHitFinder 0.378267 0.418485 0.469619 0.419258 0.0328598 10
reco:opflash:OpFlashFinder 0.19621 0.278332 0.378933 0.278324 0.055556 10
reco:opslicer:OpSlicer 0.723854 0.869161 1.08494 0.863992 0.107411 10
reco:caldata:DataPrepModule 6.24488 6.50283 6.88055 6.45614 0.160331 10
reco:wclsdatasp:WireCellToolkit 42.8729 53.7659 58.7777 56.5024 5.81002 10
reco:gaushit:GausHitFinder 0.961254 1.46546 2.04687 1.45296 0.355788 10
reco:nhitsfilter:NumberOfHitsFilter 0.000363541 0.00052493 0.000632614 0.000512484 8.49219e-05 10
reco:reco3d:SpacePointSolver 11.5723 23.6714 47.5719 22.8925 9.62324 10
reco:hitpdune:DisambigFromSpacePoints 1.0026 2.08766 3.89932 2.07833 0.826906 10
reco:pandora:StandardPandora 21.382 46.5324 105.91 43.268 22.8592 10
reco:pandoraWriter:StandardPandora 0.390843 0.525129 0.696438 0.532774 0.0889357 10
reco:pandoraTrack:LArPandoraTrackCreation 3.74474 6.59331 10.2895 7.07307 2.08115 10
reco:pandoraShower:LArPandoraShowerCreation 3.5409 8.47589 14.8763 7.7894 3.4627 10
reco:pandoracalo:Calorimetry 2.05368 3.62283 5.79488 3.60984 1.05048 10
reco:pandoracalonosce:Calorimetry 1.96872 3.42056 5.28842 3.4692 0.96462 10
reco:pandorapid:Chi2ParticleID 0.00490748 0.0065428 0.00781549 0.00666066 0.000762322 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0432826 0.0609898 0.0772765 0.061162 0.00937272 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00486902 0.00615284 0.00749651 0.00640882 0.000834047 10
reco:pandoraShowercalo:ShowerCalorimetry 3.2086 8.62365 14.7623 7.65375 3.59357 10
reco:pandoraShowercalonosce:ShowerCalorimetry 3.04707 7.99327 14.7617 7.54457 3.3043 10
reco:emtrkmichelid:EmTrackMichelId 162.581 218.478 292.073 223.778 38.7233 10
reco:crttag:SingleCRTMatchingProducer 0.0104062 0.0228562 0.0318429 0.0251835 0.0067364 10
reco:crtreco:TwoCRTMatchingProducer 0.00988978 0.0146729 0.0203269 0.0151149 0.0036499 10
reco:anodepiercerst0:T0RecoAnodePiercers 0.867323 1.21733 1.44394 1.33107 0.208636 10
reco:pandora2Track:LArPandoraTrackCreation 12.1524 22.4439 38.3471 22.0176 8.43737 10
reco:pandora2calo:Calorimetry 5.05607 11.2101 18.3215 11.1431 4.35203 10
reco:pandora2calonosce:Calorimetry 4.81504 10.442 17.5381 10.834 3.94627 10
reco:pandora2pid:Chi2ParticleID 0.0107867 0.0152297 0.0203537 0.01599 0.00292783 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0537803 0.0774935 0.103792 0.081454 0.0139123 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.0106146 0.0151312 0.0207999 0.0154715 0.0030182 10
reco:pandora2Shower:LArPandoraShowerCreation 3.99604 7.5397 12.4714 7.6898 2.60036 10
reco:pandora2Showercalo:ShowerCalorimetry 4.53033 8.87017 13.7665 9.23193 2.89973 10
reco:pandora2Showercalonosce:ShowerCalorimetry 3.8782 7.75407 11.6829 7.87135 2.40253 10
[art]:TriggerResults:TriggerResultInserter 3.7006e-05 5.2168e-05 0.000129494 4.22835e-05 2.66984e-05 10
end_path:out1:RootOutput 7.121e-06 8.5238e-06 1.4542e-05 7.982e-06 2.0708e-06 10
end_path:out1:RootOutput(write) 2.73816 3.76474 5.01919 3.89431 0.660623 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5000.39 MB
Peak resident set size usage (VmHWM): 4197.74 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.06 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.28 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 4.08 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.17 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10469120
-rw-r--r--. 1 dune009 dune 2449724687 Nov 25 19:26 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2323411699 Nov 25 19:23 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2170862864 Nov 25 20:46 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1885190794 Nov 25 16:32 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1807877638 Nov 25 16:16 g4_stage1.root
-rw-r--r--. 1 dune009 dune 82696810 Nov 25 20:38 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune 243092 Nov 25 20:46 jobscript.log
-rw-r--r--. 1 dune009 dune 217088 Nov 25 20:46 mem.db
-rw-r--r--. 1 dune009 dune 62376 Nov 25 20:46 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune 36864 Nov 25 20:46 time.db
-rw-r--r--. 1 dune009 dune 519 Nov 25 19:26 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune 519 Nov 25 16:32 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune 519 Nov 25 20:46 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune 52 Nov 25 16:09 all-input-dids.txt
-rw-r--r--. 1 dune009 dune 7 Nov 25 20:46 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune 0 Nov 25 19:32 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune 0 Nov 25 19:26 debugprod.log
