Jobsub ID 257418.1@dunegpschedd01.fnal.gov
| Jobsub ID | 257418.1@dunegpschedd01.fnal.gov |
| Workflow ID | 10685 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2025-11-25 15:08:27 |
| Site | NL_SURFsara |
| Entry | DUNE_SurfSARA_arc02 |
| Last heartbeat | 2025-11-25 19:33:42 |
| From worker node | Hostname | wn-la-08.gina.surf.nl |
| cpuinfo | AMD EPYC 9754 128-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 129600 (36 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2025-11-25 15:09:11 |
| Input files | monte-carlo-010685-000001
|
| Jobscript | Exit code | 0 |
| Real time | 4h (15819s) |
| CPU time | 4h (15609s = 98%) |
| Max RSS bytes | 4281450496 (4083 MiB) |
| Outputting started | 2025-11-25 19:32:51 |
| Output files | |
| Finished | 2025-11-25 19:33:42 |
| List job events (HTCondor job logs unavailable) |
Jobscript log (last 10,000 characters)
gitPrepService:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 111558 traces, 0 summary
Retagger: tagging trace set: gauss with 84965 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 84965 traces tagged "gauss"
FrameSaver: q=2.84055e+06 n=1487815 tag=gauss
wclsFrameSaver: saving 111558 traces tagged "wiener"
FrameSaver: q=3.02006e+06 n=1318306 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 21392 XUs and 20531 XVs -> 173 XUVs
C:0 T:1 34340 XUs and 32555 XVs -> 2383 XUVs
C:0 T:2 96466 XUs and 101387 XVs -> 5885 XUVs
C:0 T:3 44988 XUs and 34037 XVs -> 256 XUVs
C:0 T:4 8168 XUs and 11551 XVs -> 96 XUVs
C:0 T:5 30190 XUs and 34226 XVs -> 2878 XUVs
C:0 T:6 106520 XUs and 87114 XVs -> 4913 XUVs
C:0 T:7 44166 XUs and 34203 XVs -> 400 XUVs
C:0 T:8 5899 XUs and 7191 XVs -> 73 XUVs
C:0 T:9 37890 XUs and 53353 XVs -> 3504 XUVs
C:0 T:10 44740 XUs and 25073 XVs -> 5045 XUVs
C:0 T:11 377 XUs and 281 XVs -> 93 XUVs
25699 XUVs total
10877 collection wire objects
23704 potential space points
Neighbour search...
1366766 tests to find 737168 neighbours
Iterating with no regularization...
Begin: 5.35704e+07
0 4.51051e+07
1 4.43639e+07
2 4.42472e+07
3 4.42134e+07
Now with regularization...
Begin: 3.59457e+07
0 3.59057e+07
1 3.58898e+07
TestBeamParticleCreationAlgorithm::SetupTestBeamVertex - Test beam particle has no initial vertex
run: 1 ; subrun: 0 ; event: 10
evttime: 410732563
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410732563
fLifetime: 30000 [us]
25-Nov-2025 20:32:48 CET Closed output file "reco_stage1.root"
25-Nov-2025 20:32:48 CET Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 292.059 401.135 537.945 392.968 72.0703 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00124201 0.00611724 0.0130794 0.00584032 0.00340716 10
reco:ophit:OpHitFinder 0.315921 0.39551 0.559868 0.381712 0.0649364 10
reco:opflash:OpFlashFinder 0.166615 0.235485 0.300924 0.241163 0.0355143 10
reco:opslicer:OpSlicer 0.646355 0.794079 0.942309 0.806005 0.0955093 10
reco:caldata:DataPrepModule 6.18133 6.4804 6.90525 6.47059 0.21392 10
reco:wclsdatasp:WireCellToolkit 43.3288 53.2524 60.0692 55.019 6.13258 10
reco:gaushit:GausHitFinder 0.951939 1.24123 1.49426 1.24434 0.184366 10
reco:nhitsfilter:NumberOfHitsFilter 0.000364535 0.000561709 0.000772286 0.000537965 0.000132911 10
reco:reco3d:SpacePointSolver 10.2688 18.5401 34.4819 15.8216 6.77447 10
reco:hitpdune:DisambigFromSpacePoints 0.966641 1.50508 2.09365 1.40737 0.412275 10
reco:pandora:StandardPandora 18.199 31.1268 50.2556 31.0311 9.9013 10
reco:pandoraWriter:StandardPandora 0.349143 0.454565 0.574811 0.454622 0.0684284 10
reco:pandoraTrack:LArPandoraTrackCreation 3.43947 6.03429 12.8236 5.16297 2.62694 10
reco:pandoraShower:LArPandoraShowerCreation 3.12447 6.63558 12.0891 5.70043 2.45977 10
reco:pandoracalo:Calorimetry 1.74147 3.48092 6.38725 3.0341 1.40389 10
reco:pandoracalonosce:Calorimetry 1.56848 3.36375 6.29844 2.71502 1.50935 10
reco:pandorapid:Chi2ParticleID 0.00495796 0.00587095 0.00753998 0.00546606 0.000935069 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0392476 0.0531814 0.0644295 0.0547044 0.0077092 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00474701 0.00566763 0.00738972 0.00546514 0.000756708 10
reco:pandoraShowercalo:ShowerCalorimetry 2.80919 7.37336 13.6298 6.79466 3.10893 10
reco:pandoraShowercalonosce:ShowerCalorimetry 2.80052 6.79919 12.2593 6.21595 2.65084 10
reco:emtrkmichelid:EmTrackMichelId 150.258 186.686 224.105 183.905 24.3865 10
reco:crttag:SingleCRTMatchingProducer 0.0110487 0.0189149 0.0290431 0.0164093 0.00649167 10
reco:crtreco:TwoCRTMatchingProducer 0.00983915 0.0136897 0.0183469 0.0133449 0.00296419 10
reco:anodepiercerst0:T0RecoAnodePiercers 0.784495 0.980973 1.37591 0.951857 0.165268 10
reco:pandora2Track:LArPandoraTrackCreation 10.1011 19.4087 31.5965 19.0224 6.44256 10
reco:pandora2calo:Calorimetry 4.4038 9.78957 17.5332 9.37576 3.90173 10
reco:pandora2calonosce:Calorimetry 4.14835 9.30412 16.5219 9.01898 3.61499 10
reco:pandora2pid:Chi2ParticleID 0.0107859 0.0131203 0.0157709 0.0134336 0.00163284 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0506509 0.0670419 0.0789499 0.0683054 0.00925956 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.0106702 0.0132256 0.0162418 0.0133896 0.001684 10
reco:pandora2Shower:LArPandoraShowerCreation 3.7027 7.32057 11.4641 7.77416 2.44585 10
reco:pandora2Showercalo:ShowerCalorimetry 3.49288 8.7873 15.612 9.66678 3.61324 10
reco:pandora2Showercalonosce:ShowerCalorimetry 3.12001 7.4875 12.239 7.66366 2.964 10
[art]:TriggerResults:TriggerResultInserter 4.2373e-05 7.10675e-05 0.00016724 5.2018e-05 4.12433e-05 10
end_path:out1:RootOutput 7.301e-06 8.5708e-06 1.3911e-05 7.987e-06 1.82782e-06 10
end_path:out1:RootOutput(write) 2.52674 3.22821 3.82449 3.21241 0.435592 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5086.58 MB
Peak resident set size usage (VmHWM): 4281.45 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.06 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.29 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 4.04 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.17 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10490952
-rw-r--r--. 1 dune009 dune 2435605010 Nov 25 19:23 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2321147681 Nov 25 19:21 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2104259502 Nov 25 20:32 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1983129239 Nov 25 16:32 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1827959799 Nov 25 16:16 g4_stage1.root
-rw-r--r--. 1 dune009 dune 70016780 Nov 25 20:28 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune 242899 Nov 25 20:32 jobscript.log
-rw-r--r--. 1 dune009 dune 217088 Nov 25 20:32 mem.db
-rw-r--r--. 1 dune009 dune 62376 Nov 25 20:32 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune 36864 Nov 25 20:32 time.db
-rw-r--r--. 1 dune009 dune 519 Nov 25 19:23 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune 519 Nov 25 16:32 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune 519 Nov 25 20:32 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune 52 Nov 25 16:09 all-input-dids.txt
-rw-r--r--. 1 dune009 dune 7 Nov 25 20:32 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune 0 Nov 25 19:30 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune 0 Nov 25 19:23 debugprod.log
