Jobsub ID 293443.0@dunegpschedd02.fnal.gov
| Jobsub ID | 293443.0@dunegpschedd02.fnal.gov |
| Workflow ID | 12218 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-01-22 08:04:45 |
| Site | US_FNAL-FermiGrid |
| Entry | FNAL_GPGrid_ce03_mcore_op_duneonly |
| Last heartbeat | 2026-01-22 10:45:40 |
| From worker node | Hostname | dunegli-8232562-0-fnpc22009.fnal.gov |
| cpuinfo | AMD EPYC 7543 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 172800 (48 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-01-22 08:06:00 |
| Input files | monte-carlo-012218-000001
|
| Jobscript | Exit code | 0 |
| Real time | 2h (9566s) |
| CPU time | 2h (9436s = 98%) |
| Max RSS bytes | 4501131264 (4292 MiB) |
| Outputting started | 2026-01-22 10:45:26 |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12218/1/001/reco_stage1.root
|
| Finished | 2026-01-22 10:45:40 |
| Saved logs | justin-logs:293443.0-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
edRawDigitPrepService:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 173662 traces, 0 summary
Retagger: tagging trace set: gauss with 128565 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 128565 traces tagged "gauss"
FrameSaver: q=4.50292e+06 n=2213951 tag=gauss
wclsFrameSaver: saving 173662 traces tagged "wiener"
FrameSaver: q=4.80119e+06 n=1931644 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 35223 XUs and 27844 XVs -> 320 XUVs
C:0 T:1 90169 XUs and 59605 XVs -> 5509 XUVs
C:0 T:2 139525 XUs and 141070 XVs -> 7151 XUVs
C:0 T:3 62692 XUs and 55298 XVs -> 922 XUVs
C:0 T:4 17987 XUs and 19645 XVs -> 226 XUVs
C:0 T:5 137673 XUs and 95919 XVs -> 12438 XUVs
C:0 T:6 113485 XUs and 153015 XVs -> 19072 XUVs
C:0 T:7 39378 XUs and 46405 XVs -> 783 XUVs
C:0 T:8 14109 XUs and 11439 XVs -> 156 XUVs
C:0 T:9 206664 XUs and 119415 XVs -> 13551 XUVs
C:0 T:10 73912 XUs and 57607 XVs -> 11324 XUVs
C:0 T:11 76 XUs and 68 XVs -> 10 XUVs
71462 XUVs total
16934 collection wire objects
64941 potential space points
Neighbour search...
8886459 tests to find 4143448 neighbours
Iterating with no regularization...
Begin: 1.26869e+08
0 8.32219e+07
1 8.10415e+07
2 8.06467e+07
3 8.05145e+07
4 8.04468e+07
Now with regularization...
Begin: 6.496e+07
0 6.48392e+07
1 6.47673e+07
2 6.47182e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411893378
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411893378
fLifetime: 30000 [us]
22-Jan-2026 10:45:25 UTC Closed output file "reco_stage1.root"
22-Jan-2026 10:45:25 UTC Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 165.414 279.436 334.291 284.629 46.1143 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000673673 0.00310775 0.0054569 0.00346346 0.00144998 10
reco:ophit:OpHitFinder 0.202841 0.244565 0.286113 0.235953 0.0227187 10
reco:opflash:OpFlashFinder 0.132872 0.180144 0.20838 0.175062 0.0234603 10
reco:opslicer:OpSlicer 0.420459 0.545752 0.668796 0.538893 0.0724849 10
reco:caldata:DataPrepModule 3.41781 3.65042 4.05667 3.61151 0.164432 10
reco:wclsdatasp:WireCellToolkit 23.8088 24.9433 27.3646 24.2255 1.1901 10
reco:gaushit:GausHitFinder 0.583542 0.893985 1.10925 0.925004 0.129944 10
reco:nhitsfilter:NumberOfHitsFilter 0.000284037 0.000504061 0.000670417 0.000507249 0.000107343 10
reco:reco3d:SpacePointSolver 7.39574 14.8545 21.3974 15.1516 4.00586 10
reco:hitpdune:DisambigFromSpacePoints 0.725726 1.5272 2.00671 1.53847 0.33447 10
reco:pandora:StandardPandora 9.93115 31.3519 47.7335 30.5013 10.3949 10
reco:pandoraWriter:StandardPandora 0.214552 0.360694 0.431527 0.380115 0.0654296 10
reco:pandoraTrack:LArPandoraTrackCreation 1.65928 3.31903 4.40504 3.44961 0.827445 10
reco:pandoraShower:LArPandoraShowerCreation 1.70335 4.16377 5.47856 4.56396 1.13271 10
reco:pandoracalo:Calorimetry 0.910579 1.90979 2.49941 1.98492 0.443649 10
reco:pandoracalonosce:Calorimetry 0.846695 1.77548 2.26057 1.8444 0.418154 10
reco:pandorapid:Chi2ParticleID 0.00318429 0.00419979 0.00525448 0.00429988 0.000566882 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0294777 0.040158 0.0470641 0.0398617 0.00492195 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00281591 0.00396709 0.00465766 0.00400267 0.000601145 10
reco:pandoraShowercalo:ShowerCalorimetry 1.66029 4.54653 6.06019 4.70952 1.27327 10
reco:pandoraShowercalonosce:ShowerCalorimetry 1.59439 4.48881 5.88004 4.67781 1.2675 10
reco:emtrkmichelid:EmTrackMichelId 92.0939 140.013 157.81 143.048 17.9698 10
reco:crttag:SingleCRTMatchingProducer 0.00971383 0.0121911 0.0183771 0.0108176 0.00262353 10
reco:crtreco:TwoCRTMatchingProducer 0.00411351 0.00625866 0.00701762 0.00644411 0.000824048 10
reco:anodepiercerst0:T0RecoAnodePiercers 0.584079 0.797804 0.96348 0.818277 0.0985918 10
reco:pandora2Track:LArPandoraTrackCreation 4.87427 11.6947 16.4855 11.6457 3.25981 10
reco:pandora2calo:Calorimetry 2.40987 6.0178 7.59299 6.13876 1.5407 10
reco:pandora2calonosce:Calorimetry 2.29422 5.74673 7.32704 5.84007 1.46161 10
reco:pandora2pid:Chi2ParticleID 0.00681982 0.010387 0.0119169 0.0108585 0.00154965 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0334047 0.0511904 0.0598986 0.0499648 0.00728472 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.00652495 0.0101292 0.0119516 0.0105901 0.00159392 10
reco:pandora2Shower:LArPandoraShowerCreation 1.95311 3.97144 5.20155 4.01647 0.900363 10
reco:pandora2Showercalo:ShowerCalorimetry 2.14909 5.10066 6.57947 5.1603 1.24392 10
reco:pandora2Showercalonosce:ShowerCalorimetry 1.98682 4.64967 6.07936 4.72553 1.11168 10
[art]:TriggerResults:TriggerResultInserter 3.4305e-05 4.98332e-05 0.000116841 4.36775e-05 2.27666e-05 10
end_path:out1:RootOutput 6.593e-06 7.9951e-06 1.3385e-05 7.339e-06 1.89524e-06 10
end_path:out1:RootOutput(write) 1.55013 2.44162 2.82598 2.46128 0.348605 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5316.3 MB
Peak resident set size usage (VmHWM): 4501.13 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.05 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.16 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.03 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 2.31 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.08 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.03 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11981360
-rw-r--r-- 1 dunegli fnalgrid 2731214586 Jan 22 09:57 detsim_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2589587524 Jan 22 09:55 detsim_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 2498666049 Jan 22 10:45 reco_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 2224345870 Jan 22 08:20 g4_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2130407380 Jan 22 08:10 g4_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 94074880 Jan 22 10:41 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid 243556 Jan 22 10:45 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid 217088 Jan 22 10:45 mem.db
-rw-r--r-- 1 dunegli fnalgrid 62376 Jan 22 10:45 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid 36864 Jan 22 10:45 time.db
-rw-r--r-- 1 dunegli fnalgrid 519 Jan 22 09:57 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid 519 Jan 22 08:20 g4_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid 519 Jan 22 10:45 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dunegli fnalgrid 52 Jan 22 08:06 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid 7 Jan 22 10:45 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid 0 Jan 22 10:01 ProtoDUNE.log
-rw-r--r-- 1 dunegli fnalgrid 0 Jan 22 09:57 debugprod.log
