Jobsub ID 263836.0@dunegpschedd02.fnal.gov
| Jobsub ID | 263836.0@dunegpschedd02.fnal.gov |
| Workflow ID | 11310 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2025-12-22 13:46:35 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce01 |
| Last heartbeat | 2025-12-22 17:00:46 |
| From worker node | Hostname | ccwcondor0384 |
| cpuinfo | AMD EPYC 9334 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2025-12-22 13:49:48 |
| Input files | monte-carlo-011310-000001
|
| Jobscript | Exit code | 0 |
| Real time | 3h (11439s) |
| CPU time | 3h (11134s = 97%) |
| Max RSS bytes | 4237676544 (4041 MiB) |
| Outputting started | 2025-12-22 17:00:28 |
| Output files | |
| Finished | 2025-12-22 17:00:46 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare: Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 160041 traces, 0 summary
Retagger: tagging trace set: gauss with 123500 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 123500 traces tagged "gauss"
FrameSaver: q=4.12548e+06 n=2190114 tag=gauss
wclsFrameSaver: saving 160041 traces tagged "wiener"
FrameSaver: q=4.3477e+06 n=1937978 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 36577 XUs and 26837 XVs -> 486 XUVs
C:0 T:1 67427 XUs and 50455 XVs -> 5557 XUVs
C:0 T:2 127358 XUs and 140404 XVs -> 5600 XUVs
C:0 T:3 62477 XUs and 57646 XVs -> 557 XUVs
C:0 T:4 16487 XUs and 13279 XVs -> 220 XUVs
C:0 T:5 125234 XUs and 62643 XVs -> 15976 XUVs
C:0 T:6 150831 XUs and 153268 XVs -> 13081 XUVs
C:0 T:7 48818 XUs and 48610 XVs -> 582 XUVs
C:0 T:8 10200 XUs and 9916 XVs -> 88 XUVs
C:0 T:9 71963 XUs and 83322 XVs -> 5092 XUVs
C:0 T:10 41096 XUs and 27623 XVs -> 4451 XUVs
C:0 T:11 243 XUs and 177 XVs -> 18 XUVs
51708 XUVs total
14662 collection wire objects
47715 potential space points
Neighbour search...
7302465 tests to find 3240374 neighbours
Iterating with no regularization...
Begin: 7.61316e+07
0 6.56732e+07
1 6.49195e+07
2 6.48006e+07
3 6.47638e+07
Now with regularization...
Begin: 5.3645e+07
0 5.36056e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411274506
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411274506
fLifetime: 30000 [us]
22-Dec-2025 18:00:26 CET Closed output file "reco_stage1.root"
22-Dec-2025 18:00:26 CET Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 186.589 342.303 457.523 358.63 78.2733 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00104118 0.00383273 0.00596552 0.00400443 0.00159855 10
reco:ophit:OpHitFinder 0.204709 0.300673 0.354827 0.316105 0.0495446 10
reco:opflash:OpFlashFinder 0.140355 0.197495 0.25338 0.212392 0.0351958 10
reco:opslicer:OpSlicer 0.349489 0.581917 0.732694 0.617625 0.121106 10
reco:caldata:DataPrepModule 3.81984 4.92732 5.85574 4.92722 0.698108 10
reco:wclsdatasp:WireCellToolkit 28.3403 32.9488 41.26 32.58 3.72408 10
reco:gaushit:GausHitFinder 0.675475 1.05242 1.827 0.976287 0.308352 10
reco:nhitsfilter:NumberOfHitsFilter 0.000285694 0.000435285 0.000593503 0.00041049 0.000113458 10
reco:reco3d:SpacePointSolver 5.81434 16.5655 23.2558 17.9504 5.50663 10
reco:hitpdune:DisambigFromSpacePoints 0.533953 1.53218 2.93256 1.4379 0.67443 10
reco:pandora:StandardPandora 9.9257 32.2686 57.227 31.1217 13.7257 10
reco:pandoraWriter:StandardPandora 0.278194 0.444399 0.655439 0.423718 0.128905 10
reco:pandoraTrack:LArPandoraTrackCreation 1.93495 4.64976 8.23114 4.20482 1.91846 10
reco:pandoraShower:LArPandoraShowerCreation 1.19086 5.20935 8.70135 5.29406 2.16441 10
reco:pandoracalo:Calorimetry 1.02614 2.79373 4.43208 2.97426 1.11338 10
reco:pandoracalonosce:Calorimetry 0.957444 2.69298 3.96041 2.88842 0.937598 10
reco:pandorapid:Chi2ParticleID 0.00329875 0.00575463 0.00751192 0.00617512 0.00141966 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0416582 0.0608351 0.0796471 0.0591713 0.0112604 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00427071 0.00606259 0.00738442 0.00628694 0.000925046 10
reco:pandoraShowercalo:ShowerCalorimetry 1.73145 5.47559 10.0105 5.61689 2.20071 10
reco:pandoraShowercalonosce:ShowerCalorimetry 1.83477 4.67381 7.45743 4.65346 1.7811 10
reco:emtrkmichelid:EmTrackMichelId 106.687 179.118 216.746 185.671 33.2504 10
reco:crttag:SingleCRTMatchingProducer 0.00833898 0.0167745 0.0242622 0.0161044 0.00473905 10
reco:crtreco:TwoCRTMatchingProducer 0.00530664 0.0107846 0.0184806 0.0107441 0.00368434 10
reco:anodepiercerst0:T0RecoAnodePiercers 0.621022 0.874822 1.15567 0.788771 0.171975 10
reco:pandora2Track:LArPandoraTrackCreation 5.89775 14.8226 21.5471 14.8024 4.97936 10
reco:pandora2calo:Calorimetry 2.86104 6.67802 9.4821 7.00372 2.16249 10
reco:pandora2calonosce:Calorimetry 2.69648 6.64627 10.6444 6.54777 2.77569 10
reco:pandora2pid:Chi2ParticleID 0.00712782 0.0116321 0.0194937 0.0106886 0.00370251 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0374429 0.0616238 0.0987443 0.0518509 0.0207785 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.0067975 0.0114159 0.0179636 0.0102071 0.00333454 10
reco:pandora2Shower:LArPandoraShowerCreation 2.073 4.60005 7.37777 4.67102 1.50758 10
reco:pandora2Showercalo:ShowerCalorimetry 2.40131 5.60214 8.76998 5.26048 2.01963 10
reco:pandora2Showercalonosce:ShowerCalorimetry 2.13663 4.78555 7.37406 4.79906 1.64971 10
[art]:TriggerResults:TriggerResultInserter 4.2084e-05 6.95418e-05 0.000136538 6.1023e-05 2.60569e-05 10
end_path:out1:RootOutput 7.161e-06 1.03159e-05 1.4722e-05 1.001e-05 1.93243e-06 10
end_path:out1:RootOutput(write) 1.72649 2.4925 3.14797 2.54471 0.487833 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5078.29 MB
Peak resident set size usage (VmHWM): 4237.68 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.06 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.24 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 3.01 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.13 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10669344
-rw-r--r-- 1 dune001 lbno 2481906268 Dec 22 17:01 detsim_stage2.root
-rw-r--r-- 1 dune001 lbno 2350904631 Dec 22 16:59 detsim_stage1.root
-rw-r--r-- 1 dune001 lbno 2186807707 Dec 22 18:00 reco_stage1.root
-rw-r--r-- 1 dune001 lbno 1959311666 Dec 22 15:09 g4_stage2.root
-rw-r--r-- 1 dune001 lbno 1867028667 Dec 22 14:56 g4_stage1.root
-rw-r--r-- 1 dune001 lbno 78833720 Dec 22 17:55 Pandora_Events.pndr
-rw-r--r-- 1 dune001 lbno 242941 Dec 22 18:00 jobscript.log
-rw-r--r-- 1 dune001 lbno 217088 Dec 22 18:00 mem.db
-rw-r--r-- 1 dune001 lbno 62376 Dec 22 18:00 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 lbno 36864 Dec 22 18:00 time.db
-rw-r--r-- 1 dune001 lbno 519 Dec 22 17:01 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno 519 Dec 22 15:09 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno 519 Dec 22 18:00 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 lbno 52 Dec 22 14:49 all-input-dids.txt
-rw-r--r-- 1 dune001 lbno 7 Dec 22 18:00 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 lbno 0 Dec 22 17:07 ProtoDUNE.log
-rw-r--r-- 1 dune001 lbno 0 Dec 22 17:01 debugprod.log
