justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 249436.0@dunegpschedd02.fnal.gov

Jobsub ID249436.0@dunegpschedd02.fnal.gov
Workflow ID10493
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-21 21:05:48
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc01
Last heartbeat2025-11-22 01:31:30
From worker nodeHostnamewn-la-06.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-21 21:06:34
Input filesmonte-carlo-010493-000001
JobscriptExit code0
Real time4h (14886s)
CPU time4h (14733s = 98%)
Max RSS bytes4051070976 (3863 MiB)
Outputting started2025-11-22 01:14:41
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10493/1/001/reco_stage1.root
Finished2025-11-22 01:31:30
Saved logsjustin-logs:249436.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 94043 traces, 0 summary
Retagger: tagging trace set: gauss with 73888 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 73888 traces tagged "gauss"
FrameSaver: q=2.39978e+06 n=1311717 tag=gauss
wclsFrameSaver: saving 94043 traces tagged "wiener"
FrameSaver: q=2.5379e+06 n=1166584 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 46455 XUs and 36892 XVs -> 695 XUVs
C:0 T:1 131667 XUs and 106306 XVs -> 15899 XUVs
C:0 T:2 19294 XUs and 36463 XVs -> 907 XUVs
C:0 T:3 9328 XUs and 18222 XVs -> 63 XUVs
C:0 T:4 11649 XUs and 11302 XVs -> 79 XUVs
C:0 T:5 42810 XUs and 32848 XVs -> 3535 XUVs
C:0 T:6 41178 XUs and 47807 XVs -> 1399 XUVs
C:0 T:7 18217 XUs and 20952 XVs -> 101 XUVs
C:0 T:8 8226 XUs and 6804 XVs -> 109 XUVs
C:0 T:9 50698 XUs and 56780 XVs -> 4209 XUVs
C:0 T:10 10586 XUs and 4905 XVs -> 1099 XUVs
C:0 T:11 34 XUs and 30 XVs -> 3 XUVs
28098 XUVs total
8653 collection wire objects
25917 potential space points
Neighbour search...
4094183 tests to find 1703342 neighbours
Iterating with no regularization...
Begin: 4.40838e+07
0 3.91495e+07
1 3.87989e+07
2 3.87433e+07
3 3.87277e+07
Now with regularization...
Begin: 3.22736e+07
0 3.22653e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410657080
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410657080
fLifetime: 30000 [us]
22-Nov-2025 02:14:39 CET  Closed output file "reco_stage1.root"
22-Nov-2025 02:14:39 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          308.816       417.679       622.151       401.613       82.3681        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00112456    0.00501556    0.00872609    0.00458754    0.00243928       10     
reco:ophit:OpHitFinder                             0.341589       0.38562      0.434225      0.381278      0.0297313       10     
reco:opflash:OpFlashFinder                         0.190366      0.237389      0.296263      0.236457      0.0314515       10     
reco:opslicer:OpSlicer                             0.682471       0.76422      0.877604      0.759436      0.060442        10     
reco:caldata:DataPrepModule                         6.21178        6.401        6.91091       6.3651       0.182506        10     
reco:wclsdatasp:WireCellToolkit                     43.2999       54.8881       62.8592       57.4118       6.23646        10     
reco:gaushit:GausHitFinder                         0.996275       1.34981       1.78924       1.30793      0.228221        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000398737   0.000493246   0.00078236    0.000452973   0.000117349      10     
reco:reco3d:SpacePointSolver                        9.9715        20.5706       34.5971       18.4591       7.79657        10     
reco:hitpdune:DisambigFromSpacePoints               1.05908       1.5596        2.40051       1.42676      0.355705        10     
reco:pandora:StandardPandora                        21.2319       40.7655       132.929       26.6849       31.5696        10     
reco:pandoraWriter:StandardPandora                 0.375725      0.471566      0.578215      0.459219      0.0545105       10     
reco:pandoraTrack:LArPandoraTrackCreation           2.98271       5.80565       9.2182        5.36613       1.70028        10     
reco:pandoraShower:LArPandoraShowerCreation         3.21668       6.27234       10.4192       5.95268       2.15912        10     
reco:pandoracalo:Calorimetry                        1.51355       3.11094       4.41885       3.05656      0.774289        10     
reco:pandoracalonosce:Calorimetry                   1.39408       2.84459       4.1659        2.74618      0.725648        10     
reco:pandorapid:Chi2ParticleID                    0.00482975    0.00605044    0.00742282    0.00593377    0.000756569      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0427584     0.0521482     0.0612817     0.0524449    0.00452689       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00478019     0.0058602     0.007348     0.00585417    0.000616026      10     
reco:pandoraShowercalo:ShowerCalorimetry            3.21289       6.15416       9.52644       5.84705       1.82954        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.97838       5.82468       8.86825       5.80542       1.6445         10     
reco:emtrkmichelid:EmTrackMichelId                  164.325       202.159       253.509       199.415       25.4301        10     
reco:crttag:SingleCRTMatchingProducer             0.00896025     0.0132778     0.0162916     0.0137011    0.00188824       10     
reco:crtreco:TwoCRTMatchingProducer               0.00836537     0.0113178     0.0123128     0.0118731    0.00119111       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.723943      0.994967       1.13116       1.00088      0.101711        10     
reco:pandora2Track:LArPandoraTrackCreation          9.7345        17.4127       27.0756       16.8193       4.68647        10     
reco:pandora2calo:Calorimetry                       4.15816       8.36854       12.3631       8.32299       2.14255        10     
reco:pandora2calonosce:Calorimetry                  4.02907       8.1089        11.4191       8.04204       2.08588        10     
reco:pandora2pid:Chi2ParticleID                    0.0109965     0.0133835     0.0172842     0.0131034    0.00169636       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0535687     0.0661455     0.0828445     0.0655733    0.00695332       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0112009     0.0135159     0.0178769     0.013153     0.00178135       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.31995       6.20419       8.83671       6.11756       1.56363        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.52539       7.13275       10.0186       7.40109       1.69472        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.15771       6.27103       8.2775        6.36863       1.28976        10     
[art]:TriggerResults:TriggerResultInserter        3.7115e-05     5.238e-05    0.000121611   4.42565e-05   2.35023e-05      10     
end_path:out1:RootOutput                           7.461e-06     8.72e-06      1.296e-05    8.1865e-06    1.4808e-06       10     
end_path:out1:RootOutput(write)                     2.51767       3.23778       3.99559       3.22219      0.365266        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4849.72 MB
  Peak resident set size usage (VmHWM): 4051.07 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.28 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.99 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9584152
-rw-r--r--. 1 dune009 dune 2272523976 Nov 22 01:03 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2156509701 Nov 22 01:01 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 1944485820 Nov 22 02:14 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1730597667 Nov 21 22:27 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1637961519 Nov 21 22:13 g4_stage1.root
-rw-r--r--. 1 dune009 dune   71482640 Nov 22 02:11 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     243005 Nov 22 02:14 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Nov 22 02:14 mem.db
-rw-r--r--. 1 dune009 dune      62376 Nov 22 02:14 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Nov 22 02:14 time.db
-rw-r--r--. 1 dune009 dune        519 Nov 22 01:03 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov 21 22:27 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov 22 02:14 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Nov 21 22:06 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Nov 22 02:14 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Nov 22 01:09 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Nov 22 01:03 debugprod.log
justIN time: 2026-02-08 22:55:38 UTC       justIN version: 01.06.00