justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 262273.0@dunegpschedd02.fnal.gov

Jobsub ID262273.0@dunegpschedd02.fnal.gov
Workflow ID11211
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-13 17:10:51
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc02
Last heartbeat2025-12-13 21:18:44
From worker nodeHostnamewn-lb-17.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-13 17:12:52
Input filesmonte-carlo-011211-000001
JobscriptExit code0
Real time4h (14659s)
CPU time4h (14505s = 98%)
Max RSS bytes4186873856 (3992 MiB)
Outputting started2025-12-13 21:17:12
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11211/1/001/reco_stage1.root
Finished2025-12-13 21:18:44
Saved logsjustin-logs:262273.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 150174 traces, 0 summary
Retagger: tagging trace set: gauss with 113061 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 113061 traces tagged "gauss"
FrameSaver: q=3.88874e+06 n=1969059 tag=gauss
wclsFrameSaver: saving 150174 traces tagged "wiener"
FrameSaver: q=4.13252e+06 n=1744547 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 27139 XUs and 25157 XVs -> 220 XUVs
C:0 T:1 44693 XUs and 43710 XVs -> 2420 XUVs
C:0 T:2 54011 XUs and 80701 XVs -> 3274 XUVs
C:0 T:3 29570 XUs and 36456 XVs -> 196 XUVs
C:0 T:4 21253 XUs and 21629 XVs -> 271 XUVs
C:0 T:5 86501 XUs and 74462 XVs -> 6738 XUVs
C:0 T:6 138347 XUs and 161996 XVs -> 7184 XUVs
C:0 T:7 53775 XUs and 59618 XVs -> 701 XUVs
C:0 T:8 7494 XUs and 6515 XVs -> 69 XUVs
C:0 T:9 59175 XUs and 52075 XVs -> 8182 XUVs
C:0 T:10 58135 XUs and 57789 XVs -> 6001 XUVs
C:0 T:11 390 XUs and 278 XVs -> 25 XUVs
35281 XUVs total
12462 collection wire objects
32857 potential space points
Neighbour search...
2363903 tests to find 1160710 neighbours
Iterating with no regularization...
Begin: 7.09974e+07
0 6.15023e+07
1 6.07268e+07
2 6.06154e+07
3 6.05837e+07
Now with regularization...
Begin: 4.96628e+07
0 4.96503e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411096393
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411096393
fLifetime: 30000 [us]
13-Dec-2025 22:17:10 CET  Closed output file "reco_stage1.root"
13-Dec-2025 22:17:10 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          214.843       490.64        723.219       502.962       140.912        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00113024    0.00628244     0.0116715    0.00583859     0.0034622       10     
reco:ophit:OpHitFinder                             0.280401      0.349211      0.470654      0.322194      0.063637        10     
reco:opflash:OpFlashFinder                         0.142809      0.239682      0.367107      0.229456      0.0633197       10     
reco:opslicer:OpSlicer                             0.551405      0.725724       1.07819      0.641181      0.165439        10     
reco:caldata:DataPrepModule                         4.82562       5.48943       6.56405       5.20482      0.687822        10     
reco:wclsdatasp:WireCellToolkit                     34.4043       48.8291       68.646        45.4889       12.5966        10     
reco:gaushit:GausHitFinder                          0.77667       1.34127       1.82178       1.37144      0.338525        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000568888   0.000879516   0.00143984    0.00081985    0.000264998      10     
reco:reco3d:SpacePointSolver                        5.18365       20.1083       34.5439       21.5798       9.08407        10     
reco:hitpdune:DisambigFromSpacePoints              0.951951       2.01597       3.32423       1.93037      0.740009        10     
reco:pandora:StandardPandora                        13.2395       48.0196       80.6815       49.8247       19.4473        10     
reco:pandoraWriter:StandardPandora                 0.334782      0.574534      0.782541      0.582281      0.131799        10     
reco:pandoraTrack:LArPandoraTrackCreation           2.63712       10.1128       15.6048       9.80838       3.41619        10     
reco:pandoraShower:LArPandoraShowerCreation         2.64719       13.3837       27.6107       12.9421       6.56241        10     
reco:pandoracalo:Calorimetry                        1.26846       4.80752       6.79548       5.07953       1.53798        10     
reco:pandoracalonosce:Calorimetry                   1.16583       4.52349       6.38682       4.86764       1.4726         10     
reco:pandorapid:Chi2ParticleID                    0.00478906    0.00618069     0.0079634    0.00597486    0.000940565      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0320755     0.0552612     0.0777741     0.056282      0.0126987       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00345864    0.00589798    0.00798792     0.0059377    0.00123946       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.5193        11.9355       18.4117       13.6297       4.60734        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.26659       11.8879       18.6963        12.94        4.83684        10     
reco:emtrkmichelid:EmTrackMichelId                  120.241       202.811       275.713       202.144       46.5073        10     
reco:crttag:SingleCRTMatchingProducer              0.014644      0.0305014     0.0456201     0.0319791    0.00923334       10     
reco:crtreco:TwoCRTMatchingProducer               0.00824371     0.0234333     0.0346106     0.0250557    0.00916315       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.501323      0.989003       1.46512      0.962884       0.24446        10     
reco:pandora2Track:LArPandoraTrackCreation          6.6362        34.0353       54.7559       36.3068       12.9422        10     
reco:pandora2calo:Calorimetry                       2.77618       15.493        26.7026       15.6979       6.14579        10     
reco:pandora2calonosce:Calorimetry                  2.55109       14.8056       22.6985       15.5626       5.42522        10     
reco:pandora2pid:Chi2ParticleID                   0.00717458     0.0136612     0.0176595     0.0141168    0.00306386       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0383366     0.0699969     0.0897093     0.0735803     0.0155178       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0072644     0.0138109     0.0176949     0.0142756    0.00302044       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.07628       11.8067       21.7559       11.5578       5.62498        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.28403       12.1978        20.27        12.4442       5.13018        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.9915        10.3165       18.152        10.3035       4.47312        10     
[art]:TriggerResults:TriggerResultInserter        3.1907e-05    5.56133e-05   0.000111355   4.5077e-05    2.58191e-05      10     
end_path:out1:RootOutput                           6.991e-06    8.1892e-06    1.1708e-05     7.706e-06    1.41935e-06      10     
end_path:out1:RootOutput(write)                     1.87871       3.28411       4.36329       3.37899      0.760742        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4985.84 MB
  Peak resident set size usage (VmHWM): 4186.87 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.25 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.39 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10358420
-rw-r--r--. 1 dune009 dune 2426984278 Dec 13 20:53 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2300771061 Dec 13 20:51 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2145907538 Dec 13 22:17 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1867707079 Dec 13 18:32 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1782079219 Dec 13 18:19 g4_stage1.root
-rw-r--r--. 1 dune009 dune   82957730 Dec 13 22:10 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     242937 Dec 13 22:17 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Dec 13 22:17 mem.db
-rw-r--r--. 1 dune009 dune      62376 Dec 13 22:17 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Dec 13 22:17 time.db
-rw-r--r--. 1 dune009 dune        519 Dec 13 20:53 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Dec 13 18:32 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Dec 13 22:17 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Dec 13 18:12 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Dec 13 22:17 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Dec 13 20:57 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Dec 13 20:53 debugprod.log
justIN time: 2025-12-18 18:08:39 UTC       justIN version: 01.05.03