justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 255110.0@dunegpschedd01.fnal.gov

Jobsub ID255110.0@dunegpschedd01.fnal.gov
Workflow ID10490
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-21 21:05:48
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc01
Last heartbeat2025-11-22 01:16:51
From worker nodeHostnamewn-lb-07.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-21 21:07:00
Input filesmonte-carlo-010490-000001
JobscriptExit code0
Real time3h (14042s)
CPU time3h (13885s = 98%)
Max RSS bytes4225937408 (4030 MiB)
Outputting started2025-11-22 01:01:03
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10490/1/001/reco_stage1.root
Finished2025-11-22 01:16:51
Saved logsjustin-logs:255110.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

g tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 110015 traces, 0 summary
Retagger: tagging trace set: gauss with 85688 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 85688 traces tagged "gauss"
FrameSaver: q=3.04143e+06 n=1580821 tag=gauss
wclsFrameSaver: saving 110015 traces tagged "wiener"
FrameSaver: q=3.20373e+06 n=1415570 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 15088 XUs and 21369 XVs -> 183 XUVs
C:0 T:1 37285 XUs and 38623 XVs -> 2483 XUVs
C:0 T:2 69681 XUs and 65788 XVs -> 4034 XUVs
C:0 T:3 32156 XUs and 25932 XVs -> 170 XUVs
C:0 T:4 15647 XUs and 15094 XVs -> 138 XUVs
C:0 T:5 69031 XUs and 53696 XVs -> 6955 XUVs
C:0 T:6 94588 XUs and 93712 XVs -> 3800 XUVs
C:0 T:7 43584 XUs and 41030 XVs -> 315 XUVs
C:0 T:8 6369 XUs and 6850 XVs -> 22 XUVs
C:0 T:9 38544 XUs and 51779 XVs -> 2969 XUVs
C:0 T:10 38773 XUs and 27829 XVs -> 3943 XUVs
C:0 T:11 1024 XUs and 918 XVs -> 100 XUVs
25112 XUVs total
9664 collection wire objects
23332 potential space points
Neighbour search...
1266480 tests to find 682262 neighbours
Iterating with no regularization...
Begin: 5.52285e+07
0 4.81215e+07
1 4.75868e+07
2 4.75116e+07
3 4.74889e+07
Now with regularization...
Begin: 3.9164e+07
0 3.91526e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410657116
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410657116
fLifetime: 30000 [us]
22-Nov-2025 02:01:01 CET  Closed output file "reco_stage1.root"
22-Nov-2025 02:01:01 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          318.769       425.496       601.47        406.473       103.118        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00131934    0.00543239    0.00776142    0.00635029    0.00237736       10     
reco:ophit:OpHitFinder                             0.330367       0.37588      0.417849      0.373901      0.0253067       10     
reco:opflash:OpFlashFinder                         0.172159      0.227836      0.296222      0.232919      0.0363744       10     
reco:opslicer:OpSlicer                             0.569933      0.761655      0.919816      0.763889      0.104841        10     
reco:caldata:DataPrepModule                         6.21341       6.46957       7.07616       6.38444       0.23161        10     
reco:wclsdatasp:WireCellToolkit                     44.5027       61.1706       71.8023       66.4665       9.73662        10     
reco:gaushit:GausHitFinder                         0.992581       1.35285       1.6896        1.30831       0.24358        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000382651   0.000666729   0.000915247   0.000716465   0.000184119      10     
reco:reco3d:SpacePointSolver                        11.5566       18.3493       30.3904       15.839        6.30221        10     
reco:hitpdune:DisambigFromSpacePoints              0.989482       1.65618       2.62231       1.41055      0.574795        10     
reco:pandora:StandardPandora                        21.0792       39.1427       74.2649       32.8041       16.8375        10     
reco:pandoraWriter:StandardPandora                 0.343714      0.469849      0.618336      0.441245      0.0856995       10     
reco:pandoraTrack:LArPandoraTrackCreation           2.85819       5.22042       8.86115       4.87847       1.57401        10     
reco:pandoraShower:LArPandoraShowerCreation         3.21126       6.22352       11.7339       5.47696       2.7063         10     
reco:pandoracalo:Calorimetry                        1.53384       2.85106       4.78026       2.8532       0.989534        10     
reco:pandoracalonosce:Calorimetry                   1.40356       2.70916       4.67044       2.75036      0.984604        10     
reco:pandorapid:Chi2ParticleID                    0.00477133    0.00594667    0.00710346    0.00583052    0.000634437      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0424096     0.0528339     0.0671198     0.0510971     0.0080737       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0045917    0.00558648    0.00673082    0.00546295    0.000589833      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.97352       6.40996       11.594        6.00822       3.09765        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.79694       6.3341        12.7182       5.73005       3.34713        10     
reco:emtrkmichelid:EmTrackMichelId                  164.246       206.999       273.024       199.666       41.055         10     
reco:crttag:SingleCRTMatchingProducer             0.00973461      0.01349      0.0184947     0.0131922    0.00283506       10     
reco:crtreco:TwoCRTMatchingProducer               0.00855638     0.0120695     0.0191357     0.0114042    0.00315788       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.853916       1.00806       1.33323      0.962975      0.144456        10     
reco:pandora2Track:LArPandoraTrackCreation          8.7692        17.6283       30.956        16.4695       7.6596         10     
reco:pandora2calo:Calorimetry                        4.094        8.87334       15.4382       8.46192       4.07286        10     
reco:pandora2calonosce:Calorimetry                  3.86166       8.30637       14.2522       8.05907       3.7775         10     
reco:pandora2pid:Chi2ParticleID                    0.0109341     0.0139042     0.0176254     0.0136417    0.00212481       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0518474     0.0685696     0.0908513     0.0675329     0.0125223       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0111425     0.0138669     0.0173803     0.0136263    0.00202365       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.17172       6.10195       11.2925       5.63606       2.60297        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.43578       7.14311       12.6103       6.92357       3.17272        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.07675        6.079        10.1505       5.89014       2.51866        10     
[art]:TriggerResults:TriggerResultInserter        4.2704e-05    5.48092e-05   0.000132489   4.6199e-05    2.60163e-05      10     
end_path:out1:RootOutput                           7.952e-06    1.18917e-05   3.5452e-05     8.618e-06    8.11262e-06      10     
end_path:out1:RootOutput(write)                     2.50887       3.25951       4.28471       3.07672       0.60898        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5027.38 MB
  Peak resident set size usage (VmHWM): 4225.94 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.29 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.99 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9840572
-rw-r--r--. 1 dune009 dune 2331529166 Nov 22 00:48 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2219218306 Nov 22 00:45 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2006994110 Nov 22 02:01 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1757824547 Nov 21 22:23 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1688192034 Nov 21 22:12 g4_stage1.root
-rw-r--r--. 1 dune009 dune   72375070 Nov 22 01:56 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     242923 Nov 22 02:01 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Nov 22 02:01 mem.db
-rw-r--r--. 1 dune009 dune      62376 Nov 22 02:01 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Nov 22 02:01 time.db
-rw-r--r--. 1 dune009 dune        519 Nov 22 00:48 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov 21 22:23 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov 22 02:01 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Nov 21 22:07 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Nov 22 02:01 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Nov 22 00:53 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Nov 22 00:48 debugprod.log
justIN time: 2026-02-08 22:56:45 UTC       justIN version: 01.06.00