justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 259073.1@dunegpschedd02.fnal.gov

Jobsub ID259073.1@dunegpschedd02.fnal.gov
Workflow ID11083
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-08 13:28:30
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc02
Last heartbeat2025-12-08 16:49:51
From worker nodeHostnamewn-da-17.gina.surf.nl
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-08 13:30:19
Input filesmonte-carlo-011083-000001
JobscriptExit code0
Real time3h (11884s)
CPU time3h (11761s = 98%)
Max RSS bytes4373303296 (4170 MiB)
Outputting started2025-12-08 16:48:25
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11083/1/001/reco_stage1.root
Finished2025-12-08 16:49:51
Saved logsjustin-logs:259073.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 134868 traces, 0 summary
Retagger: tagging trace set: gauss with 100105 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 100105 traces tagged "gauss"
FrameSaver: q=3.47446e+06 n=1736300 tag=gauss
wclsFrameSaver: saving 134868 traces tagged "wiener"
FrameSaver: q=3.70389e+06 n=1537502 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 36409 XUs and 24702 XVs -> 761 XUVs
C:0 T:1 57154 XUs and 41937 XVs -> 5514 XUVs
C:0 T:2 91224 XUs and 130935 XVs -> 4073 XUVs
C:0 T:3 46467 XUs and 53239 XVs -> 321 XUVs
C:0 T:4 8367 XUs and 9503 XVs -> 52 XUVs
C:0 T:5 31559 XUs and 32109 XVs -> 6734 XUVs
C:0 T:6 128711 XUs and 151993 XVs -> 5171 XUVs
C:0 T:7 46119 XUs and 55790 XVs -> 457 XUVs
C:0 T:8 8700 XUs and 10375 XVs -> 122 XUVs
C:0 T:9 58637 XUs and 90572 XVs -> 13985 XUVs
C:0 T:10 30136 XUs and 20029 XVs -> 4037 XUVs
C:0 T:11 201 XUs and 140 XVs -> 22 XUVs
41249 XUVs total
10504 collection wire objects
38450 potential space points
Neighbour search...
4855540 tests to find 2292612 neighbours
Iterating with no regularization...
Begin: 7.40804e+07
0 6.24262e+07
1 6.15471e+07
2 6.143e+07
3 6.13932e+07
Now with regularization...
Begin: 5.2282e+07
0 5.22323e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410992578
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410992578
fLifetime: 30000 [us]
08-Dec-2025 17:48:23 CET  Closed output file "reco_stage1.root"
08-Dec-2025 17:48:23 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          258.715       395.436       585.778       377.563       84.1966        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000755498   0.00396171    0.00703929    0.00358723    0.00196588       10     
reco:ophit:OpHitFinder                             0.202318      0.271471      0.321169      0.274963      0.0276644       10     
reco:opflash:OpFlashFinder                         0.111748      0.185362      0.246242      0.187322      0.031146        10     
reco:opslicer:OpSlicer                             0.471247      0.665604      0.817267      0.660672      0.0963223       10     
reco:caldata:DataPrepModule                         4.34197       4.52605       4.73283       4.56644      0.129682        10     
reco:wclsdatasp:WireCellToolkit                     32.3384       52.4824       71.9245       59.0511       14.5328        10     
reco:gaushit:GausHitFinder                          0.89585       1.20622       1.66108       1.20458      0.225593        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000563759   0.00073133    0.000855871   0.000757572   0.00010065       10     
reco:reco3d:SpacePointSolver                        5.79386       13.6218       20.8191       13.5468       4.1785         10     
reco:hitpdune:DisambigFromSpacePoints              0.977491       1.56145       2.51482       1.50229      0.420688        10     
reco:pandora:StandardPandora                        20.6003       35.746        71.0605       33.5053       13.9364        10     
reco:pandoraWriter:StandardPandora                 0.304431      0.416659      0.554471      0.407685      0.0664594       10     
reco:pandoraTrack:LArPandoraTrackCreation           2.4102        4.77846       8.01876       4.58577       1.34843        10     
reco:pandoraShower:LArPandoraShowerCreation         2.93477       5.74442       11.0588       5.33371       2.2416         10     
reco:pandoracalo:Calorimetry                        1.4064        2.60404       4.21165       2.5637       0.680771        10     
reco:pandoracalonosce:Calorimetry                   1.10417       2.41321       3.91028       2.39811      0.669667        10     
reco:pandorapid:Chi2ParticleID                    0.00408486    0.00570626    0.00703641    0.00565872    0.000722049      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0363296     0.0485288     0.0632907     0.047459     0.00707685       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0040661    0.00536867    0.00664531    0.00538993    0.000614327      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.48611       5.71191       10.7027       5.26204       2.20692        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.46329       5.53481       10.3266       5.09121       2.13303        10     
reco:emtrkmichelid:EmTrackMichelId                  153.523       204.643       279.163       199.192       32.9115        10     
reco:crttag:SingleCRTMatchingProducer             0.00767415     0.0120916     0.0166634     0.0120999    0.00252467       10     
reco:crtreco:TwoCRTMatchingProducer               0.00693841     0.0105381     0.0153248     0.0105594    0.00211025       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.622108      0.846233       1.12768      0.812068      0.132041        10     
reco:pandora2Track:LArPandoraTrackCreation          7.82534       16.2432       30.2173       15.5888       5.82762        10     
reco:pandora2calo:Calorimetry                       3.49306       7.8615        14.1763       7.5778        2.78767        10     
reco:pandora2calonosce:Calorimetry                  3.18161       7.41329       13.171        7.0032        2.61991        10     
reco:pandora2pid:Chi2ParticleID                    0.0103445     0.0141212     0.0194683     0.0139899    0.00237244       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0461378     0.0626328     0.0849874     0.0606171    0.00996087       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0102643     0.0140298     0.0195699     0.0140422    0.00239224       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.06006       5.77133       9.74743       5.58916       1.82385        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.94984       6.40093       10.3453       6.0748        2.0497         10     
reco:pandora2Showercalonosce:ShowerCalorimetry       2.773        5.52662       8.37096       5.30633       1.68319        10     
[art]:TriggerResults:TriggerResultInserter        2.8235e-05    3.86108e-05   9.1647e-05    3.15055e-05   1.85022e-05      10     
end_path:out1:RootOutput                           5.22e-06     6.2839e-06    1.0209e-05    5.9715e-06    1.3663e-06       10     
end_path:out1:RootOutput(write)                     2.1519        2.92965       3.95059       2.83821      0.465098        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5175.08 MB
  Peak resident set size usage (VmHWM): 4373.3 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.19 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.81 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10823764
-rw-r--r--. 1 dune009 dune 2522294874 Dec  8 16:40 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2393412853 Dec  8 16:38 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2242826259 Dec  8 17:48 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1961741584 Dec  8 14:47 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1879935239 Dec  8 14:36 g4_stage1.root
-rw-r--r--. 1 dune009 dune   82709350 Dec  8 17:44 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     242809 Dec  8 17:48 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Dec  8 17:48 mem.db
-rw-r--r--. 1 dune009 dune      62376 Dec  8 17:48 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Dec  8 17:48 time.db
-rw-r--r--. 1 dune009 dune        519 Dec  8 16:40 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Dec  8 14:47 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Dec  8 17:48 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Dec  8 14:30 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Dec  8 17:48 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Dec  8 16:45 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Dec  8 16:40 debugprod.log
justIN time: 2025-12-18 18:08:41 UTC       justIN version: 01.05.03