justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 242114.1@dunegpschedd01.fnal.gov

Jobsub ID242114.1@dunegpschedd01.fnal.gov
Workflow ID9536
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-02 23:14:07
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc01
Last heartbeat2025-11-03 03:38:10
From worker nodeHostnamewn-la-23.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-02 23:15:12
Input filesmonte-carlo-009536-000001
JobscriptExit code0
Real time4h (15688s)
CPU time4h (15571s = 99%)
Max RSS bytes4138680320 (3946 MiB)
Outputting started2025-11-03 03:36:41
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09536/1/001/reco_stage1.root
Finished2025-11-03 03:38:10
Saved logsjustin-logs:242114.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 120458 traces, 0 summary
Retagger: tagging trace set: gauss with 91260 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 91260 traces tagged "gauss"
FrameSaver: q=3.21153e+06 n=1612349 tag=gauss
wclsFrameSaver: saving 120458 traces tagged "wiener"
FrameSaver: q=3.41578e+06 n=1429671 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 32656 XUs and 30706 XVs -> 565 XUVs
C:0 T:1 62428 XUs and 56148 XVs -> 4530 XUVs
C:0 T:2 161704 XUs and 154867 XVs -> 11939 XUVs
C:0 T:3 51840 XUs and 56396 XVs -> 553 XUVs
C:0 T:4 11120 XUs and 12675 XVs -> 112 XUVs
C:0 T:5 45877 XUs and 36881 XVs -> 5087 XUVs
C:0 T:6 41248 XUs and 64076 XVs -> 2424 XUVs
C:0 T:7 18423 XUs and 28437 XVs -> 394 XUVs
C:0 T:8 7259 XUs and 9835 XVs -> 206 XUVs
C:0 T:9 56683 XUs and 112240 XVs -> 7384 XUVs
C:0 T:10 14201 XUs and 10250 XVs -> 2427 XUVs
C:0 T:11 270 XUs and 247 XVs -> 57 XUVs
35678 XUVs total
11578 collection wire objects
32148 potential space points
Neighbour search...
2802724 tests to find 1414266 neighbours
Iterating with no regularization...
Begin: 6.95665e+07
0 5.80512e+07
1 5.71574e+07
2 5.70244e+07
3 5.69849e+07
Now with regularization...
Begin: 4.63712e+07
0 4.63334e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410276650
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410276650
fLifetime: 30000 [us]
03-Nov-2025 04:36:39 CET  Closed output file "reco_stage1.root"
03-Nov-2025 04:36:39 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                            361         446.684       533.645       435.69        53.2355        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00135415    0.00610453     0.0104391     0.0067612    0.00290923       10     
reco:ophit:OpHitFinder                             0.369173      0.430276      0.512659      0.422459      0.0383134       10     
reco:opflash:OpFlashFinder                         0.227961       0.29977      0.360843      0.301766      0.0382505       10     
reco:opslicer:OpSlicer                             0.699364      0.877945       1.03259      0.885676      0.0991425       10     
reco:caldata:DataPrepModule                         6.30175       6.61786       7.89229       6.41788      0.459659        10     
reco:wclsdatasp:WireCellToolkit                     42.5889       53.6541       59.8518       54.1371       4.51767        10     
reco:gaushit:GausHitFinder                          1.13291       1.36262       1.53772       1.4385       0.154739        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000456763   0.00077762    0.00129156     0.0007094    0.000268713      10     
reco:reco3d:SpacePointSolver                        9.36113       17.492        28.8246       16.5193       5.02563        10     
reco:hitpdune:DisambigFromSpacePoints               1.0726        1.64462       2.19491       1.65321       0.35534        10     
reco:pandora:StandardPandora                        23.958        37.5684       58.2147       33.2528       12.1668        10     
reco:pandoraWriter:StandardPandora                 0.428306      0.514696      0.577825      0.516819      0.0491843       10     
reco:pandoraTrack:LArPandoraTrackCreation           5.05091       8.2217        10.3116       8.67394       1.60395        10     
reco:pandoraShower:LArPandoraShowerCreation         6.36801       9.19312       11.8231       9.33715       1.72966        10     
reco:pandoracalo:Calorimetry                        2.50049       4.53659       5.73089       4.73345      0.937159        10     
reco:pandoracalonosce:Calorimetry                   2.4118        4.28099       5.82473       4.28348      0.914394        10     
reco:pandorapid:Chi2ParticleID                    0.00526826    0.00664258     0.0082837    0.00638405    0.000875291      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0476873     0.0558682     0.0636754     0.0554708    0.00462653       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00513983     0.0063124    0.00805112    0.00619644    0.000724736      10     
reco:pandoraShowercalo:ShowerCalorimetry            4.64855       8.93513       13.0439       9.37968       2.14225        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.49003       8.46468       12.1275       8.59589       1.89575        10     
reco:emtrkmichelid:EmTrackMichelId                  179.307       211.929       237.952       209.646       18.9348        10     
reco:crttag:SingleCRTMatchingProducer              0.0114303     0.0213361     0.0332505     0.0216602    0.00659741       10     
reco:crtreco:TwoCRTMatchingProducer                0.010732      0.0141891     0.0248994     0.0129005    0.00392414       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.777374       1.04884       1.24212       1.05472      0.118411        10     
reco:pandora2Track:LArPandoraTrackCreation          16.3193       21.0557       28.6365       19.2755       4.00046        10     
reco:pandora2calo:Calorimetry                       7.56777       10.2467       15.1165       9.89516       2.16108        10     
reco:pandora2calonosce:Calorimetry                  7.29009       10.0003       13.6358       9.45772       2.05208        10     
reco:pandora2pid:Chi2ParticleID                    0.0116188     0.014066      0.018035       0.01409      0.0018117       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0594435     0.0708702     0.0789314     0.0725161    0.00665388       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0113572     0.0141471     0.018179      0.014375     0.00186834       10     
reco:pandora2Shower:LArPandoraShowerCreation        6.2199        8.22694       11.3583       7.90868       1.72688        10     
reco:pandora2Showercalo:ShowerCalorimetry           6.3382        8.62059       11.1759       8.49598       1.41699        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.70455       7.55042       9.52177       7.49001       1.22392        10     
[art]:TriggerResults:TriggerResultInserter        3.6736e-05    6.11632e-05   0.000138787   4.96835e-05    2.868e-05       10     
end_path:out1:RootOutput                           8.312e-06    1.00161e-05   1.7647e-05    9.0485e-06    2.65248e-06      10     
end_path:out1:RootOutput(write)                     2.74316       3.42676       4.40458       3.38959      0.448765        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4944.85 MB
  Peak resident set size usage (VmHWM): 4138.68 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.29 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.11 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10025400
-rw-r--r--. 1 dune009 dune 2359332941 Nov  3 03:20 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2235000586 Nov  3 03:17 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2039723889 Nov  3 04:36 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1820723375 Nov  3 00:37 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1737071774 Nov  3 00:22 g4_stage1.root
-rw-r--r--. 1 dune009 dune   73543490 Nov  3 04:31 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     242922 Nov  3 04:36 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Nov  3 04:36 mem.db
-rw-r--r--. 1 dune009 dune      62376 Nov  3 04:36 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Nov  3 04:36 time.db
-rw-r--r--. 1 dune009 dune        519 Nov  3 03:20 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov  3 00:37 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov  3 04:36 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Nov  3 00:15 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Nov  3 04:36 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Nov  3 03:26 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Nov  3 03:20 debugprod.log
justIN time: 2025-11-04 01:35:25 UTC       justIN version: 01.05.01