justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 243133.1@dunegpschedd01.fnal.gov

Jobsub ID243133.1@dunegpschedd01.fnal.gov
Workflow ID9601
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-04 19:30:27
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc01
Last heartbeat2025-11-04 23:55:30
From worker nodeHostnamewn-db-11.gina.surf.nl
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-04 19:31:16
Input filesmonte-carlo-009601-000001
JobscriptExit code0
Real time4h (15760s)
CPU time4h (15612s = 99%)
Max RSS bytes4505403392 (4296 MiB)
Outputting started2025-11-04 23:53:57
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09601/1/001/reco_stage1.root
Finished2025-11-04 23:55:30
Saved logsjustin-logs:243133.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

er
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 142724 traces, 0 summary
Retagger: tagging trace set: gauss with 107092 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 107092 traces tagged "gauss"
FrameSaver: q=3.79634e+06 n=1882616 tag=gauss
wclsFrameSaver: saving 142724 traces tagged "wiener"
FrameSaver: q=4.03017e+06 n=1661262 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 28558 XUs and 28900 XVs -> 209 XUVs
C:0 T:1 51983 XUs and 46381 XVs -> 2647 XUVs
C:0 T:2 120251 XUs and 159307 XVs -> 9666 XUVs
C:0 T:3 52232 XUs and 59517 XVs -> 706 XUVs
C:0 T:4 14145 XUs and 15360 XVs -> 123 XUVs
C:0 T:5 99295 XUs and 87002 XVs -> 12817 XUVs
C:0 T:6 87508 XUs and 131662 XVs -> 7950 XUVs
C:0 T:7 35975 XUs and 49076 XVs -> 509 XUVs
C:0 T:8 8649 XUs and 9588 XVs -> 96 XUVs
C:0 T:9 108543 XUs and 164976 XVs -> 32172 XUVs
C:0 T:10 30468 XUs and 19298 XVs -> 3734 XUVs
C:0 T:11 717 XUs and 2420 XVs -> 364 XUVs
70993 XUVs total
13057 collection wire objects
66338 potential space points
Neighbour search...
17557026 tests to find 6856790 neighbours
Iterating with no regularization...
Begin: 8.39426e+07
0 6.47617e+07
1 6.35148e+07
2 6.33312e+07
3 6.32778e+07
Now with regularization...
Begin: 5.16961e+07
0 5.16261e+07
1 5.15788e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410313792
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410313792
fLifetime: 30000 [us]
05-Nov-2025 00:53:55 CET  Closed output file "reco_stage1.root"
05-Nov-2025 00:53:55 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          349.897       534.778       715.483       532.793       106.141        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00124135    0.00617221     0.011262     0.00627935     0.0032618       10     
reco:ophit:OpHitFinder                             0.277272      0.322713      0.363855      0.324613      0.0301614       10     
reco:opflash:OpFlashFinder                         0.169519      0.237268      0.297165      0.234706      0.0417083       10     
reco:opslicer:OpSlicer                             0.656039      0.782918      0.970247       0.78755      0.104057        10     
reco:caldata:DataPrepModule                         4.93975       5.3003        5.67376       5.28416      0.194442        10     
reco:wclsdatasp:WireCellToolkit                     41.5956       68.4034       83.1593       79.3378       16.5583        10     
reco:gaushit:GausHitFinder                          1.03357       1.52229       1.92562       1.51896      0.236896        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000567835   0.00121273    0.00294422    0.000894213   0.000687422      10     
reco:reco3d:SpacePointSolver                        9.19583       24.8561       31.9796       26.7415       7.20919        10     
reco:hitpdune:DisambigFromSpacePoints               1.10869       2.00278       2.71254       2.06513       0.49585        10     
reco:pandora:StandardPandora                        21.0797       52.2231       84.4641       47.1266       19.1539        10     
reco:pandoraWriter:StandardPandora                  0.45577      0.548742      0.681944       0.55142      0.0735289       10     
reco:pandoraTrack:LArPandoraTrackCreation           4.21433       8.54582       11.3876       8.90789       2.41176        10     
reco:pandoraShower:LArPandoraShowerCreation         5.15909       10.6962       19.6615       9.26796       4.34288        10     
reco:pandoracalo:Calorimetry                        2.03222       4.48637       6.51883        4.316        1.37849        10     
reco:pandoracalonosce:Calorimetry                   1.98362       4.18163       6.12309       3.87356       1.32168        10     
reco:pandorapid:Chi2ParticleID                    0.00542635    0.00752924    0.00903218    0.00769706    0.00100918       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.043964      0.0614382     0.0805279     0.0624214     0.0101754       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00525559    0.00690716    0.00784326    0.00709461    0.000776728      10     
reco:pandoraShowercalo:ShowerCalorimetry            4.61268       10.3007       18.9357       8.74837       4.21643        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.15676       10.0882       17.8001       8.64432       3.9758         10     
reco:emtrkmichelid:EmTrackMichelId                  174.968       244.965       307.313       247.421       37.2243        10     
reco:crttag:SingleCRTMatchingProducer              0.0107321     0.023285      0.0351837     0.0221794    0.00754867       10     
reco:crtreco:TwoCRTMatchingProducer               0.00755014     0.0196746     0.0388324     0.0185498    0.00780077       10     
reco:anodepiercerst0:T0RecoAnodePiercers            0.73752       1.02351       1.23874       1.02713      0.147921        10     
reco:pandora2Track:LArPandoraTrackCreation          10.4615       27.4091       36.2899       28.3399       7.51502        10     
reco:pandora2calo:Calorimetry                       4.69131       12.5896       17.6828       12.0197       3.72276        10     
reco:pandora2calonosce:Calorimetry                  4.17373       12.3423       19.3842       11.8403       4.00677        10     
reco:pandora2pid:Chi2ParticleID                    0.0121948     0.0180068     0.0222311     0.018169     0.00276375       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.053347      0.0778026     0.0937035     0.0771586     0.0121062       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0123844     0.0179149     0.0214384     0.0179111    0.00252812       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.82741       9.27696       14.5691       8.22147       2.94704        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.82112       10.0683       15.8332       9.46244       3.22275        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.2974        8.60468       13.8644       8.20234       2.81271        10     
[art]:TriggerResults:TriggerResultInserter        3.0017e-05    5.29989e-05   0.000140859   4.34025e-05   3.00495e-05      10     
end_path:out1:RootOutput                           6.503e-06    9.6143e-06    2.3444e-05     7.555e-06    5.10957e-06      10     
end_path:out1:RootOutput(write)                     2.26747       3.5088        4.38492       3.50076       0.57401        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5307.08 MB
  Peak resident set size usage (VmHWM): 4505.4 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.23 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.28 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11386904
-rw-r--r--. 1 dune009 dune 2637702228 Nov  4 23:22 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2502685999 Nov  4 23:19 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2366516296 Nov  5 00:53 reco_stage1.root
-rw-r--r--. 1 dune009 dune 2080829473 Nov  4 20:55 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1986755010 Nov  4 20:39 g4_stage1.root
-rw-r--r--. 1 dune009 dune   85080840 Nov  5 00:47 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     243592 Nov  5 00:53 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Nov  5 00:53 mem.db
-rw-r--r--. 1 dune009 dune      62376 Nov  5 00:53 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Nov  5 00:53 time.db
-rw-r--r--. 1 dune009 dune        519 Nov  4 23:22 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov  4 20:55 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov  5 00:53 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Nov  4 20:31 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Nov  5 00:53 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Nov  4 23:28 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Nov  4 23:22 debugprod.log
justIN time: 2026-02-11 07:36:31 UTC       justIN version: 01.06.00