justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 232542.1@dunegpschedd02.fnal.gov

Jobsub ID232542.1@dunegpschedd02.fnal.gov
Workflow ID9304
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-24 07:36:44
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce15-multicore
Last heartbeat2025-10-24 13:54:56
From worker nodeHostnamehnode53.pic.es
cpuinfoAMD EPYC 7402P 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-24 07:37:26
Input filesmonte-carlo-009304-000001
JobscriptExit code0
Real time6h (21959s)
CPU time3h (12806s = 58%)
Max RSS bytes4182630400 (3988 MiB)
Outputting started2025-10-24 13:43:26
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09304/1/001/reco_stage1.root
Finished2025-10-24 13:54:56
Saved logsjustin-logs:232542.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

pare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 120921 traces, 0 summary
Retagger: tagging trace set: gauss with 94092 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 94092 traces tagged "gauss"
FrameSaver: q=3.01554e+06 n=1619077 tag=gauss
wclsFrameSaver: saving 120921 traces tagged "wiener"
FrameSaver: q=3.18328e+06 n=1430955 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 34425 XUs and 38345 XVs -> 404 XUVs
C:0 T:1 60544 XUs and 63011 XVs -> 3716 XUVs
C:0 T:2 78554 XUs and 49421 XVs -> 6760 XUVs
C:0 T:3 30359 XUs and 17588 XVs -> 89 XUVs
C:0 T:4 10282 XUs and 10767 XVs -> 120 XUVs
C:0 T:5 37545 XUs and 27337 XVs -> 2616 XUVs
C:0 T:6 84032 XUs and 121347 XVs -> 4894 XUVs
C:0 T:7 31976 XUs and 41027 XVs -> 270 XUVs
C:0 T:8 8029 XUs and 6311 XVs -> 64 XUVs
C:0 T:9 41217 XUs and 49001 XVs -> 3100 XUVs
C:0 T:10 32859 XUs and 22450 XVs -> 3798 XUVs
C:0 T:11 100 XUs and 64 XVs -> 8 XUVs
25839 XUVs total
10087 collection wire objects
24063 potential space points
Neighbour search...
2576705 tests to find 1134300 neighbours
Iterating with no regularization...
Begin: 5.30823e+07
0 4.56594e+07
1 4.51413e+07
2 4.5066e+07
3 4.50451e+07
Now with regularization...
Begin: 3.73905e+07
0 3.73745e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410082119
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410082119
fLifetime: 30000 [us]
24-Oct-2025 15:43:25 CEST  Closed output file "reco_stage1.root"
24-Oct-2025 15:43:25 CEST  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          388.327       673.795       934.278       641.138       181.32         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000807004   0.00415912      0.01461     0.00315988    0.00370759       10     
reco:ophit:OpHitFinder                             0.215779      0.300676      0.365366      0.313367      0.0468253       10     
reco:opflash:OpFlashFinder                         0.123523      0.176512      0.213912      0.181398      0.0262365       10     
reco:opslicer:OpSlicer                             0.484904      0.778074      0.986932      0.800367      0.178039        10     
reco:caldata:DataPrepModule                         4.41265       17.9585       128.15        6.37707       36.7403        10     
reco:wclsdatasp:WireCellToolkit                     33.3261       75.3385       165.383       38.2257       56.7205        10     
reco:gaushit:GausHitFinder                          0.72992       1.27737       1.70895       1.3304        0.27548        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000500485   0.000593316   0.000771364   0.000577983   7.98054e-05      10     
reco:reco3d:SpacePointSolver                        5.55525       39.3196       137.565       16.7495       48.6988        10     
reco:hitpdune:DisambigFromSpacePoints              0.699757       1.55149       2.96602       1.52121      0.597929        10     
reco:pandora:StandardPandora                        17.481        49.4643       229.035       32.8822       60.3103        10     
reco:pandoraWriter:StandardPandora                  0.25404      0.429433       0.61358      0.415907      0.114236        10     
reco:pandoraTrack:LArPandoraTrackCreation           2.19177       16.3778       125.793       3.96625       36.4939        10     
reco:pandoraShower:LArPandoraShowerCreation         1.37669       4.56365       8.08356       4.49058       2.08898        10     
reco:pandoracalo:Calorimetry                        1.0517        2.39552       4.09224       2.55403      0.810088        10     
reco:pandoracalonosce:Calorimetry                  0.973528       2.19943       4.04216       2.09242      0.827367        10     
reco:pandorapid:Chi2ParticleID                    0.00485303    0.00606888     0.0102818    0.00534137    0.00157675       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0325959     0.0559894     0.103416      0.0461039     0.0219738       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00464147    0.00625357    0.00981042    0.00558565    0.00162691       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.30596       4.86318       9.82289       4.31885       2.60094        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.26378       4.87888       9.92616       4.37589       2.86103        10     
reco:emtrkmichelid:EmTrackMichelId                  173.315       368.728       628.569       330.285       138.996        10     
reco:crttag:SingleCRTMatchingProducer             0.00812241     0.0200887     0.0551509     0.0155321     0.013913        10     
reco:crtreco:TwoCRTMatchingProducer               0.00685776     0.0103309     0.0131556     0.0104472    0.00235489       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.744787       13.4199       125.505      0.984442       37.362         10     
reco:pandora2Track:LArPandoraTrackCreation          6.84714       13.6248       20.5046       13.1994       4.16402        10     
reco:pandora2calo:Calorimetry                       2.96048       18.2619       125.43        6.95359       35.757         10     
reco:pandora2calonosce:Calorimetry                  2.41916       6.07986       9.18315       6.29985       1.95825        10     
reco:pandora2pid:Chi2ParticleID                    0.0110416     0.0161772     0.0234556     0.0153914    0.00394359       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0387343     0.0723003     0.0892866     0.0771339     0.0161868       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00946289     0.0149216     0.0197298     0.0148673    0.00301097       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.03698       5.1509        8.0018        5.43651       1.68519        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.93093       5.61004       8.42849       5.63387       1.56694        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.22453       17.6593       130.58        5.1927        37.6852        10     
[art]:TriggerResults:TriggerResultInserter         2.435e-05    3.95922e-05   0.000118843   2.8581e-05    2.77731e-05      10     
end_path:out1:RootOutput                           4.851e-06    5.8351e-06     1.124e-05     5.28e-06     1.82205e-06      10     
end_path:out1:RootOutput(write)                     2.41066       3.05035       3.87283       3.15507      0.421783        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4995.52 MB
  Peak resident set size usage (VmHWM): 4182.63 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.25 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:  15.86 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9946096
-rw-r--r-- 1 dune001 dune 2353195550 Oct 24 13:44 detsim_stage2.root
-rw-r--r-- 1 dune001 dune 2234662028 Oct 24 13:42 detsim_stage1.root
-rw-r--r-- 1 dune001 dune 2026127249 Oct 24 15:43 reco_stage1.root
-rw-r--r-- 1 dune001 dune 1785132096 Oct 24 10:28 g4_stage2.root
-rw-r--r-- 1 dune001 dune 1712797698 Oct 24 09:50 g4_stage1.root
-rw-r--r-- 1 dune001 dune   72254070 Oct 24 15:33 Pandora_Events.pndr
-rw-r--r-- 1 dune001 dune     243277 Oct 24 15:43 jobscript.log
-rw-r--r-- 1 dune001 dune     217088 Oct 24 15:43 mem.db
-rw-r--r-- 1 dune001 dune      62376 Oct 24 15:43 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 dune      36864 Oct 24 15:43 time.db
-rw-r--r-- 1 dune001 dune        519 Oct 24 13:44 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Oct 24 10:28 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Oct 24 15:43 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 dune         52 Oct 24 09:37 all-input-dids.txt
-rw-r--r-- 1 dune001 dune          7 Oct 24 15:43 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune          0 Oct 24 13:55 ProtoDUNE.log
-rw-r--r-- 1 dune001 dune          0 Oct 24 13:44 debugprod.log
justIN time: 2025-11-03 18:30:39 UTC       justIN version: 01.05.01